Once you have built and installed a package, you can create a binary package which can be installed on another system with &man.pkg.add.1;. This saves having to build the same package on a group of hosts and wasting CPU time. It also provides a simple means for others to install your package, should you distribute it. To create a binary package, change into the appropriate directory in pkgsrc, and run make package: &rprompt; cd misc/figlet &rprompt; make package This will build and install your package (if not already done), and then build a binary package from what was installed. You can then use the pkg_* tools to manipulate it. Binary packages are created by default in /usr/pkgsrc/packages, in the form of a gzipped tar file. See for a continuation of the above misc/figlet example. See for information on how to submit such a binary package. See . If you want to get a full set of precompiled binary packages, this section describes how to get them. Beware that the bulk build will remove all currently installed packages from your system! Having an FTP server configured either on the machine doing the bulk builds or on a nearby NFS server can help to make the packages available to everyone. See &man.ftpd.8; for more information. If you use a remote NFS server's storage, be sure to not actually compile on NFS storage, as this slows things down a lot. You may want to set variables in /etc/mk.conf. Look at pkgsrc/mk/defaults/mk.conf for details of the default settings. You will want to ensure that ACCEPTABLE_LICENSES meet your local policy. As used in this example, _ACCEPTABLE=yes accepts all licenses. PACKAGES?= ${_PKGSRCDIR}/packages/${MACHINE_ARCH} WRKOBJDIR?= /usr/tmp/pkgsrc # build here instead of in pkgsrc BSDSRCDIR= /usr/src BSDXSRCDIR= /usr/xsrc # for x11/xservers OBJHOSTNAME?= yes # use work.`hostname` FAILOVER_FETCH= yes # insist on the correct checksum PKG_DEVELOPER?= yes _ACCEPTABLE= yes In pkgsrc/mk/bulk, copy build.conf-example to build.conf and edit it, following the comments in that file. This is the config file that determines where log files are generated after the build, where to mail the build report to, where your pkgsrc tree is located and the user to which user to &man.su.8; to do a cvs update. It is possible to configure the bulk build to perform certain site-specific tasks at the end of the pre-build stage. If the file pre-build.local exists in /usr/pkgsrc/mk/bulk, it will be executed (as a &man.sh.1; script) at the end of the usual pre-build stage. An example use of pre-build.local is to have the line: &rprompt; echo "I do not have enough disk space to build this pig." \ > pkgsrc/misc/openoffice/$BROKENF to prevent the system from trying to build a particular package which requires nearly 3 GB of disk space. As /usr/pkg will be completely deleted at the start of bulk builds, make sure your login shell is placed somewhere else. Either drop it into /usr/local/bin (and adjust your login shell in the passwd file), or (re-)install it via &man.pkg.add.1; from /etc/rc.local, so you can login after a reboot (remember that your current process won't die if the package is removed, you just can't start any new instances of the shell any more). Also, if you use &os; earlier than 1.5, or you still want to use the pkgsrc version of ssh for some reason, be sure to install ssh before starting it from rc.local: ( cd /usr/pkgsrc/security/ssh ; make bulk-install ) if [ -f /usr/pkg/etc/rc.d/sshd ]; then /usr/pkg/etc/rc.d/sshd fi Not doing so will result in you being not able to log in via ssh after the bulk build is finished or if the machine gets rebooted or crashes. You have been warned! :) Make sure you don't need any of the packages still installed. During the bulk build, all packages will be removed! Be sure to remove all other things that might interfere with builds, like some libs installed in /usr/local, etc. then become root and type: &rprompt; cd /usr/pkgsrc &rprompt; sh mk/bulk/build If for some reason your last build didn't complete (power failure, system panic, ...), you can continue it by running: &rprompt; sh mk/bulk/build restart At the end of the bulk build, you will get a summary via mail, and find build logs in the directory specified by FTP in the build.conf file. The bulk builds consist of three steps: 1. pre-build The script updates your pkgsrc tree via (anon)cvs, then cleans out any broken distfiles, and removes all packages installed. 2. the bulk build This is basically make bulk-package with an optimised order in which packages will be built. Packages that don't require other packages will be built first, and packages with many dependencies will be built later. 3. post-build Generates a report that's placed in the directory specified in the build.conf file named broken.html, a short version of that report will also be mailed to the build's admin. During the build, a list of broken packages will be compiled in /usr/pkgsrc/.broken (or .../.broken.${MACHINE} if OBJMACHINE is set), individual build logs of broken builds can be found in the package's directory. These files are used by the bulk-targets to mark broken builds to not waste time trying to rebuild them, and they can be used to debug these broken package builds later. Currently, roughly the following requirements are valid for NetBSD 2.0/i386: 10 GB - distfiles (NFS ok) 8 GB - full set of all binaries (NFS ok) 5 GB - temp space for compiling (local disk recommended) Note that all pkgs will be de-installed as soon as they are turned into a binary package, and that sources are removed, so there is no excessively huge demand to disk space. Afterwards, if the package is needed again, it will be installed via &man.pkg.add.1; instead of building again, so there are no cycles wasted by recompiling. If you don't want all the packages nuked from a machine (rendering it useless for anything but pkg compiling), there is the possibility of doing the package bulk build inside a chroot environment. The first step is to set up a chroot sandbox, e.g. /usr/sandbox. This can be done by using null mounts, or manually. There is a shell script called pkgsrc/mk/bulk/mksandbox which will set up the sandbox environment using null mounts. It will also create a script called sandbox in the root of the sandbox environment, which will allow the null mounts to be activated using the sandbox mount command and deactivated using the sandbox umount command. To set up a sandbox environment by hand, after extracting all the sets from a &os; installation or doing a make distribution DESTDIR=/usr/sandbox in /usr/src/etc, be sure the following items are present and properly configured: Kernel &rprompt; cp /netbsd /usr/sandbox /dev/* &rprompt; cd /usr/sandbox/dev ; sh MAKEDEV all /etc/resolv.conf (for security/smtpd and mail): &rprompt; cp /etc/resolv.conf /usr/sandbox/etc Working(!) mail config (hostname, sendmail.cf): &rprompt; cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail /etc/localtime (for security/smtpd): &rprompt; ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime /usr/src (system sources, for sysutils/aperture, net/ppp-mppe): &rprompt; ln -s ../disk1/cvs . &rprompt; ln -s cvs/src-2.0 src Create /var/db/pkg (not part of default install): &rprompt; mkdir /usr/sandbox/var/db/pkg Create /usr/pkg (not part of default install): &rprompt; mkdir /usr/sandbox/usr/pkg Checkout pkgsrc via cvs into /usr/sandbox/usr/pkgsrc: &rprompt; cd /usr/sandbox/usr &rprompt; cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc Do not mount/link this to the copy of your pkgsrc tree you do development in, as this will likely cause problems! Make /usr/sandbox/usr/pkgsrc/packages and .../distfiles point somewhere appropriate. NFS- and/or nullfs-mounts may come in handy! Edit /etc/mk.conf, see . Adjust mk/bulk/build.conf to suit your needs. If you have set CVS_USER in build.conf, make sure that account exists and can do a cvs ${CVS_FLAGS} update properly! When the chroot sandbox is set up, you can start the build with the following steps: &rprompt; cd /usr/sandbox/usr/pkgsrc &rprompt; sh mk/bulk/do-sandbox-build This will just jump inside the sandbox and start building. At the end of the build, mail will be sent with the results of the build. Created binary pkgs will be in /usr/sandbox/usr/pkgsrc/packages (wherever that points/mounts to/from). In addition to building a complete set of all packages in pkgsrc, the pkgsrc/mk/bulk/build script may be used to build a subset of the packages contained in pkgsrc. By setting SPECIFIC_PKGS in /etc/mk.conf, the variables SITE_SPECIFIC_PKGS HOST_SPECIFIC_PKGS GROUP_SPECIFIC_PKGS USER_SPECIFIC_PKGS will define the set of packages which should be built. The bulk build code will also include any packages which are needed as dependencies for the explicitly listed packages. One use of this is to do a bulk build with SPECIFIC_PKGS in a chroot sandbox periodically to have a complete set of the binary packages needed for your site available without the overhead of building extra packages that are not needed. This section describes how pkgsrc developers can upload binary pkgs built by bulk builds to ftp.NetBSD.org. If you would like to automatically create checksum files for the binary packages you intend to upload, remember to set MKSUMS=yes in your mk/bulk/build.conf. If you would like to PGP sign the checksum files (highly recommended!), remember to set SIGN_AS=username@NetBSD.org in your mk/bulk/build.conf. This will prompt you for your GPG password to sign the files before uploading everything. Then, make sure that you have RSYNC_DST set properly in your mk/bulk/build.conf file, i.e. adjust it to something like one of the following: RSYNC_DST=$CVS_USER@ftp.NetBSD.org:/pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch/upload Please use appropriate values for "pkgsrc-200xQy", "NetBSD-a.b.c" and "arch" here. If your login on ftp.NetBSD.org is different from CVS_USER, write your login directly into the variable, e.g. my local account is "feyrer", but for my login "hubertf", I use: RSYNC_DST=hubertf@ftp.NetBSD.org:/pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch/upload A separate upload directory is used here to allow "closing" the directory during upload. To do so, run the following command on ftp.NetBSD.org next: nbftp% mkdir -p -m 750 /pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch/upload Please note that /pub/NetBSD/packages is only appropriate for packages for the NetBSD operating system. Binary packages for other operating systems should go into /pub/pkgsrc. Before uploading the binary pkgs, ssh authentication needs to be set up. This example shows how to set up temporary keys for the root account inside the sandbox (assuming that no keys should be present there usually): &rprompt; chroot /usr/sandbox chroot-&rprompt; rm $HOME/.ssh/id-dsa* chroot-&rprompt; ssh-keygen -t dsa chroot-&rprompt; cat $HOME/.ssh/id-dsa.pub Now take the output of id-dsa.pub and append it to your ~/.ssh/authorized_keys file on ftp.NetBSD.org. You can remove the key after the upload is done! Next, test if your ssh connection really works: chroot-&rprompt; ssh ftp.NetBSD.org date Use "-l yourNetBSDlogin" here as appropriate! Now after all this works, you can exit the sandbox and start the upload: chroot-&rprompt; exit &rprompt; cd /usr/sandbox/usr/pkgsrc &rprompt; sh mk/bulk/do-sandbox-upload The upload process may take quite some time. Use &man.ls.1; or &man.du.1; on the FTP server to monitor progress of the upload. The upload script will take care of not uploading restricted packages and putting vulnerable packages into the vulnerable subdirectory. After the upload has ended, first thing is to revoke ssh access: nbftp% vi ~/.ssh/authorized_keys Gdd:x! Use whatever is needed to remove the key you've entered before! Last, move the uploaded packages out of the upload directory to have them accessible to everyone: nbftp% cd /pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch nbftp% mv upload/* . nbftp% rmdir upload nbftp% chmod 755 . After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM set of the resulting binary packages to assist in installing packages on other machines. The pkgtools/cdpack package provides a simple tool for creating the ISO 9660 images. cdpack arranges the packages on the CD-ROMs in a way that keeps all the dependencies for a given package on the same CD as that package. Complete documentation for cdpack is found in the cdpack(1) man page. The following short example assumes that the binary packages are left in /usr/pkgsrc/packages/All and that sufficient disk space exists in /u2 to hold the ISO 9660 images. &rprompt; mkdir /u2/images &rprompt; pkg_add /usr/pkgsrc/packages/All/cdpack &rprompt; cdpack /usr/pkgsrc/packages/All /u2/images If you wish to include a common set of files (COPYRIGHT, README, etc.) on each CD in the collection, then you need to create a directory which contains these files. e.g. &rprompt; mkdir /tmp/common &rprompt; echo "This is a README" > /tmp/common/README &rprompt; echo "Another file" > /tmp/common/COPYING &rprompt; mkdir /tmp/common/bin &rprompt; echo "#!/bin/sh" > /tmp/common/bin/myscript &rprompt; echo "echo Hello world" >> /tmp/common/bin/myscript &rprompt; chmod 755 /tmp/common/bin/myscript Now create the images: &rprompt; cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images Each image will contain README, COPYING, and bin/myscript in their root directories.