Creating binary packages
Building a single binary package
Once you have built and installed a package, you can create a
binary package which can be installed on another
system with &man.pkg.add.1; This saves having to build the same package on
a group of hosts and wasting CPU time. It also provides a simple means
for others to install your package, should you distribute it.
To create a binary package, change into the appropriate
directory in pkgsrc, and run make package:
&rprompt; cd misc/figlet
&rprompt; make package
This will build and install your package (if not already done), and then
build a binary package from what was installed. You can then use the
pkg_* tools to manipulate it. Binary packages are
created by default in /usr/pkgsrc/packages, in the
form of a gzipped tar file. See for
a continuation of the above misc/figlet example.
See for information on how to submit such a
binary package.
Settings for creation of binary packages
See .
Doing a bulk build of all packages
If you want to get a full set of precompiled binary packages, this section
describes how to get them. Beware that the bulk build will remove all
currently installed packages from your system! Having a FTP server
configured either on the machine doing the bulk builds or on a nearby NFS
server can help to make the packages available to everyone. See
&man.ftpd.8; for
more information. If you use a remote NFS server's storage, be sure to not
actually compile on NFS storage, as this slows things down a lot.
Configuration
/etc/mk.conf
You may want to set things in /etc/mk.conf.
Look at pkgsrc/mk/bsd.pkg.defaults.mk for
details of the default settings. You will want to ensure that
ACCEPTABLE_LICENSES meet your local policy.
As used in this example, _ACCEPTABLE=yes
accepts all licenses.
PACKAGES?= ${_PKGSRCDIR}/packages/${MACHINE_ARCH}
WRKOBJDIR?= /usr/tmp/pkgsrc # build here instead of in pkgsrc
BSDSRCDIR= /usr/src
BSDXSRCDIR= /usr/xsrc # for x11/xservers
OBJHOSTNAME?= yes # use work.`hostname`
FAILOVER_FETCH= yes # insist on the correct checksum
PKG_DEVELOPER?= yes
_ACCEPTABLE= yes
build.conf
In pkgsrc/mk/bulk, copy
build.conf-example to
build.conf and
edit it, following the comments in that file. This is the config
file that determines where log files are generated after the build,
where to mail the build report to, where your pkgsrc tree is located and
which user to &man.su.8; to to do a cvs update.
pre-build.local
It is possible to configure the bulk build to perform certain site
specific tasks at the end of the pre-build stage. If the file
pre-build.local exists in
/usr/pkgsrc/mk/bulk it will be executed
(as a sh(1) script) at the end of the usual pre-build stage. An
example use of pre-build.local is to have the
line:
&rprompt; echo "I do not have enough disk space to build this pig." \
> pkgsrc/games/crafty-book-enormous/$BROKENF
to prevent the system from trying to build a particular package
which requires nearly 3 GB of disk space.
Other environmental considerations
As /usr/pkg will be completely deleted at the
start of bulk builds, make sure your login shell is placed somewhere
else. Either drop it into /usr/local/bin
(and adjust your login shell in the passwd file), or (re-)install
it via &man.pkg.add.1; from
/etc/rc.local, so you can login after a reboot
(remember that your current process won't die if the package is
removed, you just can't start any new instances of the shell any more).
Also, if you use &os; earlier than 1.5, or you still want to use the
pkgsrc version of ssh for some reason, be sure to install ssh before
starting it from rc.local:
( cd /usr/pkgsrc/security/ssh ; make bulk-install )
if [ -f /usr/pkg/etc/rc.d/sshd ]; then
/usr/pkg/etc/rc.d/sshd
fi
Not doing so will result in you being not able to log in via ssh
after the bulk build is finished or if the machine gets rebooted
or crashes. You have been warned! :)
Operation
Make sure you don't need any of the packages still installed.
During the bulk build, all packages will be
removed!
Be sure to remove all other things that might
interfere with builds, like some libs installed in
/usr/local, etc. then become root and type:
&rprompt; cd /usr/pkgsrc
&rprompt; sh mk/bulk/build
If for some reason your last build didn't complete (power failure,
system panic, ...), you can continue it by running:
&rprompt; sh mk/bulk/build restart
At the end of the bulk build, you will get a summary via mail, and find
build logs in the directory specified by FTP in the
build.conf file.
What it does
The bulk builds consist of three steps:
1. pre-build
The script updates your pkgsrc tree via (anon)cvs, then cleans
out any broken distfiles, and removes all packages installed.
2. the bulk build
This is basically
make bulk-package
with an optimised
order in which packages will be built. Packages that don't require
other packages will be built first, and packages with many dependencies
will be built later.
3. post-build
Generates a report that's placed in the directory specified
in the build.conf file named
broken.html, a short version of
that report will also be mailed to the build's admin.
During the build, a list of broken packages will be compiled in
/usr/pkgsrc/.broken (or
.../.broken.${MACHINE} if
OBJMACHINE is set),
individual build logs of broken builds can be found in the package's
directory. These files are used by the bulk-targets to mark broken builds
to not waste time trying to rebuild them, and they can be used to debug
these broken package builds later.
Disk space requirements
Currently, roughly the following requirements are valid for
NetBSD 2.0/i386:
10 GB - distfiles (NFS ok)
8 GB - full set of all binaries (NFS ok)
5 GB - temp space for compiling (local disk recommended)
Note that all pkgs will be de-installed as soon as they are turned into a
binary package, and that sources are removed, so there is no excessively huge
demand to disk space. Afterwards, if the package is needed again, it will
be installed via &man.pkg.add.1; instead of building again, so
there are no cycles wasted by recompiling.
Setting up a sandbox for chroot'ed builds
If you don't want all the pkgs nuked from a machine (rendering it useless
for anything but pkg compiling), there is the possibility of doing the pkg
bulk build inside a chroot environment.
The first step to do so is setting up a chroot
sandbox, e.g. /usr/sandbox.
After extracting all the sets from a &os; installation or doing a
make distribution DESTDIR=/usr/sandbox in
/usr/src/etc, be sure the following
items are present and properly configured:
Kernel
&rprompt; cp /netbsd /usr/sandbox
/dev/*
&rprompt; cd /usr/sandbox/dev ; sh MAKEDEV all
/etc/resolv.conf (for security/smtpd and mail):
&rprompt; cp /etc/resolv.conf /usr/sandbox/etc
Working(!) mail config (hostname, sendmail.cf):
&rprompt; cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail
/etc/localtime (for security/smtpd):
&rprompt; ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime
/usr/src (system sources, for sysutils/aperture,
net/ppp-mppe):
&rprompt; ln -s ../disk1/cvs .
&rprompt; ln -s cvs/src-1.6 src
Create /var/db/pkg (not part of default install):
&rprompt; mkdir /usr/sandbox/var/db/pkg
Create /usr/pkg (not part of default install):
&rprompt; mkdir /usr/sandbox/usr/pkg
Checkout pkgsrc via cvs into
/usr/sandbox/usr/pkgsrc:
&rprompt; cd /usr/sandbox/usr
&rprompt; cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc
Do not mount/link this to the copy of your pkgsrc tree
you do development in, as this will likely cause problems!
Make
/usr/sandbox/usr/pkgsrc/packages and
.../distfiles point somewhere
appropriate. NFS- and/or nullfs-mounts may come in handy!
Edit /etc/mk.conf, see .
Adjust mk/bulk/build.conf to suit your needs.
If you have set CVS_USER in
build.conf, make sure that
account exists and can do a cvs ${CVS_FLAGS}
update properly!
When the chroot sandbox is setup, you can start
the build with the following steps:
&rprompt; cd /usr/sandbox/usr/pkgsrc
&rprompt; sh mk/bulk/do-sandbox-build
This will just jump inside the sandbox and start building.
At the end of the build, mail will be sent with the results of the build.
Created binary pkgs will be in
/usr/sandbox/usr/pkgsrc/packages (wherever
that points/mounts to/from).
Building a partial set of packages
In addition to building a complete set of all packages in
pkgsrc, the pkgsrc/mk/bulk/build script
may be used to build a subset of the packages contained in
pkgsrc. By setting defining SPECIFIC_PKGS
in /etc/mk.conf, the variables
SITE_SPECIFIC_PKGS
HOST_SPECIFIC_PKGS
GROUP_SPECIFIC_PKGS
USER_SPECIFIC_PKGS
will define the set of packages which should be built.
The bulk build code will also include any packages which are
needed as dependencies for the explicitly listed packages.
One use of this is to do a bulk build with
SPECIFIC_PKGS in a chroot sandbox
periodically to have a complete set of the binary packages
needed for your site available without the overhead of
building extra packages that are not needed.
Creating a multiple CD-ROM packages collection
After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM
set of the resulting binary packages to assist in installing packages on
other machines. The pkgtools/cdpack package provides a simple
tool for creating the ISO 9660 images. cdpack arranges
the packages on the CD-ROMs in a way that keeps all the dependencies for
given package on the same CD as that package.
Example of cdpack
Complete documentation for cdpack is found in the
cdpack(1) manpage. The following
short example assumes that the binary packages are left in
/usr/pkgsrc/packages/All and that sufficient disk
space exists in /u2 to hold the ISO 9660 images.
&rprompt; mkdir /u2/images
&rprompt; pkg_add /usr/pkgsrc/packages/All/cdpack
&rprompt; cdpack /usr/pkgsrc/packages/All /u2/images
If you wish to include a common set of files
(COPYRIGHT, README, etc.)
on each CD in the collection, then you need to create a directory which
contains these files. e.g.
&rprompt; mkdir /tmp/common
&rprompt; echo "This is a README" > /tmp/common/README
&rprompt; echo "Another file" > /tmp/common/COPYING
&rprompt; mkdir /tmp/common/bin
&rprompt; echo "#!/bin/sh" > /tmp/common/bin/myscript
&rprompt; echo "echo Hello world" >> /tmp/common/bin/myscript
&rprompt; chmod 755 /tmp/common/bin/myscript
Now create the images:
&rprompt; cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images
Each image will contain README,
COPYING, and bin/myscript
in their root directories.