$NetBSD: patch-ar,v 1.3 2011/11/20 16:20:24 marino Exp $ --- ../FreeBSD/README 2011-11-20 13:18:28.333065000 +0000 +++ README @@ -2,23 +2,30 @@ Tinderbox is a package building system for FreeBSD ports, based on official Portbuild scripts used on pointyhat building cluster. - Tinderbox was written by Joe Marcus Clarke. + Tinderbox was written by Joe Marcus Clarke, and has + been ported to DragonFly BSD by John Marino. You can define multiple jails (base system versions) and multiple portstrees. The combination of jail and portstree is called a build. A Tinderbox jail is not what is understood as a jail in - FreeBSD, it is in fact a given world in a chroot. Tinderbox - supports automatic tracking of dependencies and only rebuilds - packages that changed since last run. Tinderbox has support for - email notification of failed builds. Tinderbox also integrates - well with ccache. + FreeBSD and DragonFly, it is in fact a given world in a + chroot. DragonFly uses NetBSD's pkgsrc to build software which + is known as "packages" rather than "ports". Rather than update + DragonFly Tinderbox to change numerous references to "ports" and + "portstrees", the users of pkgsrc are asked to equate the term + "port" with "package" whenever they encounter it. + + Tinderbox supports automatic tracking of dependencies and only + rebuilds packages that changed since last run. Tinderbox has + support for email notification of failed builds. Tinderbox also + integrates well with ccache. Tinderbox is designed to easily provide package sets of ports you need, for platforms and architectures you need. Tinderbox is also excellent tool for testing new ports and port upgrades, especially for testing dependencies and packing lists. It's also useful for - testing ports on various releases of FreeBSD, since you can run - FreeBSD 7.X/8.X world as a jail on a FreeBSD 9.X host. + testing ports on various releases of DragonFly, since you can run + earlier releases as a jail on the DragonFly-current. -------------------------------------------------------------- @@ -96,34 +103,28 @@ * ports-mgmt/tinderbox which tracks the latest stable release (plus sometimes back-ported fixes from the STABLE branch) - * ports-mgmt/tinderbox-devel which tracks HEAD and might contain - additional experimental patches not (yet) present in the CVS. + The pkgsrc/DragonFly version of Tinderbox repository is + located at https://github.com/jrmarino/pkgsrc-tinderbox -------------------------------------------------------------- Chapter 2 Requirements - Recent FreeBSD (development happens on 9-CURRENT only, but - Tinderbox is known to work on RELENG_7 and RELENG_8 too), Perl 5.8 - or later (lang/perl5.8), and either MySQL 4.1 or later - (databases/mysql41-server) or PostgreSQL 8.2 or later - (databases/postgresql82-server). + Recent DragonFly, Perl 5.8 or later (lang/perl5), and either + MySQL 5.0 or later (databases/mysql51-server) or + PostgreSQL 8.4 or later (databases/postgres84-server). If MySQL is used, the MySQL Perl module must also be installed - (databases/p5-DBD-mysql41). If PostgreSQL is used, the Pg Perl - module must also be installed (databases/p5-DBD-Pg). + (databases/p5-DBD-mysql). If PostgreSQL is used, the Pg Perl + module must also be installed (databases/p5-DBD-postgresql). - Note: Please note, however, that is is recommended that for - MySQL version 5.0 or later be used. There are currently no plans - to make this a requirement, but it may happen in the future. - - PHP 4.x is now EOL, so the web front-end (found in the webui/ - subdirectory) requires PHP 5 (lang/php5), PEAR::MDB2_Driver_mysql - (databases/pear-MDB2_Driver_mysql) or PEAR::MDB2_Driver_pgsql - (databases/pear-MDB2_Driver_pgsql), PHP Session - (www/php5-session), and one of the PHP MySQL - (databases/php5-mysql) or PHP Pgsql (databases/php5-pgsql) - depending on your choice of backend database above. + The web front-end (found in the webui/ subdirectory) + requires PHP 5 (lang/php5), + PEAR::MDB2_Driver_mysql (databases/pear-MDB2_Driver_mysql) or + PEAR::MDB2_Driver_pgsql (databases/pear-MDB2_Driver_pgsql), + and one of the PHP MySQL (databases/php-mysql) or + PHP Pgsql (databases/php-pgsql) depending on your choice of + backend database above. If you will be sending emails from Tinderbox (e.g. build failure or build completion emails) you must install net/p5-Net to get the @@ -218,12 +219,12 @@ 9. Tinderbox can use either NFS or nullfs to mount the required file systems within the build chroots (called Builds in - Tinderbox). If you wish to use nullfs, skip to step 12. + Tinderbox). If you wish to use nullfs (default), skip to step 12. 10. Setup the Tinderbox server as an NFS server by adding the following to /etc/exports: - ${pb} -alldirs -maproot=0:0 localhost + ${pb} -alldirs -maproot=root localhost Important: ${pb} CANNOT be a symlink. It should be a real, fully qualified path (hint: use realpath on your desired @@ -238,84 +239,78 @@ nfs_server_enable="YES" nfs_reserved_port_only="YES" + NullFS will still be used unless the mount point is defined, and + it's defined in the format server:directory. + 12. Create the required Jails using the tc command. A Jail is - nothing more than a version of FreeBSD. For example, to create - a Jail for FreeBSD 8.2-RELEASE: + nothing more than a version of DragonFly. For example, to create + a Jail for DragonFly 2.12-RELEASE: - # cd ${pb}/scripts && ./tc createJail -j 8.2 -d "FreeBSD 8.2-RELEASE" \ - -t RELENG_8_2_0_RELEASE -u CVSUP + # cd ${pb}/scripts && ./tc createJail -j 2.12 -d "DragonFly 2.12-RELEASE" \ + -t DragonFly_RELEASE_2_12 -u GIT or - # cd ${pb}/scripts && ./tc createJail -j 8.2 -d "FreeBSD 8.2-RELEASE" \ - -t 8.2-RELEASE -u LFTP -H ftp.freebsd.org - - The first method will download source via cvsup and use make - world to compile a complete FreeBSD installation. +# cd ${pb}/scripts && ./tc createJail -j 2.12 -d "DragonFly 2.12-RELEASE" \ + -t 2.12.1 -u RELEASE -H http://ftp.dragonflybsd.org/dragonflybsd - The second method will instead download binary release sets - (used on CDs) and install them, making the process much - shorter. As the command suggests, the second method requires - lftp (ftp/lftp) to be installed. You need to specify what - release (not the CVS tag, as opposed to the first method) you - want to download with the -t option. Also note that you need - to provide an FTP server to download the sets from (with the - -H option). - - Important: All Jail names MUST begin with their FreeBSD - major version number. That is, the following is an illegal - jail name: "FreeBSD-8.2". - - Tip: It is recommended that the Jail begin with the FreeBSD - major.minor version (i.e. "8.2-FreeBSD" instead of just - "8-FreeBSD") as this may prove useful when using things such + Note:The protocol for DragonFly host servers can be either + http or ftp. The protocol and the path up but not including the + repository directory must be provided. For the RELEASE type, the + directory is called "iso-images" and for the SNAPSHOT type the + directory is called "snapshots". For the RELEASE type, the + must be the release represented by three numbers in + format, and for the SNAPSHOT type, the + must either be the value "LATEST" for the latest snapshot or + in the pattern "YYYYMMDD-DEV-va.b.c.d.gxxxxxx" (YYYYMMDD is the + calendar date, a.b.c is major.minor.point release, d is an + index, and xxxxxx is the first 5 characters of the git commit hash. + + The first method will download source via git and use + make world to compile a complete DragonFly installation. + The next method will download binary release and snapshot ISO + images and install them, making the process much shorter. You + can provide an alternative to the default FTP server to download + the sets from (with the -H option). + + Important: All Jail names MUST begin with their DragonFly + major version number. That is, the following is an illegal + jail name: "DragonFly-2.12". + + Tip: It is recommended that the Jail begin with the DragonFly + major.minor version (i.e. "2.12-DragonFly" instead of just + "2-DragonFly") as this may prove useful when using things such as Hooks (Section 8.8). 13. Create the required PortsTrees using the tc command. A PortsTree is a set of ports you wish to build. A PortsTree - does not have to be a complete FreeBSD ports tree. However, + does not have to be a complete pkgsrc tree. However, all ports within a tree must have all of their dependencies within the same tree. - For example, to create a portstree that tracks the full - FreeBSD ports tree: + For example, to create a portstree based on 2011Q3 quarterly + branch that tracks the full pkgsrc tree: + +# cd ${pb}/scripts && ./tc createPortsTree -p pkgsrc \ + -d "pkgsrc tree" \ + -t "pkgsrc-2011Q3" \ + -w http://cvsweb.se.netbsd.org/cgi-bin/bsdweb.cgi/pkgsrc/ - # cd ${pb}/scripts && ./tc createPortsTree -p FreeBSD \ - -d "FreeBSD ports tree" \ - -w http://www.freebsd.org/cgi/cvsweb.cgi/ports/ + Omit the "-t" option if you want the HEAD, or alternatively, + use -t "." instead. 14. Create the required Builds using the tc command. A Build is a combination of a Jail and a PortsTree. The build is the object in which packages are created. To create a build that combines - a 8.2 Jail with the FreeBSD ports tree: + a 2.12 Jail with the DragonFly pkgsrc tree: - # cd ${pb}/scripts && ./tc createBuild -b 8.2-FreeBSD -j 8.2 \ - -p FreeBSD -d "8.2-RELEASE with FreeBSD ports tree" +# cd ${pb}/scripts && ./tc createBuild -b 2.12-DragonFly -j 2.12 \ + -p pkgsrc -d "2.12-RELEASE with DragonFly pkgsrc tree" Note: the recommended way to name Builds is "Jail-PortsTree". - All builds must also begin with their FreeBSD major version + All builds must also begin with their DragonFly major version number. - Note: tc uses cvsup12 as its default cvsup mirror. If you would - like to use another server or enable cvsup compression use the - -H and -C line arguments to the tc script respectively. For - example, to use cvsup2.freebsd.org and enable cvsup compression - for all Jail updates, use the following command: - - # cd ${pb}/scripts && ./tc createJail -j 8-STABLE \ - -d "FreeBSD 8-STABLE" -t RELENG_8 -u CVSUP -C \ - -H cvsup2.freebsd.org - - Tip: It is possible to suppress spurious setuid warnings from - periodic if ${pb} is on a dedicated partition. Simply add the - nosuid flag to the partition; for UFS in /etc/fstab: - - /dev/ad5s1f /usr/local/tinderbox ufs rw,nosuid 1 1 - - and for ZFS: - - # zfs set setuid=off $(zfs list ${pb} | tail -n 1 | cut -d ' ' -f 1) - -------------------------------------------------------------- Chapter 4 Upgrading @@ -359,9 +354,9 @@ Where {BUILD} is the name of the Build for which this port should be built and {PORT DIRECTORY} is the directory within the PortsTree where this port can be found. For example, to build the - GNOME 2 Desktop port for the Build "8.2-FreeBSD": + Joe editor port for the Build "2.12-DragonFly": - # cd ${pb}/scripts && ./tc addPort -b 8.2-FreeBSD -d x11/gnome2 + # cd ${pb}/scripts && ./tc addPort -b 2.12-DragonFly -d editors/joe Note: A port does not have to be added to the database for Tinderbox to build it. If you just want to do a quick ad hoc @@ -376,10 +371,9 @@ # cd ${pb}/scripts && ./tc tinderbuild -b {BUILD} {PORT DIRECTORY} - For example, to build the GNOME 2 Desktop for the Build - 8.2-FreeBSD: + For example, to build the Joe editor for the Build 2.12-DragonFly: - # cd ${pb}/scripts && ./tc tinderbuild -b 8.2-FreeBSD x11/gnome2 +# cd ${pb}/scripts && ./tc tinderbuild -b 2.12-DragonFly editors/joe Tip: The example above will run the build in the foreground with all messages and errors echoing to the terminal. To capture all @@ -388,13 +382,13 @@ * Bourne shell equivalents: - # cd ${pb}/scripts && ./tc tinderbuild -b 8.2-FreeBSD \ - x11/gnome2 > ${pb}/builds/8.2-FreeBSD/build.log 2>&1 & +# cd ${pb}/scripts && ./tc tinderbuild -b 2.12-DragonFly \ + editors/joe > ${pb}/builds/2.12-DragonFly/build.log 2>&1 & * C shell equivalents: - # cd ${pb}/scripts && ./tc tinderbuild -b 8.2-FreeBSD \ - x11/gnome2 >& ${pb}/builds/8.2-FreeBSD/build.log & +# cd ${pb}/scripts && ./tc tinderbuild -b 2.12-DragonFly \ + editors/joe >& ${pb}/builds/2.12-DragonFly/build.log & More advanced logging options are also available. All log files including tinderbuild output, make logs, and individual port @@ -404,7 +398,7 @@ If set to a directory, a subdirectory will be created in the format of {BUILD}-{DATE} where {BUILD} is the Build name, and {DATE} is the date in the format YYYYMMDDHHMMSS (e.g. - 8.2-FreeBSD-20091012112105). Within this subdirectory will be + 2.6-DragonFly-20091012112105). Within this subdirectory will be the tinderbuild log, and symlinks to the make logs and individual port build logs. If you would rather have the log files copied to this location (instead of using symlinks), set @@ -432,8 +426,6 @@ Table 5-2. tinderbuild Commands -init updates the Jail then updates the Build - -nullfs uses nullfs instead of NFS to mount Jail and - PortsTree file systems -cleanpackages removes all packages already built fors the specified Build -updateports updates the Build's PortsTree (NOTE: dangerous if @@ -452,8 +444,8 @@ -fetch-original ignores the distfile cache, and fetches all distfiles from their respective sources -nolog disables log analysis code - -onceonly only performs one build pass (i.e. tinderbuild - Phase 1) + -twice Performs a second build pass (tinderbuild Phase 1) + after a build failure. -norebuild do not force a rebuild of packages specified on the command line @@ -472,23 +464,23 @@ To update existing Jails: - # cd ${pb}/scripts && ./tc makeJail -j 8.2 + # cd ${pb}/scripts && ./tc makeJail -j 2.12 The output of the Jail build will go to stdout. The output of - update (e.g cvsup) will go to ${pb}/jails/{JAIL}/update.log (where + update (e.g git) will go to ${pb}/jails/{JAIL}/update.log (where {JAIL} is the name of the Jail in question). To update existing PortsTrees, use tc with the updatePortsTree command. For example: - # cd ${pb}/scripts && ./tc updatePortsTree -p FreeBSD +# cd ${pb}/scripts && ./tc updatePortsTree -p DragonFly If you want to clone various aspects of an existing Build, use the tc application with the copyBuild command. For example: - # cd ${pb}/scripts && ./tc copyBuild -s 8.2-FreeBSD -d 9-FreeBSD +# cd ${pb}/scripts && ./tc copyBuild -s 2.12-DragonFly -d 2.13-DragonFly - The source and destination Builds (i.e. 8.2-FreeBSD and 9-FreeBSD + The source and destination Builds (i.e. 2.12-DFly and 2.13-DFly respectively in this example) must already exist. By default, copyBuild will copy the environment file, OPTIONS, and ports from the source Build to the destination Build. It can also optionally @@ -597,47 +589,48 @@ 8.1 Alternative Means of Creating Jail Sources and PortsTrees - By default, Jails and PortsTrees are updated using csup - (/usr/bin/csup). Every time a new Jail or PortsTree is created, - they will inherit the default update type. This update type is - stored in ${pb}/scripts/etc/env/GLOBAL as defaultUpdateType. The - allowed values are CVSUP, CSUP, LFTP, USER, and NONE. For CVSUP - and CSUP there is also a defaultUpdateHost. This can also be - changed in ${pb}/scripts/etc/env/GLOBAL. + PortsTrees are updated using csup (/usr/pkg/bin/csup) and Jails + are updated by fetch or git. Variables controlling how these + are updated are stored at ${pb}/scripts/etc/env/GLOBAL: + + defaultUpdateHost: The host name of the cvsup server that + creates and updates the NetBSD pkgsrc tree + defaultUpdateType: CSUP. Don't modify. + defaultDragonHost: The protocol + host + path-to-iso-images + directory that contains DragonFly release ISO + images. The protocol can be http:// or ftp:// + and the path-to-iso-images (which does not + include "/iso-images") can be omitted if the + it's a top level directory. + defaultDragonType: RELEASE (for Release ISO) + SNAPSHOT (for daily build ISOs) + GIT (to build world from source) + defaultGitSrcHost: The host name of the DragonFly Git mirror A Jail or PortsTree's update type can also be set when creating the Jail or PortsTree. To do this, specify the -u option to the - appropriate create command. If the value of the update type is - CVSUP, then /usr/local/bin/cvsup will be used to update the Jail - or PortsTree. If the value is NONE, then no updates can be + appropriate create command. If the value is NONE, then no updates can be performed once the Jail or PortsTree is created. Instead, it is assumed these trees already exist in the appropriate format. If the value is USER, then an update.sh script must be created under the Jail or PortsTree's root directory. This script will be called when an update is required. For example, if you have a Jail - called "8.2", an executable update.sh script must be placed under - ${pb}/jails/8.2. If you have a PortsTree called "FreeBSD", an + called "2.2", an executable update.sh script must be placed under + ${pb}/jails/2.2. If you have a PortsTree called "DragonFly", an executable update.sh script must be placed under - ${pb}/portstrees/FreeBSD. + ${pb}/portstrees/DragonFly. -------------------------------------------------------------- 8.2 Alternative Mounting If you want to mount /ports inside your PortsTree or /src inside - your Jail via nullfs or NFS from another location, use -m switch - to create. + your Jail via NFS from another location, use -m switch to create. Examples for NFS: - - # ./tc createPortsTree -p FreeBSD -m server:/directory - # ./tc createJail -j 8-FreeBSD -m server:/directory - - Examples for nullfs: - - # ./tc createPortsTree -p FreeBSD -m /directory - # ./tc createJail -j 8-FreeBSD -m /directory +# ./tc createPortsTree -p DragonFly -m server:/directory +# ./tc createJail -j 2.8-DragonFly -m server:/directory Tinderbuild will ensure that these file systems are correctly mounted so you do not need to mount them by your own before @@ -662,7 +655,7 @@ # ./tc configDistfile -c localhost:${distfiles} - * nullfs: + * nullfs (default): # ./tc configDistfile -c ${distfiles} @@ -719,8 +712,8 @@ PERL_VERSION=5.6.2 Likewise, if you want to enable debugging for a particular - PortsTree (e.g. "FreeBSD-debug"), create the following - portstree.FreeBSD-debug in ${pb}/scripts/etc/env/: + PortsTree (e.g. "DragonFly-debug"), create the following + portstree.DragonFly-debug in ${pb}/scripts/etc/env/: CFLAGS="-O -g -pipe" STRIP= @@ -740,27 +733,14 @@ 8.7 Configuring port OPTIONS - It is possible to manipulate port OPTIONS on a per-Build basis. To - do this, create a directory under which all build OPTIONS - sub-directories will go (e.g. ${pb}/options/). For each Build that - you wish to use to use OPTIONS, create a subdirectory named for - that build (e.g. 8-STABLE-FreeBSD/). This directory should look - like /var/db/ports/ in that it contains sub-directories for each - OPTIONS-supporting port. - - For example, if you wanted to build net/wireshark with RTP support - for the Build 8-STABLE-FreeBSD, you would have the following - directory structure: - - ${pb}/options/8-STABLE-FreeBSD/wireshark/options - - The contents of the wireshark/options file would be: - - _OPTIONS_READ=wireshark-0.99.4 - WITH_RTP=true - WITH_SNMP=true - WITH_ADNS=true - WITH_PCRE=true + It is possible to manipulate port OPTIONS on a per-Build basis. + To do this, create a directory under which all build OPTIONS + sub-directories will go (e.g. ${pb}/options/). For each Build + that you wish to use to use OPTIONS, create a subdirectory + named for that build (e.g. 2.12-DragonFly/), and place a single + text file named "pkg_options" in that directory. This file will + contain the customized options for every port in the build on a + per-distname basis. Once this structure is setup, then enable OPTIONS support in Tinderbox, and specify the path to the OPTIONS directory @@ -776,15 +756,52 @@ -e enable OPTIONS support -d disable OPTIONS support - -o set an OPTIONS source directory tree + -o set an OPTIONS source directory tree - If you would like to interactively choose OPTIONS for your ports, - specify the -o option to "./tc addPort". This will perform a make - config on all ports, displaying the ncurses OPTIONS dialog. All - OPTIONS settings will be automatically saved to the correct - location. Each time addPort or rescanPorts is run with the -o - flag, all existing OPTIONS settings will be purged. If you want to - retain existing OPTIONS, use the -O flag instead. + As an example of a per-distname basis, net/wget-1.12 and + net/wget-1.13.4 are considered two separate ports. If you had set + the options for version 1.12, then you would have to create a new + entry for net/wget-1.13.4 when it's time to upgrade that port. + + Each line reports a port, and the line format is as follows: + :: + + The would be wget-1.13.4 in this example, not + wget-1.13.4nb2. Leave off the nbX suffixes, otherwise the entry + will be ignored. + + The is normally empty. For single CPU systems, + there is no point to set this. SMP systems will assume a job + number of 2 * number of CPUs + 1. If the port crashes and it's + suspect that it's an jobs-unsafe issue, set to 1 + and try the port again. + + The is the same as you'd see in the + $LOCALBASE/etc/mk.conf file. + + sample pkg_options file: + gawk-3.1.8::portals + gcc-4.6.1:5:-gcc-fortran -gcc-objc nls + heirloom-070715:1: + + Note that heirloom doesn't use the options framework, but we + still need to have an entry for it to override the MAKE_JOBS. + + Unique to Tinderbox/pkgsrc is a file named + ${pb}/scripts/lib/boilerplate_mk.conf, and it contains settings + for mk.conf that aren't found in the standard ISO/snapshot make.conf + files. For example, it sets all licenses to be skipped and to + build vulnerable ports. It should not be modified since the digest + hash won't match when it's time to deinstall tinderbox-dragonfly. + + There is another way to add settings, but on a per-build basis. In the + same directory as the pkg_options file (${optionsDir}/${build}), + one may add a file named "more_mk.conf". This file gets appended + to the standard + boilerplate make.conf. It actively filters out + MAKE_JOBS and PKG_OPTIONS* settings. Remember, pkg_options takes care + of those. The file more_mk.conf should contain settings like: + PYTHON_VERSION_DEFAULT= 27 + PGSQL_VERSION_DEFAULT= 90 -------------------------------------------------------------- @@ -904,14 +921,14 @@ 8.11 Running Tinderbox in a Jail - Tinderbox can be run in a FreeBSD jail with some concessions. + Tinderbox can be run in a DragonFly jail with some concessions. Since neither NFS nor nullfs (at the time of writing) are usable in a jail, we cannot simply install and use as normal. Instead, we simply use a chroot. 1. Create a jail for your Tinderbox using the instructions in the - FreeBSD Handbook. + DragonFly Handbook. 2. Install dependencies into the jail (Chapter 2). @@ -938,7 +955,6 @@ tinderd_enable="YES" tinderd_chroot="${jail_dir}" tinderd_directory="${pb}" - tinderd_flags="-nullfs" 7. Install a web server of your choice in the jail, and follow the webserver setup.