diff options
| author | asau <asau@pkgsrc.org> | 2014-05-31 21:10:04 +0000 |
|---|---|---|
| committer | asau <asau@pkgsrc.org> | 2014-05-31 21:10:04 +0000 |
| commit | 5619fb6ef29f235884ced33c57c109a2d8dac3ee (patch) | |
| tree | 13325d4aeb11b987870da27506e15c99b1f02207 | |
| parent | 7c1104f6d43a769ced74d58be351a4bdbbbbab55 (diff) | |
| download | pkgsrc-5619fb6ef29f235884ced33c57c109a2d8dac3ee.tar.gz | |
regen
| -rw-r--r-- | doc/pkgsrc.html | 625 | ||||
| -rw-r--r-- | doc/pkgsrc.txt | 553 |
2 files changed, 201 insertions, 977 deletions
diff --git a/doc/pkgsrc.html b/doc/pkgsrc.html index 9375031b9b3..4ded3648214 100644 --- a/doc/pkgsrc.html +++ b/doc/pkgsrc.html @@ -135,24 +135,13 @@ builds)</a></span></dt> <dd><dl> <dt><span class="sect1"><a href="#bulk.pre">7.1. Think first, build later</a></span></dt> <dt><span class="sect1"><a href="#bulk.req">7.2. Requirements of a bulk build</a></span></dt> -<dt><span class="sect1"><a href="#bulk.old">7.3. Running an old-style bulk build</a></span></dt> +<dt><span class="sect1"><a href="#bulk.pbulk">7.3. Running a pbulk-style bulk build</a></span></dt> <dd><dl> -<dt><span class="sect2"><a href="#binary.configuration">7.3.1. Configuration</a></span></dt> -<dt><span class="sect2"><a href="#other-environmental-considerations">7.3.2. Other environmental considerations</a></span></dt> -<dt><span class="sect2"><a href="#operation">7.3.3. Operation</a></span></dt> -<dt><span class="sect2"><a href="#what-it-does">7.3.4. What it does</a></span></dt> -<dt><span class="sect2"><a href="#disk-space-requirements">7.3.5. Disk space requirements</a></span></dt> -<dt><span class="sect2"><a href="#setting-up-a-sandbox">7.3.6. Setting up a sandbox for chrooted builds</a></span></dt> -<dt><span class="sect2"><a href="#building-a-partial-set">7.3.7. Building a partial set of packages</a></span></dt> -<dt><span class="sect2"><a href="#bulk-upload">7.3.8. Uploading results of a bulk build</a></span></dt> +<dt><span class="sect2"><a href="#bulk.pbulk.prepare">7.3.1. Preparation</a></span></dt> +<dt><span class="sect2"><a href="#bulk.pbulk.conf">7.3.2. Configuration</a></span></dt> </dl></dd> -<dt><span class="sect1"><a href="#bulk.pbulk">7.4. Running a pbulk-style bulk build</a></span></dt> -<dd><dl> -<dt><span class="sect2"><a href="#bulk.pbulk.prepare">7.4.1. Preparation</a></span></dt> -<dt><span class="sect2"><a href="#bulk.pbulk.conf">7.4.2. Configuration</a></span></dt> -</dl></dd> -<dt><span class="sect1"><a href="#creating-cdroms">7.5. Creating a multiple CD-ROM packages collection</a></span></dt> -<dd><dl><dt><span class="sect2"><a href="#cdpack-example">7.5.1. Example of cdpack</a></span></dt></dl></dd> +<dt><span class="sect1"><a href="#creating-cdroms">7.4. Creating a multiple CD-ROM packages collection</a></span></dt> +<dd><dl><dt><span class="sect2"><a href="#cdpack-example">7.4.1. Example of cdpack</a></span></dt></dl></dd> </dl></dd> <dt><span class="chapter"><a href="#files">8. Directory layout of the installed files</a></span></dt> <dd><dl> @@ -189,7 +178,7 @@ builds)</a></span></dt> <dt><span class="sect1"><a href="#creating.common">10.1. Common types of packages</a></span></dt> <dd><dl> <dt><span class="sect2"><a href="#creating.perl-module">10.1.1. Perl modules</a></span></dt> -<dt><span class="sect2"><a href="#creating.kde-app">10.1.2. KDE applications</a></span></dt> +<dt><span class="sect2"><a href="#creating.kde-app">10.1.2. KDE3 applications</a></span></dt> <dt><span class="sect2"><a href="#creating.python-module">10.1.3. Python modules and programs</a></span></dt> </dl></dd> <dt><span class="sect1"><a href="#creating.examples">10.2. Examples</a></span></dt> @@ -877,24 +866,13 @@ builds)</a></span></dt> <dd><dl> <dt><span class="sect1"><a href="#bulk.pre">7.1. Think first, build later</a></span></dt> <dt><span class="sect1"><a href="#bulk.req">7.2. Requirements of a bulk build</a></span></dt> -<dt><span class="sect1"><a href="#bulk.old">7.3. Running an old-style bulk build</a></span></dt> -<dd><dl> -<dt><span class="sect2"><a href="#binary.configuration">7.3.1. Configuration</a></span></dt> -<dt><span class="sect2"><a href="#other-environmental-considerations">7.3.2. Other environmental considerations</a></span></dt> -<dt><span class="sect2"><a href="#operation">7.3.3. Operation</a></span></dt> -<dt><span class="sect2"><a href="#what-it-does">7.3.4. What it does</a></span></dt> -<dt><span class="sect2"><a href="#disk-space-requirements">7.3.5. Disk space requirements</a></span></dt> -<dt><span class="sect2"><a href="#setting-up-a-sandbox">7.3.6. Setting up a sandbox for chrooted builds</a></span></dt> -<dt><span class="sect2"><a href="#building-a-partial-set">7.3.7. Building a partial set of packages</a></span></dt> -<dt><span class="sect2"><a href="#bulk-upload">7.3.8. Uploading results of a bulk build</a></span></dt> -</dl></dd> -<dt><span class="sect1"><a href="#bulk.pbulk">7.4. Running a pbulk-style bulk build</a></span></dt> +<dt><span class="sect1"><a href="#bulk.pbulk">7.3. Running a pbulk-style bulk build</a></span></dt> <dd><dl> -<dt><span class="sect2"><a href="#bulk.pbulk.prepare">7.4.1. Preparation</a></span></dt> -<dt><span class="sect2"><a href="#bulk.pbulk.conf">7.4.2. Configuration</a></span></dt> +<dt><span class="sect2"><a href="#bulk.pbulk.prepare">7.3.1. Preparation</a></span></dt> +<dt><span class="sect2"><a href="#bulk.pbulk.conf">7.3.2. Configuration</a></span></dt> </dl></dd> -<dt><span class="sect1"><a href="#creating-cdroms">7.5. Creating a multiple CD-ROM packages collection</a></span></dt> -<dd><dl><dt><span class="sect2"><a href="#cdpack-example">7.5.1. Example of cdpack</a></span></dt></dl></dd> +<dt><span class="sect1"><a href="#creating-cdroms">7.4. Creating a multiple CD-ROM packages collection</a></span></dt> +<dd><dl><dt><span class="sect2"><a href="#cdpack-example">7.4.1. Example of cdpack</a></span></dt></dl></dd> </dl></dd> <dt><span class="chapter"><a href="#files">8. Directory layout of the installed files</a></span></dt> <dd><dl> @@ -2249,7 +2227,7 @@ works.</p> <li class="listitem"><p><code class="varname">DEPENDS_TARGET</code>: By default, dependencies are only installed, and no binary package is created for them. You can set this variable to - <code class="literal">package</code> to automatically create binary + <code class="literal">package-install</code> to automatically create binary packages after installing dependencies.</p></li> </ul></div> </div> @@ -2271,14 +2249,6 @@ works.</p> you can set <code class="varname">USE_DESTDIR=no</code>; this setting will be deprecated though, so it's preferable to convert a package to DESTDIR instead.</p> -<p>DESTDIR support changes the behaviour of various targets - slightly. To install a package after building it, use - <code class="literal">package-install</code>. <code class="literal">package</code> and - <code class="literal">install</code> don't do that any - longer. <code class="literal">package-install</code> can be used as - <code class="varname">DEPENDS_TARGET</code>. <code class="literal">bin-install</code> - will ask for the root password to install the package and fail, - <code class="literal">package-install</code> will ask again.</p> <p>With basic DESTDIR support, <strong class="userinput"><code>make clean</code></strong> needs to be run as root.</p> <p>Considering the <code class="filename">foo/bar</code> package, @@ -2298,7 +2268,7 @@ uid=1000(myusername) gid=100(users) groups=100(users),0(wheel) </p> <pre class="programlisting"> -<code class="prompt">$</code> make USE_DESTDIR=yes install +<code class="prompt">$</code> make stage-install </pre> <p> @@ -2306,7 +2276,7 @@ uid=1000(myusername) gid=100(users) groups=100(users),0(wheel) </p> <pre class="programlisting"> -<code class="prompt">$</code> make USE_DESTDIR=yes PACKAGES=$HOME/packages package +<code class="prompt">$</code> make PACKAGES=$HOME/packages package </pre> <p> @@ -2315,7 +2285,7 @@ uid=1000(myusername) gid=100(users) groups=100(users),0(wheel) </p> <pre class="programlisting"> -<code class="prompt">$</code> make USE_DESTDIR=yes PACKAGES=$HOME/packages package-install +<code class="prompt">$</code> make PACKAGES=$HOME/packages install </pre> <p> @@ -2344,18 +2314,32 @@ uid=1000(myusername) gid=100(users) groups=100(users),0(wheel) compilers to invoke when building packages. Valid values are:</p> <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> -<li class="listitem"><p><code class="varname">distcc</code>: - distributed C/C++ (chainable)</p></li> +<li class="listitem"><p><code class="varname">ccc</code>: + Compaq C Compilers (Tru64)</p></li> <li class="listitem"><p><code class="varname">ccache</code>: compiler cache (chainable)</p></li> +<li class="listitem"><p><code class="varname">clang</code>: + Clang C and Objective-C compiler</p></li> +<li class="listitem"><p><code class="varname">distcc</code>: + distributed C/C++ (chainable)</p></li> +<li class="listitem"><p><code class="varname">f2c</code>: + Fortran 77 to C compiler (chainable)</p></li> +<li class="listitem"><p><code class="varname">icc</code>: + Intel C++ Compiler (Linux)</p></li> +<li class="listitem"><p><code class="varname">ido</code>: + SGI IRIS Development Option cc (IRIX 5)</p></li> <li class="listitem"><p><code class="varname">gcc</code>: GNU C/C++ Compiler</p></li> +<li class="listitem"><p><code class="varname">hp</code>: + HP-UX C/aC++ compilers</p></li> <li class="listitem"><p><code class="varname">mipspro</code>: Silicon Graphics, Inc. MIPSpro (n32/n64)</p></li> -<li class="listitem"><p><code class="varname">mipspro</code>: +<li class="listitem"><p><code class="varname">mipspro-ucode</code>: Silicon Graphics, Inc. MIPSpro (o32)</p></li> <li class="listitem"><p><code class="varname">sunpro</code>: Sun Microsystems, Inc. WorkShip/Forte/Sun ONE Studio</p></li> +<li class="listitem"><p><code class="varname">xlc</code>: + IBM's XL C/C++ compiler suite (Darwin/MacOSX)</p></li> </ul></div> <p>The default is <span class="quote">“<span class="quote"><code class="varname">gcc</code></span>”</span>. You can use @@ -2563,31 +2547,20 @@ builds)</h2></div></div></div> <dl> <dt><span class="sect1"><a href="#bulk.pre">7.1. Think first, build later</a></span></dt> <dt><span class="sect1"><a href="#bulk.req">7.2. Requirements of a bulk build</a></span></dt> -<dt><span class="sect1"><a href="#bulk.old">7.3. Running an old-style bulk build</a></span></dt> -<dd><dl> -<dt><span class="sect2"><a href="#binary.configuration">7.3.1. Configuration</a></span></dt> -<dt><span class="sect2"><a href="#other-environmental-considerations">7.3.2. Other environmental considerations</a></span></dt> -<dt><span class="sect2"><a href="#operation">7.3.3. Operation</a></span></dt> -<dt><span class="sect2"><a href="#what-it-does">7.3.4. What it does</a></span></dt> -<dt><span class="sect2"><a href="#disk-space-requirements">7.3.5. Disk space requirements</a></span></dt> -<dt><span class="sect2"><a href="#setting-up-a-sandbox">7.3.6. Setting up a sandbox for chrooted builds</a></span></dt> -<dt><span class="sect2"><a href="#building-a-partial-set">7.3.7. Building a partial set of packages</a></span></dt> -<dt><span class="sect2"><a href="#bulk-upload">7.3.8. Uploading results of a bulk build</a></span></dt> -</dl></dd> -<dt><span class="sect1"><a href="#bulk.pbulk">7.4. Running a pbulk-style bulk build</a></span></dt> +<dt><span class="sect1"><a href="#bulk.pbulk">7.3. Running a pbulk-style bulk build</a></span></dt> <dd><dl> -<dt><span class="sect2"><a href="#bulk.pbulk.prepare">7.4.1. Preparation</a></span></dt> -<dt><span class="sect2"><a href="#bulk.pbulk.conf">7.4.2. Configuration</a></span></dt> +<dt><span class="sect2"><a href="#bulk.pbulk.prepare">7.3.1. Preparation</a></span></dt> +<dt><span class="sect2"><a href="#bulk.pbulk.conf">7.3.2. Configuration</a></span></dt> </dl></dd> -<dt><span class="sect1"><a href="#creating-cdroms">7.5. Creating a multiple CD-ROM packages collection</a></span></dt> -<dd><dl><dt><span class="sect2"><a href="#cdpack-example">7.5.1. Example of cdpack</a></span></dt></dl></dd> +<dt><span class="sect1"><a href="#creating-cdroms">7.4. Creating a multiple CD-ROM packages collection</a></span></dt> +<dd><dl><dt><span class="sect2"><a href="#cdpack-example">7.4.1. Example of cdpack</a></span></dt></dl></dd> </dl> </div> <p>When you have multiple machines that should run the same packages, it is wasted time if they all build their packages themselves from -source. There are two ways of getting a set of binary packages: The old -bulk build system, or the new (as of 2007) parallel bulk build (pbulk) -system. This chapter describes how to set them up so that the packages +source. There is a ways of getting a set of binary packages: +The bulk build system, or pbulk ("p" stands for "parallel). +This chapter describes how to set it up so that the packages are most likely to be usable later.</p> <div class="sect1"> <div class="titlepage"><div><div><h2 class="title" style="clear: both"> @@ -2643,404 +2616,7 @@ temporary filesystems, others must survive a sudden reboot.</p> </div> <div class="sect1"> <div class="titlepage"><div><div><h2 class="title" style="clear: both"> -<a name="bulk.old"></a>7.3. Running an old-style bulk build</h2></div></div></div> -<div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> -<h3 class="title">Note</h3> -<p>There are two ways of doing a bulk build. The old-style -one and the new-style <span class="quote">“<span class="quote">pbulk</span>”</span>. The latter is the recommended -way.</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="binary.configuration"></a>7.3.1. Configuration</h3></div></div></div> -<div class="sect3"> -<div class="titlepage"><div><div><h4 class="title"> -<a name="binary.bulk.build.conf"></a>7.3.1.1. <code class="filename">build.conf</code> -</h4></div></div></div> -<p>The <code class="filename">build.conf</code> file is the main - configuration file for bulk builds. You can configure how your - copy of pkgsrc is kept up to date, how the distfiles are - downloaded, how the packages are built and how the report is - generated. You can find an annotated example file in - <code class="filename">pkgsrc/mk/bulk/build.conf-example</code>. To use - it, copy <code class="filename">build.conf-example</code> to - <code class="filename">build.conf</code> and edit it, following the - comments in that file.</p> -</div> -<div class="sect3"> -<div class="titlepage"><div><div><h4 class="title"> -<a name="binary.mk.conf"></a>7.3.1.2. <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a> -</h4></div></div></div> -<p>You may want to set variables in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>. - Look at <code class="filename">pkgsrc/mk/defaults/mk.conf</code> for - details of the default settings. You will want to ensure that - <code class="varname">ACCEPTABLE_LICENSES</code> meet your local policy. - As used in this example, <code class="varname">SKIP_LICENSE_CHECK=yes</code> - completely bypasses the license check.</p> -<pre class="programlisting"> -PACKAGES?= ${_PKGSRCDIR}/packages/${MACHINE_ARCH} -WRKOBJDIR?= /usr/tmp/pkgsrc # build here instead of in pkgsrc -BSDSRCDIR= /usr/src -BSDXSRCDIR= /usr/xsrc # for x11/xservers -OBJHOSTNAME?= yes # use work.`hostname` -FAILOVER_FETCH= yes # insist on the correct checksum -PKG_DEVELOPER?= yes -SKIP_LICENSE_CHECK= yes -</pre> -<p>Some options that are especially useful for bulk builds - can be found at the top lines of the file - <code class="filename">mk/bulk/bsd.bulk-pkg.mk</code>. The most useful - options of these are briefly described here.</p> -<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> -<li class="listitem"><p>If you are on a slow machine, you may want to - set <code class="varname">USE_BULK_BROKEN_CHECK</code> to - <span class="quote">“<span class="quote">no</span>”</span>.</p></li> -<li class="listitem"><p>If you are doing bulk builds from a read-only - copy of pkgsrc, you have to set <code class="varname">BULKFILESDIR</code> - to the directory where all log files are created. Otherwise the - log files are created in the pkgsrc directory.</p></li> -<li class="listitem"><p>Another important variable is - <code class="varname">BULK_PREREQ</code>, which is a list of packages that - should be always available while building other - packages.</p></li> -</ul></div> -<p>Some other options are scattered in the pkgsrc - infrastructure:</p> -<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> -<li class="listitem"><p><code class="varname">ALLOW_VULNERABLE_PACKAGES</code> - should be set to <code class="literal">yes</code>. The purpose of the - bulk builds is creating binary packages, no matter if they - are vulnerable or not. Leaving this variable unset would - prevent the bulk build system from even trying to build - them, so possible building errors would not show - up.</p></li> -<li class="listitem"><p><code class="varname">CHECK_FILES</code> - (<code class="filename">pkgsrc/mk/check/check-files.mk</code>) can be set to - <span class="quote">“<span class="quote">yes</span>”</span> to check that the installed set of files - matches the <code class="filename">PLIST</code>.</p></li> -<li class="listitem"><p><code class="varname">CHECK_INTERPRETER</code> - (<code class="filename">pkgsrc/mk/check/check-interpreter.mk</code>) can be set to - <span class="quote">“<span class="quote">yes</span>”</span> to check that the installed - <span class="quote">“<span class="quote">#!</span>”</span>-scripts will find their - interpreter.</p></li> -<li class="listitem"><p><code class="varname">PKGSRC_RUN_TEST</code> can be - set to <span class="quote">“<span class="quote"><code class="literal">yes</code></span>”</span> to run each - package's self-test before installing it. Note that some - packages make heavy use of <span class="quote">“<span class="quote">good</span>”</span> random - numbers, so you need to assure that the machine on which you - are doing the bulk builds is not completely idle. Otherwise - some test programs will seem to hang, while they are just - waiting for new random data to be - available.</p></li> -</ul></div> -</div> -<div class="sect3"> -<div class="titlepage"><div><div><h4 class="title"> -<a name="pre-build.local"></a>7.3.1.3. <code class="filename">pre-build.local</code> -</h4></div></div></div> -<p>It is possible to configure the bulk build to perform - certain site-specific tasks at the end of the pre-build - stage. If the file - <code class="filename">pre-build.local</code> exists in - <code class="filename">/usr/pkgsrc/mk/bulk</code>, it will be executed - (as a <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?sh+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">sh</span>(1)</span></a> script) at the end of the usual pre-build - stage. An example use of - <code class="filename">pre-build.local</code> is to have the line:</p> -<pre class="screen">echo "I do not have enough disk space to build this pig." \ - > misc/openoffice/$BROKENF</pre> -<p>to prevent the system from trying to build a particular package - which requires nearly 3 GB of disk space.</p> -</div> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="other-environmental-considerations"></a>7.3.2. Other environmental considerations</h3></div></div></div> -<p>As <code class="filename">/usr/pkg</code> will be completely - deleted at the start of bulk builds, make sure your login - shell is placed somewhere else. Either drop it into - <code class="filename">/usr/local/bin</code> (and adjust your login - shell in the passwd file), or (re-)install it via - <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?pkg_add+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">pkg_add</span>(1)</span></a> from <code class="filename">/etc/rc.local</code>, so - you can login after a reboot (remember that your current - process won't die if the package is removed, you just can't - start any new instances of the shell any more). Also, if you - use NetBSD earlier than 1.5, or you still want to use the pkgsrc - version of ssh for some reason, be sure to install ssh before - starting it from <code class="filename">rc.local</code>:</p> -<pre class="programlisting"> -(cd /usr/pkgsrc/security/ssh && make bulk-install) -if [ -f /usr/pkg/etc/rc.d/sshd ]; then - /usr/pkg/etc/rc.d/sshd -fi -</pre> -<p>Not doing so will result in you being not able to log in - via ssh after the bulk build is finished or if the machine - gets rebooted or crashes. You have been warned! :)</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="operation"></a>7.3.3. Operation</h3></div></div></div> -<p>Make sure you don't need any of the packages still - installed.</p> -<div class="warning" style="margin-left: 0.5in; margin-right: 0.5in;"> -<h3 class="title">Warning</h3> -<p>During the bulk build, <span class="emphasis"><em>all packages, their - configuration files and some more files from - <code class="filename">/var</code>, <code class="filename">/home</code> and - possibly other locations will be removed! So don't run a bulk - build with privileges that might harm your - system.</em></span></p> -</div> -<p>Be sure to remove all other things that might - interfere with builds, like some libs installed in - <code class="filename">/usr/local</code>, etc. then become root and type:</p> -<pre class="screen"> -<code class="prompt">#</code> <strong class="userinput"><code>cd /usr/pkgsrc</code></strong> -<code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/build</code></strong> - </pre> -<p>If for some reason your last build didn't complete (power - failure, system panic, ...), you can continue it by - running:</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/build restart</code></strong></pre> -<p>At the end of the bulk build, you will get a summary via mail, - and find build logs in the directory specified by - <code class="varname">FTP</code> in the <code class="filename">build.conf</code> - file.</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="what-it-does"></a>7.3.4. What it does</h3></div></div></div> -<p>The bulk builds consist of three steps:</p> -<div class="variablelist"><dl class="variablelist"> -<dt><span class="term">1. pre-build</span></dt> -<dd><p>The script updates your pkgsrc tree via (anon)cvs, then - cleans out any broken distfiles, and removes all - packages installed.</p></dd> -<dt><span class="term">2. the bulk build</span></dt> -<dd><p>This is basically <span class="quote">“<span class="quote">make bulk-package</span>”</span> with - an optimised order in which packages will be - built. Packages that don't require other packages will - be built first, and packages with many dependencies will - be built later.</p></dd> -<dt><span class="term">3. post-build</span></dt> -<dd><p>Generates a report that's placed in the directory - specified in the <code class="filename">build.conf</code> file - named <code class="filename">broken.html</code>, a short version - of that report will also be mailed to the build's - admin.</p></dd> -</dl></div> -<p>During the build, a list of broken packages will be compiled - in <code class="filename">/usr/pkgsrc/.broken</code> (or - <code class="filename">.../.broken.${MACHINE}</code> if - <code class="varname">OBJMACHINE</code> is set), individual build logs - of broken builds can be found in the package's - directory. These files are used by the bulk-targets to mark - broken builds to not waste time trying to rebuild them, and - they can be used to debug these broken package builds - later.</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="disk-space-requirements"></a>7.3.5. Disk space requirements</h3></div></div></div> -<p>Currently, roughly the following requirements are valid for - NetBSD 6.99/amd64:</p> -<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> -<li class="listitem"><p>40 GB - distfiles (NFS ok)</p></li> -<li class="listitem"><p>30 GB - full set of all binaries (NFS ok)</p></li> -<li class="listitem"><p>5 GB - temp space for compiling (local disk recommended)</p></li> -</ul></div> -<p>Note that all pkgs will be de-installed as soon as they are - turned into a binary package, and that sources are removed, - so there is no excessively huge demand to disk - space. Afterwards, if the package is needed again, it will - be installed via <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?pkg_add+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">pkg_add</span>(1)</span></a> instead of building again, so - there are no cycles wasted by recompiling.</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="setting-up-a-sandbox"></a>7.3.6. Setting up a sandbox for chrooted builds</h3></div></div></div> -<p>If you don't want all the packages nuked from a machine - (rendering it useless for anything but pkg compiling), there - is the possibility of doing the package bulk build inside a - chroot environment.</p> -<p>The first step is to set up a chroot sandbox, - e.g. <code class="filename">/usr/sandbox</code>. This can be done by - using null mounts, or manually.</p> -<p>There is a shell script called - <code class="filename">mksandbox</code> installed by the <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/pkgtools/mksandbox/README.html" target="_top"><code class="filename">pkgtools/mksandbox</code></a> package, which will set - up the sandbox environment using null mounts. It will also - create a script called <code class="filename">sandbox</code> in the - root of the sandbox environment, which will allow the null - mounts to be activated using the <span class="command"><strong>sandbox - mount</strong></span> command and deactivated using the - <span class="command"><strong>sandbox umount</strong></span> command.</p> -<p>To set up a sandbox environment by hand, after extracting all - the sets from a NetBSD installation or doing a <span class="command"><strong>make - distribution DESTDIR=/usr/sandbox</strong></span> in - <code class="filename">/usr/src/etc</code>, be sure the following items - are present and properly configured:</p> -<div class="procedure"><ol class="procedure" type="1"> -<li class="step"> -<p>Kernel</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cp /netbsd /usr/sandbox</code></strong></pre> -</li> -<li class="step"> -<p><code class="filename">/dev/*</code></p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/dev ; sh MAKEDEV all</code></strong></pre> -</li> -<li class="step"> -<p><code class="filename">/etc/resolv.conf</code> (for <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/security/smtpd/README.html" target="_top"><code class="filename">security/smtpd</code></a> and mail):</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cp /etc/resolv.conf /usr/sandbox/etc</code></strong></pre> -</li> -<li class="step"> -<p>Working(!) mail config (hostname, sendmail.cf):</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail</code></strong></pre> -</li> -<li class="step"> -<p><code class="filename">/etc/localtime</code> (for <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/security/smtpd/README.html" target="_top"><code class="filename">security/smtpd</code></a>):</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime</code></strong></pre> -</li> -<li class="step"> -<p><code class="filename">/usr/src</code> (system sources, - rarely used by packages if at all:</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>ln -s ../disk1/cvs .</code></strong> - <code class="prompt">#</code> <strong class="userinput"><code>ln -s cvs/src-2.0 src</code></strong></pre> -</li> -<li class="step"> -<p>Create <code class="filename">/var/db/pkg</code> (not part of default install):</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>mkdir /usr/sandbox/var/db/pkg</code></strong></pre> -</li> -<li class="step"> -<p>Create <code class="filename">/usr/pkg</code> (not part of default install):</p> -<pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>mkdir /usr/sandbox/usr/pkg</code></strong></pre> -</li> -<li class="step"> -<p>Checkout pkgsrc via cvs into - <code class="filename">/usr/sandbox/usr/pkgsrc</code>:</p> -<pre class="screen"> -<code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/usr</code></strong> -<code class="prompt">#</code> <strong class="userinput"><code>cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc</code></strong> - </pre> -<p>Do not mount/link this to the copy of your pkgsrc tree - you do development in, as this will likely cause problems!</p> -</li> -<li class="step"><p>Make - <code class="filename">/usr/sandbox/usr/pkgsrc/packages</code> and - <code class="filename">.../distfiles</code> point somewhere - appropriate. NFS- and/or nullfs-mounts may come in handy!</p></li> -<li class="step"><p>Edit <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>, see <a class="xref" href="#binary.mk.conf" title="7.3.1.2. mk.conf">Section 7.3.1.2, “<code class="filename">mk.conf</code>”</a>.</p></li> -<li class="step"><p>Adjust <code class="filename">mk/bulk/build.conf</code> to suit your needs.</p></li> -</ol></div> -<p>When the chroot sandbox is set up, you can start - the build with the following steps:</p> -<pre class="screen"> -<code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/usr/pkgsrc</code></strong> -<code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/do-sandbox-build</code></strong> - </pre> -<p>This will just jump inside the sandbox and start building. At - the end of the build, mail will be sent with the results of - the build. Created binary pkgs will be in - <code class="filename">/usr/sandbox/usr/pkgsrc/packages</code> - (wherever that points/mounts to/from).</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="building-a-partial-set"></a>7.3.7. Building a partial set of packages</h3></div></div></div> -<p>In addition to building a complete set of all packages in - pkgsrc, the <code class="filename">pkgsrc/mk/bulk/build</code> script - may be used to build a subset of the packages contained in - pkgsrc. By setting <code class="varname">SPECIFIC_PKGS</code> - in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>, the variables</p> -<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> -<li class="listitem"><p>SITE_SPECIFIC_PKGS</p></li> -<li class="listitem"><p>HOST_SPECIFIC_PKGS</p></li> -<li class="listitem"><p>GROUP_SPECIFIC_PKGS</p></li> -<li class="listitem"><p>USER_SPECIFIC_PKGS</p></li> -</ul></div> -<p>will define the set of packages which should be built. - The bulk build code will also include any packages which are - needed as dependencies for the explicitly listed packages.</p> -<p>One use of this is to do a bulk build with - <code class="varname">SPECIFIC_PKGS</code> in a chroot sandbox - periodically to have a complete set of the binary packages - needed for your site available without the overhead of - building extra packages that are not needed.</p> -</div> -<div class="sect2"> -<div class="titlepage"><div><div><h3 class="title"> -<a name="bulk-upload"></a>7.3.8. Uploading results of a bulk build</h3></div></div></div> -<p>This section describes how pkgsrc developers can upload binary - pkgs built by bulk builds to ftp.NetBSD.org.</p> -<p>If you would like to automatically create checksum files for the - binary packages you intend to upload, remember to set - <code class="varname">MKSUMS=yes</code> in your - <code class="filename">mk/bulk/build.conf</code>.</p> -<p>If you would like to PGP sign the checksum files (highly - recommended!), remember to set - <code class="varname">SIGN_AS=username@NetBSD.org</code> in your - <code class="filename">mk/bulk/build.conf</code>. This will prompt you for - your GPG password to sign the files before uploading everything.</p> -<p>Then, make sure that you have <code class="varname">RSYNC_DST</code> - set properly in your <code class="filename">mk/bulk/build.conf</code> - file, i.e. adjust it to something like one of the following:</p> -<pre class="screen">RSYNC_DST=ftp.NetBSD.org:/pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy/upload</pre> -<p>Please use appropriate values for "20xxQy" (the branch), - "a.b.c" (the OS version) and "arch" here. If your login on ftp.NetBSD.org - is different from your local login, write your login directly - into the variable, e.g. my local account is "feyrer", but for my - login "hubertf", I use:</p> -<pre class="screen">RSYNC_DST=hubertf@ftp.NetBSD.org:/pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy/upload</pre> -<p>A separate <code class="filename">upload</code> directory is used - here to allow "closing" the directory during upload. To do - so, run the following command on ftp.NetBSD.org next:</p> -<pre class="screen">nbftp% <strong class="userinput"><code>mkdir -p -m 750 /pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy/upload</code></strong></pre> -<p>Before uploading the binary pkgs, ssh authentication needs - to be set up. This example shows how to set up temporary keys - for the root account <span class="emphasis"><em>inside the sandbox</em></span> - (assuming that no keys should be present there usually):</p> -<pre class="screen"> -<code class="prompt">#</code> <strong class="userinput"><code>chroot /usr/sandbox</code></strong> -chroot-<code class="prompt">#</code> <strong class="userinput"><code>rm $HOME/.ssh/id-dsa*</code></strong> -chroot-<code class="prompt">#</code> <strong class="userinput"><code>ssh-keygen -t rsa</code></strong> -chroot-<code class="prompt">#</code> <strong class="userinput"><code>cat $HOME/.ssh/id-rsa.pub</code></strong> - </pre> -<p>Now take the output of <code class="filename">id-rsa.pub</code> and - append it to your <code class="filename">~/.ssh/authorized_keys</code> - file on ftp.NetBSD.org. You should remove the key after the - upload is done!</p> -<p>Next, test if your ssh connection really works:</p> -<pre class="screen">chroot-<code class="prompt">#</code> <strong class="userinput"><code>ssh ftp.NetBSD.org date</code></strong> </pre> -<p>Use "-l yourNetBSDlogin" here as appropriate!</p> -<p>Now after all this works, you can exit the sandbox and start - the upload:</p> -<pre class="screen"> -chroot-<code class="prompt">#</code> <strong class="userinput"><code>exit</code></strong> -<code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/usr/pkgsrc</code></strong> -<code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/do-sandbox-upload</code></strong> - </pre> -<p>The upload process may take quite some time. Use <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?ls+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">ls</span>(1)</span></a> or - <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?du+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">du</span>(1)</span></a> on the FTP server to monitor progress of the - upload. The upload script will take care of not uploading - restricted packages.</p> -<p>After the upload has ended, first thing is to revoke ssh access:</p> -<pre class="screen">nbftp% <strong class="userinput"><code>vi ~/.ssh/authorized_keys</code></strong> - Gdd:x! </pre> -<p>Use whatever is needed to remove the key you've entered - before! Last, move the uploaded packages out of the - <code class="filename">upload</code> directory to have them accessible - to everyone:</p> -<pre class="screen"> -nbftp% <strong class="userinput"><code>cd /pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy</code></strong> -nbftp% <strong class="userinput"><code>mv upload/* .</code></strong> -nbftp% <strong class="userinput"><code>rmdir upload</code></strong> -nbftp% <strong class="userinput"><code>chgrp -R netbsd .</code></strong> -nbftp% <strong class="userinput"><code>find . -type d | xargs chmod 775</code></strong> - </pre> -</div> -</div> -<div class="sect1"> -<div class="titlepage"><div><div><h2 class="title" style="clear: both"> -<a name="bulk.pbulk"></a>7.4. Running a pbulk-style bulk build</h2></div></div></div> +<a name="bulk.pbulk"></a>7.3. Running a pbulk-style bulk build</h2></div></div></div> <p>Running a pbulk-style bulk build works roughly as follows:</p> <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> <li class="listitem"><p>First, build the pbulk infrastructure in a fresh pkgsrc location.</p></li> @@ -3048,7 +2624,7 @@ nbftp% <strong class="userinput"><code>find . -type d | xargs chmod 775</code></ </ul></div> <div class="sect2"> <div class="titlepage"><div><div><h3 class="title"> -<a name="bulk.pbulk.prepare"></a>7.4.1. Preparation</h3></div></div></div> +<a name="bulk.pbulk.prepare"></a>7.3.1. Preparation</h3></div></div></div> <p>First, you need to create a pkgsrc installation for the pbulk infrastructure. No matter on which platform you are (even on NetBSD), you should bootstrap into its own directory. Let's take the directory <code class="filename">/usr/pbulk</code> or <code class="filename">$HOME/pbulk</code> for it. This installation will be bootstrapped and all the tools that are required for the bulk build will be installed there.</p> <pre class="screen"> $ <strong class="userinput"><code>cd /usr/pkgsrc</code></strong> @@ -3073,14 +2649,14 @@ $ <strong class="userinput"><code>rm -rf /tmp/pbulk-outer</code></strong> </div> <div class="sect2"> <div class="titlepage"><div><div><h3 class="title"> -<a name="bulk.pbulk.conf"></a>7.4.2. Configuration</h3></div></div></div> +<a name="bulk.pbulk.conf"></a>7.3.2. Configuration</h3></div></div></div> <p>TODO; see pkgsrc/doc/HOWTO-pbulk for more information.</p> <p>TODO: continue writing</p> </div> </div> <div class="sect1"> <div class="titlepage"><div><div><h2 class="title" style="clear: both"> -<a name="creating-cdroms"></a>7.5. Creating a multiple CD-ROM packages collection</h2></div></div></div> +<a name="creating-cdroms"></a>7.4. Creating a multiple CD-ROM packages collection</h2></div></div></div> <p>After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM set of the resulting binary packages to assist in installing packages on other machines. The @@ -3091,7 +2667,7 @@ $ <strong class="userinput"><code>rm -rf /tmp/pbulk-outer</code></strong> CD as that package.</p> <div class="sect2"> <div class="titlepage"><div><div><h3 class="title"> -<a name="cdpack-example"></a>7.5.1. Example of cdpack</h3></div></div></div> +<a name="cdpack-example"></a>7.4.1. Example of cdpack</h3></div></div></div> <p>Complete documentation for cdpack is found in the cdpack(1) man page. The following short example assumes that the binary packages are left in @@ -3726,7 +3302,7 @@ anymore, you can remove that file and run <span class="command"><strong>cvs -q u <dt><span class="sect1"><a href="#creating.common">10.1. Common types of packages</a></span></dt> <dd><dl> <dt><span class="sect2"><a href="#creating.perl-module">10.1.1. Perl modules</a></span></dt> -<dt><span class="sect2"><a href="#creating.kde-app">10.1.2. KDE applications</a></span></dt> +<dt><span class="sect2"><a href="#creating.kde-app">10.1.2. KDE3 applications</a></span></dt> <dt><span class="sect2"><a href="#creating.python-module">10.1.3. Python modules and programs</a></span></dt> </dl></dd> <dt><span class="sect1"><a href="#creating.examples">10.2. Examples</a></span></dt> @@ -3957,7 +3533,7 @@ anymore, you can remove that file and run <span class="command"><strong>cvs -q u <dt><span class="sect1"><a href="#creating.common">10.1. Common types of packages</a></span></dt> <dd><dl> <dt><span class="sect2"><a href="#creating.perl-module">10.1.1. Perl modules</a></span></dt> -<dt><span class="sect2"><a href="#creating.kde-app">10.1.2. KDE applications</a></span></dt> +<dt><span class="sect2"><a href="#creating.kde-app">10.1.2. KDE3 applications</a></span></dt> <dt><span class="sect2"><a href="#creating.python-module">10.1.3. Python modules and programs</a></span></dt> </dl></dd> <dt><span class="sect1"><a href="#creating.examples">10.2. Examples</a></span></dt> @@ -4002,7 +3578,7 @@ Your package may then look like this:</p> <pre class="programlisting"> [...] -BUILD_DEPENDS+= lua>=5.0:../../lang/lua +BUILD_DEPENDS+= libxslt-[0-9]*:../../textproc/libxslt DEPENDS+= screen-[0-9]*:../../misc/screen DEPENDS+= screen>=4.0:../../misc/screen @@ -4060,10 +3636,10 @@ package from the set of installed files.</p></li> </div> <div class="sect2"> <div class="titlepage"><div><div><h3 class="title"> -<a name="creating.kde-app"></a>10.1.2. KDE applications</h3></div></div></div> -<p>KDE applications should always include +<a name="creating.kde-app"></a>10.1.2. KDE3 applications</h3></div></div></div> +<p>KDE3 applications should always include <code class="filename">meta-pkgs/kde3/kde3.mk</code>, which contains numerous -settings that are typical of KDE packages.</p> +settings that are typical of KDE3 packages.</p> </div> <div class="sect2"> <div class="titlepage"><div><div><h3 class="title"> @@ -4397,9 +3973,10 @@ sections.</p> distribution file to be downloaded from the package's website.</p></li> <li class="listitem"><p><code class="varname">PKGNAME</code> is the name of the - package, as used by pkgsrc. You only need to provide it if + package, as used by pkgsrc. You need to provide it if <code class="varname">DISTNAME</code> (which is the default) is not a good - name for the package in pkgsrc. Usually it is the pkgsrc + name for the package in pkgsrc or <code class="varname">DISTNAME</code> is not + provided (no distribution file is required). Usually it is the pkgsrc directory name together with the version number. It must match the regular expression <code class="varname">^[A-Za-z0-9][A-Za-z0-9-_.+]*$</code>, that is, it @@ -4869,8 +4446,16 @@ MESSAGE_SUBST+= SOMEVAR="somevalue" exists.</p> </dd> <dt><span class="term"><code class="filename">ALTERNATIVES</code></span></dt> -<dd><p>FIXME: There is no documentation on the - alternatives framework.</p></dd> +<dd> +<p>This file is used by the alternatives framework. + It creates, configures, and destroys generic wrappers used to + run programs with similar interfaces. + See pkg_alternatives(8) from pkgtools/pkg_alternatives + for more information.</p> +<p>Each line of the file contains two filenames, first + the wrapper and then the alternative provided by the package. + Both paths are relative to <code class="varname">PREFIX</code>.</p> +</dd> </dl></div> </div> <div class="sect2"> @@ -6853,7 +6438,8 @@ GTKDIR_DEFAULT= ${LOCALBASE} below.</p> <p>The variable <code class="varname">DISTFILES</code> specifies the list of distfiles that have to be fetched. Its value - defaults to <code class="literal">${DISTNAME}${EXTRACT_SUFX}</code>, + defaults to <code class="literal">${DEFAULT_DISTFILES}</code> and + its value is <code class="literal">${DISTNAME}${EXTRACT_SUFX}</code>, so that most packages don't need to define it at all. <code class="varname">EXTRACT_SUFX</code> is <code class="literal">.tar.gz</code> by default, but can be changed @@ -6862,7 +6448,7 @@ GTKDIR_DEFAULT= ${LOCALBASE} additional filenames using the <code class="literal">+=</code> operator, but you have write for example:</p> <pre class="programlisting"> -DISTFILES= ${DISTNAME}${EXTRACT_SUFX} additional-files.tar.gz +DISTFILES= ${DEFAULT_DISTFILES} additional-files.tar.gz </pre> <p>Each distfile is fetched from a list of sites, usually <code class="varname">MASTER_SITES</code>. If the package has multiple @@ -6893,8 +6479,19 @@ http://www.somewhereelse.com/mirror/somehow/ MASTER_SITES= http://www.example.com/download.cgi?file= </pre> <p> The exception to this rule are URLs starting with a dash. - In that case the URL is taken as is, fetched and the result stored - under the name of the distfile.</p> + In that case the URL is taken as is, fetched and the result + stored under the name of the distfile. You can use this style + for the case when the download URL style does not match the + above common case. For example, if permanent download URL is a + redirector to the real download URL, or the download file name + is offered by an HTTP Content-Disposition header. In the + following example, <code class="filename">foo-1.0.0.tar.gz</code> will be + created instead of the default + <code class="filename">v1.0.0.tar.gz</code>.</p> +<pre class="programlisting"> +DISTNAME= foo-1.0.0 +MASTER_SITES= -http://www.example.com/archive/v1.0.0.tar.gz +</pre> <p>There are some predefined values for <code class="varname">MASTER_SITES</code>, which can be used in packages. The names of the variables should speak for @@ -6910,13 +6507,18 @@ ${MASTER_SITE_GENTOO} ${MASTER_SITE_GNOME} ${MASTER_SITE_GNU} ${MASTER_SITE_GNUSTEP} +${MASTER_SITE_HASKELL_HACKAGE} ${MASTER_SITE_IFARCHIVE} ${MASTER_SITE_KDE} ${MASTER_SITE_MOZILLA} +${MASTER_SITE_MOZILLA_ALL} +${MASTER_SITE_MOZILLA_ESR} ${MASTER_SITE_MYSQL} +${MASTER_SITE_NETLIB} ${MASTER_SITE_OPENOFFICE} ${MASTER_SITE_PERL_CPAN} ${MASTER_SITE_PGSQL} +${MASTER_SITE_RUBYGEMS} ${MASTER_SITE_R_CRAN} ${MASTER_SITE_SOURCEFORGE} ${MASTER_SITE_SOURCEFORGE_JP} @@ -6925,6 +6527,7 @@ ${MASTER_SITE_SUSE} ${MASTER_SITE_TEX_CTAN} ${MASTER_SITE_XCONTRIB} ${MASTER_SITE_XEMACS} +${MASTER_SITE_XORG} </pre> <p>Some explanations for the less self-explaining ones: <code class="varname">MASTER_SITE_BACKUP</code> contains backup sites @@ -7621,7 +7224,8 @@ ${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS} package already exists, no action is taken. If not, this target will compile, install and package it (and its depends, if <code class="varname">PKG_DEPENDS</code> is set - properly. See <a class="xref" href="#binary.configuration" title="7.3.1. Configuration">Section 7.3.1, “Configuration”</a>). + properly. See <a class="xref" href="#bulk" title="Chapter 7. Creating binary packages for everything in pkgsrc (bulk builds)">Chapter 7, <i>Creating binary packages for everything in pkgsrc (bulk +builds)</i></a>). After creating the binary package, the sources, the just-installed package and its required packages are removed, preserving free disk space.</p> @@ -7736,7 +7340,7 @@ TOOLS_PLATFORM.true?= true # shell builtin <div class="titlepage"><div><div><h2 class="title" style="clear: both"> <a name="tools.questions"></a>18.4. Questions regarding the tools</h2></div></div></div> <div class="qandaset"> -<a name="idm73061424"></a><dl> +<a name="idm25028420"></a><dl> <dt>18.4.1. <a href="#tools.new">How do I add a new tool?</a> </dt> <dt>18.4.2. <a href="#tools.listall">How do I get a list of all available @@ -7755,7 +7359,7 @@ TOOLS_PLATFORM.true?= true # shell builtin <tbody> <tr class="question"> <td align="left" valign="top"> -<a name="tools.new"></a><a name="idm73061040"></a><p><b>18.4.1.</b></p> +<a name="tools.new"></a><a name="idm25028228"></a><p><b>18.4.1.</b></p> </td> <td align="left" valign="top"><p>How do I add a new tool?</p></td> </tr> @@ -7765,7 +7369,7 @@ TOOLS_PLATFORM.true?= true # shell builtin </tr> <tr class="question"> <td align="left" valign="top"> -<a name="tools.listall"></a><a name="idm73060016"></a><p><b>18.4.2.</b></p> +<a name="tools.listall"></a><a name="idm25027716"></a><p><b>18.4.2.</b></p> </td> <td align="left" valign="top"><p>How do I get a list of all available tools?</p></td> @@ -7776,7 +7380,7 @@ TOOLS_PLATFORM.true?= true # shell builtin </tr> <tr class="question"> <td align="left" valign="top"> -<a name="tools.used"></a><a name="idm73058992"></a><p><b>18.4.3.</b></p> +<a name="tools.used"></a><a name="idm25027204"></a><p><b>18.4.3.</b></p> </td> <td align="left" valign="top"><p>How can I get a list of all the tools that a package is using while being built? I want to know whether it @@ -9651,7 +9255,8 @@ PERL5_PACKLIST= auto/Pg/.packlist protect our users! You're still free to put up your home-made binary packages and tell the world where to get them. NetBSD developers doing bulk builds and wanting to upload them please - see <a class="xref" href="#bulk-upload" title="7.3.8. Uploading results of a bulk build">Section 7.3.8, “Uploading results of a bulk build”</a>.</p> + see <a class="xref" href="#bulk" title="Chapter 7. Creating binary packages for everything in pkgsrc (bulk builds)">Chapter 7, <i>Creating binary packages for everything in pkgsrc (bulk +builds)</i></a>.</p> </div> <div class="sect1"> <div class="titlepage"><div><div><h2 class="title" style="clear: both"> @@ -9716,7 +9321,7 @@ PERL5_PACKLIST= auto/Pg/.packlist not the same as your NetBSD login name. The target also automatically removes possibly existing entries for the package in the <code class="filename">TODO</code> file. Don't forget to commit - the changes, e.g. by using <span class="command"><strong>make changes-entry-commit</strong></span>! + the changes, e.g. by using <span class="command"><strong>make commit-changes-entry</strong></span>! If you are not using a checkout directly from cvs.NetBSD.org, but e.g. a local copy of the repository, you can set USE_NETBSD_REPO=yes. This makes the cvs commands use the main repository. @@ -9743,6 +9348,9 @@ you can check by running <span class="quote">“<span class="quote">cvs stat <code class="prompt">$</code> cd .. <code class="prompt">$</code> vi Makefile # add SUBDIRS+=pkgname line <code class="prompt">$</code> cvs commit Makefile +<code class="prompt">$</code> cd pkgname +<code class="prompt">$</code> make CTYPE=Added changes-entry +<code class="prompt">$</code> make commit-changes-entry </pre> <p>The commit message of the initial import should include part of the <code class="filename">DESCR</code> file, so people reading the mailing lists know @@ -9851,7 +9459,7 @@ place.</p></li> and if you still don't have the answer, ask on the <code class="literal">pkgsrc-users</code> mailing list.</p> <div class="qandaset"> -<a name="idm74589232"></a><dl> +<a name="idm30948932"></a><dl> <dt>22.1. <a href="#devfaq.makeflags">What is the difference between MAKEFLAGS, .MAKEFLAGS and MAKE_FLAGS?</a> @@ -9896,7 +9504,7 @@ do?</a> <tbody> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.makeflags"></a><a name="idm74588848"></a><p><b>22.1.</b></p> +<a name="devfaq.makeflags"></a><a name="idm30948740"></a><p><b>22.1.</b></p> </td> <td align="left" valign="top"><p>What is the difference between <code class="varname">MAKEFLAGS</code>, <code class="varname">.MAKEFLAGS</code> and @@ -9912,7 +9520,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.make"></a><a name="idm74584880"></a><p><b>22.2.</b></p> +<a name="devfaq.make"></a><a name="idm30938372"></a><p><b>22.2.</b></p> </td> <td align="left" valign="top"><p>What is the difference between <code class="varname">MAKE</code>, <code class="varname">GMAKE</code> and @@ -9930,7 +9538,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.cc"></a><a name="idm74580400"></a><p><b>22.3.</b></p> +<a name="devfaq.cc"></a><a name="idm30936068"></a><p><b>22.3.</b></p> </td> <td align="left" valign="top"><p>What is the difference between <code class="varname">CC</code>, <code class="varname">PKG_CC</code> and @@ -9948,7 +9556,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.bl3flags"></a><a name="idm74576304"></a><p><b>22.4.</b></p> +<a name="devfaq.bl3flags"></a><a name="idm30933956"></a><p><b>22.4.</b></p> </td> <td align="left" valign="top"><p>What is the difference between <code class="varname">BUILDLINK_LDFLAGS</code>, @@ -9961,7 +9569,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.bl3prefix"></a><a name="idm74574128"></a><p><b>22.5.</b></p> +<a name="devfaq.bl3prefix"></a><a name="idm30932868"></a><p><b>22.5.</b></p> </td> <td align="left" valign="top"><p>Why does <span class="command"><strong>make show-var VARNAME=BUILDLINK_PREFIX.<em class="replaceable"><code>foo</code></em></strong></span> @@ -9977,7 +9585,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.master_sites"></a><a name="idm74570928"></a><p><b>22.6.</b></p> +<a name="devfaq.master_sites"></a><a name="idm30931268"></a><p><b>22.6.</b></p> </td> <td align="left" valign="top"><p>What does <code class="literal">${MASTER_SITE_SOURCEFORGE:=package/}</code> mean? I @@ -10001,7 +9609,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.mailinglists"></a><a name="idm74554672"></a><p><b>22.7.</b></p> +<a name="devfaq.mailinglists"></a><a name="idm30927300"></a><p><b>22.7.</b></p> </td> <td align="left" valign="top"><p>Which mailing lists are there for package developers?</p></td> @@ -10026,7 +9634,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.documentation"></a><a name="idm74550960"></a><p><b>22.8.</b></p> +<a name="devfaq.documentation"></a><a name="idm30925252"></a><p><b>22.8.</b></p> </td> <td align="left" valign="top"><p>Where is the pkgsrc documentation?</p></td> @@ -10074,7 +9682,7 @@ do?</a> </tr> <tr class="question"> <td align="left" valign="top"> -<a name="devfaq.too-much-time"></a><a name="idm74544432"></a><p><b>22.9.</b></p> +<a name="devfaq.too-much-time"></a><a name="idm30921988"></a><p><b>22.9.</b></p> </td> <td align="left" valign="top"><p>I have a little time to kill. What shall I do?</p></td> @@ -10855,23 +10463,6 @@ CFLAGS+= -Wall definitions that are used by pkgsrc. Start by copying one of the other files and edit it to your needs.</p></dd> -<dt><span class="term"><code class="filename">mk/platform/<em class="replaceable"><code>MyOS</code></em>.pkg.dist</code></span></dt> -<dd><p>This file contains a list of directories, - together with their permission bits and ownership. These - directories will be created automatically with every package - that explicitly sets <code class="varname">USE_MTREE</code>. This feature will - be removed.</p></dd> -<dt><span class="term"><code class="filename">mk/platform/<em class="replaceable"><code>MyOS</code></em>.x11.dist</code></span></dt> -<dd><p>Just copy one of the pre-existing x11.dist files - to your - <code class="filename"><em class="replaceable"><code>MyOS</code></em>.x11.dist</code>.</p></dd> -<dt><span class="term"><code class="filename">mk/tools/bootstrap.mk</code></span></dt> -<dd><p>On some operating systems, the tools that are - provided with the base system are not good enough for pkgsrc. - For example, there are many versions of <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?sed+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">sed</span>(1)</span></a> that have a - narrow limit on the line length they can process. Therefore - pkgsrc brings its own tools, which can be enabled - here.</p></dd> <dt><span class="term"><code class="filename">mk/tools/tools.<em class="replaceable"><code>MyOS</code></em>.mk</code></span></dt> <dd><p>This file defines the paths to all the tools that are needed by one or the other package in pkgsrc, as well diff --git a/doc/pkgsrc.txt b/doc/pkgsrc.txt index 5ed5f49b1ae..493a9e43f11 100644 --- a/doc/pkgsrc.txt +++ b/doc/pkgsrc.txt @@ -115,25 +115,14 @@ I. The pkgsrc user's guide 7.1. Think first, build later 7.2. Requirements of a bulk build - 7.3. Running an old-style bulk build + 7.3. Running a pbulk-style bulk build - 7.3.1. Configuration - 7.3.2. Other environmental considerations - 7.3.3. Operation - 7.3.4. What it does - 7.3.5. Disk space requirements - 7.3.6. Setting up a sandbox for chrooted builds - 7.3.7. Building a partial set of packages - 7.3.8. Uploading results of a bulk build + 7.3.1. Preparation + 7.3.2. Configuration - 7.4. Running a pbulk-style bulk build + 7.4. Creating a multiple CD-ROM packages collection - 7.4.1. Preparation - 7.4.2. Configuration - - 7.5. Creating a multiple CD-ROM packages collection - - 7.5.1. Example of cdpack + 7.4.1. Example of cdpack 8. Directory layout of the installed files @@ -170,7 +159,7 @@ II. The pkgsrc developer's guide 10.1. Common types of packages 10.1.1. Perl modules - 10.1.2. KDE applications + 10.1.2. KDE3 applications 10.1.3. Python modules and programs 10.2. Examples @@ -772,25 +761,14 @@ Table of Contents 7.1. Think first, build later 7.2. Requirements of a bulk build - 7.3. Running an old-style bulk build - - 7.3.1. Configuration - 7.3.2. Other environmental considerations - 7.3.3. Operation - 7.3.4. What it does - 7.3.5. Disk space requirements - 7.3.6. Setting up a sandbox for chrooted builds - 7.3.7. Building a partial set of packages - 7.3.8. Uploading results of a bulk build + 7.3. Running a pbulk-style bulk build - 7.4. Running a pbulk-style bulk build + 7.3.1. Preparation + 7.3.2. Configuration - 7.4.1. Preparation - 7.4.2. Configuration + 7.4. Creating a multiple CD-ROM packages collection - 7.5. Creating a multiple CD-ROM packages collection - - 7.5.1. Example of cdpack + 7.4.1. Example of cdpack 8. Directory layout of the installed files @@ -1962,8 +1940,8 @@ XXX up settings used by builds in /usr/src. * DEPENDS_TARGET: By default, dependencies are only installed, and no binary - package is created for them. You can set this variable to package to - automatically create binary packages after installing dependencies. + package is created for them. You can set this variable to package-install + to automatically create binary packages after installing dependencies. 5.3. Variables affecting the installation process @@ -1981,12 +1959,6 @@ DESTDIR support is now the default. To switch back to non-DESTDIR, you can set USE_DESTDIR=no; this setting will be deprecated though, so it's preferable to convert a package to DESTDIR instead. -DESTDIR support changes the behaviour of various targets slightly. To install a -package after building it, use package-install. package and install don't do -that any longer. package-install can be used as DEPENDS_TARGET. bin-install -will ask for the root password to install the package and fail, package-install -will ask again. - With basic DESTDIR support, make clean needs to be run as root. Considering the foo/bar package, DESTDIR full support can be tested using the @@ -1999,15 +1971,15 @@ $ cd $PKGSRCDIR/foo/bar Verify DESTDIR full support, no root privileges should be needed -$ make USE_DESTDIR=yes install +$ make stage-install Create a package without root privileges -$ make USE_DESTDIR=yes PACKAGES=$HOME/packages package +$ make PACKAGES=$HOME/packages package For the following command, you must be able to gain root privileges using su(1) -$ make USE_DESTDIR=yes PACKAGES=$HOME/packages package-install +$ make PACKAGES=$HOME/packages install Then, as a simple user @@ -2025,18 +1997,32 @@ PKGSRC_COMPILER: This is a list of values specifying the chain of compilers to invoke when building packages. Valid values are: - + distcc: distributed C/C++ (chainable) + + ccc: Compaq C Compilers (Tru64) + ccache: compiler cache (chainable) + + clang: Clang C and Objective-C compiler + + + distcc: distributed C/C++ (chainable) + + + f2c: Fortran 77 to C compiler (chainable) + + + icc: Intel C++ Compiler (Linux) + + + ido: SGI IRIS Development Option cc (IRIX 5) + + gcc: GNU C/C++ Compiler + + hp: HP-UX C/aC++ compilers + + mipspro: Silicon Graphics, Inc. MIPSpro (n32/n64) - + mipspro: Silicon Graphics, Inc. MIPSpro (o32) + + mipspro-ucode: Silicon Graphics, Inc. MIPSpro (o32) + sunpro: Sun Microsystems, Inc. WorkShip/Forte/Sun ONE Studio + + xlc: IBM's XL C/C++ compiler suite (Darwin/MacOSX) + The default is "gcc". You can use ccache and/or distcc with an appropriate PKGSRC_COMPILER setting, e.g. "ccache gcc". This variable should always be terminated with a value for a real compiler. Note that only one real @@ -2193,31 +2179,20 @@ Table of Contents 7.1. Think first, build later 7.2. Requirements of a bulk build -7.3. Running an old-style bulk build - - 7.3.1. Configuration - 7.3.2. Other environmental considerations - 7.3.3. Operation - 7.3.4. What it does - 7.3.5. Disk space requirements - 7.3.6. Setting up a sandbox for chrooted builds - 7.3.7. Building a partial set of packages - 7.3.8. Uploading results of a bulk build +7.3. Running a pbulk-style bulk build -7.4. Running a pbulk-style bulk build + 7.3.1. Preparation + 7.3.2. Configuration - 7.4.1. Preparation - 7.4.2. Configuration +7.4. Creating a multiple CD-ROM packages collection -7.5. Creating a multiple CD-ROM packages collection - - 7.5.1. Example of cdpack + 7.4.1. Example of cdpack When you have multiple machines that should run the same packages, it is wasted -time if they all build their packages themselves from source. There are two -ways of getting a set of binary packages: The old bulk build system, or the new -(as of 2007) parallel bulk build (pbulk) system. This chapter describes how to -set them up so that the packages are most likely to be usable later. +time if they all build their packages themselves from source. There is a ways +of getting a set of binary packages: The bulk build system, or pbulk ("p" +stands for "parallel). This chapter describes how to set it up so that the +packages are most likely to be usable later. 7.1. Think first, build later @@ -2269,354 +2244,7 @@ others must survive a sudden reboot. * 5 GB for temporary files (read-write, local, temporary) -7.3. Running an old-style bulk build - -Note - -There are two ways of doing a bulk build. The old-style one and the new-style " -pbulk". The latter is the recommended way. - -7.3.1. Configuration - -7.3.1.1. build.conf - -The build.conf file is the main configuration file for bulk builds. You can -configure how your copy of pkgsrc is kept up to date, how the distfiles are -downloaded, how the packages are built and how the report is generated. You can -find an annotated example file in pkgsrc/mk/bulk/build.conf-example. To use it, -copy build.conf-example to build.conf and edit it, following the comments in -that file. - -7.3.1.2. mk.conf - -You may want to set variables in mk.conf. Look at pkgsrc/mk/defaults/mk.conf -for details of the default settings. You will want to ensure that -ACCEPTABLE_LICENSES meet your local policy. As used in this example, -SKIP_LICENSE_CHECK=yes completely bypasses the license check. - -PACKAGES?= ${_PKGSRCDIR}/packages/${MACHINE_ARCH} -WRKOBJDIR?= /usr/tmp/pkgsrc # build here instead of in pkgsrc -BSDSRCDIR= /usr/src -BSDXSRCDIR= /usr/xsrc # for x11/xservers -OBJHOSTNAME?= yes # use work.`hostname` -FAILOVER_FETCH= yes # insist on the correct checksum -PKG_DEVELOPER?= yes -SKIP_LICENSE_CHECK= yes - -Some options that are especially useful for bulk builds can be found at the top -lines of the file mk/bulk/bsd.bulk-pkg.mk. The most useful options of these are -briefly described here. - - * If you are on a slow machine, you may want to set USE_BULK_BROKEN_CHECK to - "no". - - * If you are doing bulk builds from a read-only copy of pkgsrc, you have to - set BULKFILESDIR to the directory where all log files are created. - Otherwise the log files are created in the pkgsrc directory. - - * Another important variable is BULK_PREREQ, which is a list of packages that - should be always available while building other packages. - -Some other options are scattered in the pkgsrc infrastructure: - - * ALLOW_VULNERABLE_PACKAGES should be set to yes. The purpose of the bulk - builds is creating binary packages, no matter if they are vulnerable or - not. Leaving this variable unset would prevent the bulk build system from - even trying to build them, so possible building errors would not show up. - - * CHECK_FILES (pkgsrc/mk/check/check-files.mk) can be set to "yes" to check - that the installed set of files matches the PLIST. - - * CHECK_INTERPRETER (pkgsrc/mk/check/check-interpreter.mk) can be set to "yes - " to check that the installed "#!"-scripts will find their interpreter. - - * PKGSRC_RUN_TEST can be set to "yes" to run each package's self-test before - installing it. Note that some packages make heavy use of "good" random - numbers, so you need to assure that the machine on which you are doing the - bulk builds is not completely idle. Otherwise some test programs will seem - to hang, while they are just waiting for new random data to be available. - -7.3.1.3. pre-build.local - -It is possible to configure the bulk build to perform certain site-specific -tasks at the end of the pre-build stage. If the file pre-build.local exists in -/usr/pkgsrc/mk/bulk, it will be executed (as a sh(1) script) at the end of the -usual pre-build stage. An example use of pre-build.local is to have the line: - -echo "I do not have enough disk space to build this pig." \ - > misc/openoffice/$BROKENF - -to prevent the system from trying to build a particular package which requires -nearly 3 GB of disk space. - -7.3.2. Other environmental considerations - -As /usr/pkg will be completely deleted at the start of bulk builds, make sure -your login shell is placed somewhere else. Either drop it into /usr/local/bin -(and adjust your login shell in the passwd file), or (re-)install it via -pkg_add(1) from /etc/rc.local, so you can login after a reboot (remember that -your current process won't die if the package is removed, you just can't start -any new instances of the shell any more). Also, if you use NetBSD earlier than -1.5, or you still want to use the pkgsrc version of ssh for some reason, be -sure to install ssh before starting it from rc.local: - -(cd /usr/pkgsrc/security/ssh && make bulk-install) -if [ -f /usr/pkg/etc/rc.d/sshd ]; then - /usr/pkg/etc/rc.d/sshd -fi - -Not doing so will result in you being not able to log in via ssh after the bulk -build is finished or if the machine gets rebooted or crashes. You have been -warned! :) - -7.3.3. Operation - -Make sure you don't need any of the packages still installed. - -Warning - -During the bulk build, all packages, their configuration files and some more -files from /var, /home and possibly other locations will be removed! So don't -run a bulk build with privileges that might harm your system. - -Be sure to remove all other things that might interfere with builds, like some -libs installed in /usr/local, etc. then become root and type: - -# cd /usr/pkgsrc -# sh mk/bulk/build - - -If for some reason your last build didn't complete (power failure, system -panic, ...), you can continue it by running: - -# sh mk/bulk/build restart - -At the end of the bulk build, you will get a summary via mail, and find build -logs in the directory specified by FTP in the build.conf file. - -7.3.4. What it does - -The bulk builds consist of three steps: - -1. pre-build - - The script updates your pkgsrc tree via (anon)cvs, then cleans out any - broken distfiles, and removes all packages installed. - -2. the bulk build - - This is basically "make bulk-package" with an optimised order in which - packages will be built. Packages that don't require other packages will be - built first, and packages with many dependencies will be built later. - -3. post-build - - Generates a report that's placed in the directory specified in the - build.conf file named broken.html, a short version of that report will also - be mailed to the build's admin. - -During the build, a list of broken packages will be compiled in /usr/pkgsrc -/.broken (or .../.broken.${MACHINE} if OBJMACHINE is set), individual build -logs of broken builds can be found in the package's directory. These files are -used by the bulk-targets to mark broken builds to not waste time trying to -rebuild them, and they can be used to debug these broken package builds later. - -7.3.5. Disk space requirements - -Currently, roughly the following requirements are valid for NetBSD 6.99/amd64: - - * 40 GB - distfiles (NFS ok) - - * 30 GB - full set of all binaries (NFS ok) - - * 5 GB - temp space for compiling (local disk recommended) - -Note that all pkgs will be de-installed as soon as they are turned into a -binary package, and that sources are removed, so there is no excessively huge -demand to disk space. Afterwards, if the package is needed again, it will be -installed via pkg_add(1) instead of building again, so there are no cycles -wasted by recompiling. - -7.3.6. Setting up a sandbox for chrooted builds - -If you don't want all the packages nuked from a machine (rendering it useless -for anything but pkg compiling), there is the possibility of doing the package -bulk build inside a chroot environment. - -The first step is to set up a chroot sandbox, e.g. /usr/sandbox. This can be -done by using null mounts, or manually. - -There is a shell script called mksandbox installed by the pkgtools/mksandbox -package, which will set up the sandbox environment using null mounts. It will -also create a script called sandbox in the root of the sandbox environment, -which will allow the null mounts to be activated using the sandbox mount -command and deactivated using the sandbox umount command. - -To set up a sandbox environment by hand, after extracting all the sets from a -NetBSD installation or doing a make distribution DESTDIR=/usr/sandbox in /usr/ -src/etc, be sure the following items are present and properly configured: - - 1. Kernel - - # cp /netbsd /usr/sandbox - - 2. /dev/* - - # cd /usr/sandbox/dev ; sh MAKEDEV all - - 3. /etc/resolv.conf (for security/smtpd and mail): - - # cp /etc/resolv.conf /usr/sandbox/etc - - 4. Working(!) mail config (hostname, sendmail.cf): - - # cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail - - 5. /etc/localtime (for security/smtpd): - - # ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime - - 6. /usr/src (system sources, rarely used by packages if at all: - - # ln -s ../disk1/cvs . - # ln -s cvs/src-2.0 src - - 7. Create /var/db/pkg (not part of default install): - - # mkdir /usr/sandbox/var/db/pkg - - 8. Create /usr/pkg (not part of default install): - - # mkdir /usr/sandbox/usr/pkg - - 9. Checkout pkgsrc via cvs into /usr/sandbox/usr/pkgsrc: - - # cd /usr/sandbox/usr - # cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc - - - Do not mount/link this to the copy of your pkgsrc tree you do development - in, as this will likely cause problems! - -10. Make /usr/sandbox/usr/pkgsrc/packages and .../distfiles point somewhere - appropriate. NFS- and/or nullfs-mounts may come in handy! - -11. Edit mk.conf, see Section 7.3.1.2, "mk.conf". - -12. Adjust mk/bulk/build.conf to suit your needs. - -When the chroot sandbox is set up, you can start the build with the following -steps: - -# cd /usr/sandbox/usr/pkgsrc -# sh mk/bulk/do-sandbox-build - - -This will just jump inside the sandbox and start building. At the end of the -build, mail will be sent with the results of the build. Created binary pkgs -will be in /usr/sandbox/usr/pkgsrc/packages (wherever that points/mounts to/ -from). - -7.3.7. Building a partial set of packages - -In addition to building a complete set of all packages in pkgsrc, the pkgsrc/mk -/bulk/build script may be used to build a subset of the packages contained in -pkgsrc. By setting SPECIFIC_PKGS in mk.conf, the variables - - * SITE_SPECIFIC_PKGS - - * HOST_SPECIFIC_PKGS - - * GROUP_SPECIFIC_PKGS - - * USER_SPECIFIC_PKGS - -will define the set of packages which should be built. The bulk build code will -also include any packages which are needed as dependencies for the explicitly -listed packages. - -One use of this is to do a bulk build with SPECIFIC_PKGS in a chroot sandbox -periodically to have a complete set of the binary packages needed for your site -available without the overhead of building extra packages that are not needed. - -7.3.8. Uploading results of a bulk build - -This section describes how pkgsrc developers can upload binary pkgs built by -bulk builds to ftp.NetBSD.org. - -If you would like to automatically create checksum files for the binary -packages you intend to upload, remember to set MKSUMS=yes in your mk/bulk/ -build.conf. - -If you would like to PGP sign the checksum files (highly recommended!), -remember to set SIGN_AS=username@NetBSD.org in your mk/bulk/build.conf. This -will prompt you for your GPG password to sign the files before uploading -everything. - -Then, make sure that you have RSYNC_DST set properly in your mk/bulk/build.conf -file, i.e. adjust it to something like one of the following: - -RSYNC_DST=ftp.NetBSD.org:/pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy/upload - -Please use appropriate values for "20xxQy" (the branch), "a.b.c" (the OS -version) and "arch" here. If your login on ftp.NetBSD.org is different from -your local login, write your login directly into the variable, e.g. my local -account is "feyrer", but for my login "hubertf", I use: - -RSYNC_DST=hubertf@ftp.NetBSD.org:/pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy/upload - -A separate upload directory is used here to allow "closing" the directory -during upload. To do so, run the following command on ftp.NetBSD.org next: - -nbftp% mkdir -p -m 750 /pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy/upload - -Before uploading the binary pkgs, ssh authentication needs to be set up. This -example shows how to set up temporary keys for the root account inside the -sandbox (assuming that no keys should be present there usually): - -# chroot /usr/sandbox -chroot-# rm $HOME/.ssh/id-dsa* -chroot-# ssh-keygen -t rsa -chroot-# cat $HOME/.ssh/id-rsa.pub - - -Now take the output of id-rsa.pub and append it to your ~/.ssh/authorized_keys -file on ftp.NetBSD.org. You should remove the key after the upload is done! - -Next, test if your ssh connection really works: - -chroot-# ssh ftp.NetBSD.org date - -Use "-l yourNetBSDlogin" here as appropriate! - -Now after all this works, you can exit the sandbox and start the upload: - -chroot-# exit -# cd /usr/sandbox/usr/pkgsrc -# sh mk/bulk/do-sandbox-upload - - -The upload process may take quite some time. Use ls(1) or du(1) on the FTP -server to monitor progress of the upload. The upload script will take care of -not uploading restricted packages. - -After the upload has ended, first thing is to revoke ssh access: - -nbftp% vi ~/.ssh/authorized_keys - Gdd:x! - -Use whatever is needed to remove the key you've entered before! Last, move the -uploaded packages out of the upload directory to have them accessible to -everyone: - -nbftp% cd /pub/pkgsrc/packages/NetBSD/arch/a.b.c-20xxQy -nbftp% mv upload/* . -nbftp% rmdir upload -nbftp% chgrp -R netbsd . -nbftp% find . -type d | xargs chmod 775 - - -7.4. Running a pbulk-style bulk build +7.3. Running a pbulk-style bulk build Running a pbulk-style bulk build works roughly as follows: @@ -2625,7 +2253,7 @@ Running a pbulk-style bulk build works roughly as follows: * Then, build each of the packages from a clean installation directory using the infrastructure. -7.4.1. Preparation +7.3.1. Preparation First, you need to create a pkgsrc installation for the pbulk infrastructure. No matter on which platform you are (even on NetBSD), you should bootstrap into @@ -2665,13 +2293,13 @@ Now the pbulk infrastructure is built and installed. It still needs to be configured, and after some more preparation, we will be able to start the real bulk build. -7.4.2. Configuration +7.3.2. Configuration TODO; see pkgsrc/doc/HOWTO-pbulk for more information. TODO: continue writing -7.5. Creating a multiple CD-ROM packages collection +7.4. Creating a multiple CD-ROM packages collection After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM set of the resulting binary packages to assist in installing packages on other @@ -2679,7 +2307,7 @@ machines. The pkgtools/cdpack package provides a simple tool for creating the ISO 9660 images. cdpack arranges the packages on the CD-ROMs in a way that keeps all the dependencies for a given package on the same CD as that package. -7.5.1. Example of cdpack +7.4.1. Example of cdpack Complete documentation for cdpack is found in the cdpack(1) man page. The following short example assumes that the binary packages are left in /usr/ @@ -3240,7 +2868,7 @@ Table of Contents 10.1. Common types of packages 10.1.1. Perl modules - 10.1.2. KDE applications + 10.1.2. KDE3 applications 10.1.3. Python modules and programs 10.2. Examples @@ -3470,7 +3098,7 @@ Table of Contents 10.1. Common types of packages 10.1.1. Perl modules - 10.1.2. KDE applications + 10.1.2. KDE3 applications 10.1.3. Python modules and programs 10.2. Examples @@ -3511,7 +3139,7 @@ package involves only a few steps. [...] - BUILD_DEPENDS+= lua>=5.0:../../lang/lua + BUILD_DEPENDS+= libxslt-[0-9]*:../../textproc/libxslt DEPENDS+= screen-[0-9]*:../../misc/screen DEPENDS+= screen>=4.0:../../misc/screen @@ -3563,10 +3191,10 @@ package involves only a few steps. Simple Perl modules are handled automatically by url2pkg, including dependencies. -10.1.2. KDE applications +10.1.2. KDE3 applications -KDE applications should always include meta-pkgs/kde3/kde3.mk, which contains -numerous settings that are typical of KDE packages. +KDE3 applications should always include meta-pkgs/kde3/kde3.mk, which contains +numerous settings that are typical of KDE3 packages. 10.1.3. Python modules and programs @@ -3846,12 +3474,13 @@ mostly historical and has no further meaning. * DISTNAME is the basename of the distribution file to be downloaded from the package's website. - * PKGNAME is the name of the package, as used by pkgsrc. You only need to - provide it if DISTNAME (which is the default) is not a good name for the - package in pkgsrc. Usually it is the pkgsrc directory name together with - the version number. It must match the regular expression ^[A-Za-z0-9] - [A-Za-z0-9-_.+]*$, that is, it starts with a letter or digit, and contains - only letters, digits, dashes, underscores, dots and plus signs. + * PKGNAME is the name of the package, as used by pkgsrc. You need to provide + it if DISTNAME (which is the default) is not a good name for the package in + pkgsrc or DISTNAME is not provided (no distribution file is required). + Usually it is the pkgsrc directory name together with the version number. + It must match the regular expression ^[A-Za-z0-9][A-Za-z0-9-_.+]*$, that + is, it starts with a letter or digit, and contains only letters, digits, + dashes, underscores, dots and plus signs. * SVR4_PKGNAME is the name of the package file to create if the PKGNAME isn't unique on a SVR4 system. The default is PKGNAME, which may be shortened @@ -4186,7 +3815,13 @@ MESSAGE ALTERNATIVES - FIXME: There is no documentation on the alternatives framework. + This file is used by the alternatives framework. It creates, configures, + and destroys generic wrappers used to run programs with similar interfaces. + See pkg_alternatives(8) from pkgtools/pkg_alternatives for more + information. + + Each line of the file contains two filenames, first the wrapper and then + the alternative provided by the package. Both paths are relative to PREFIX. 11.5.2. Files affecting the build process @@ -5800,13 +5435,14 @@ name is derived from the DISTNAME variable, is fetched. The more complicated cases are described below. The variable DISTFILES specifies the list of distfiles that have to be fetched. -Its value defaults to ${DISTNAME}${EXTRACT_SUFX}, so that most packages don't -need to define it at all. EXTRACT_SUFX is .tar.gz by default, but can be -changed freely. Note that if your package requires additional distfiles to the -default one, you cannot just append the additional filenames using the += -operator, but you have write for example: +Its value defaults to ${DEFAULT_DISTFILES} and its value is ${DISTNAME}$ +{EXTRACT_SUFX}, so that most packages don't need to define it at all. +EXTRACT_SUFX is .tar.gz by default, but can be changed freely. Note that if +your package requires additional distfiles to the default one, you cannot just +append the additional filenames using the += operator, but you have write for +example: -DISTFILES= ${DISTNAME}${EXTRACT_SUFX} additional-files.tar.gz +DISTFILES= ${DEFAULT_DISTFILES} additional-files.tar.gz Each distfile is fetched from a list of sites, usually MASTER_SITES. If the package has multiple DISTFILES or multiple PATCHFILES from different sites, you @@ -5830,6 +5466,14 @@ MASTER_SITES= http://www.example.com/download.cgi?file= The exception to this rule are URLs starting with a dash. In that case the URL is taken as is, fetched and the result stored under the name of the distfile. +You can use this style for the case when the download URL style does not match +the above common case. For example, if permanent download URL is a redirector +to the real download URL, or the download file name is offered by an HTTP +Content-Disposition header. In the following example, foo-1.0.0.tar.gz will be +created instead of the default v1.0.0.tar.gz. + +DISTNAME= foo-1.0.0 +MASTER_SITES= -http://www.example.com/archive/v1.0.0.tar.gz There are some predefined values for MASTER_SITES, which can be used in packages. The names of the variables should speak for themselves. @@ -5844,13 +5488,18 @@ ${MASTER_SITE_GENTOO} ${MASTER_SITE_GNOME} ${MASTER_SITE_GNU} ${MASTER_SITE_GNUSTEP} +${MASTER_SITE_HASKELL_HACKAGE} ${MASTER_SITE_IFARCHIVE} ${MASTER_SITE_KDE} ${MASTER_SITE_MOZILLA} +${MASTER_SITE_MOZILLA_ALL} +${MASTER_SITE_MOZILLA_ESR} ${MASTER_SITE_MYSQL} +${MASTER_SITE_NETLIB} ${MASTER_SITE_OPENOFFICE} ${MASTER_SITE_PERL_CPAN} ${MASTER_SITE_PGSQL} +${MASTER_SITE_RUBYGEMS} ${MASTER_SITE_R_CRAN} ${MASTER_SITE_SOURCEFORGE} ${MASTER_SITE_SOURCEFORGE_JP} @@ -5859,6 +5508,7 @@ ${MASTER_SITE_SUSE} ${MASTER_SITE_TEX_CTAN} ${MASTER_SITE_XCONTRIB} ${MASTER_SITE_XEMACS} +${MASTER_SITE_XORG} Some explanations for the less self-explaining ones: MASTER_SITE_BACKUP contains backup sites for packages that are maintained in ftp://ftp.NetBSD.org/ @@ -6421,10 +6071,10 @@ bulk-package Used to do bulk builds. If an appropriate binary package already exists, no action is taken. If not, this target will compile, install and package it - (and its depends, if PKG_DEPENDS is set properly. See Section 7.3.1, - "Configuration"). After creating the binary package, the sources, the - just-installed package and its required packages are removed, preserving - free disk space. + (and its depends, if PKG_DEPENDS is set properly. See Chapter 7, Creating + binary packages for everything in pkgsrc (bulk builds)). After creating the + binary package, the sources, the just-installed package and its required + packages are removed, preserving free disk space. Beware that this target may deinstall all packages installed on a system! @@ -8046,8 +7696,8 @@ Our policy is that we accept binaries only from pkgsrc developers to guarantee that the packages don't contain any trojan horses etc. This is not to annoy anyone but rather to protect our users! You're still free to put up your home-made binary packages and tell the world where to get them. NetBSD -developers doing bulk builds and wanting to upload them please see -Section 7.3.8, "Uploading results of a bulk build". +developers doing bulk builds and wanting to upload them please see Chapter 7, +Creating binary packages for everything in pkgsrc (bulk builds). 21.2. Submitting source packages (for non-NetBSD-developers) @@ -8973,23 +8623,6 @@ mk/platform/MyOS.mk This file contains the platform-specific definitions that are used by pkgsrc. Start by copying one of the other files and edit it to your needs. -mk/platform/MyOS.pkg.dist - - This file contains a list of directories, together with their permission - bits and ownership. These directories will be created automatically with - every package that explicitly sets USE_MTREE. This feature will be removed. - -mk/platform/MyOS.x11.dist - - Just copy one of the pre-existing x11.dist files to your MyOS.x11.dist. - -mk/tools/bootstrap.mk - - On some operating systems, the tools that are provided with the base system - are not good enough for pkgsrc. For example, there are many versions of sed - (1) that have a narrow limit on the line length they can process. Therefore - pkgsrc brings its own tools, which can be enabled here. - mk/tools/tools.MyOS.mk This file defines the paths to all the tools that are needed by one or the |