Moved the explanation of the various variables in the "fetch" phase from

components.xml to build.xml. Added a list of variable names and a link
to the old place. Rewrote and extended the existing documentation to
cover most common cases.

2 files changed, 137 insertions, 95 deletions

diff --git a/doc/guide/files/build.xml b/doc/guide/files/build.xml
index 52004c732a1..4c05b5d329a 100644
--- a/doc/guide/files/build.xml
+++ b/doc/guide/files/build.xml
@@ -1,4 +1,4 @@
-<!-- $NetBSD: build.xml,v 1.29 2006/06/27 11:07:12 rillig Exp $ -->
+<!-- $NetBSD: build.xml,v 1.30 2006/07/24 10:32:36 rillig Exp $ -->
 
 <chapter id="build">
 <title>The build process</title>
@@ -207,27 +207,136 @@ the package will be built, but not installed.</para>
 <sect1 id="build.fetch">
 <title>The <emphasis>fetch</emphasis> phase</title>
 
-	  <para>This will check if the file(s) given in the variables
-	    <varname>DISTFILES</varname> and <varname>PATCHFILES</varname> (as
-	    defined in the package's Makefile) are present on the
-	    local system in <filename>/usr/pkgsrc/distfiles</filename>. If they
-	    are not present, an attempt will be made to fetch them using commands
-	    of the form:</para>
+	<para>The first step in building a package is to fetch the
+	distribution files (distfiles) from the sites that are providing
+	them. This is the task of the <emphasis>fetch</emphasis>
+	phase.</para>
+
+<sect2 id="build.fetch.what">
+<title>What to fetch and where to get it from</title>
+
+	<para>In simple cases, <varname>MASTER_SITES</varname> defines
+	all URLs from where the distfile, whose name is derived from the
+	<varname>DISTNAME</varname> variable, is fetched. The more
+	complicated cases are described below.</para>
+
+	<para>The variable <varname>DISTFILES</varname> specifies the
+	list of distfiles that have to be fetched. Its value defaults to
+	<literal>${DISTNAME}${EXTRACT_SUFX}</literal>, so that most
+	packages don't need to define it at all.
+	<varname>EXTRACT_SUFX</varname> is <literal>.tar.gz</literal> by
+	default, but can be changed freely. Note that if your package
+	requires additional distfiles to the default one, you cannot
+	just append the additional filenames using the
+	<literal>+=</literal> operator, but you have write for
+	example:</para>
 
 <programlisting>
-    ${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS}
+    DISTFILES=      ${DISTNAME}${EXTRACT_SUFX} additional-files.tar.gz
+</programlisting>
+
+	<para>Each of the distfiles is fetched from a list of sites,
+	usually <varname>MASTER_SITES</varname>. If the package has
+	multiple <varname>DISTFILES</varname> or multiple
+	<varname>PATCHFILES</varname> from different sites, you can set
+	<varname>SITES.<replaceable>distfile</replaceable></varname> to
+	the list of URLs where the file
+	<filename><replaceable>distfile</replaceable></filename>
+	(including the suffix) can be found.</para>
+
+<programlisting>
+    DISTFILES=      ${DISTNAME}${EXTRACT_SUFX}
+    DISTFILES+=     foo-file.tar.gz
+    SITES.foo-file.tar.gz= \
+            http://www.somewhere.com/somehow/ \
+            http://www.somewhereelse.com/mirror/somehow/
+</programlisting>
+
+	<para>When actually fetching the distfiles, each item from
+	<varname>MASTER_SITES</varname> or <varname>SITES.*</varname>
+	gets the name of each distfile appended to it, without an
+	intermediate slash. Therefore, all site values have to end with
+	a slash or other separator character. This allows for example to
+	set <varname>MASTER_SITES</varname> to a URL of a CGI script
+	that gets the name of the distfile as a parameter. In this case,
+	the definition would look like:</para>
+
+<programlisting>
+    MASTER_SITES=   http://www.example.com/download.cgi?file=
 </programlisting>
 
-	  <para>where ${site} varies through several possibilities in turn: first,
-	    <varname>MASTER_SITE_OVERRIDE</varname> is tried, then the sites
-	    specified in either <varname>SITES.file</varname> if defined, else
-	    <varname>MASTER_SITES</varname> or <varname>PATCH_SITES</varname>, as
-	    applies, then finally the value of
-	    <varname>MASTER_SITE_BACKUP</varname>. The order of all except the
-	    first can be optionally sorted by the user, via setting either
-	    <varname>MASTER_SORT_AWK</varname> or
-	    <varname>MASTER_SORT_REGEX</varname>.</para>
+	<para>There are some predefined values for
+	<varname>MASTER_SITES</varname>, which can be used in packages.
+	The names of the variables should speak for themselves.</para>
+
+<!-- sort mk/fetch/sites.mk | sed -n 's/\(^MA[A-Z_]*\).*/    ${\1}/p' -->
+<programlisting>
+    ${MASTER_SITE_APACHE}
+    ${MASTER_SITE_BACKUP}
+    ${MASTER_SITE_CYGWIN}
+    ${MASTER_SITE_DEBIAN}
+    ${MASTER_SITE_FREEBSD}
+    ${MASTER_SITE_FREEBSD_LOCAL}
+    ${MASTER_SITE_GNOME}
+    ${MASTER_SITE_GNU}
+    ${MASTER_SITE_GNUSTEP}
+    ${MASTER_SITE_IFARCHIVE}
+    ${MASTER_SITE_KDE}
+    ${MASTER_SITE_MOZILLA}
+    ${MASTER_SITE_MYSQL}
+    ${MASTER_SITE_OPENOFFICE}
+    ${MASTER_SITE_PERL_CPAN}
+    ${MASTER_SITE_PGSQL}
+    ${MASTER_SITE_R_CRAN}
+    ${MASTER_SITE_SOURCEFORGE}
+    ${MASTER_SITE_SOURCEFORGE_JP}
+    ${MASTER_SITE_SUNSITE}
+    ${MASTER_SITE_SUSE}
+    ${MASTER_SITE_TEX_CTAN}
+    ${MASTER_SITE_XCONTRIB}
+    ${MASTER_SITE_XEMACS}
+</programlisting>
+
+	<para>If you choose one of these predefined sites, you may want
+	to specify a subdirectory of that site. Since these macros may
+	expand to more than one actual site, you
+	<emphasis>must</emphasis> use the following construct to specify
+	a subdirectory:</para>
+
+<programlisting>
+    MASTER_SITES=   ${MASTER_SITE_GNU:=subdirectory/name/}
+    MASTER_SITES=   ${MASTER_SITE_SOURCEFORGE:=project_name/}
+</programlisting>
+
+	<para>Note the trailing slash after the subdirectory
+	name.</para>
+
+</sect2>
+<sect2 id="build.fetch.how">
+<title>How are the files fetched?</title>
+
+	<para>The <emphasis>fetch</emphasis> phase makes sure that all
+	the distfiles exist in a local directory
+	(<varname>DISTDIR</varname>), which can be set by the pkgsrc
+	user). If the files do not exist, they are fetched using
+	commands of the form
+
+<programlisting>
+    ${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS}
+</programlisting>
 
+	where <literal>${site}</literal> varies through several
+	possibilities in turn: first,
+	<varname>MASTER_SITE_OVERRIDE</varname> is tried, then the sites
+	specified in either <varname>SITES.file</varname> if defined,
+	else <varname>MASTER_SITES</varname> or
+	<varname>PATCH_SITES</varname>, as applies, then finally the
+	value of <varname>MASTER_SITE_BACKUP</varname>. The order of all
+	except the first can be optionally sorted by the user, via
+	setting either <varname>MASTER_SORT_AWK</varname> or
+	<varname>MASTER_SORT_REGEX</varname>.</para>
+
+</sect2>
 </sect1>
 
 <sect1 id="build.checksum">
diff --git a/doc/guide/files/components.xml b/doc/guide/files/components.xml
index 713331af9a3..52814689bbd 100644
--- a/doc/guide/files/components.xml
+++ b/doc/guide/files/components.xml
@@ -1,4 +1,4 @@
-<!-- $NetBSD: components.xml,v 1.25 2006/06/25 10:33:33 rillig Exp $ -->
+<!-- $NetBSD: components.xml,v 1.26 2006/07/24 10:32:36 rillig Exp $ -->
 
 <chapter id="components"> <?dbhtml filename="components.html"?>
   <title>Package components - files, directories and contents</title>
@@ -21,7 +21,10 @@
      sections that describe the package.</para>
 
     <para>In the first section there are the following variables, which
-      should appear exactly in the order given here.
+    should appear exactly in the order given here. The ordering and
+    grouping of variables is mostly historical and has no further
+    meaning.</para>
+
       <itemizedlist>
 
       <listitem><para><varname>DISTNAME</varname> is the basename of the
@@ -56,83 +59,13 @@
 </programlisting>
 	</listitem>
 
-	<listitem><para><varname>MASTER_SITES</varname> is a list of URLs where
-	    the distribution files can be downloaded. Each URL must end with a
-	    slash.</para>
-	  <para>The <varname>MASTER_SITES</varname> may make use of
-	    the following predefined sites:</para>
-
-<!-- sort bsd.sites.mk | sed -n 's/\(^MA[A-Z_]*\).*/        ${\1}/p' -->
-<programlisting>
-    ${MASTER_SITE_APACHE}
-    ${MASTER_SITE_BACKUP}
-    ${MASTER_SITE_CYGWIN}
-    ${MASTER_SITE_DEBIAN}
-    ${MASTER_SITE_FREEBSD}
-    ${MASTER_SITE_FREEBSD_LOCAL}
-    ${MASTER_SITE_GNOME}
-    ${MASTER_SITE_GNU}
-    ${MASTER_SITE_GNUSTEP}
-    ${MASTER_SITE_IFARCHIVE}
-    ${MASTER_SITE_MOZILLA}
-    ${MASTER_SITE_OPENOFFICE}
-    ${MASTER_SITE_PERL_CPAN}
-    ${MASTER_SITE_R_CRAN}
-    ${MASTER_SITE_SOURCEFORGE}
-    ${MASTER_SITE_SUNSITE}
-    ${MASTER_SITE_SUSE}
-    ${MASTER_SITE_TEX_CTAN}
-    ${MASTER_SITE_XCONTRIB}
-    ${MASTER_SITE_XEMACS}
-</programlisting>
-
-	  <para>If one of these predefined sites is chosen, you may
-	    want to specify a subdirectory of that
-	    site.  Since these macros may expand to more than one
-	    actual site, you <emphasis>must</emphasis> use the
-	    following construct to specify a subdirectory:</para>
-
-<programlisting>
-    ${MASTER_SITE_GNU:=subdirectory/name/}
-    ${MASTER_SITE_SOURCEFORGE:=project_name/}
-</programlisting>
-
-	  <para>Note the trailing slash after the subdirectory name.</para>
-
-	  <para>If the package has multiple
-	    <varname>DISTFILES</varname> or multiple
-	    <varname>PATCHFILES</varname> from different
-	    sites, set <varname>SITES.foo</varname> to a list of URIs
-	    where file <quote>foo</quote> may be
-	    found. <quote>foo</quote> includes the suffix, e.g.:</para>
+	<listitem><para><varname>MASTER_SITES</varname>,
+	<varname>DYNAMIC_MASTER_SITES</varname>,
+	<varname>DIST_SUBDIR</varname>, <varname>EXTRACT_SUFX</varname>
+	and <varname>DISTFILES</varname> are discussed in detail in
+	<xref linkend="build.fetch"/>.</para></listitem>
 
-<programlisting>
-    DISTFILES=      ${DISTNAME}${EXTRACT_SUFX}
-    DISTFILES+=     foo-file.tar.gz
-    SITES.foo-file.tar.gz= \
-            http://www.somewhere.com/somehow/ \
-            http://www.somewhereelse.com/mirror/somehow/
-</programlisting>
-	</listitem>
-
-	<listitem><para><varname>DISTFILES</varname>: Name(s)
-	    of archive file(s) containing distribution. The default is
-	    <filename>${DISTNAME}${EXTRACT_SUFX}</filename>. Should only
-	    be set if you have more than one distfile.</para>
-	  <para>Note that the normal default setting of
-	    <varname>DISTFILES</varname> must be made explicit if you
-	    want to add to it (rather than replace it), as you usually
-	    would.</para>
-	</listitem>
-
-	<listitem><para><varname>EXTRACT_SUFX</varname>: Suffix of the
-	    distribution file, will be appended to
-	    <varname>DISTNAME</varname>. Defaults to
-	    <filename>.tar.gz</filename>.
-	  </para>
-	</listitem>
-      </itemizedlist>
-    </para>
+	</itemizedlist>
 
     <para>The second section contains information about separately
       downloaded patches, if any.