path: root/doc/guide/files/build.xml

<!-- $NetBSD: build.xml,v 1.71 2014/05/31 21:08:50 asau Exp $ -->

<chapter id="build">
  <title>The build process</title>

  <sect1 id="build.intro">
    <title>Introduction</title>

    <para>This chapter gives a detailed description on how a package is
    built. Building a package is separated into different
    <emphasis>phases</emphasis> (for example <varname>fetch</varname>,
    <varname>build</varname>, <varname>install</varname>), all of which are
    described in the following sections. Each phase is split into
    so-called <emphasis>stages</emphasis>, which take the name of the
    containing phase, prefixed by one of <varname>pre-</varname>,
    <varname>do-</varname> or <varname>post-</varname>. (Examples are
    <varname>pre-configure</varname>, <varname>post-build</varname>.) Most
    of the actual work is done in the <varname>do-*</varname> stages.</para>

    <para>Never override the regular targets (like
    <varname>fetch</varname>), if you have to, override the
    <varname>do-*</varname> ones instead.</para>

    <para>The basic steps for building a program are always the same.  First
    the program's source (<emphasis>distfile</emphasis>) must be brought to
    the local system and then extracted. After any pkgsrc-specific patches
    to compile properly are applied, the software can be configured, then
    built (usually by compiling), and finally the generated binaries, etc.
    can be put into place on the system.</para>

    <para>To get more details about what is happening at each step,
    you can set the <varname>PKG_VERBOSE</varname> variable, or the
    <varname>PATCH_DEBUG</varname> variable if you are just interested
    in more details about the <emphasis>patch</emphasis> step.</para>
  </sect1>

  <sect1 id="build.prefix">
    <title>Program location</title>

    <para>Before outlining the process performed by the &os; package system in
    the next section, here's a brief discussion on where programs are
    installed, and which variables influence this.</para>

    <para>The automatic variable <varname>PREFIX</varname> indicates
    where all files of the final program shall be installed. It is
    usually set to <varname>LOCALBASE</varname>
    (<filename>/usr/pkg</filename>), or <varname>CROSSBASE</varname>
    for pkgs in the <filename>cross</filename> category.  The value of
    <varname>PREFIX</varname> needs to be put
    into the various places in the program's source where paths to
    these files are encoded.  See <xref
    linkend="components.patches"/> and <xref
    linkend="fixes.libtool"/> for more details.</para>

    <para>When choosing which of these variables to use,
    follow the following rules:</para>

    <itemizedlist>
      <listitem>
	<para><varname>PREFIX</varname> always points to the location
	where the current pkg will be installed.  When referring to a
	pkg's own installation path, use
	<quote>${PREFIX}</quote>.</para>
      </listitem>

      <listitem>
	<para><varname>LOCALBASE</varname> is where all non-X11 pkgs
	are installed.  If you need to construct a -I or -L argument
	to the compiler to find includes and libraries installed by
	another non-X11 pkg, use <quote>${LOCALBASE}</quote>. The name
	<varname>LOCALBASE</varname> stems from FreeBSD, which
	installed all packages in <filename>/usr/local</filename>. As
	pkgsrc leaves <filename>/usr/local</filename> for the system
	administrator, this variable is a misnomer.</para>
      </listitem>

      <listitem>
	<para><varname>X11BASE</varname> is where the actual X11
	distribution (from xsrc, etc.) is installed. When looking for
	<emphasis>standard</emphasis> X11 includes (not those
	installed by a package), use <quote>${X11BASE}</quote>.</para>
      </listitem>

      <listitem>
	<para>X11-based packages are special in that they may be
	installed in either <varname>X11BASE</varname> or
	<varname>LOCALBASE</varname>.</para>

	<para>Usually, X11 packages should be installed under
	<varname>LOCALBASE</varname> whenever possible.  Note that you
	will need to include
	<filename>../../mk/x11.buildlink3.mk</filename> in them to
	request the presence of X11 and to get the right compilation
	flags.</para>

	<para>Even though, there are some packages that cannot be installed
	under <varname>LOCALBASE</varname>: those that come with app-defaults
	files. These packages are special and they must be placed under
	<varname>X11BASE</varname>. To accomplish this, set either
	<varname>USE_X11BASE</varname> or <varname>USE_IMAKE</varname> in
	your package.</para>

	<para>Some notes: If you need
	to find includes or libraries installed by a pkg that has
	<varname>USE_IMAKE</varname> or <varname>USE_X11BASE</varname> in
	its pkg <filename>Makefile</filename>, you need to look in
	<emphasis>both</emphasis> <filename>${X11BASE}</filename> and
	<filename>${LOCALBASE}</filename>. To force installation of
	all X11 packages in <varname>LOCALBASE</varname>, the
	<filename role="pkg">pkgtools/xpkgwedge</filename> package
	is enabled by default.</para>
      </listitem>

      <listitem>
	<para><varname>X11PREFIX</varname> should be used to refer to
	the installed location of an X11
	package. <varname>X11PREFIX</varname> will be set to
	<varname>X11BASE</varname> if xpkgwedge is not installed, and
	to <varname>LOCALBASE</varname> if xpkgwedge is
	installed.</para>
      </listitem>

      <listitem>
	<para>If xpkgwedge is installed, it is possible to have some
	packages installed in <varname>X11BASE</varname> and some in
	<varname>LOCALBASE</varname>.  To determine the prefix of an
	installed package, the <varname>EVAL_PREFIX</varname>
	definition can be used. It takes pairs in the format
	<quote>DIRNAME=&lt;package&gt;</quote>, and the &man.make.1;
	variable <varname>DIRNAME</varname> will be set to the prefix
	of the installed package &lt;package&gt;, or
	<quote>${X11PREFIX}</quote> if the package is not
	installed.</para>

	<para>This is best illustrated by example.</para>

	<para>The following lines are taken from
	<filename>pkgsrc/wm/scwm/Makefile</filename>:</para>

<programlisting>
EVAL_PREFIX+=           GTKDIR=gtk+
CONFIGURE_ARGS+=        --with-guile-prefix=${LOCALBASE:Q}
CONFIGURE_ARGS+=        --with-gtk-prefix=${GTKDIR:Q}
CONFIGURE_ARGS+=        --enable-multibyte
</programlisting>

	<para>Specific defaults can be defined for the packages
	evaluated using <varname>EVAL_PREFIX</varname>, by using a
	definition of the form:</para>

<programlisting>
GTKDIR_DEFAULT= ${LOCALBASE}
</programlisting>

	<para>where <varname>GTKDIR</varname> corresponds
	to the first definition in
	the <varname>EVAL_PREFIX</varname> pair.</para>
      </listitem>

      <listitem>
	<para>Within <filename>${PREFIX}</filename>, packages should
	install files according to &man.hier.7;, with the exception that
	manual pages go into <filename>${PREFIX}/man</filename>, not
	<filename>${PREFIX}/share/man</filename>.</para>
      </listitem>
    </itemizedlist>
  </sect1>

  <sect1 id="build.builddirs">
    <title>Directories used during the build process</title>

    <para>When building a package, various directories are used to store
    source files, temporary files, pkgsrc-internal files, and so on. These
    directories are explained here.</para>

    <para>Some of the directory variables contain relative pathnames. There
    are two common base directories for these relative directories:
    <varname>PKGSRCDIR/PKGPATH</varname> is used for directories that are
    pkgsrc-specific. <varname>WRKSRC</varname> is used for directories
    inside the package itself.</para>

    <variablelist>

      <varlistentry><term><varname>PKGSRCDIR</varname></term>
      <listitem><para>This is an absolute pathname that points to the pkgsrc
      root directory. Generally, you don't need
      it.</para></listitem></varlistentry>

      <varlistentry><term><varname>PKGDIR</varname></term>
      <listitem><para>This is an absolute pathname that points to the
      current package.</para></listitem></varlistentry>

      <varlistentry><term><varname>PKGPATH</varname></term>
      <listitem><para>This is a pathname relative to
      <varname>PKGSRCDIR</varname> that points to the current
      package.</para></listitem></varlistentry>

      <varlistentry><term><varname>WRKDIR</varname></term>
      <listitem><para>This is an absolute pathname pointing to the directory
      where all work takes place. The distfiles are extracted to this
      directory. It also contains temporary directories and log files used by
      the various pkgsrc frameworks, like <emphasis>buildlink</emphasis> or
      the <emphasis>wrappers</emphasis>.</para></listitem></varlistentry>

      <varlistentry><term><varname>WRKSRC</varname></term>
      <listitem><para>This is an absolute pathname pointing to the directory
      where the distfiles are extracted. It is usually a direct subdirectory
      of <varname>WRKDIR</varname>, and often it's the only directory entry
      that isn't hidden. This variable may be changed by a package
      <filename>Makefile</filename>.</para></listitem></varlistentry>

    </variablelist>

    <para>The <varname>CREATE_WRKDIR_SYMLINK</varname> definition takes either
    the value <emphasis>yes</emphasis> or <emphasis>no</emphasis> and defaults
    to <emphasis>no</emphasis>. It indicates whether a symbolic link to the
    <varname>WRKDIR</varname> is to be created in the pkgsrc entry's directory.
    If users would like to have their pkgsrc trees behave in a
    read-only manner, then the value of
    <varname>CREATE_WRKDIR_SYMLINK</varname> should be set to
    <emphasis>no</emphasis>.</para>
  </sect1>

  <sect1 id="build.running">
    <title>Running a phase</title>

    <para>You can run a particular phase by typing <command>make
    phase</command>, where <emphasis>phase</emphasis> is the name of the
    phase. This will automatically run all phases that are required for this
    phase. The default phase is <varname>build</varname>, that is, when you
    run <command>make</command> without parameters in a package directory,
    the package will be built, but not installed.</para>

  </sect1>

  <sect1 id="build.fetch">
    <title>The <emphasis>fetch</emphasis> phase</title>

    <para>The first step in building a package is to fetch the
    distribution files (distfiles) from the sites that are providing
    them. This is the task of the <emphasis>fetch</emphasis>
    phase.</para>

    <sect2 id="build.fetch.what">
      <title>What to fetch and where to get it from</title>

      <para>In simple cases, <varname>MASTER_SITES</varname>
      defines all URLs from where the distfile, whose name is
      derived from the <varname>DISTNAME</varname> variable, is
      fetched. The more complicated cases are described
      below.</para>

      <para>The variable <varname>DISTFILES</varname> specifies
      the list of distfiles that have to be fetched. Its value
      defaults to <literal>${DEFAULT_DISTFILES}</literal> and
      its value is <literal>${DISTNAME}${EXTRACT_SUFX}</literal>,
      so that most packages don't need to define it at all.
      <varname>EXTRACT_SUFX</varname> is
      <literal>.tar.gz</literal> by default, but can be changed
      freely. Note that if your package requires additional
      distfiles to the default one, you cannot just append the
      additional filenames using the <literal>+=</literal>
      operator, but you have write for example:</para>

<programlisting>
DISTFILES=      ${DEFAULT_DISTFILES} additional-files.tar.gz
</programlisting>

      <para>Each distfile is fetched from a list of sites, usually
      <varname>MASTER_SITES</varname>. If the package has multiple
      <varname>DISTFILES</varname> or multiple
      <varname>PATCHFILES</varname> from different sites, you can
      set
      <varname>SITES.<replaceable>distfile</replaceable></varname>
      to the list of URLs where the file
      <filename><replaceable>distfile</replaceable></filename>
      (including the suffix) can be found.</para>

<programlisting>
DISTFILES=      ${DISTNAME}${EXTRACT_SUFX}
DISTFILES+=     foo-file.tar.gz
SITES.foo-file.tar.gz= \
http://www.somewhere.com/somehow/ \
http://www.somewhereelse.com/mirror/somehow/
</programlisting>

      <para>When actually fetching the distfiles, each item from
      <varname>MASTER_SITES</varname> or
      <varname>SITES.*</varname> gets the name of each distfile
      appended to it, without an intermediate slash. Therefore,
      all site values have to end with a slash or other separator
      character. This allows for example to set
      <varname>MASTER_SITES</varname> to a URL of a CGI script
      that gets the name of the distfile as a parameter. In this
      case, the definition would look like:</para>

<programlisting>
MASTER_SITES=   http://www.example.com/download.cgi?file=
</programlisting>

      <para> The exception to this rule are URLs starting with a dash.
      In that case the URL is taken as is, fetched and the result
      stored under the name of the distfile. You can use this style
      for the case when the download URL style does not match the
      above common case. For example, if permanent download URL is a
      redirector to the real download URL, or the download file name
      is offered by an HTTP Content-Disposition header. In the
      following example, <filename>foo-1.0.0.tar.gz</filename> will be
      created instead of the default
      <filename>v1.0.0.tar.gz</filename>.</para>

<programlisting>
DISTNAME=       foo-1.0.0
MASTER_SITES=   -http://www.example.com/archive/v1.0.0.tar.gz
</programlisting>

      <para>There are some predefined values for
      <varname>MASTER_SITES</varname>, which can be used in
      packages.  The names of the variables should speak for
      themselves.</para>

      <!-- sort mk/fetch/sites.mk | sed -n 's/\(^MA[A-Z_]*\).*/    ${\1}/p' -->

<programlisting>
${MASTER_SITE_APACHE}
${MASTER_SITE_BACKUP}
${MASTER_SITE_CYGWIN}
${MASTER_SITE_DEBIAN}
${MASTER_SITE_FREEBSD}
${MASTER_SITE_FREEBSD_LOCAL}
${MASTER_SITE_GENTOO}
${MASTER_SITE_GNOME}
${MASTER_SITE_GNU}
${MASTER_SITE_GNUSTEP}
${MASTER_SITE_HASKELL_HACKAGE}
${MASTER_SITE_IFARCHIVE}
${MASTER_SITE_KDE}
${MASTER_SITE_MOZILLA}
${MASTER_SITE_MOZILLA_ALL}
${MASTER_SITE_MOZILLA_ESR}
${MASTER_SITE_MYSQL}
${MASTER_SITE_NETLIB}
${MASTER_SITE_OPENOFFICE}
${MASTER_SITE_PERL_CPAN}
${MASTER_SITE_PGSQL}
${MASTER_SITE_RUBYGEMS}
${MASTER_SITE_R_CRAN}
${MASTER_SITE_SOURCEFORGE}
${MASTER_SITE_SOURCEFORGE_JP}
${MASTER_SITE_SUNSITE}
${MASTER_SITE_SUSE}
${MASTER_SITE_TEX_CTAN}
${MASTER_SITE_XCONTRIB}
${MASTER_SITE_XEMACS}
${MASTER_SITE_XORG}
</programlisting>

      <para>Some explanations for the less self-explaining ones:
      <varname>MASTER_SITE_BACKUP</varname> contains backup sites
      for packages that are maintained in <ulink
      url="ftp://ftp.NetBSD.org/pub/pkgsrc/distfiles/${DIST_SUBDIR}"
      />.  <varname>MASTER_SITE_LOCAL</varname> contains local
      package source distributions that are maintained in <ulink
      url="ftp://ftp.NetBSD.org/pub/pkgsrc/distfiles/LOCAL_PORTS/"
      />.</para>

      <para>If you choose one of these predefined sites, you may
      want to specify a subdirectory of that site. Since these
      macros may expand to more than one actual site, you
      <emphasis>must</emphasis> use the following construct to
      specify a subdirectory:</para>

<programlisting>
MASTER_SITES=   ${MASTER_SITE_GNU:=subdirectory/name/}
MASTER_SITES=   ${MASTER_SITE_SOURCEFORGE:=project_name/}
</programlisting>

      <para>Note the trailing slash after the subdirectory
      name.</para>

    </sect2>
    <sect2 id="build.fetch.how">
      <title>How are the files fetched?</title>

      <para>The <emphasis>fetch</emphasis> phase makes sure that
      all the distfiles exist in a local directory
      (<varname>DISTDIR</varname>, which can be set by the pkgsrc
      user). If the files do not exist, they are fetched using
      commands of the form</para>

<programlisting>
${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS}
</programlisting>

      <para>where <literal>${site}</literal> varies through
      several possibilities in turn: first,
      <varname>MASTER_SITE_OVERRIDE</varname> is tried, then the
      sites specified in either <varname>SITES.file</varname> if
      defined, else <varname>MASTER_SITES</varname> or
      <varname>PATCH_SITES</varname>, as applies, then finally the
      value of <varname>MASTER_SITE_BACKUP</varname>. The order of
      all except the first and the last can be optionally sorted
      by the user, via setting either
      <varname>MASTER_SORT_RANDOM</varname>, and
      <varname>MASTER_SORT_AWK</varname> or
      <varname>MASTER_SORT_REGEX</varname>.</para>

      <para> The specific command and arguments used depend on the
      <varname>FETCH_USING</varname> parameter. The example above is
      for <literal>FETCH_USING=custom</literal>.</para>

      <para>The distfiles mirror run by the NetBSD Foundation uses the
      <emphasis>mirror-distfiles</emphasis> target to mirror the
      distfiles, if they are freely distributable.  Packages setting
      <varname>NO_SRC_ON_FTP</varname> (usually to
      <quote>${RESTRICTED}</quote>) will not have their distfiles
      mirrored.</para>
    </sect2>

  </sect1>

  <sect1 id="build.checksum">
    <title>The <emphasis>checksum</emphasis> phase</title>

    <para>After the distfile(s) are fetched, their checksum is
    generated and compared with the checksums stored in the
    distinfo file. If the checksums don't match, the build is
    aborted. This is to ensure the same distfile is used for
    building, and that the distfile wasn't changed, e.g. by some
    malign force, deliberately changed distfiles on the master
    distribution site or network lossage.</para>

  </sect1>

  <sect1 id="build.extract">
    <title>The <emphasis>extract</emphasis> phase</title>

    <para>When the distfiles are present on the local system, they
    need to be extracted, as they usually come in the form of some
    compressed archive format.</para>

    <para>By default, all <varname>DISTFILES</varname> are
    extracted. If you only need some of them, you can set the
    <varname>EXTRACT_ONLY</varname> variable to the list of those
    files.</para>

    <para>Extracting the files is usually done by a little
    program, <filename>mk/extract/extract</filename>, which
    already knows how to extract various archive formats, so most
    likely you will not need to change anything here. But if you
    need, the following variables may help you:</para>

    <variablelist>

      <varlistentry><term><varname>EXTRACT_OPTS_{BIN,LHA,PAX,RAR,TAR,ZIP,ZOO}</varname></term>
      <listitem><para>Use these variables to override the default
      options for an extract command, which are defined in
      <filename>mk/extract/extract</filename>.</para></listitem></varlistentry>

      <varlistentry><term><varname>EXTRACT_USING</varname></term>
      <listitem><para>This variable can be set to
      <literal>bsdtar</literal>, <literal>gtar</literal>, <literal>nbtar</literal>
      (which is the default value), <literal>pax</literal>, or an
      absolute pathname pointing to the command with which tar
      archives should be extracted.  It is preferred to choose bsdtar over gtar
      if NetBSD's pax-as-tar is not good enough.</para></listitem></varlistentry>

    </variablelist>

    <para>If the <filename>extract</filename> program doesn't
    serve your needs, you can also override the
    <varname>EXTRACT_CMD</varname> variable, which holds the
    command used for extracting the files. This command is
    executed in the <filename>${WRKSRC}</filename>
    directory. During execution of this command, the shell
    variable <varname>extract_file</varname> holds the absolute
    pathname of the file that is going to be extracted.</para>

    <para>And if that still does not suffice, you can override the
    <varname>do-extract</varname> target in the package
    Makefile.</para>

  </sect1>

  <sect1 id="build.patch">
    <title>The <emphasis>patch</emphasis> phase</title>

    <para>After extraction, all the patches named by the
    <varname>PATCHFILES</varname>, those present in the patches
    subdirectory of the package as well as in
    $LOCALPATCHES/$PKGPATH (e.g.
    <filename>/usr/local/patches/graphics/png</filename>) are
    applied.  Patchfiles ending in <filename>.Z</filename> or
    <filename>.gz</filename> are uncompressed before they are
    applied, files ending in <filename>.orig</filename> or
    <filename>.rej</filename> are ignored. Any special options to
    &man.patch.1; can be handed in
    <varname>PATCH_DIST_ARGS</varname>.  See <xref
    linkend="components.patches"/> for more details.</para>

    <para>By default &man.patch.1; is given special args to make
    it fail if the patches apply with some lines of fuzz. Please
    fix (regen) the patches so that they apply cleanly. The
    rationale behind this is that patches that don't apply cleanly
    may end up being applied in the wrong place, and cause severe
    harm there.</para>

  </sect1>

  <sect1 id="build.tools">
    <title>The <emphasis>tools</emphasis> phase</title>

    <para>This is covered in <xref linkend="tools"/>.
    </para>

  </sect1>

  <sect1 id="build.wrapper">
    <title>The <emphasis>wrapper</emphasis> phase</title>

    <para>This phase creates wrapper programs for the compilers and
    linkers. The following variables can be used to tweak the
    wrappers.</para>

    <variablelist>

      <varlistentry><term><varname>ECHO_WRAPPER_MSG</varname></term>
      <listitem><para>The command used to print progress
      messages. Does nothing by default. Set to
      <literal>${ECHO}</literal> to see the progress
      messages.</para></listitem></varlistentry>

      <varlistentry><term><varname>WRAPPER_DEBUG</varname></term>
      <listitem><para>This variable can be set to
      <literal>yes</literal> (default) or <literal>no</literal>,
      depending on whether you want additional information in the
      wrapper log file.</para></listitem></varlistentry>

      <varlistentry><term><varname>WRAPPER_UPDATE_CACHE</varname></term>
      <listitem><para>This variable can be set to
      <literal>yes</literal> or <literal>no</literal>, depending
      on whether the wrapper should use its cache, which will
      improve the speed. The default value is
      <literal>yes</literal>, but is forced to
      <literal>no</literal> if the platform does not support
      it.</para></listitem></varlistentry>

      <varlistentry><term><varname>WRAPPER_REORDER_CMDS</varname></term>

      <listitem><para>A list of reordering commands. A reordering
      command has the form
      <literal>reorder:l:<replaceable>lib1</replaceable>:<replaceable>lib2</replaceable></literal>.
      It ensures that that
      <literal>-l<replaceable>lib1</replaceable></literal> occurs
      before <literal>-l<replaceable>lib2</replaceable></literal>.
      </para></listitem></varlistentry>

      <varlistentry><term><varname>WRAPPER_TRANSFORM_CMDS</varname></term>
      <listitem><para>A list of transformation commands. [TODO:
      investigate further]</para></listitem></varlistentry>

      <!-- These should probably be internal variables
	   <varlistentry><term><varname>WRAPPEES</varname></term>
	   <listitem><para></para></listitem></varlistentry>
	   <varlistentry><term><varname>UNWRAP_PATTERNS</varname></term>
	   <listitem><para></para></listitem></varlistentry>
	   <varlistentry><term><varname>UNWRAP_FILES</varname></term>
	   <listitem><para></para></listitem></varlistentry>
      -->

    </variablelist>
  </sect1>

  <sect1 id="build.configure">
    <title>The <emphasis>configure</emphasis> phase</title>

    <para>Most pieces of software need information on the header
    files, system calls, and library routines which are available
    on the platform they run on. The process of determining this
    information is known as configuration, and is usually
    automated. In most cases, a script is supplied with the
    distfiles, and its invocation results in generation of header
    files, Makefiles, etc.</para>

    <para>If the package contains a configure script, this can be
    invoked by setting <varname>HAS_CONFIGURE</varname> to
    <quote>yes</quote>. If the configure script is a GNU autoconf
    script, you should set <varname>GNU_CONFIGURE</varname> to
    <quote>yes</quote> instead. What happens in the
    <emphasis>configure</emphasis> phase is roughly:</para>

<programlisting>
.for d in ${CONFIGURE_DIRS}
        cd ${WRKSRC} \
        &amp;&amp; cd ${d} \
        &amp;&amp; env ${CONFIGURE_ENV} ${CONFIGURE_SCRIPT} ${CONFIGURE_ARGS}
.endfor
</programlisting>

    <para><varname>CONFIGURE_DIRS</varname> (default:
    <quote>.</quote>) is a list of pathnames relative to
    <varname>WRKSRC</varname>. In each of these directories, the
    configure script is run with the environment
    <varname>CONFIGURE_ENV</varname> and arguments
    <varname>CONFIGURE_ARGS</varname>. The variables
    <varname>CONFIGURE_ENV</varname>,
    <varname>CONFIGURE_SCRIPT</varname> (default:
    <quote>./configure</quote>) and
    <varname>CONFIGURE_ARGS</varname> may all be changed by the
    package.</para>

    <para>If the program uses the Perl way of configuration (mainly Perl
    modules, but not only), i.e. a file called
    <filename>Makefile.PL</filename>, it should include
    <filename>../../lang/perl5/module.mk</filename>. To set any parameter for
    <filename>Makefile.PL</filename> use the <varname>MAKE_PARAMS</varname>
    variable (e.g., <literal>MAKE_PARAMS+=foo=bar</literal></para>

    <para>If the program uses an <filename>Imakefile</filename>
    for configuration, the appropriate steps can be invoked by
    setting <varname>USE_IMAKE</varname> to
    <quote>yes</quote>. (If you only want the package installed in
    <varname>${X11PREFIX}</varname> but xmkmf not being run, set
    <varname>USE_X11BASE</varname> instead.)  You can add variables to
    xmkmf's environment by adding them to the
    <varname>SCRIPTS_ENV</varname> variable.</para>

    <para>If the program uses <filename>cmake</filename>
    for configuration, the appropriate steps can be invoked by
    setting <varname>USE_CMAKE</varname> to <quote>yes</quote>.
    You can add variables to cmake's environment by adding them to the
    <varname>CONFIGURE_ENV</varname> variable and arguments to cmake
    by adding them to the <varname>CMAKE_ARGS</varname> variable.
    The top directory argument is given by the
    <varname>CMAKE_ARG_PATH</varname> variable, that defaults to
    <quote>.</quote> (relative to <varname>CONFIGURE_DIRS</varname>)</para>

    <para>If there is no configure step at all, set
    <varname>NO_CONFIGURE</varname> to <quote>yes</quote>.</para>
  </sect1>

  <sect1 id="build.build">
    <title>The <emphasis>build</emphasis> phase</title>

    <para>For building a package, a rough equivalent of the
    following code is executed.</para>

<programlisting>
.for d in ${BUILD_DIRS}
        cd ${WRKSRC} \
        &amp;&amp; cd ${d} \
        &amp;&amp; env ${MAKE_ENV} \
            ${MAKE_PROGRAM} ${BUILD_MAKE_FLAGS} \
                -f ${MAKE_FILE} \
                ${BUILD_TARGET}
.endfor
</programlisting>

    <para><varname>BUILD_DIRS</varname> (default:
    <quote>.</quote>) is a list of pathnames relative to
    <varname>WRKSRC</varname>. In each of these directories,
    <varname>MAKE_PROGRAM</varname> is run with the environment
    <varname>MAKE_ENV</varname> and arguments
    <varname>BUILD_MAKE_FLAGS</varname>. The variables
    <varname>MAKE_ENV</varname>,
    <varname>BUILD_MAKE_FLAGS</varname>,
    <varname>MAKE_FILE</varname> and
    <varname>BUILD_TARGET</varname> may all be changed by the
    package.</para>

    <para>The default value of <varname>MAKE_PROGRAM</varname> is
    <quote>gmake</quote> if <varname>USE_TOOLS</varname> contains
    <quote>gmake</quote>, <quote>make</quote> otherwise. The
    default value of <varname>MAKE_FILE</varname> is
    <quote>Makefile</quote>, and <varname>BUILD_TARGET</varname>
    defaults to <quote>all</quote>.</para>

    <para>If there is no build step at all, set
    <varname>NO_BUILD</varname> to <quote>yes</quote>.</para>
  </sect1>

  <sect1 id="build.test">
    <title>The <emphasis>test</emphasis> phase</title>

    <para>[TODO]</para>

  </sect1>

  <sect1 id="build.install">
    <title>The <emphasis>install</emphasis> phase</title>

    <para>Once the build stage has completed, the final step is to
    install the software in public directories, so users can
    access the programs and files.</para>

    <para>In the <emphasis>install</emphasis> phase, a rough
    equivalent of the following code is executed. Additionally,
    before and after this code, much magic is performed to do
    consistency checks, registering the package, and so on.</para>

<programlisting>
.for d in ${INSTALL_DIRS}
        cd ${WRKSRC} \
        &amp;&amp; cd ${d} \
        &amp;&amp; env ${MAKE_ENV} \
            ${MAKE_PROGRAM} ${INSTALL_MAKE_FLAGS} \
                -f ${MAKE_FILE} \
                ${INSTALL_TARGET}
.endfor
</programlisting>

    <para>The variable's meanings are analogous to the ones in the
    <emphasis>build</emphasis> phase.
    <varname>INSTALL_DIRS</varname> defaults to
    <varname>BUILD_DIRS</varname>. <varname>INSTALL_TARGET</varname>
    is <quote>install</quote> by default, plus
    <quote>install.man</quote> if <varname>USE_IMAKE</varname> is
    defined and <varname>NO_INSTALL_MANPAGES</varname> is not
    defined.</para>

    <para>In the <emphasis>install</emphasis> phase, the following
    variables are useful. They are all variations of the
    &man.install.1; command that have the owner, group and
    permissions preset. <varname>INSTALL</varname> is the plain
    install command. The specialized variants, together with their
    intended use, are:</para>

    <variablelist>
      <varlistentry><term><varname>INSTALL_PROGRAM_DIR</varname></term>
      <listitem><para>directories that contain
      binaries</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_SCRIPT_DIR</varname></term>
      <listitem><para>directories that contain
      scripts</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_LIB_DIR</varname></term>
      <listitem><para>directories that contain shared and static
      libraries</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_DATA_DIR</varname></term>
      <listitem><para>directories that contain data
      files</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_MAN_DIR</varname></term>
      <listitem><para>directories that contain man
      pages</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_PROGRAM</varname></term>
      <listitem><para>binaries that can be stripped from debugging
      symbols</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_SCRIPT</varname></term>
      <listitem><para>binaries that cannot be
      stripped</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_GAME</varname></term>
      <listitem><para>game
      binaries</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_LIB</varname></term>
      <listitem><para>shared and static
      libraries</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_DATA</varname></term>
      <listitem><para>data files</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_GAME_DATA</varname></term>
      <listitem><para>data files for
      games</para></listitem></varlistentry>

      <varlistentry><term><varname>INSTALL_MAN</varname></term>
      <listitem><para>man pages</para></listitem></varlistentry>
    </variablelist>

    <para>Some other variables are:</para>

    <variablelist>
      <varlistentry><term><varname>INSTALLATION_DIRS</varname></term>

      <listitem><para>A list of directories relative to
      <varname>PREFIX</varname> that are created by pkgsrc at the
      beginning of the <emphasis>install</emphasis> phase.
      The package is supposed to create all needed directories itself
      before installing files to it and list all other directories here.
      </para></listitem></varlistentry>

    </variablelist>

    <para>In the rare cases that a package shouldn't install anything,
    set <varname>NO_INSTALL</varname> to <quote>yes</quote>. This is
    mostly relevant for packages in the <filename>regress</filename>
    category.</para>
  </sect1>

  <sect1 id="build.package">
    <title>The <emphasis>package</emphasis> phase</title>

    <para>Once the install stage has completed, a binary package of
    the installed files can be built.  These binary packages can be
    used for quick installation without previous compilation, e.g. by
    the <command>make bin-install</command> or by using
    <command>pkg_add</command>.</para>

    <para>By default, the binary packages are created in
    <filename>${PACKAGES}/All</filename> and symlinks are created in
    <filename>${PACKAGES}/<replaceable>category</replaceable></filename>,
    one for each category in the <varname>CATEGORIES</varname>
    variable.  <varname>PACKAGES</varname> defaults to
    <filename>pkgsrc/packages</filename>.</para>
  </sect1>

  <sect1 id="build.clean">
    <title>Cleaning up</title>

    <para>Once you're finished with a package, you can clean the work
    directory by running <command>make clean</command>.  If you want
    to clean the work directories of all dependencies too, use
    <command>make clean-depends</command>.</para>
  </sect1>

  <sect1 id="build.helpful-targets">
    <title>Other helpful targets</title>

    <variablelist>
      <varlistentry>
	<term>pre/post-*</term>

	<listitem>
	  <para>For any of the main targets described in the
	  previous section, two auxiliary targets exist with
	  <quote>pre-</quote> and <quote>post-</quote> used as a
	  prefix for the main target's name.  These targets are
	  invoked before and after the main target is called,
	  allowing extra configuration or installation steps be
	  performed from a package's Makefile, for example, which
	  a program's configure script or install target
	  omitted.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>do-*</term>

	<listitem>
	  <para>Should one of the main targets do the wrong thing,
	  and should there be no variable to fix this, you can
	  redefine it with the do-* target.  (Note that redefining
	  the target itself instead of the do-* target is a bad
	  idea, as the pre-* and post-* targets won't be called
	  anymore, etc.) You will not usually need to do
	  this.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>reinstall</term>

	<listitem>
	  <para>If you did a <command>make install</command> and
	  you noticed some file was not installed properly, you
	  can repeat the installation with this target, which will
	  ignore the <quote>already installed</quote> flag.</para>

	  <para>This is the default value of
	  <varname>DEPENDS_TARGET</varname> except in the case of
	  <command>make update</command> and <command>make
	  package</command>, where the defaults are
	  <quote>package</quote> and <quote>update</quote>,
	  respectively.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>deinstall</term>

	<listitem>
	  <para>This target does a &man.pkg.delete.1; in the
	  current directory, effectively de-installing the
	  package. The following variables can be used to tune the
	  behaviour:</para>

	  <variablelist>
	    <varlistentry>
	      <term><varname>PKG_VERBOSE</varname></term>

	      <listitem>
		<para>Add a "-v" to the &man.pkg.delete.1; command.</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
	      <term><varname>DEINSTALLDEPENDS</varname></term>

	      <listitem>
		<para>Remove all packages that require (depend on)
		the given package.  This can be used to remove any
		packages that may have been pulled in by a given
		package, e.g. if <command>make deinstall
		DEINSTALLDEPENDS=1</command> is done in
		<filename>pkgsrc/x11/kde</filename>, this is
		likely to remove whole KDE. Works by adding
		<quote>-R</quote> to the &man.pkg.delete.1;
		command line.</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>bin-install</term>

	<listitem>
	  <para>Install a binary package from local disk and via FTP
	  from a list of sites (see the
	  <varname>BINPKG_SITES</varname> variable), and do a
	  <command>make package</command> if no binary package is
	  available anywhere.  The arguments given to
	  <command>pkg_add</command> can be set via
	  <varname>BIN_INSTALL_FLAGS</varname> e.g., to do verbose
	  operation, etc.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>update</term>

	<listitem>
	  <para>This target causes the current package to be
	  updated to the latest version.  The package and all
	  depending packages first get de-installed, then current
	  versions of the corresponding packages get compiled and
	  installed.  This is similar to manually noting which
	  packages are currently installed, then performing a
	  series of <command>make deinstall</command> and
	  <command>make install</command> (or whatever
	  <varname>UPDATE_TARGET</varname> is set to) for these
	  packages.</para>

	  <para>You can use the <quote>update</quote> target to
	  resume package updating in case a previous <command>make
	  update</command> was interrupted for some reason.
	  However, in this case, make sure you don't call
	  <command>make clean</command> or otherwise remove the
	  list of dependent packages in <varname>WRKDIR</varname>.
	  Otherwise, you lose the ability to automatically update
	  the current package along with the dependent packages
	  you have installed.</para>

	  <para>Resuming an interrupted <command>make
	  update</command> will only work as long as the package
	  tree remains unchanged.  If the source code for one of
	  the packages to be updated has been changed, resuming
	  <command>make update</command> will most certainly
	  fail!</para>

	  <para>The following variables can be used either on the
	  command line or in &mk.conf; to
	  alter the behaviour of <command>make
	  update</command>:</para>

	  <variablelist>
	    <varlistentry>
	      <term><varname>UPDATE_TARGET</varname></term>

	      <listitem>
		<para>Install target to recursively use for the
		updated package and the dependent packages.
		Defaults to <varname>DEPENDS_TARGET</varname> if
		set, <quote>install</quote> otherwise for
		<command>make update</command>.  Other good
		targets are <quote>package</quote> or
		<quote>bin-install</quote>.  Do not set this to
		<quote>update</quote> or you will get stuck in an
		endless loop!</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
	      <term><varname>NOCLEAN</varname></term>

	      <listitem>
		<para>Don't clean up after updating.  Useful if
		you want to leave the work sources of the updated
		packages around for inspection or other purposes.
		Be sure you eventually clean up the source tree
		(see the <quote>clean-update</quote> target below)
		or you may run into troubles with old source code
		still lying around on your next
		<command>make</command> or <command>make
		update</command>.</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
	      <term><varname>REINSTALL</varname></term>

	      <listitem>
		<para>Deinstall each package before installing
		(making <varname>DEPENDS_TARGET</varname>). This
		may be necessary if the
		<quote>clean-update</quote> target (see below) was
		called after interrupting a running <command>make
		update</command>.</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
	      <term><varname>DEPENDS_TARGET</varname></term>

	      <listitem>
		<para>Allows you to disable recursion and hardcode
		the target for packages.  The default is
		<quote>update</quote> for the update target,
		facilitating a recursive update of prerequisite
		packages.  Only set
		<varname>DEPENDS_TARGET</varname> if you want to
		disable recursive updates. Use
		<varname>UPDATE_TARGET</varname> instead to just
		set a specific target for each package to be
		installed during <command>make update</command>
		(see above).</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>clean-update</term>

	<listitem>
	  <para>Clean the source tree for all packages that would
	  get updated if <command>make update</command> was called
	  from the current directory.  This target should not be
	  used if the current package (or any of its depending
	  packages) have already been de-installed (e.g., after
	  calling <command>make update</command>) or you may lose
	  some packages you intended to update. As a rule of
	  thumb: only use this target <emphasis>before</emphasis>
	  the first time you run <command>make update</command>
	  and only if you have a dirty package tree (e.g., if you
	  used <varname>NOCLEAN</varname>).</para>

	  <para>If you are unsure about whether your tree is
	  clean, you can either perform a <command>make
	  clean</command> at the top of the tree, or use the
	  following sequence of commands from the directory of the
	  package you want to update (<emphasis>before</emphasis>
	  running <command>make update</command> for the first
	  time, otherwise you lose all the packages you wanted to
	  update!):</para>

	  <screen>
&rprompt; <userinput>make clean-update</userinput>
&rprompt; <userinput>make clean CLEANDEPENDS=YES</userinput>
&rprompt; <userinput>make update</userinput>
	  </screen>

	  <para>The following variables can be used either on the
	  command line or in &mk.conf; to alter the behaviour of
	  <command>make clean-update</command>:</para>

	  <variablelist>
	    <varlistentry>
	      <term><varname>CLEAR_DIRLIST</varname></term>

	      <listitem>
		<para>After <command>make clean</command>, do not
		reconstruct the list of directories to update for
		this package.  Only use this if <command>make
		update</command> successfully installed all
		packages you wanted to update.  Normally, this is
		done automatically on <command>make
		update</command>, but may have been suppressed by
		the <varname>NOCLEAN</varname> variable (see
		above).</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>replace</term>

	<listitem>
	  <para>Update the installation of the current package.  This
	  differs from update in that it does not replace dependent
	  packages.  You will need to install <filename
	  role="pkg">pkgtools/pkg_tarup</filename> for this
	  target to work.</para>

	  <para><emphasis>Be careful when using this
	  target!</emphasis> There are no guarantees that dependent
	  packages will still work, in particular they will most
	  certainly break if you <command>make replace</command> a
	  library package whose shared library major version changed
	  between your installed version and the new one.  For this
	  reason, this target is not officially supported and only
	  recommended for advanced users.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>info</term>

	<listitem>
	  <para>This target invokes &man.pkg.info.1; for the current
	  package. You can use this to check which version of a
	  package is installed.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>index</term>

	<listitem>
	  <para>This is a top-level command, i.e. it should be used in
	  the <filename>pkgsrc</filename> directory.  It creates a
	  database of all packages in the local pkgsrc tree, including
	  dependencies, comment, maintainer, and some other useful
	  information.  Individual entries are created by running
	  <command>make describe</command> in the packages'
	  directories.  This index file is saved as
	  <filename>pkgsrc/INDEX</filename>.  It can be displayed in
	  verbose format by running <command>make
	  print-index</command>.  You can search in it with
	  <command>make search
	  key=<replaceable>something</replaceable></command>.  You can
	  extract a list of all packages that depend on a particular
	  one by running <command>make show-deps
	  PKG=<replaceable>somepackage</replaceable></command>.</para>

	  <para>Running this command takes a very long time, some
	  hours even on fast machines!</para>
	</listitem>
      </varlistentry>


      <varlistentry>
	<term>readme</term>

	<listitem>
	  <para>This target generates a
	  <filename>README.html</filename> file, which can be
	  viewed using a browser such as <filename
	  role="pkg">www/firefox</filename> or <filename
	  role="pkg">www/links</filename>.  The generated files
	  contain references to any packages which are in the
	  <varname>PACKAGES</varname> directory on the local
	  host. The generated files can be made to refer to URLs
	  based on <varname>FTP_PKG_URL_HOST</varname> and
	  <varname>FTP_PKG_URL_DIR</varname>. For example, if I
	  wanted to generate <filename>README.html</filename>
	  files which pointed to binary packages on the local
	  machine, in the directory
	  <filename>/usr/packages</filename>, set
	  <varname>FTP_PKG_URL_HOST=file://localhost</varname> and
	  <varname>FTP_PKG_URL_DIR=/usr/packages</varname>. The
	  <varname>${PACKAGES}</varname> directory and its
	  subdirectories will be searched for all the binary
	  packages.</para>

	  <para>The target can be run at the toplevel or in category
	  directories, in which case it descends recursively.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>readme-all</term>

	<listitem>
	  <para>This is a top-level command, run it in
	  <filename>pkgsrc</filename>.  Use this target to create a
	  file <filename>README-all.html</filename> which contains a
	  list of all packages currently available in the &os;
	  Packages Collection, together with the category they belong
	  to and a short description. This file is compiled from the
	  <filename>pkgsrc/*/README.html</filename> files, so be sure
	  to run this <emphasis>after</emphasis> a <command>make
	  readme</command>.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>cdrom-readme</term>

	<listitem>
	  <para>This is very much the same as the
	  <quote>readme</quote> target (see above), but is to be
	  used when generating a pkgsrc tree to be written to a
	  CD-ROM.  This target also produces
	  <filename>README.html</filename> files, and can be made
	  to refer to URLs based on
	  <varname>CDROM_PKG_URL_HOST</varname> and
	  <varname>CDROM_PKG_URL_DIR</varname>.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>show-distfiles</term>

	<listitem>
	  <para>This target shows which distfiles and patchfiles
	  are needed to build the package
	  (<varname>ALLFILES</varname>, which contains all
	  <varname>DISTFILES</varname> and
	  <varname>PATCHFILES</varname>, but not
	  <filename>patches/*</filename>).</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>show-downlevel</term>

	<listitem>
	  <para>This target shows nothing if the package is not
	  installed. If a version of this package is installed,
	  but is not the version provided in this version of
	  pkgsrc, then a warning message is displayed. This target
	  can be used to show which of your installed packages are
	  downlevel, and so the old versions can be deleted, and
	  the current ones added.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>show-pkgsrc-dir</term>

	<listitem>
	  <para>This target shows the directory in the pkgsrc
	  hierarchy from which the package can be built and
	  installed. This may not be the same directory as the one
	  from which the package was installed. This target is
	  intended to be used by people who may wish to upgrade
	  many packages on a single host, and can be invoked from
	  the top-level pkgsrc Makefile by using the
	  <quote>show-host-specific-pkgs</quote> target.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>show-installed-depends</term>

	<listitem>
	  <para>This target shows which installed packages match
	  the current package's <varname>DEPENDS</varname>. Useful
	  if out of date dependencies are causing build
	  problems.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>check-shlibs</term>

	<listitem>
	  <para>After a package is installed, check all its
	  binaries and (on ELF platforms) shared libraries to see
	  if they find the shared libs they need.  Run by default
	  if <varname>PKG_DEVELOPER</varname> is set in &mk.conf;.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>print-PLIST</term>

	<listitem>
	  <para>After a <quote>make install</quote> from a new or
	  upgraded pkg, this prints out an attempt to generate a
	  new <filename>PLIST</filename> from a <command>find
	  -newer work/.extract_done</command>.  An attempt is made
	  to care for shared libs etc., but it is
	  <emphasis>strongly</emphasis> recommended to review the
	  result before putting it into
	  <filename>PLIST</filename>. On upgrades, it's useful to
	  diff the output of this command against an already
	  existing <filename>PLIST</filename> file.</para>

	  <para>If the package installs files via &man.tar.1; or
	  other methods that don't update file access times, be
	  sure to add these files manually to your
	  <filename>PLIST</filename>, as the <quote>find
	  -newer</quote> command used by this target won't catch
	  them!</para>

	  <para>See <xref linkend="print-PLIST"/> for more
	  information on this target.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<!-- TODO: old bulk build? --> 
	<term>bulk-package</term>

	<listitem>
	  <para>Used to do bulk builds. If an appropriate binary
	  package already exists, no action is taken. If not, this
	  target will compile, install and package it (and its
	  depends, if <varname>PKG_DEPENDS</varname> is set
	  properly. See <xref linkend="bulk"/>).
	  After creating the binary package, the sources, the
	  just-installed package and its required packages are
	  removed, preserving free disk space.</para>

	  <para><emphasis>Beware that this target may deinstall
	  all packages installed on a system!</emphasis></para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>bulk-install</term>

	<listitem>
	  <para>Used during bulk-installs to install required
	  packages. If an up-to-date binary package is available,
	  it will be installed via &man.pkg.add.1;. If not,
	  <command>make bulk-package</command> will be executed,
	  but the installed binary won't be removed.</para>

	  <para>A binary package is considered
	  <quote>up-to-date</quote> to be installed via
	  &man.pkg.add.1; if:</para>

	  <itemizedlist>
	    <listitem>
	      <para>None of the package's files
	      (<filename>Makefile</filename>, ...) were modified
	      since it was built.</para>
	    </listitem>

	    <listitem>
	      <para>None of the package's required (binary)
	      packages were modified since it was built.</para>
	    </listitem>
	  </itemizedlist>

	  <para><emphasis>Beware that this target may deinstall
	  all packages installed on a system!</emphasis></para>
	</listitem>
      </varlistentry>
    </variablelist>
  </sect1>
</chapter>