path: root/doc/guide/files/components.xml

<!-- $NetBSD: components.xml,v 1.59 2020/05/17 00:54:00 tnn Exp $ -->

<chapter id="components"> <?dbhtml filename="components.html"?>
<title>Package components - files, directories and contents</title>

<para>Whenever you're preparing a package, there are a number of
files involved which are described in the following
sections.</para>

<sect1 id="components.Makefile">
  <title><filename>Makefile</filename></title>

  <para>Building, installation and creation of a binary package are all
  controlled by the package's <filename>Makefile</filename>.
  The <filename>Makefile</filename> describes various things about
  a package, for example from where to get it, how to configure,
  build, and install it.</para>

  <para>A package <filename>Makefile</filename> contains several
  sections that describe the package.</para>

  <para>In the first section there are the following variables, which
  should appear exactly in the order given here.  The order and
  grouping of the variables is mostly historical and has no further
  meaning.</para>
  <itemizedlist>

    <listitem><para><varname>DISTNAME</varname> is the basename of the
    distribution file to be downloaded from the package's
    website.</para></listitem>

    <listitem><para><varname>PKGNAME</varname> is the name of the
    package, as used by pkgsrc. You need to provide it if
    <varname>DISTNAME</varname> (which is the default) is not a good
    name for the package in pkgsrc or <varname>DISTNAME</varname> is not
    provided (no distribution file is required).  Usually it is the pkgsrc
    directory name together with the version number. It must match the
    regular expression
    <varname>^[A-Za-z0-9][A-Za-z0-9-_.+]*$</varname>, that is, it
    starts with a letter or digit, and contains only letters, digits,
    dashes, underscores, dots and plus signs.</para></listitem>

    <listitem><para><varname>CATEGORIES</varname> is a list of categories
    which the package fits in. You can choose any of the top-level
    directories of pkgsrc for it.</para>

    <para>Currently the following values are available for
    <varname>CATEGORIES</varname>. If more than
    one is used, they need to be separated by spaces:</para>

<programlisting>
archivers     cross         geography     meta-pkgs     security
audio         databases     graphics      misc          shells
benchmarks    devel         ham           multimedia    sysutils
biology       editors       inputmethod   net           textproc
cad           emulators     lang          news          time
chat          finance       mail          parallel      wm
comms         fonts         math          pkgtools      www
converters    games         mbone         print         x11
</programlisting>
    </listitem>

    <listitem><para><varname>MASTER_SITES</varname>,
    <varname>DYNAMIC_MASTER_SITES</varname>,
    <varname>DIST_SUBDIR</varname>, <varname>EXTRACT_SUFX</varname>
    and <varname>DISTFILES</varname> are discussed in detail in
    <xref linkend="build.fetch"/>.</para></listitem>

  </itemizedlist>

  <para>The second section contains information about separately
  downloaded patches, if any.
  <itemizedlist>
    <listitem><para><varname>PATCHFILES</varname>:
    Name(s) of additional files that contain distribution patches.
    There is no default. pkgsrc will look for them at
    <varname>PATCH_SITES</varname>.
    They will automatically be uncompressed before patching if
    the names end with <filename>.gz</filename> or
    <filename>.Z</filename>.</para>
    </listitem>
    <listitem><para><varname>PATCH_SITES</varname>:
    Primary location(s) for distribution patch files (see
    <varname>PATCHFILES</varname> above) if not found locally.</para>
    </listitem>
    <listitem><para><varname>PATCH_DIST_STRIP</varname>:
    an argument to &man.patch.1; that sets the pathname strip count to
    help find the correct files to patch. It defaults to
    <command>-p0</command>.</para>
    </listitem>
  </itemizedlist></para>
  <para>The third section contains the following variables.
  <itemizedlist>

    <listitem><para><varname>MAINTAINER</varname> is the email
    address of the person who feels responsible for this package,
    and who is most likely to look at problems or questions regarding
    this package which have been reported with &man.send-pr.1;.
    Other developers may contact the <varname>MAINTAINER</varname>
    before making changes to the package, but are not required to
    do so. When packaging a new program, set <varname>MAINTAINER</varname>
    to yourself. If you really can't maintain the package for future
    updates, set it to
    <email>pkgsrc-users@NetBSD.org</email>.</para></listitem>

    <listitem><para><varname>OWNER</varname> should be used instead
    of <varname>MAINTAINER</varname> when you do not want other
    developers to update or change the package without contacting
    you first. A package Makefile should contain one of
    <varname>MAINTAINER</varname> or <varname>OWNER</varname>, but
    not both.  </para></listitem>

    <listitem><para><varname>HOMEPAGE</varname> is a URL where users can
    find more information about the package.</para></listitem>

    <listitem><para><varname>COMMENT</varname> is a one-line
    description of the package (should not include the package
    name).</para></listitem>

    <listitem><para><varname>LICENSE</varname> indicates the license(s)
    applicable for the package. See <xref
    linkend="handling-licenses"/> for further details.</para>
    </listitem>
  </itemizedlist></para>

  <para>Other variables that affect the build:
  <itemizedlist>

    <listitem>
      <para><varname>WRKSRC</varname>: The directory where the
      interesting distribution files of the package are found. The
      default is <filename>${WRKDIR}/${DISTNAME}</filename>, which
      works for most packages.</para>

      <para>If a package doesn't create a subdirectory for itself
      (most GNU software does, for instance), but extracts itself in
      the current directory, you should set
      <varname>WRKSRC=${WRKDIR}</varname>.</para>

      <para>If a package doesn't create a subdirectory with the
      name of <varname>DISTNAME</varname> but some different name,
      set <varname>WRKSRC</varname> to point to the proper name in
      <filename>${WRKDIR}</filename>, for example
      <varname>WRKSRC=${WRKDIR}/${DISTNAME}/unix</varname>. See
      <filename role="pkg">lang/tcl</filename> and <filename
      role="pkg">x11/tk</filename> for other examples.</para>

      <para>The name of the working directory created by pkgsrc is
      taken from the <varname>WRKDIR_BASENAME</varname>
      variable. By default, its value is
      <filename>work</filename>. If you want to use the same
      pkgsrc tree for building different kinds of binary packages,
      you can change the variable according to your needs. Two
      other variables handle common cases of setting
      <varname>WRKDIR_BASENAME</varname> individually. If
      <varname>OBJHOSTNAME</varname> is defined in
      &mk.conf;, the first component of
      the host's name is attached to the directory name. If
      <varname>OBJMACHINE</varname> is defined, the platform name
      is attached, which might look like
      <filename>work.i386</filename> or
      <filename>work.sparc</filename>.</para>

    </listitem>

  </itemizedlist></para>

  <para>Please pay attention to the following gotchas:</para>
  <itemizedlist>
    <listitem>
      <para>Add <varname>MANCOMPRESSED</varname> if man pages are
      installed in compressed form by the package.  For packages using
      BSD-style makefiles which honor MANZ, there is
      <varname>MANCOMPRESSED_IF_MANZ</varname>.</para>
    </listitem>

    <listitem>
      <para>Replace <filename>/usr/local</filename> with
      <quote>${PREFIX}</quote> in all files (see patches,
      below).</para>
    </listitem>

    <listitem>
      <para>If the package installs any info files, see <xref
      linkend="faq.info-files"/>.</para>
    </listitem>

  </itemizedlist>
</sect1>

<sect1 id="components.distinfo">
  <title><filename>distinfo</filename></title>

  <para>The <filename>distinfo</filename> file contains the message
  digest, or checksum, of each distfile needed for the package. This
  ensures that the distfiles retrieved from the Internet have not been
  corrupted during transfer or altered by a malign force to introduce
  a security hole. To provide maximum security, all distfiles are
  protected using three different message digest algorithms (SHA1,
  RMD160, SHA512), as well as the file size.</para>

  <para>The <filename>distinfo</filename> file also contains the
  checksums for all the patches found in the
  <filename>patches</filename> directory (see <xref
  linkend="components.patches"/>). These checksums ensure that patches
  are only applied intentionally and that they don't accidentally change,
  e.g. when merging different changes together. They also make sure that
  new patches are actually added to CVS and old ones are removed.
  Too see whether the patches and the <filename>distinfo</filename> file
  match, run <command>pkglint</command> after changing the patches.</para>

  <para>To regenerate the <filename>distinfo</filename> file, use the
  <command>make distinfo</command> command.</para>

  <para>Some packages have different sets of distfiles depending on
  the platform, for example <filename
  role="pkg">lang/openjdk8</filename>. These are kept in the same
  <filename>distinfo</filename> file and care should be taken when
  upgrading such a package to ensure distfile information is not
  lost.</para>

</sect1>

<sect1 id="components.patches">
  <title><filename>patches/*</filename></title>

  <para>Some packages don't work out-of-the box on the various
  platforms that are supported by pkgsrc. These packages need
  to be patched to make them work. The patch files can be
  found in the <filename>patches/</filename> directory.</para>

  <para>In the <emphasis>patch</emphasis> phase, these patches are
  applied to the files in <varname>WRKSRC</varname> directory after
  extracting them, in alphabetic order.</para>

<sect2 id="components.patch.structure">
<title>Structure of a single patch file</title>

  <para>The <filename>patch-*</filename> files should be in
  <command>diff -bu</command> format, and apply without a fuzz to avoid
  problems. (To force patches to apply with fuzz you can set
  <varname>PATCH_FUZZ_FACTOR=-F2</varname>). Furthermore, each patch
  should contain only changes for a single file, and no file should be
  patched by more than one patch file. This helps to keep future
  modifications simple.</para>

  <para>Each patch file is structured as follows: In the first line,
  there is the RCS Id of the patch itself. The second line should be
  empty for aesthetic reasons. After that, there should be a comment for
  each change that the patch does. There are a number of standard
  cases:</para>

  <itemizedlist>

  <listitem><para>Patches for commonly known vulnerabilities should
  mention the vulnerability ID (CAN, CVE).</para></listitem>

  <listitem><para>Patches that change source code should mention the
  platform and other environment (for example, the compiler) that the
  patch is needed for.</para></listitem>

  </itemizedlist>

  <para>The patch should be commented so that any
  developer who knows the code of the application can make some use of
  the patch. Special care should be taken for the upstream developers,
  since we generally want that they accept our patches, so we have less
  work in the future.</para>

</sect2>
<sect2 id="components.patches.caveats">
<title>Creating patch files</title>

  <para>One important thing to mention is to pay attention that no RCS
  IDs get stored in the patch files, as these will cause problems when
  later checked into the &os; CVS tree. Use the
  <command>pkgdiff</command> command from the <filename
  role="pkg">pkgtools/pkgdiff</filename> package to avoid these
  problems.</para>

  <para>For even more automation, we recommend using
  <command>mkpatches</command> from the same package to make a
  whole set of patches. You just have to backup files before you
  edit them to <filename>filename.orig</filename>, e.g. with
  <command>cp -p filename filename.orig</command> or, easier, by
  using <command>pkgvi</command> again from the same package. If
  you upgrade a package this way, you can easily compare the new
  set of patches with the previously existing one with
  <command>patchdiff</command>. The files in <filename>patches</filename>
  are replaced by new files, so carefully check if you want to take all
  the changes.</para>

  <para>When you have finished a package, remember to generate
  the checksums for the patch files by using the <command>make
  makepatchsum</command> command, see <xref
  linkend="components.distinfo"/>.</para>

  <para>When adding a patch that corrects a problem in the
  distfile (rather than e.g. enforcing pkgsrc's view of where
  man pages should go), send the patch as a bug report to the
  maintainer.  This benefits non-pkgsrc users of the package,
  and usually makes it possible to remove the patch in future
  version.</para>

  <para>The file names of the patch files are usually of the form
  <filename>patch-<replaceable>path_to_file__with__underscores.c</replaceable></filename>.
  Many packages still use the previous convention
  <filename>patch-<replaceable>[a-z][a-z]</replaceable></filename>,
  but new patches should be of the form containing the filename.
  <command>mkpatches</command> included in <filename
  role="pkg">pkgtools/pkgdiff</filename> takes care of the name
  automatically.</para>

</sect2>
<sect2 id="components.patches.sources">
<title>Sources where the patch files come from</title>

  <para>If you want to share patches between multiple packages
  in pkgsrc, e.g. because they use the same distfiles, set
  <varname>PATCHDIR</varname> to the path where the patch files
  can be found, e.g.:</para>
<programlisting>
PATCHDIR=       ../../editors/xemacs/patches
</programlisting>

    <para>Patch files that are distributed by the author or other
    maintainers can be listed in
    <varname>PATCHFILES</varname>.</para>

    <para>If it is desired to store any patches that should not be
    committed into pkgsrc, they can be kept outside the pkgsrc
    tree in the <filename>$LOCALPATCHES</filename> directory. The
    directory tree there is expected to have the same
    <quote>category/package</quote> structure as pkgsrc, and
    patches are expected to be stored inside these dirs (also
    known as <filename>$LOCALPATCHES/$PKGPATH</filename>). For
    example, if you want to keep a private patch for
    <filename>pkgsrc/graphics/png</filename>, keep it in
    <filename>$LOCALPATCHES/graphics/png/mypatch</filename>. All
    files in the named directory are expected to be patch files,
    and <emphasis>they are applied after pkgsrc patches are
    applied</emphasis>.</para>

</sect2>
    <sect2 id="components.patches.guidelines">
      <title>Patching guidelines</title>

      <para>When fixing a portability issue in the code do not use
      preprocessor magic to check for the current operating system nor
      platform.  Doing so hurts portability to other platforms because
      the OS-specific details are not abstracted appropriately.</para>

      <para>The general rule to follow is: instead of checking for the
      operating system the application is being built on, check for the
      specific <emphasis>features</emphasis> you need.  For example,
      instead of assuming that kqueue is available under NetBSD and
      using the <varname>__NetBSD__</varname> macro to conditionalize
      kqueue support, add a check that detects kqueue itself &mdash;
      yes, this generally involves patching the
      <command>configure</command> script.  There is absolutely nothing
      that prevents some OSes from adopting interfaces from other OSes
      (e.g. Linux implementing kqueue), something that the above checks
      cannot take into account.</para>

      <para>Of course, checking for features generally involves more
      work on the developer's side, but the resulting changes are
      cleaner and there are chances they will work on many other
      platforms.  Not to mention that there are higher chances of being
      later integrated into the mainstream sources.  Remember:
      <emphasis>It doesn't work unless it is right!</emphasis></para>

      <para>Some typical examples:</para>

      <table id="patch-examples">
	<title>Patching examples</title>

	<tgroup cols="3">
	  <thead>
	    <row>
	      <entry>Where</entry>
	      <entry>Incorrect</entry>
	      <entry>Correct</entry>
	    </row>
	  </thead>

	  <tbody>
	    <row>
	      <entry>configure script</entry>
	      <entry>
<programlisting>
case ${target_os} in
netbsd*) have_kvm=yes ;;
*)       have_kvm=no  ;;
esac
</programlisting>
	      </entry>

	      <entry>
<programlisting>
AC_CHECK_LIB(kvm, kvm_open, have_kvm=yes, have_kvm=no)
</programlisting>
	      </entry>
	    </row>

	    <row>
	      <entry>C source file</entry>
	      <entry>
<programlisting>
#if defined(__NetBSD__)
#  include &lt;sys/event.h&gt;
#endif
</programlisting>
	      </entry>
	      <entry>
<programlisting>
#if defined(HAVE_SYS_EVENT_H)
#  include &lt;sys/event.h&gt;
#endif
</programlisting>
	      </entry>
	    </row>

	    <row>
	      <entry>C source file</entry>
	      <entry>
<programlisting>
int
monitor_file(...)
{
#if defined(__NetBSD__)
        int fd = kqueue();
        ...
#else
        ...
#endif
}
</programlisting>
	      </entry>
	      <entry>
<programlisting>
int
monitor_file(...)
{
#if defined(HAVE_KQUEUE)
        int fd = kqueue();
        ...
#else
        ...
#endif
}
</programlisting>
	      </entry>
	    </row>
	  </tbody>
	</tgroup>
      </table>
    </sect2>

    <sect2 id="components.patches.feedback">
      <title>Feedback to the author</title>

      <para>Always, always, <emphasis role="strong">always</emphasis>
      feed back any <emphasis>portability fixes</emphasis> or
      improvements you do to a package to the mainstream developers.
      This is the only way to get their attention on portability issues
      and to ensure that future versions can be built out-of-the box on
      NetBSD.  Furthermore, any user that gets newer distfiles will get
      the fixes straight from the packaged code.</para>

      <para>This generally involves cleaning up the patches
      (because sometimes the patches that are
      added to pkgsrc are quick hacks), filing bug reports in the
      appropriate trackers for the projects and working with the
      mainstream authors to accept your changes.  It is
      <emphasis>extremely important</emphasis> that you do it so that
      the packages in pkgsrc are kept simple and thus further changes
      can be done without much hassle.</para>

      <para>When you have done this, please add a URL to the upstream
      bug report to the patch comment.</para>

      <para>Support the idea of free software!</para>

    </sect2>

  </sect1>

  <sect1 id="other-mandatory-files">
    <title>Other mandatory files</title>

    <variablelist>
      <varlistentry>
	<term><filename>DESCR</filename></term>

	<listitem>
	  <para>A multi-line description of the piece of software.  This should include
	  any credits where they are due.  Please bear in mind that others do not
	  share your sense of humour (or spelling idiosyncrasies), and that others
	  will read everything that you write here.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><filename>PLIST</filename></term>

	<listitem>
	  <para>This file governs the files that are installed on your
	  system: all the binaries, manual pages, etc. There are other
	  directives which may be entered in this file, to control the
	  creation and deletion of directories, and the location of
	  inserted files.  See <xref linkend="plist"/> for more
	  information.</para>
	</listitem>
      </varlistentry>
    </variablelist>
  </sect1>

  <sect1 id="components.optional">
    <title>Optional files</title>

    <sect2 id="components.optional.bin">
      <title>Files affecting the binary package</title>

      <variablelist>
	<varlistentry>
	  <term><filename>INSTALL</filename></term>

	  <listitem>
	    <para>This shell script is invoked twice by &man.pkg.add.1;.
	    First time after package extraction and before files are
	    moved in place, the second time after the files to install
	    are moved in place. This can be used to do any custom
	    procedures not possible with @exec commands in
	    <filename>PLIST</filename>. See &man.pkg.add.1; and
	    &man.pkg.create.1; for more information.  See also <xref
	    linkend="files-and-dirs-outside-prefix" />.
	    Please note that you can modify variables in it easily by using
	    <varname>FILES_SUBST</varname> in the package's
	    <filename>Makefile</filename>:</para>

<programlisting>
FILES_SUBST+=  SOMEVAR="somevalue"
</programlisting>

	    <para>replaces "@SOMEVAR@" with <quote>somevalue</quote> in the
	    <filename>INSTALL</filename>.  By default, substitution is
	    performed for <varname>PREFIX</varname>,
	    <varname>LOCALBASE</varname>, <varname>X11BASE</varname>,
	    <varname>VARBASE</varname>, and a few others, type
	    <command>make help topic=FILES_SUBST</command> for a
	    complete list.</para>
 	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term><filename>DEINSTALL</filename></term>

	  <listitem>
	    <para>This script is executed before and after any files are removed.  It is
	    this script's responsibility to clean up any additional messy details
	    around the package's installation, since all pkg_delete knows is how to
	    delete the files created in the original distribution.
	    See &man.pkg.delete.1;
	    and &man.pkg.create.1; for more information.
	    The same methods to replace variables can be used as for
	    the <filename>INSTALL</filename> file.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term><filename>MESSAGE</filename></term>

	  <listitem>
	    <para>This file is displayed after installation of the package.
	    Useful for things like legal notices on almost-free
	    software and hints for updating config files after
	    installing modules for apache, PHP etc.
	    Please note that you can modify variables in it easily by using
	    <varname>MESSAGE_SUBST</varname> in the package's
	    <filename>Makefile</filename>:</para>

<programlisting>
MESSAGE_SUBST+=  SOMEVAR="somevalue"
</programlisting>

	    <para>replaces "${SOMEVAR}" with <quote>somevalue</quote> in
	    <filename>MESSAGE</filename>.  By default, substitution is
	    performed for <varname>PKGNAME</varname>,
	    <varname>PKGBASE</varname>, <varname>PREFIX</varname>,
	    <varname>LOCALBASE</varname>, <varname>X11BASE</varname>,
	    <varname>PKG_SYSCONFDIR</varname>,
	    <varname>ROOT_GROUP</varname>, and
	    <varname>ROOT_USER</varname>.</para>

	    <para>You can display a different or additional files by
	    setting the <varname>MESSAGE_SRC</varname> variable.  Its
	    default is <filename>MESSAGE</filename>, if the file
	    exists.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term><filename>ALTERNATIVES</filename></term>

	  <listitem>
	    <para>This file is used by the alternatives framework.
	    It creates, configures, and destroys generic wrappers used to
	    run programs with similar interfaces.
	    See pkg_alternatives(8) from pkgtools/pkg_alternatives
            for more information.</para>

	    <para>Each line of the file contains two filenames, first
	    the wrapper and then the alternative provided by the package.
	    Both paths are relative to <varname>PREFIX</varname>.</para>
	  </listitem>
	</varlistentry>

      </variablelist>

    </sect2>
    <sect2 id="components.optional.build">
      <title>Files affecting the build process</title>

      <variablelist>

	<varlistentry><term><filename>Makefile.common</filename></term>
	<listitem><para>This file contains arbitrary things that could
	also go into a <filename>Makefile</filename>, but its purpose is
	to be used by more than one package. This file should only be
	used when the packages that will use the file are known in
	advance. For other purposes it is often better to write a
	<filename>*.mk</filename> file and give it a good name that
	describes what it does.</para></listitem></varlistentry>

	<varlistentry><term><filename>buildlink3.mk</filename></term>
	<listitem><para>This file contains the dependency information
	for the buildlink3 framework (see <xref
	linkend="buildlink"/>).</para></listitem></varlistentry>

	<varlistentry><term><filename>hacks.mk</filename></term>
	<listitem><para>This file contains workarounds for compiler bugs
	and similar things. It is included automatically by the pkgsrc
	infrastructure, so you don't need an extra
	<literal>.include</literal> line for
	it.</para></listitem></varlistentry>

	<varlistentry><term><filename>options.mk</filename></term>
	<listitem><para>This file contains the code for the
	package-specific options (see <xref linkend="options"/>) that can be
	selected by the user. If a package has only one or two options,
	it is equally acceptable to put the code directly into the
	<filename>Makefile</filename>.</para></listitem></varlistentry>

      </variablelist>

    </sect2>
    <sect2 id="components.optional.none">
      <title>Files affecting nothing at all</title>

      <variablelist>

	<varlistentry><term><filename>README*</filename></term>
	<listitem><para>These files do not take place in the creation of
	a package and thus are purely informative to the package
	developer.</para></listitem></varlistentry>

	<varlistentry><term><filename>TODO</filename></term>
	<listitem><para>This file contains things that need to be done
	to make the package even
	better.</para></listitem></varlistentry>

      </variablelist>

    </sect2>
  </sect1>

  <sect1 id="work-dir">
    <title><filename>work*</filename></title>

    <para>When you type <command>make</command>, the distribution files are
    unpacked into the directory denoted by
    <varname>WRKDIR</varname>. It can be removed by running
    <command>make clean</command>. Besides the sources, this
    directory is also used to keep various timestamp files.
    The directory gets <emphasis>removed completely</emphasis> on clean.
    The default is <filename>${.CURDIR}/work</filename>
    or <filename>${.CURDIR}/work.${MACHINE_ARCH}</filename>
    if <varname>OBJMACHINE</varname> is set.</para>
  </sect1>

  <sect1 id="files-dir">
    <title><filename>files/*</filename></title>

    <para>If you have any files that you wish to be placed in the package
    prior to configuration or building, you can place these files here
    and use a <command>${CP}</command> command in the
    <quote>post-extract</quote> target to achieve this.</para>

    <para>If you want to share files in this way with other
    packages, set the <varname>FILESDIR</varname> variable to point
    to the other package's <filename>files</filename> directory,
    e.g.:</para>
<programlisting>
FILESDIR=       ../../editors/xemacs/files
</programlisting>
  </sect1>
</chapter>