path: root/doc/guide/files/using.xml

<!-- $NetBSD: using.xml,v 1.51 2020/05/22 20:57:15 rillig Exp $ -->

<chapter id="using"> <?dbhtml filename="using.html"?>
<title>Using pkgsrc</title>

<para>Basically, there are two ways of using pkgsrc. The first
is to only install the package tools and to use binary packages
that someone else has prepared. This is the <quote>pkg</quote>
in pkgsrc. The second way is to install the <quote>src</quote>
of pkgsrc, too. Then you are able to build your own packages,
and you can still use binary packages from someone else.</para>

<sect1 id="using-pkg">
  <title>Using binary packages</title>

  <!-- this URL needs to be kept at http, not https, since pkg_add cannot use https. -->
  <para>On the <ulink url="http://cdn.NetBSD.org/">cdn.NetBSD.org</ulink>
  site and mirrors, there are collections of binary packages,
  ready to be installed. These binary packages have been built using the
  default settings for the directories, that is:</para>

  <itemizedlist>
  <listitem><para><filename>/usr/pkg</filename> for <varname>LOCALBASE</varname>, where most of the files are installed,</para></listitem>
  <listitem><para><filename>/usr/pkg/etc</filename> for configuration files,</para></listitem>
  <listitem><para><filename>/var</filename> for <varname>VARBASE</varname>, where those files are installed that may change after installation.</para></listitem>
  </itemizedlist>

  <para>If you cannot use these directories for whatever reasons (maybe
  because you're not root), you cannot use these binary packages, but
  have to build the packages yourself, which is explained in <xref
  linkend="bootstrapping-pkgsrc" />.</para>

  <sect2 id="finding-binary-packages">
    <title>Finding binary packages</title>

    <para>To install binary packages, you first need to know from where
    to get them. The first place where you should look is on the main
    <!-- this URL needs to be kept at http, not https, since pkg_add cannot use https. -->
    pkgsrc FTP server in the directory <ulink
    url="http://cdn.NetBSD.org/pub/pkgsrc/packages/"><filename>/pub/pkgsrc/packages</filename></ulink>.</para>

    <para>This directory contains binary packages for multiple
    platforms. First, select your operating system. (Ignore the
    directories with version numbers attached to it, they just exist for
    legacy reasons.) Then, select your hardware architecture, and in the
    third step, the OS version and the <quote>version</quote> of pkgsrc.</para>

    <para>In this directory, you often find a file called
    <filename>bootstrap.tar.gz</filename> which contains the package
    management tools. If the file is missing, it is likely that your
    operating system already provides those tools. Download the file and
    extract it in the <filename>/</filename> directory. It will create
    the directories <filename>/usr/pkg</filename> (containing the tools
    for managing binary packages and the database of installed packages).</para>
  </sect2>

  <sect2 id="installing-binary-packages">
    <title>Installing binary packages</title>

    <para>In the directory from the last section, there is a
    subdirectory called <filename>All/</filename>, which contains all the
    binary packages that are available for the platform, excluding those
    that may not be distributed via FTP or CDROM (depending on which
    medium you are using).</para>

    <para>To install packages directly from an FTP or HTTP server, run
    the following commands in a Bourne-compatible shell (be sure to
    <command>su</command> to root first):</para>

<screen>
&rprompt; <userinput>PATH="/usr/pkg/sbin:$PATH"</userinput>
<!-- this URL needs to be kept at http, not https, since pkg_add cannot use https. -->
&rprompt; <userinput>PKG_PATH="http://cdn.NetBSD.org/pub/pkgsrc/packages"</userinput>
&rprompt; <userinput>PKG_PATH="$PKG_PATH/<replaceable>OPSYS</replaceable>/<replaceable>ARCH</replaceable>/<replaceable>VERSIONS</replaceable>/All/"</userinput>
&rprompt; <userinput>export PATH PKG_PATH</userinput>
</screen>

    <para>Instead of URLs, you can also use local paths, for example if
    you are installing from a set of CDROMs, DVDs or an NFS-mounted
    repository. If you want to install packages from multiple sources,
    you can separate them by a semicolon in
    <varname>PKG_PATH</varname>.</para>

    <para>After these preparations, installing a package is very
    easy:</para>

<screen>
&rprompt; <userinput>pkg_add libreoffice</userinput>
&rprompt; <userinput>pkg_add ap24-php71-*</userinput>
</screen>

    <para>Note that any prerequisite packages needed to run the
    package in question will be installed, too, assuming they are
    present where you install from.</para>

    <para>Adding packages might install vulnerable packages.
    Thus you should run <command>pkg_admin audit</command>
    regularly, especially after installing new packages, and verify
    that the vulnerabilities are acceptable for your configuration.</para>

    <para>After you've installed packages, be sure to have
    <filename>/usr/pkg/bin</filename> and <filename>/usr/pkg/sbin</filename> in your
    <varname>PATH</varname> so you can actually start the just
    installed program.</para>
  </sect2>

  <sect2 id="using.pkg_delete">
    <title>Deinstalling packages</title>

    <para>To deinstall a package, it does not matter whether it was
    installed from source code or from a binary package. The
    <command>pkg_delete</command> command does not know it anyway.
    To delete a package, you can just run <command>pkg_delete
    <replaceable>package-name</replaceable></command>. The package
    name can be given with or without version number. Wildcards can
    also be used to deinstall a set of packages, for example
    <literal>*emacs*</literal>. Be sure to include them in quotes,
    so that the shell does not expand them before
    <literal>pkg_delete</literal> sees them.</para>

    <para>The <option>-r</option> option is very powerful: it
    removes all the packages that require the package in question
    and then removes the package itself. For example:

    <screen>
&rprompt; <userinput>pkg_delete -r jpeg</userinput>
    </screen>

    will remove jpeg and all the packages that used it; this allows
    upgrading the jpeg package.</para>

  </sect2>

  <sect2 id="using.pkg_info">
    <title>Getting information about installed packages</title>

    <para>The <command>pkg_info</command> shows information about
    installed packages or binary package files.</para>

  </sect2>

  <sect2 id="vulnerabilities">
    <title>Checking for security vulnerabilities in installed packages</title>

    <para>
      The pkgsrc Security Team and Packages Groups maintain a list of
      known security vulnerabilities to packages which are (or have been)
      included in pkgsrc.  The list is available from the NetBSD
      <!-- this URL needs to be kept at http, not https, since pkg_add cannot use https. -->
      FTP site at <ulink url="http://ftp.NetBSD.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities"/>.
    </para>

    <para>
      Through <command>pkg_admin fetch-pkg-vulnerabilities</command>,
      this list can be downloaded
      automatically, and a security audit of all packages installed on a system
      can take place.
    </para>

    <para>
      There are two components to auditing.  The first
      step, <command>pkg_admin fetch-pkg-vulnerabilities</command>,
      is for downloading
      the list of vulnerabilities from the NetBSD FTP site.  The second
      step, <command>pkg_admin audit</command>, checks to see if any of your
      installed packages are vulnerable.  If a package is vulnerable, you
      will see output similar to the following:
    </para>

    <screen>Package samba-2.0.9 has a local-root-shell vulnerability, see
    https://www.samba.org/samba/whatsnew/macroexploit.html</screen>

    <para>
      You may wish to have the
      <!-- this URL needs to be kept at http, not https, since pkg_add cannot use https. -->
      <ulink url="http://ftp.NetBSD.org/pub/pkgsrc/distfiles/vulnerabilities">vulnerabilities</ulink>
      file downloaded daily so that
      it remains current.  This may be done by adding an appropriate entry
      to the root users &man.crontab.5; entry.  For example the entry
      <screen>
# Download vulnerabilities file
0 3 * * * /usr/pkg/sbin/pkg_admin fetch-pkg-vulnerabilities >/dev/null 2>&1
# Audit the installed packages and email results to root
9 3 * * * /usr/pkg/sbin/pkg_admin audit |mail -s "Installed package audit result" \
	    root >/dev/null 2>&1
      </screen>
      will update the vulnerability list every day at 3AM, followed by an audit
      at 3:09AM. The result of the audit are then emailed to root.

      On NetBSD this may be accomplished instead by adding the following
      line to <filename>/etc/daily.conf</filename>:
      <screen>
fetch_pkg_vulnerabilities=YES
      </screen>
      to fetch the vulnerability list from the daily security script. The system
      is set to audit the packages by default but can be set explicitly, if
      desired (not required), by adding the following line to
<filename>/etc/security.conf</filename>:
      <screen>
check_pkg_vulnerabilities=YES
      </screen>
      see &man.daily.conf.5; and &man.security.conf.5; for more details.
    </para>
  </sect2>

  <sect2 id="pkg_versions">
    <title>Finding if newer versions of your installed packages are in pkgsrc</title>
    <para>
      Install <filename role="pkg">pkgtools/lintpkgsrc</filename> and run
      <command>lintpkgsrc</command> with the <quote>-i</quote>
      argument to check if your packages are up-to-date, e.g.
    </para>
    <screen>
&cprompt; <userinput>lintpkgsrc -i</userinput>
...
Version mismatch: 'tcsh' 6.09.00 vs 6.10.00
    </screen>
    <para>You can then use <command>make update</command> to update the
    package on your system and rebuild any dependencies.
    </para>
  </sect2>

  <sect2 id="using.pkg_admin">
    <title>Other administrative functions</title>

    <para>The <command>pkg_admin</command> executes various
    administrative functions on the package system.</para>

  </sect2>
</sect1>

<sect1 id="building-packages-from-source">
  <title>Building packages from source</title>

  <para>After obtaining pkgsrc, the <filename>pkgsrc</filename>
  directory now contains a set of packages, organized into
  categories. You can browse the online index of packages, or run
  <command>make readme</command> from the <filename>pkgsrc</filename>
  directory to build local <filename>README.html</filename> files for
  all packages, viewable with any web browser such as <filename
  role="pkg">www/lynx</filename> or <filename
  role="pkg">www/firefox</filename>.</para>

  <para>The default <emphasis>prefix</emphasis> for installed packages
  is <filename>/usr/pkg</filename>. If you wish to change this, you
  should do so by setting <varname>LOCALBASE</varname> in
  &mk.conf;. You should not try to use multiple
  different <varname>LOCALBASE</varname> definitions on the same
  system (inside a chroot is an exception). </para>

  <para>The rest of this chapter assumes that the package is already
  in pkgsrc. If it is not, see <xref linkend="developers-guide"/> for
  instructions how to create your own packages.</para>

  <sect2 id="requirements">
    <title>Requirements</title>

    <para>To build packages from source, you need a working C
    compiler. On NetBSD, you need to install the
    <quote>comp</quote> and the <quote>text</quote> distribution
    sets. If you want to build X11-related packages, the
    <quote>xbase</quote> and <quote>xcomp</quote> distribution
    sets are required, too.</para>
    <!-- FIXME: what about installing x11/XFree86-*? -->
  </sect2>


  <sect2 id="fetching-distfiles">
    <title>Fetching distfiles</title>

    <para>The first step for building a package is downloading the
    distfiles (i.e. the unmodified source). If they have not yet been
    downloaded, pkgsrc will fetch them automatically.</para>

    <para>If you have all files that you need in the
    <filename>distfiles</filename> directory,
    you don't need to connect. If the distfiles are on CD-ROM, you can
    mount the CD-ROM on <filename>/cdrom</filename> and add:
    <screen>DISTDIR=/cdrom/pkgsrc/distfiles</screen>
    to your &mk.conf;.</para>

    <para>By default a list of distribution sites will be randomly
    intermixed to prevent huge load on servers which holding popular
    packages (for example, SourceForge.net mirrors). Thus, every
    time when you need to fetch yet another distfile all the mirrors
    will be tried in new (random) order. You can turn this feature
    off by setting <varname>MASTER_SORT_RANDOM=NO</varname> (for
    <varname>PKG_DEVELOPER</varname>s it's already disabled).</para>

    <para>You can overwrite some of the major distribution sites to
    fit to sites that are close to your own.  By setting one or two
    variables you can modify the order in which the master sites are
    accessed.  <varname>MASTER_SORT</varname> contains a whitespace
    delimited list of domain suffixes.
    <varname>MASTER_SORT_REGEX</varname> is even more flexible, it
    contains a whitespace delimited list of regular expressions.  It
    has higher priority than <varname>MASTER_SORT</varname>.  Have a
    look at <filename>pkgsrc/mk/defaults/mk.conf</filename> to find
    some examples.  This may save some of your bandwidth and
    time.</para>

    <para>You can change these settings either in your shell's environment, or,
    if you want to keep the settings, by editing the
    &mk.conf; file,
    and adding the definitions there.</para>

    <para>
      If a package depends on many other packages (such as
      <filename role="pkg">meta-pkgs/kde4</filename>), the build process may
      alternate between periods of
      downloading source, and compiling. To ensure you have all the source
      downloaded initially you can run the command:
      <screen>&cprompt; <userinput>make fetch-list | sh</userinput></screen>
      which will output and run a set of shell commands to fetch the
      necessary files into the <filename>distfiles</filename> directory.  You can
      also choose to download the files manually.
    </para>

  </sect2>

  <sect2 id="how-to-build-and-install">
    <title>How to build and install</title>

    <para>
      Once the software has downloaded, any patches will be applied, then it
      will be compiled for you. This may take some time depending on your
      computer, and how many other packages the software depends on and their
      compile time.
    </para>

    <note><para>If using bootstrap or pkgsrc on a non-NetBSD system,
    use the pkgsrc <command>bmake</command> command instead of
    <quote>make</quote> in the examples in this guide.</para></note>

    <para>For example, type</para>

    <screen>
&cprompt; <userinput>cd misc/figlet</userinput>
&cprompt; <userinput>make</userinput>
    </screen>

    <para>at the shell prompt to build the various components of the
    package.</para>

    <para>The next stage is to actually install the newly compiled
    program onto your system. Do this by entering:

    <screen>
&cprompt; <userinput>make install</userinput>
    </screen>

    while you are still in the directory for whatever package you
    are installing.</para>

    <para>Installing the package on your system may require you to
    be root.  However, pkgsrc has a
    <emphasis>just-in-time-su</emphasis> feature, which allows you
    to only become root for the actual installation step.</para>

    <para>That's it, the software should now be installed and setup for use.
    You can now enter:

    <screen>
&cprompt; <userinput>make clean</userinput>
    </screen>

    to remove the compiled files in the work directory, as you shouldn't need
    them any more. If other packages were also added to your system
    (dependencies) to allow your program to compile, you can tidy these up
    also with the command:</para>

    <screen>
&cprompt; <userinput>make clean-depends</userinput>
    </screen>

    <para>Taking the figlet utility as an example, we can install it on our
    system by building as shown in <xref linkend="logs"/>.</para>

    <para>The program is installed under the default root of the
    packages tree - <filename>/usr/pkg</filename>. Should this not
    conform to your tastes, set the <varname>LOCALBASE</varname>
    variable in your environment, and it will use that value as the
    root of your packages tree. So, to use
    <filename>/usr/local</filename>, set
    <varname>LOCALBASE=/usr/local</varname> in your environment.
    Please note that you should use a directory which is dedicated to
    packages and not shared with other programs (i.e., do not try and
    use <varname>LOCALBASE=/usr</varname>).  Also, you should not try
    to add any of your own files or directories (such as
    <filename>src/</filename>, <filename>obj/</filename>, or
    <filename>pkgsrc/</filename>) below the
    <varname>LOCALBASE</varname> tree.  This is to prevent possible
    conflicts between programs and other files installed by the
    package system and whatever else may have been installed
    there.</para>

    <para>Some packages look in &mk.conf; to
    alter some configuration options at build time.  Have a look at
    <filename>pkgsrc/mk/defaults/mk.conf</filename> to get an overview
    of what will be set there by default.  Environment variables such
    as <varname>LOCALBASE</varname> can be set in
    &mk.conf; to save having to remember to
    set them each time you want to use pkgsrc.</para>

    <para>Occasionally, people want to <quote>look under the
    covers</quote> to see what is going on when a package is building
    or being installed. This may be for debugging purposes, or out of
    simple curiosity. A number of utility values have been added to
    help with this.</para>

    <orderedlist>
      <listitem>
	<para>If you invoke the &man.make.1; command with
	<varname>PKG_DEBUG_LEVEL=2</varname>, then a huge amount of
	information will be displayed. For example,</para>

	<screen><userinput>make patch PKG_DEBUG_LEVEL=2</userinput></screen>

	<para>will show all the commands that are invoked, up to and
	including the <quote>patch</quote> stage.</para>
      </listitem>

      <listitem>
	<para>If you want to know the value of a certain &man.make.1;
	definition, then the <varname>VARNAME</varname> definition
	should be used, in conjunction with the show-var
	target. e.g. to show the expansion of the &man.make.1;
	variable <varname>LOCALBASE</varname>:</para>

	<screen>
&cprompt; <userinput>make show-var VARNAME=LOCALBASE</userinput>
/usr/pkg
&cprompt;
	</screen>

      </listitem>
    </orderedlist>

    <para>If you want to install a binary package that you've either
    created yourself (see next section), that you put into
    pkgsrc/packages manually or that is located on a remote FTP
    server, you can use the "bin-install" target. This target will
    install a binary package - if available - via &man.pkg.add.1;,
    else do a <command>make package</command>.  The list of remote FTP
    sites searched is kept in the variable
    <varname>BINPKG_SITES</varname>, which defaults to
    ftp.NetBSD.org. Any flags that should be added to &man.pkg.add.1;
    can be put into <varname>BIN_INSTALL_FLAGS</varname>.  See
    <filename>pkgsrc/mk/defaults/mk.conf</filename> for more
    details.</para>

    <para>A final word of warning: If you set up a system that has a
    non-standard setting for <varname>LOCALBASE</varname>, be sure to
    set that before any packages are installed, as you cannot use
    several directories for the same purpose. Doing so will result in
    pkgsrc not being able to properly detect your installed packages,
    and fail miserably. Note also that precompiled binary packages are
    usually built with the default <varname>LOCALBASE</varname> of
    <filename>/usr/pkg</filename>, and that you should
    <emphasis>not</emphasis> install any if you use a non-standard
    <varname>LOCALBASE</varname>.</para>
  </sect2>
</sect1>
</chapter>