regen.

1 files changed, 183 insertions, 57 deletions

diff --git a/doc/pkgsrc.txt b/doc/pkgsrc.txt
index aebb802e54e..9c56a1178f5 100644
--- a/doc/pkgsrc.txt
+++ b/doc/pkgsrc.txt
@@ -14,7 +14,7 @@ The pkgsrc Developers
 
 Copyright (C) 1994-2005 The NetBSD Foundation, Inc
 
-$NetBSD: pkgsrc.xml,v 1.14 2006/05/10 20:56:00 rillig Exp $
+$NetBSD: pkgsrc.xml,v 1.16 2006/05/12 23:03:22 rillig Exp $
 
 Abstract
 
@@ -291,21 +291,33 @@ II. The pkgsrc developer's guide
 
     19. Frequently Asked Questions
 
-III. The pkgsrc infrastructure
+III. The pkgsrc infrastructure internals
 
-    20. Regression tests
+    20. Design of the pkgsrc infrastructure
 
-        20.1. The regression tests framework
-        20.2. Running the regression tests
-        20.3. Adding a new regression test
+        20.1. Variable evaluation
 
-            20.3.1. Overridable functions
-            20.3.2. Helper functions
+            20.1.1. At load time
+            20.1.2. At runtime
 
-    21. Porting pkgsrc
+        20.2. Designing interfaces for Makefile fragments
 
-        21.1. Porting pkgsrc to a new operating system
-        21.2. Adding support for a new compiler
+            20.2.1. Procedures with parameters
+            20.2.2. Actions taken on behalf of parameters
+
+    21. Regression tests
+
+        21.1. The regression tests framework
+        21.2. Running the regression tests
+        21.3. Adding a new regression test
+
+            21.3.1. Overridable functions
+            21.3.2. Helper functions
+
+    22. Porting pkgsrc
+
+        22.1. Porting pkgsrc to a new operating system
+        22.2. Adding support for a new compiler
 
 A. A simple example package: bison
 
@@ -393,22 +405,16 @@ platforms:
 
 1.2. Overview
 
-This document is divided into two parts. The first, The pkgsrc user's guide,
+This document is divided into three parts. The first, The pkgsrc user's guide,
 describes how one can use one of the packages in the Package Collection, either
 by installing a precompiled binary package, or by building one's own copy using
 the NetBSD package system. The second part, The pkgsrc developer's guide,
 explains how to prepare a package so it can be easily built by other NetBSD
-users without knowing about the package's building details.
-
-This document is available in various formats:
-
-  * HTML
+users without knowing about the package's building details. The third part, The
+pkgsrc infrastructure internals is intended for those who want to understand
+how pkgsrc is implemented.
 
-  * PDF
-
-  * PS
-
-  * TXT
+This document is available in various formats: HTML, PDF, PS, TXT.
 
 1.3. Terminology
 
@@ -3919,21 +3925,21 @@ The following example shows how bsd.options.mk should be used by the
 hypothetical ``wibble'' package, either in the package Makefile, or in a file,
 e.g. options.mk, that is included by the main package Makefile.
 
-    PKG_OPTIONS_VAR=            PKG_OPTIONS.wibble
-    PKG_SUPPORTED_OPTIONS=              wibble-foo ldap
-    PKG_OPTIONS_OPTIONAL_GROUPS=        database
-    PKG_OPTIONS_GROUP.database= mysql pgsql
-    PKG_SUGGESTED_OPTIONS=              wibble-foo
-    PKG_OPTIONS_LEGACY_VARS+=   WIBBLE_USE_OPENLDAP:ldap
-    PKG_OPTIONS_LEGACY_OPTS+=   foo:wibble-foo
+    PKG_OPTIONS_VAR=                PKG_OPTIONS.wibble
+    PKG_SUPPORTED_OPTIONS=          wibble-foo ldap
+    PKG_OPTIONS_OPTIONAL_GROUPS=    database
+    PKG_OPTIONS_GROUP.database=     mysql pgsql
+    PKG_SUGGESTED_OPTIONS=          wibble-foo
+    PKG_OPTIONS_LEGACY_VARS+=       WIBBLE_USE_OPENLDAP:ldap
+    PKG_OPTIONS_LEGACY_OPTS+=       foo:wibble-foo
 
     .include "../../mk/bsd.prefs.mk"
 
     # this package was previously named wibble2
     .if defined(PKG_OPTIONS.wibble2)
-    PKG_LEGACY_OPTIONS+=  ${PKG_OPTIONS.wibble2}
+    PKG_LEGACY_OPTIONS+=            ${PKG_OPTIONS.wibble2}
     PKG_OPTIONS_DEPRECATED_WARNINGS+= \
-            "Deprecated variable PKG_OPTIONS.wibble2 used, use "${PKG_OPTIONS_VAR:Q}" instead."
+            "Deprecated variable PKG_OPTIONS.wibble2 used, use ${PKG_OPTIONS_VAR instead."
     .endif
 
     .include "../../mk/bsd.options.mk"
@@ -6052,34 +6058,154 @@ pkgsrc-users mailing list.
       "wrapper" phase and later. To "simulate" the wrapper phase, append
       PKG_PHASE=wrapper to the above command.
 
-Part III. The pkgsrc infrastructure
+Part III. The pkgsrc infrastructure internals
+
+This part of the guide deals with everything from the infrastructure that is
+behind the interfaces described in the developer's guide. A casual package
+maintainer should not need anything from this part.
 
 Table of Contents
 
-20. Regression tests
+20. Design of the pkgsrc infrastructure
+
+    20.1. Variable evaluation
+
+        20.1.1. At load time
+        20.1.2. At runtime
+
+    20.2. Designing interfaces for Makefile fragments
+
+        20.2.1. Procedures with parameters
+        20.2.2. Actions taken on behalf of parameters
+
+21. Regression tests
+
+    21.1. The regression tests framework
+    21.2. Running the regression tests
+    21.3. Adding a new regression test
+
+        21.3.1. Overridable functions
+        21.3.2. Helper functions
+
+22. Porting pkgsrc
+
+    22.1. Porting pkgsrc to a new operating system
+    22.2. Adding support for a new compiler
+
+Chapter 20. Design of the pkgsrc infrastructure
+
+Table of Contents
+
+20.1. Variable evaluation
+
+    20.1.1. At load time
+    20.1.2. At runtime
+
+20.2. Designing interfaces for Makefile fragments
+
+    20.2.1. Procedures with parameters
+    20.2.2. Actions taken on behalf of parameters
+
+The pkgsrc infrastructure consists of many small Makefile fragments. Each such
+fragment needs a properly specified interface. This chapter explains how such
+an interface looks like.
+
+20.1. Variable evaluation
+
+20.1.1. At load time
+
+Variable evaluation takes place either at load time or at runtime, depending on
+the context in which they occur. The contexts where variables are evaluated at
+load time are:
+
+  * The right hand side of the := and != operators,
+
+  * Make directives like .if or .for,
+
+  * Dependency lines.
+
+A special exception are references to the iteration variables of .for loops,
+which are expanded inline, no matter in which context they appear.
+
+As the values of variables may change during load time, care must be taken not
+to evaluate them by accident. Typical examples for variables that should not be
+evaluated at load time are DEPENDS and CONFIGURE_ARGS. To make the effect more
+clear, here is an example:
+
+    CONFIGURE_ARGS=         # none
+    CFLAGS=                 -O
+    CONFIGURE_ARGS+=        CFLAGS=${CFLAGS:Q}
+
+    CONFIGURE_ARGS:=        ${CONFIGURE_ARGS}
+
+    CFLAGS+=                -Wall
+
+
+This code shows how the use of the := operator can quickly lead to unexpected
+results. The first paragraph is fairly common code. The second paragraph
+evaluates the CONFIGURE_ARGS variable, which results in CFLAGS=-O. In the third
+paragraph, the -Wall is appended to the CFLAGS, but this addition will not
+appear in CONFIGURE_ARGS. In actual code, the three paragraphs from above
+typically occur in completely unrelated files.
+
+20.1.2. At runtime
+
+After all the files have been loaded, the values of the variables cannot be
+changed anymore. Variables that are used in the shell commands are expanded at
+this point.
+
+20.2. Designing interfaces for Makefile fragments
+
+Most of the .mk files fall into one of the following classes. Cases where a
+file falls into more than one class should be avoided as it often leads to
+subtle bugs.
+
+20.2.1. Procedures with parameters
+
+In a traditional imperative programming language some of the .mk files could be
+described as procedures. They take some input parameters and?after inclusion?
+provide a result in output parameters. Since all variables in Makefiles have
+global scope care must be taken not to use parameter names that have already
+another meaning. For example, PKGNAME is a bad choice for a parameter name.
+
+Procedures are completely evaluated at preprocessing time. That is, when
+calling a procedure all input parameters must be completely resolvable. For
+example, CONFIGURE_ARGS should never be an input parameter since it is very
+likely that further text will be added after calling the procedure, which would
+effectively apply the procedure to only a part of the variable. Also,
+references to other variables wit will be modified after calling the procedure.
+
+A procedure can declare its output parameters either as suitable for use in
+preprocessing directives or as only available at runtime. The latter
+alternative is for variables that contain references to other runtime
+variables.
+
+Procedures shall be written such that it is possible to call the procedure more
+than once. That is, the file must not contain multiple-inclusion guards.
 
-    20.1. The regression tests framework
-    20.2. Running the regression tests
-    20.3. Adding a new regression test
+Examples for procedures are mk/bsd.options.mk and mk/buildlink3/bsd.builtin.mk.
+To express that the parameters are evaluated at load time, they should be
+assigned using the := operator, which should be used only for this purpose.
 
-        20.3.1. Overridable functions
-        20.3.2. Helper functions
+20.2.2. Actions taken on behalf of parameters
 
-21. Porting pkgsrc
+Action files take some input parameters and may define runtime variables. They
+shall not define loadtime variables. There are action files that are included
+implicitly by the pkgsrc infrastructure, while other must be included
+explicitly.
 
-    21.1. Porting pkgsrc to a new operating system
-    21.2. Adding support for a new compiler
+An example for action files is mk/subst.mk.
 
-Chapter 20. Regression tests
+Chapter 21. Regression tests
 
 Table of Contents
 
-20.1. The regression tests framework
-20.2. Running the regression tests
-20.3. Adding a new regression test
+21.1. The regression tests framework
+21.2. Running the regression tests
+21.3. Adding a new regression test
 
-    20.3.1. Overridable functions
-    20.3.2. Helper functions
+    21.3.1. Overridable functions
+    21.3.2. Helper functions
 
 The pkgsrc infrastructure consists of a large codebase, and there are many
 corners where every little bit of a file is well thought out, making pkgsrc
@@ -6088,22 +6214,22 @@ changes from breaking anything, a suite of regression tests should go along
 with every important part of the pkgsrc infrastructure. This chapter describes
 how regression tests work in pkgsrc and how you can add new tests.
 
-20.1. The regression tests framework
+21.1. The regression tests framework
 
-20.2. Running the regression tests
+21.2. Running the regression tests
 
 You first need to install the pkgtools/pkg_regress package, which provides the
 pkg_regress command. Then you can simply run that command, which will run all
 tests in the regress category.
 
-20.3. Adding a new regression test
+21.3. Adding a new regression test
 
 Every directory in the regress category that contains a file called spec is
 considered a regression test. This file is a shell program that is included by
 the pkg_regress command. The following functions can be overridden to suit your
 needs.
 
-20.3.1. Overridable functions
+21.3.1. Overridable functions
 
 These functions do not take any parameters. They are all called in "set -e"
 mode, so you should be careful to check the exitcodes of any commands you run
@@ -6131,7 +6257,7 @@ do_cleanup()
     This function cleans everything up after the test has been run. By default
     it does nothing.
 
-20.3.2. Helper functions
+21.3.2. Helper functions
 
 exit_status(expected)
 
@@ -6150,18 +6276,18 @@ output_prohibit(regex...)
     () does not match the extended regular expression. If any of the regular
     expressions matches, the test will fail.
 
-Chapter 21. Porting pkgsrc
+Chapter 22. Porting pkgsrc
 
 Table of Contents
 
-21.1. Porting pkgsrc to a new operating system
-21.2. Adding support for a new compiler
+22.1. Porting pkgsrc to a new operating system
+22.2. Adding support for a new compiler
 
 The pkgsrc system has already been ported to many operating systems, hardware
 architectures and compilers. This chapter explains the necessary steps to make
 pkgsrc even more portable.
 
-21.1. Porting pkgsrc to a new operating system
+22.1. Porting pkgsrc to a new operating system
 
 To port pkgsrc to a new operating system (called MyOS in this example), you
 need to touch the following files:
@@ -6209,7 +6335,7 @@ mk/tools/MyOS.mk
 Now, you should be able to build some basic packages, like lang/perl5, shells/
 bash.
 
-21.2. Adding support for a new compiler
+22.2. Adding support for a new compiler
 
 TODO