diff options
Diffstat (limited to 'mk')
| -rw-r--r-- | mk/buildlink3/BUILDLINK3_DG | 231 | ||||
| -rw-r--r-- | mk/buildlink3/PKGVIEWS_UG | 222 |
2 files changed, 453 insertions, 0 deletions
diff --git a/mk/buildlink3/BUILDLINK3_DG b/mk/buildlink3/BUILDLINK3_DG new file mode 100644 index 00000000000..a878984c48e --- /dev/null +++ b/mk/buildlink3/BUILDLINK3_DG @@ -0,0 +1,231 @@ +$NetBSD: BUILDLINK3_DG,v 1.1 2004/01/29 06:03:15 jlam Exp $ + + 0 Developer's guide to buildlink3 + ================================= + +This is a tutorial for pkgsrc developers to understand and to use the +buildlink3 framework in pkgsrc. + + + 1 Changes between buildlink2 and buildlink3 + =========================================== + +The buildlink3 framework is a evolutionary descendant of the +buildlink2 framework that does a better job of adhering to the +fundamental buildlink principle: only allow the software build +process to see what we choose to allow it to see. + + + 1.1 Better behavior with libtool + ================================ + +One of the biggest problems in buildlink2 is handling packages that +install libtool archive files for libraries that are also present in +the base system. buildlink3 is significantly better at this as it +more tightly controls where libtool can find libtool archives. One +side effect of this is that we no longer need to create fake libtool +archives to work around cases where the pkgsrc libraries were being +used instead of the system libraries if they shared the same name. + + + 1.3 Support for native compilers + ================================ + +The buildlink3 wrapper scripts have better support for using SunPro +and MIPSpro compilers to build pkgsrc software. For the most part, +packages can use any compiler, but some third-party software is +written assuming that it will be compiled using GCC. The buildlink3 +wrapper scripts can capture some common GCC options and convert them +into native toolchain equivalents. + + + 1.4 New buildlink3.mk file structure + ==================================== + +buildlink3.mk files have two major differences over buildlink2.mk +files. The first, most noticeable difference is that buildlink3.mk +generally don't have contain a BUILDLINK_FILES definition. This is +because buildlink3 automatically determines which files to symlink +into ${BUILDLINK_DIR} by examining the PLIST of the installed package. +The second difference is that buildlink3.mk files keep track of how +"deep" we are in including buildlink3.mk files, and only creates +dependencies on packages encountered at depth 1. This means that +packages that want to add a dependency must directly include the +buildlink3.mk file for that dependency. + + + 1.5 Support for pkgviews + ======================== + +When building pkgviews packages, buildlink3 doesn't symlink files +into ${BUILDLINK_DIR} since it can safely refer to only a specific +package's files by passing the appropriate -I<dir> and -L<dir> flags +to the compiler, where <dir> points to a location in the package's +depot directory. When building "overwrite" packages, buildlink3 will +act and feel very much like buildlink2 but with more advanced wrapper +scripts, and there are provisions for allowing an "overwrite" package +to build against the viewed instance of a depoted package. + + + 2 Writing buildlink3.mk files + ============================= + +A package's buildlink3.mk file is included by Makefiles to indicate +the need to compile and link against header files and libraries +provided by the package. A buildlink3.mk file should always provide +enough information to add the correct type of dependency relationship +and include any other buildlink3.mk files that it needs to find +headers and libraries that it needs in turn. + + + 2.1 Simple packages + =================== + +To generate an initial buildlink3.mk file for further editting, Rene +Hexel's pkgtools/createbuildlink package is highly recommended. For +most packages, the following command will generate a good starting +point for buildlink3.mk files: + + cd pkgsrc/category/pkgdir; createbuildlink -3 > buildlink3.mk + +The following real-life example buildlink3.mk is taken from +graphics/tiff: + +------------8<------------8<------------8<------------8<------------ +# $NetBSD: BUILDLINK3_DG,v 1.1 2004/01/29 06:03:15 jlam Exp $ + +BUILDLINK_DEPTH:= ${BUILDLINK_DEPTH}+ +TIFF_BUILDLINK3_MK:= ${TIFF_BUILDLINK3_MK}+ + +.if !empty(BUILDLINK_DEPTH:M+) +BUILDLINK_DEPENDS+= tiff +.endif + +.if !empty(TIFF_BUILDLINK3_MK:M+) +BUILDLINK_PACKAGES+= tiff +BUILDLINK_DEPENDS.tiff+= tiff>=3.5.4 +BUILDLINK_PKGSRCDIR.tiff?= ../../graphics/tiff + +. include "../../devel/zlib/buildlink3.mk" +. include "../../graphics/jpeg/buildlink3.mk" +.endif # TIFF_BUILDLINK3_MK + +BUILDLINK_DEPTH:= ${BUILDLINK_DEPTH:S/+$//} +------------8<------------8<------------8<------------8<------------ + +The header and footer manipulate BUILDLINK_DEPTH, which is common +across all buildlink3.mk files and is used to track at what depth we +are in including buildlink3.mk files. + +The first section controls if the dependency on tiff is added. +BUILDLINK_DEPENDS is the global list of packages for which +dependencies are added by buildlink3. The tiff package is only +appended to this list if the buildlink3.mk is included directly by a +package Makefile. + +The second section is protected from multiple inclusion and control +how the dependency on tiff is added. Several important variables +are set in the section: + + (1) BUILDLINK_PACKAGES is the global list of packages for which + buildlink3.mk files have been included. It must _always_ be + appended to within a buildlink3.mk; + + (2) BUILDLINK_DEPENDS.tiff is the actual dependency recorded in the + installed package; this should always be set using += to ensure + that we're appending to any pre-existing list of values. + + (3) BUILDLINK_PKGSRCDIR.tiff is the location of the tiff pkgsrc + directory; + + (4) BUILDLINK_DEPMETHOD.tiff (not shown above) controls whether we + use BUILD_DEPENDS or DEPENDS to add the dependency on tiff. The + build dependency is selected by setting BUILDLINK_DEPMETHOD.tiff + to "build". By default, the full dependency is used. + + (5) BUILDLINK_INCDIRS.tiff and BUILDLINK_LIBDIRS.tiff (not shown + above) are lists of subdirectories of ${BUILDLINK_PREFIX.tiff} to + add the header and library search paths. These default to + "include" and "lib" respectively. + + (6) BUILDLINK_CPPFLAGS.tiff is the list of preprocessor flags to add + CPPFLAGS, which are passed on to the configure and build phases. + -I should be avoided and instead be added using + BUILDLINK_INCDIRS.tiff as above. + +Any buildlink3.mk for tiff's dependencies are also included at this +point. Including these buildlink3.mk files means that the headers +and libraries for these dependencies are also symlinked into +${BUILDLINK_DIR} whenever the tiff buildlink3.mk file is included. + +There are several considerations that arise when figuring out how +to set BUILDLINK_DEPENDS.<pkg> correctly: + + (1) If the package has a pre-existing buildlink2.mk file, then + match the BUILDLINK_DEPENDS.<pkg> lines between the + buildlink2.mk file and the newly-created buildlink3.mk file. + + (2) If there is no pre-existing buildlink2.mk file, then set + BUILDLINK_DEPENDS.foo to the first version of the package that + had the last change in the major number of a shared library or + that had a major API change. + + + 2.1 Packages that coincide with base system software + ==================================================== + +Some packages in pkgsrc install headers and libraries that coincide +with headers and libraries present in the base system. The best +recommendation for writing buildlink3.mk files for these packages is +to use graphics/MesaLib/buildlink3.mk as a template. + + + 3 bl3ifying a package + ===================== + +The process of "bl3ifying" a package, or converting a package to use +the buildlink3 framework, is surprisingly easy. The things to keep in +mind are: + + (1) Set USE_BUILDLINK3=yes. + + (2) Change references to buildlink2.mk files into buildlink3.mk + files. + + (3) Ensure that the build always calls the wrapper scripts instead of + the actual toolchain. Some are tricky, e.g. openssl, cdrecord, + ocaml, and the only way to know for sure is the check .work.log + to see if the wrappers are being invoked. + + (4) Don't override PREFIX from within the package Makefile, e.g. + Java VMs, standalone shells, etc., because the code to symlink + files into ${BUILDLINK_DIR} looks for files relative to + "pkg_info -qp <pkgname>". + + + 4 Troubleshooting + ================= + +Q1: I'm trying to bl3ify a package but I get an error that looks like: + + make: don't know how to make _BUILDLINK_USE. Stop + +A1: You forgot to change a reference to a buildlink2.mk file into a + buildlink3.mk file. + + +Q2: Dependencies are added for every single buildlink3.mk file I + include, including for when it's supposed to use the base system + software. What's going on? + +A2: You forgot to change USE_BUILDLINK2 to USE_BUILDLINK3 in the + package Makefile. + + +Q3: Where can I see the actual command executed by the wrapper + scripts? + +A3: You should examine the contents of the ${WRKDIR}/.work.log file. + The lines preceded with [*] are the commands that are intercepted + by the wrapper scripts, and the lines preceded with <.> are the + commands that are executed by the wrapper scripts. diff --git a/mk/buildlink3/PKGVIEWS_UG b/mk/buildlink3/PKGVIEWS_UG new file mode 100644 index 00000000000..6e586e3d898 --- /dev/null +++ b/mk/buildlink3/PKGVIEWS_UG @@ -0,0 +1,222 @@ +$NetBSD: PKGVIEWS_UG,v 1.1 2004/01/29 06:03:15 jlam Exp $ + + 0 User's Guide to pkgviews + ========================== + +This is a tutorial for pkgsrc users who wish to experiment with the +new "pkgviews" implementation in pkgsrc. More information about +pkgviews may be found in pkgsrc/mk/buildlink3/README. That document +also explains why you might want to use pkgviews. Some reasons +include: + + * fully dynamic PLISTs + * multiple version of the same package can co-exist + * no or non-fatal conflicting packages + + + 0.1 CAVEAT (USE AT YOUR OWN RISK!) + ================================== + +Pkgviews is *completely experimental* at this point in time. Bug +reports on pkgviews will be treated with a fairly low priority by the +general pkgsrc developers, though I may personally be more responsive. +However, the major thrust of the next several weeks of pkgsrc +development for me will revolve around testing and integrating the +buildlink3 framework into pkgsrc and deprecating buildlink2. Any +work on pkgviews during that time is strictly happenstance. + + + 0.1 Preparing your system to use pkgviews + ========================================= + +You will need to start with a clean system to use pkgviews. Depoted +packages, a.k.a "pkgviews" packages (packages that are built using +pkgviews) cannot depend on non-depoted packages, a.k.a. "overwrite" +packages, although the reverse is possible. If you have _any_ +packages installed, you will need to pkg_delete them before you can +start building depoted packages. In fact, it's best to completely +nuke /usr/pkg (or wherever you choose for your ${LOCALBASE}) as +pkgviews manages all of its own directories. + +Next you will need to add the following line to /etc/mk.conf: + + PKG_INSTALLATION_PREFS= pkgviews overwrite + +This creates pkgviews packages instead of overwrite packages for any +packages that support it. The packages that do support pkgviews +can be identified by searching the package Makefile for the the +following line: + + PKG_INSTALLATION_TYPES= overwrite pkgviews + + + 0.2 Installing your first pkgviews package + ========================================== + +The first package you will need to install is pkgsrc/pkgtools/digest. + +===> Checking for vulnerabilities in digest-20021220 +===> Extracting for digest-20021220 +===> Checking for vulnerabilities in digest-20021220 +===> Patching for digest-20021220 +===> Overriding tools for digest-20021220 +===> Buildlinking for digest-20021220 +===> Configuring for digest-20021220 +configure: WARNING: If you wanted to set the --build type, don't use --host. + If a cross compiler is detected then cross compile mode will be used. +checking build system type... i386-unknown-netbsdelf1.6.2. +checking host system type... i386--netbsdelf +... +configure: creating ./config.status +config.status: creating Makefile +config.status: creating config.h +===> Building for digest-20021220 +cc -DHAVE_CONFIG_H -I. -I. -DHOST=\"i386--netbsdelf\" -DVERSION=\"20021220\" -O2 -mcpu=pentiumpro -c bits.c -o bits.o +... +cc -O2 -mcpu=pentiumpro -o digest digest.o md5c.o rmd160.o rmd160hl.o sha2.o sha2hl.o md5hl.o sha1.o sha1hl.o +=> Fixing buildlink references in files-to-be-installed. +===> Installing for digest-20021220 +===> Becoming root@blossom.hq.williamlam.com to install digest. +sudo /bin/sh ./mkinstalldirs /usr/pkg/packages/digest-20021220/bin +mkdir /usr/pkg/packages/digest-20021220/bin +install -c -s -o root -g wheel -m 555 digest /usr/pkg/packages/digest-20021220/bin/digest +/bin/sh ./mkinstalldirs /usr/pkg/packages/digest-20021220/man/man1 +mkdir /usr/pkg/packages/digest-20021220/man +mkdir /usr/pkg/packages/digest-20021220/man/man1 +install -c -o root -g wheel -m 444 digest.1 /usr/pkg/packages/digest-20021220/man/man1/digest.1 +===> Registering installation for digest-20021220 +===> Building views for digest-20021220 +=> Performing package view clash check for digest-20021220 in standard view +=> Performing package view overwrite check for digest-20021220 in standard view +=> Linking package into standard view + +Notice in the transcript that the digest package has been installed +into /usr/pkg/packages/digest-20021220 and then subsequently linked +into the "standard view". The standard view is simply /usr/pkg, which +means that the digest executable is accessible as /usr/pkg/bin/digest. + + + 0.3 The contents of /usr/pkg (${LOCALBASE}) + =========================================== + +There is a directory /usr/pkg/packages that contains all of the +depoted packages. The contents of /usr/pkg/packages after installing +digest are: + +/usr/pkg/packages/digest-20021220/+BUILD_INFO +/usr/pkg/packages/digest-20021220/+BUILD_VERSION +/usr/pkg/packages/digest-20021220/+COMMENT +/usr/pkg/packages/digest-20021220/+CONTENTS +/usr/pkg/packages/digest-20021220/+DESC +/usr/pkg/packages/digest-20021220/+SIZE_ALL +/usr/pkg/packages/digest-20021220/+SIZE_PKG +/usr/pkg/packages/digest-20021220/+VIEWS +/usr/pkg/packages/digest-20021220/bin/digest +/usr/pkg/packages/digest-20021220/man/man1/digest.1 + +Note that all of the files related to the package, including the +package metadata files used by the pkg_* tools, are stored in the +"depot directory" for digest (/usr/pkg/packages/digest-20021220). + +The digest package was also linked into the standard view. Looking +inside /usr/pkg shows us the following symlinks: + +/usr/pkg/bin/digest -> /usr/pkg/packages/digest-20021220/bin/digest +/usr/pkg/man/man1/digest.1 -> /usr/pkg/packages/digest-20021220/man/man1/digest.1 + +Since digest is present in /usr/pkg/bin, it can be used as always +without any changes needed by the typical user with /usr/pkg/bin in +his PATH, and "man digest" will continue to work as expected. + + + 0.4 Creating new views + ====================== + +The following commands will install both pkgsrc/www/lynx and +pkgsrc/www/lynx-current onto the same system using pkgviews: + + cd /usr/pkgsrc/www/lynx; make install + cd /usr/pkgsrc/www/lynx-current; make install PKGVIEWS=devel + +The stable version of lynx (2.8.4.1nb2) is linked into the standard +view, while the -current version of lynx (2.8.5.0.7nb3) is linked into +the "devel" view. The lynx-related files in /usr/pkg are: + +/usr/pkg/bin/lynx -> /usr/pkg/packages/lynx-2.8.4.1nb2/bin/lynx +/usr/pkg/man/man1/lynx.1 -> /usr/pkg/packages/lynx-2.8.4.1nb2/man/man1/lynx.1 +/usr/pkg/share/locale/cs/LC_MESSAGES/lynx.mo -> ... +... + +/usr/pkg/devel/bin/lynx -> /usr/pkg/packages/lynx-2.8.5.0.7nb3/bin/lynx +/usr/pkg/devel/man/man1/lynx.1 -> /usr/pkg/packages/lynx-2.8.5.0.7nb3/man/man1/lynx.1 +/usr/pkg/devel/share/locale/cs/LC_MESSAGES/lynx.mo -> ... +... + +Note that the files for pkgsrc/www/lynx have been symlinked into +/usr/pkg as usual, but the files for pkgsrc/www/lynx-current have been +symlinked into the "devel" subdirectory of /usr/pkg. The files in +in the "devel" view can be accessed by adding /usr/pkg/devel/bin to +the PATH and adding /usr/pkg/devel/man to MANPATH in the appropriate +place. + + + 0.5 Managing views using pkg_* tools + ==================================== + +Adding and removing package instances to and from views is always a +safe operation as the true package files are left untouched. All that +happens is that linkfarms are being added or removed, thus changes are +always easy to reverse. + +Adding a depoted package into a view may be accomplished with: + + pkg_view -w devel add gmake-3.80nb2 + +This adds the gmake-3.80nb2 package (already present on the system) to +the "devel" view. The "devel" view is created if it doesn't already +exist. + +Removing a package from a view may be accomplished with either of the +following commands: + + pkg_view -w devel delete gmake-3.80nb2 + pkg_delete -K /usr/pkg/devel/.pkgdb gmake + +Either of these will remove the gmake-3.80nb2 package from the "devel" +view if the package exists in the view. + +Removing a package from the standard view may be accomplished with +either of the following commands + + pkg_view delete gmake-3.80nb2 + pkg_delete gmake + +The latter one is recommended for speed. + + + 0.6 Completely removing a package + ================================= + +Removing a depoted package from the system (removing it from all views +and removing the depot directory) is done with: + + pkg_delete -K /usr/pkg/packages gmake-3.80nb2 + +If you know that only a single gmake is present on your system, then +you can specify "gmake" instead of the full package name to +pkg_delete(1). However, I recommend being explicit about the package +you are removing to avoid any surprises. + + + 0.6 Where to learn more about pkgviews + ====================================== + +There are several shell environment variables that may be set to +affect the default behaviour of the pkg_* tools. Please read the +pkg_view(1) man page for more information. + +For a more complete understanding of the motivations and principles +behind pkgviews, the following paper by Alistair Crooks is highly +recommended: + + http://www.NetBSD.org/Documentation/software/pkgviews.pdf |