First pass at implementing support for package system flavors other

than pkgsrc's current one.  This is an important lead-up to any project
that redesigns the pkg_* tools in that it doesn't tie us to past design
(mis)choices.  This commit mostly deals with rearranging code, although
there was a considerable amount of rewriting done in cases where I
thought the code was somewhat messy and was difficult to understand.

The design I chose for supporting multiple package system flavors is
that the various depends, install, package, etc.  modules would define
default targets and variables that may be overridden in files from
pkgsrc/mk/flavor/${PKG_FLAVOR}.  The default targets would do the
sensible thing of doing nothing, and pkgsrc infrastructure would rely
on the appropriate things to be defined in pkgsrc/mk/flavor to do the
real work.  The pkgsrc/mk/flavor directory contains subdirectories
corresponding to each package system flavor that we support.  Currently,
I only have "pkg" which represents the current pkgsrc-native package
flavor.  I've separated out most of the code where we make assumptions
about the package system flavor, mostly either because we directly
use the pkg_* tools, or we make assumptions about the package meta-data
directory, or we directly manipulate the package meta-data files, and
placed it into pkgsrc/mk/flavor/pkg.

There are several new modules that have been refactored out of bsd.pkg.mk
as part of these changes: check, depends, install, package, and update.
Each of these modules has been slimmed down by rewriting them to avoid
some recursive make calls.  I've also religiously documented which
targets are "public" and which are "private" so that users won't rely
on reaching into pkgsrc innards to call a private target.

The "depends" module is a complete overhaul of the way that we handle
dependencies.  There is now a separate "depends" phase that occurs
before the "extract" phase where dependencies are installed.  This
differs from the old way where dependencies were installed just before
extraction occurred.  The reduce-depends.mk file is now replaced by
a script that is invoked only once during the depends phase and is
used to generate a cookie file that holds the full set of reduced
dependencies.  It is now possible to type "make depends" in a package
directory and all missing dependencies will be installed.

Future work on this project include:

    * Resolve the workflow design in anticipation of future work on
      staged installations where "package" conceptually happens before
      "install".

    * Rewrite the buildlink3 framework to not assume the use of the
      pkgsrc pkg_* tools.

    * Rewrite the pkginstall framework to provide a standard pkg_*
      tool to perform the actions, and allowing a purely declarative
      file per package to describe what actions need to be taken at
      install or deinstall time.

    * Implement support for the SVR4 package flavor.  This will be
      proof that the appropriate abstractions are in place to allow
      using a completely different set of package management tools.

5 files changed, 510 insertions, 0 deletions

diff --git a/mk/install/bsd.install-vars.mk b/mk/install/bsd.install-vars.mk
new file mode 100644
index 00000000000..ec6c429021e
--- /dev/null
+++ b/mk/install/bsd.install-vars.mk
@@ -0,0 +1,65 @@
+# $NetBSD: bsd.install-vars.mk,v 1.1 2006/06/03 23:11:42 jlam Exp $
+#
+# This Makefile fragment is included by bsd.prefs.mk and defines some
+# variables which must be defined earlier than where bsd.install.mk
+# is included.
+#
+
+# If a package sets INSTALLATION_DIRS, then it's known to pre-create
+# all of the directories that it needs at install-time, so we don't need
+# mtree to do it for us.
+#
+.if defined(INSTALLATION_DIRS) && !empty(INSTALLATION_DIRS)
+NO_MTREE=	yes
+.endif
+#
+# Certain classes of packages never need to run mtree during installation
+# because they manage the creation of their own directories.
+#
+.if (${PKG_INSTALLATION_TYPE} == "pkgviews") && defined(CROSSBASE)
+NO_MTREE=	yes
+.endif
+.if !defined(NO_MTREE)  
+USE_TOOLS+=	mtree 
+.endif
+
+# If MANZ is defined, then we want the final man pages to be compressed.
+# If MANZ is not defined, then we want the final man pages to be
+# uncompressed.
+#
+# We need to figure out if during installation, we need either gunzip
+# or gzip to decompress or compress the installed man pages.  If a
+# package sets MANCOMPRESSED to "yes" or "no", then it's an indication
+# to the install code that the package itself installed the man pages
+# either compressed or uncompressed.  If a package sets
+# MANCOMPRESSED_IF_MANZ, then the package uses BSD-style makefiles,
+# so we need to determine if the BSD-style makefile causes the man
+# pages to be compressed or not.
+#
+.if !defined(_MANCOMPRESSED)
+.  if defined(MANCOMPRESSED) && !empty(MANCOMPRESSED:M[yY][eE][sS])
+_MANCOMPRESSED=		yes
+.  else
+_MANCOMPRESSED=		no
+.  endif
+.  if defined(MANCOMPRESSED_IF_MANZ) && defined(PKGMAKECONF)
+_MANCOMPRESSED!=							\
+	{ ${ECHO} ".include \""${PKGMAKECONF:Q}"\"";			\
+	  ${ECHO} "all:";						\
+	  ${ECHO} ".if defined(MANZ)";					\
+	  ${ECHO} "	@${ECHO} yes";					\
+	  ${ECHO} ".else";						\
+	  ${ECHO} "	@${ECHO} no";					\
+	  ${ECHO} ".endif";						\
+	} | ${MAKE} -f - all
+.  endif
+.endif
+_MANZ=		${MANZ:Dyes:Uno}
+MAKEVARS+=	_MANCOMPRESSED _MANZ
+
+.if !empty(_MANCOMPRESSED:M[yY][eE][sS]) && empty(_MANZ:M[yY][eE][sS])
+USE_TOOLS+=	gunzip
+.endif
+.if empty(_MANCOMPRESSED:M[yY][eE][sS]) && !empty(_MANZ:M[yY][eE][sS])
+USE_TOOLS+=	gzip
+.endif
diff --git a/mk/install/bsd.install.mk b/mk/install/bsd.install.mk
new file mode 100644
index 00000000000..2cfcb186e04
--- /dev/null
+++ b/mk/install/bsd.install.mk
@@ -0,0 +1,40 @@
+# $NetBSD: bsd.install.mk,v 1.1 2006/06/03 23:11:42 jlam Exp $
+#
+# This Makefile fragment is included by bsd.pkg.mk and provides all
+# variables and targets related to installing packages.
+#
+# The following are the "public" targets provided this module:
+#
+#    install, deinstall, reinstall, replace, undo-replace
+#
+# The following targets may be overridden in a package Makefile:
+#
+#    pre-install, do-install, post-install
+#
+
+_INSTALL_COOKIE=	${WRKDIR}/.install_done
+
+######################################################################
+### install (PUBLIC)
+######################################################################
+### install is a public target to install the package.
+###
+.if defined(NO_INSTALL)
+.PHONY: install
+install: ${_PKGSRC_BUILD_TARGETS} install-cookie
+.else
+.  include "${PKGSRCDIR}/mk/install/install.mk"
+.endif
+
+.include "${PKGSRCDIR}/mk/install/deinstall.mk"
+.include "${PKGSRCDIR}/mk/install/replace.mk"
+
+######################################################################
+### install-cookie (PRIVATE)
+######################################################################
+### install-cookie creates the "install" cookie file.
+###
+.PHONY: install-cookie
+install-cookie:
+	${_PKG_SILENT}${_PKG_DEBUG}${ECHO} ${PKGNAME} >> ${_INSTALL_COOKIE}
+
diff --git a/mk/install/deinstall.mk b/mk/install/deinstall.mk
new file mode 100644
index 00000000000..eccbd639911
--- /dev/null
+++ b/mk/install/deinstall.mk
@@ -0,0 +1,44 @@
+# $NetBSD: deinstall.mk,v 1.1 2006/06/03 23:11:42 jlam Exp $
+
+# DEINSTALLDEPENDS controls whether dependencies and dependents are also
+# removed when a package is de-installed.  The valid values are:
+#
+#    no		only the package is removed (if dependencies allow it)
+#    yes	dependent packages are also removed
+#    all	dependent packages and unused dependencies are also removed
+#
+DEINSTALLDEPENDS?=	no
+
+######################################################################
+### deinstall, su-deinstall (PUBLIC)
+######################################################################
+### deinstall is a public target to remove an installed package.
+### It will acquire elevated privileges just-in-time.
+###
+.PHONY: deinstall su-deinstall
+deinstall: su-target
+	@${ECHO_MSG} "${_PKGSRC_IN}> Deinstalling for ${PKGNAME}"
+
+su-deinstall: deinstall-pkg install-clean
+
+MAKEFLAGS.su-deinstall=	DEINSTALLDEPENDS=${DEINSTALLDEPENDS}
+
+######################################################################
+### reinstall (PUBLIC)
+######################################################################
+### reinstall is a special target to re-run the install target.
+### It will acquire elevated privileges just-in-time.
+###
+.PHONY: reinstall
+reinstall: install-clean install
+
+######################################################################
+### deinstall-pkg (PRIVATE, override)
+######################################################################
+### deinstall-pkg removes the package from the system.  This should
+### be overridden per package system flavor.
+###
+.if !target(deinstall-pkg)
+deinstall-pkg:
+	@${DO_NADA}
+.endif
diff --git a/mk/install/install.mk b/mk/install/install.mk
new file mode 100644
index 00000000000..3ccfc610702
--- /dev/null
+++ b/mk/install/install.mk
@@ -0,0 +1,302 @@
+# $NetBSD: install.mk,v 1.1 2006/06/03 23:11:42 jlam Exp $
+
+######################################################################
+### install (PUBLIC)
+######################################################################
+### install is a public target to install the package.  It will
+### acquire elevated privileges just-in-time.
+###
+_INSTALL_TARGETS+=	${_PKGSRC_BUILD_TARGETS}
+_INSTALL_TARGETS+=	acquire-install-lock
+_INSTALL_TARGETS+=	${_INSTALL_COOKIE}
+_INSTALL_TARGETS+=	release-install-lock
+
+.PHONY: install
+install: ${_INSTALL_TARGETS}
+
+.PHONY: acquire-install-lock release-install-lock
+acquire-install-lock: acquire-lock
+release-install-lock: release-lock
+
+${_INSTALL_COOKIE}:
+.if !empty(INTERACTIVE_STAGE:Minstall) && defined(BATCH)
+	@${ECHO} "*** The installation stage of this package requires user interaction"
+	@${ECHO} "*** Please install manually with \"cd ${.CURDIR} && ${MAKE} install\""
+	@${TOUCH} ${_INTERACTIVE_COOKIE}
+	@${FALSE}
+.else
+	${_PKG_SILENT}${_PKG_DEBUG}cd ${.CURDIR} && ${MAKE} ${MAKEFLAGS} real-install PKG_PHASE=install
+.endif
+
+######################################################################
+### real-install (PRIVATE)
+######################################################################
+### real-install is a helper target to set the PKG_PHASE explicitly to
+### "install" before running the remainder of the install targets.
+###
+_REAL_INSTALL_TARGETS+=	install-check-version
+_REAL_INSTALL_TARGETS+=	install-message
+_REAL_INSTALL_TARGETS+=	install-vars
+_REAL_INSTALL_TARGETS+=	unprivileged-install-hook
+_REAL_INSTALL_TARGETS+=	install-all
+_REAL_INSTALL_TARGETS+=	install-cookie
+
+.PHONY: real-install
+real-install: ${_REAL_INSTALL_TARGETS}
+
+.PHONY: install-message
+install-message:
+	@${ECHO_MSG} "${_PKGSRC_IN}> Installing for ${PKGNAME}"
+
+######################################################################
+### unprivileged-install-hook (PRIVATE, override, hook)
+######################################################################
+### unprivileged-install-hook is a generic hook target that is run just
+### before pkgsrc elevates privileges for install-all.
+###
+.PHONY: unprivileged-install-hook
+.if !target(unprivileged-install-hook)
+unprivileged-install-hook:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### install-check-version (PRIVATE)
+######################################################################
+### install-check-version will verify that the built package located in
+### ${WRKDIR} matches the version specified in the package Makefile.
+### This is a check against stale work directories.
+###
+.PHONY: install-check-version
+install-check-version: ${_EXTRACT_COOKIE}
+	${_PKG_SILENT}${_PKG_DEBUG}					\
+	extractname=`${CAT} ${_EXTRACT_COOKIE}`;			\
+	pkgname=${PKGNAME};						\
+	case "$$extractname" in						\
+	"")	${ECHO_MSG} "*** Warning: ${WRKDIR} may contain an older version of ${PKGBASE}" ;; \
+	"$$pkgname")	;;						\
+	*)	${ECHO_MSG} "*** Warning: Package version $$extractname in ${WRKDIR}"; \
+		${ECHO_MSG} "*** Current version $$pkgname in ${PKG_PATH}"; \
+		${ECHO_MSG} "*** Cleaning and rebuilding $$pkgname...";	\
+		${MAKE} clean && ${MAKE} build ;;			\
+	esac
+
+######################################################################
+### The targets below are run with elevated privileges.
+######################################################################
+
+######################################################################
+### install-all, su-install-all (PRIVATE)
+######################################################################
+### install-all is a helper target to run the install target of
+### the built software, register the software installation, and run
+### some sanity checks.
+###
+.if !defined(NO_PKG_REGISTER) && !defined(FORCE_PKG_REGISTER)
+_INSTALL_ALL_TARGETS+=		install-check-conflicts
+_INSTALL_ALL_TARGETS+=		install-check-installed
+.endif
+_INSTALL_ALL_TARGETS+=		install-check-umask
+.if empty(CHECK_FILES:M[nN][oO])
+_INSTALL_ALL_TARGETS+=		check-files-pre
+.endif
+_INSTALL_ALL_TARGETS+=		install-makedirs
+_INSTALL_ALL_TARGETS+=		pre-install-script
+_INSTALL_ALL_TARGETS+=		pre-install
+_INSTALL_ALL_TARGETS+=		do-install
+_INSTALL_ALL_TARGETS+=		post-install
+_INSTALL_ALL_TARGETS+=		plist
+_INSTALL_ALL_TARGETS+=		install-doc-handling
+_INSTALL_ALL_TARGETS+=		install-script-data
+.if empty(CHECK_FILES:M[nN][oO])
+_INSTALL_ALL_TARGETS+=		check-files-post
+.endif
+_INSTALL_ALL_TARGETS+=		post-install-script
+.if !defined(NO_PKG_REGISTER)
+_INSTALL_ALL_TARGETS+=		register-pkg
+.endif
+_INSTALL_ALL_TARGETS+=		privileged-install-hook
+
+.if empty(CHECK_SHLIBS:M[nN][oO])
+privileged-install-hook: check-shlibs
+.endif
+.if empty(CHECK_WRKREF:M[nN][oO])
+privileged-install-hook: check-wrkref
+.endif
+.if empty(CHECK_FILES:M[nN][oO])
+privileged-install-hook: check-files
+.endif
+.if empty(CHECK_INTERPRETER:M[nN][oO])
+privileged-install-hook: check-interpreter
+.endif
+
+.PHONY: install-all su-install-all
+install-all: su-target
+su-install-all: ${_INSTALL_ALL_TARGETS}
+
+######################################################################
+### install-check-conflicts (PRIVATE, override)
+######################################################################
+### install-check-conflicts check for conflicts between the package and
+### any installed packages.  This should be overridden per package
+### system flavor.
+###
+.PHONY: install-check-conflicts
+.if !target(install-check-conflicts)
+install-check-conflicts:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### install-check-installed (PRIVATE, override)
+######################################################################
+### install-check-installed checks if the package (perhaps an older
+### version) is already installed on the system.  This should be
+### overridden per package system flavor.
+###
+.PHONY: install-check-installed
+.if !target(install-check-installed)
+install-check-installed:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### install-check-umask (PRIVATE)
+######################################################################
+### install-check-umask tests whether the umask is properly set and
+### emits a non-fatal warning otherwise.
+###
+.PHONY: install-check-umask
+install-check-umask:
+	${_PKG_SILENT}${_PKG_DEBUG}					\
+	umask=`${SH} -c umask`;						\
+	if ${TEST} "$$umask" -ne ${DEF_UMASK}; then			\
+		${ECHO_MSG} "${_PKGSRC_IN}> Warning: your umask is \`\`$$umask''."; \
+		${ECHO_MSG} "If this is not desired, set it to an appropriate value (${DEF_UMASK}) and install"; \
+		${ECHO_MSG} "this package again by \`\`${MAKE} deinstall reinstall''."; \
+        fi
+
+######################################################################
+### install-makedirs (PRIVATE)
+######################################################################
+### install-makedirs is a target to create directories expected to
+### exist prior to installation.  If a package sets INSTALLATION_DIRS,
+### then it's known to pre-create all of the directories that it needs
+### at install-time, so we don't need mtree to do it for us.
+###
+MTREE_FILE?=	${PKGSRCDIR}/mk/platform/${OPSYS}.pkg.dist
+MTREE_ARGS?=	-U -f ${MTREE_FILE} -d -e -p
+
+.PHONY: install-makedirs
+install-makedirs:
+	${_PKG_SILENT}${_PKG_DEBUG}${TEST} -d ${PREFIX} || ${MKDIR} ${PREFIX}
+.if !defined(NO_MTREE)
+	${_PKG_SILENT}${_PKG_DEBUG}${TEST} ! -f ${MTREE_FILE} ||	\
+		${MTREE} ${MTREE_ARGS} ${PREFIX}/
+.endif
+.if defined(INSTALLATION_DIRS) && !empty(INSTALLATION_DIRS)
+	@${ECHO_MSG} "${_PKGSRC_IN}> Creating installation directories"
+	${_PKG_SILENT}${_PKG_DEBUG}					\
+	for dir in ${INSTALLATION_DIRS}; do				\
+		case "$$dir" in						\
+		${PREFIX}/*)						\
+			dir=`${ECHO} $$dir | ${SED} "s|^${PREFIX}/||"` ;; \
+		/*)	continue ;;					\
+		esac;							\
+		case "$$dir" in						\
+		*bin|*bin/*|*libexec|*libexec/*)			\
+			${INSTALL_PROGRAM_DIR} ${PREFIX}/$$dir ;;	\
+		${PKGMANDIR}/*)						\
+			${INSTALL_MAN_DIR} ${PREFIX}/$$dir ;;		\
+		*)							\
+			${INSTALL_DATA_DIR} ${PREFIX}/$$dir ;;		\
+		esac;							\
+	done
+.endif	# INSTALLATION_DIRS
+
+######################################################################
+### pre-install, do-install, post-install (PUBLIC, override)
+######################################################################
+### {pre,do,post}-install are the heart of the package-customizable
+### install targets, and may be overridden within a package Makefile.
+###
+.PHONY: pre-install do-install post-install
+
+INSTALL_DIRS?=		${BUILD_DIRS}
+INSTALL_MAKE_FLAGS?=	${MAKE_FLAGS}
+INSTALL_TARGET?=	install ${USE_IMAKE:D${NO_INSTALL_MANPAGES:D:Uinstall.man}}
+
+.if !target(do-install)
+do-install:
+.  for _dir_ in ${INSTALL_DIRS}
+	${_PKG_SILENT}${_PKG_DEBUG}${_ULIMIT_CMD}			\
+	cd ${WRKSRC} && cd ${_dir_} &&					\
+	${SETENV} ${MAKE_ENV} ${MAKE_PROGRAM} ${INSTALL_MAKE_FLAGS}	\
+		-f ${MAKEFILE} ${INSTALL_TARGET}
+.  endfor
+.endif
+
+.if !target(pre-install)
+pre-install:
+	@${DO_NADA}
+.endif
+.if !target(post-install)
+post-install:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### install-doc-handling (PRIVATE)
+######################################################################
+### install-doc-handling does automatic document (de)compression based
+### on the contents of the PLIST.
+###
+_PLIST_REGEXP.info=	\
+	^([^\/]*\/)*${PKGINFODIR}/[^/]*(\.info)?(-[0-9]+)?(\.gz)?$$
+_PLIST_REGEXP.man=	\
+	^([^/]*/)+(man[1-9ln]/[^/]*\.[1-9ln]|cat[1-9ln]/[^/]*\.[0-9])(\.gz)?$$
+
+_DOC_COMPRESS=								\
+	${SETENV} PATH=${PATH:Q}					\
+		MANZ=${_MANZ}						\
+		PKG_VERBOSE=${PKG_VERBOSE}				\
+		TEST=${TOOLS_TEST:Q}					\
+	${SH} ${PKGSRCDIR}/mk/plist/doc-compress ${PREFIX}
+
+.PHONY: install-doc-handling
+install-doc-handling: plist
+	@${ECHO_MSG} "${_PKGSRC_IN}> [Automatic manual page handling]"
+	${_PKG_SILENT}${_PKG_DEBUG}${CAT} ${PLIST} | ${GREP} -v "^@" |	\
+	${EGREP} ${_PLIST_REGEXP.man:Q} | ${_DOC_COMPRESS}
+
+######################################################################
+### register-pkg (PRIVATE, override)
+######################################################################
+### register-pkg registers the package as being installed on the system.
+###
+.PHONY: register-pkg
+.if !target(register-pkg)
+register-pkg:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### privileged-install-hook (PRIVATE, override, hook)
+######################################################################
+### privileged-install-hook is a generic hook target that is run just
+### before pkgsrc drops elevated privileges.
+###
+.PHONY: privileged-install-hook
+.if !target(privileged-install-hook)
+privileged-install-hook:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### install-clean (PRIVATE)
+######################################################################
+### install-clean removes the state files for the "install" and
+### later phases so that the "install" target may be re-invoked.
+###
+install-clean: package-clean check-clean
+	${_PKG_SILENT}${_PKG_DEBUG}${RM} -f ${PLIST} ${_INSTALL_COOKIE}
diff --git a/mk/install/replace.mk b/mk/install/replace.mk
new file mode 100644
index 00000000000..18623adf950
--- /dev/null
+++ b/mk/install/replace.mk
@@ -0,0 +1,59 @@
+# $NetBSD: replace.mk,v 1.1 2006/06/03 23:11:42 jlam Exp $
+
+######################################################################
+### replace (PUBLIC)
+######################################################################
+### replace is a public target to update a package in-place on the
+### system.  It will acquire elevated privileges just-in-time.
+###
+_REPLACE_TARGETS+=	${_PKGSRC_BUILD_TARGETS}
+_REPLACE_TARGETS+=	replace-message
+_REPLACE_TARGETS+=	unprivileged-install-hook
+
+.PHONY: replace su-replace
+replace: ${_REPLACE_TARGETS} su-target
+
+replace-message:
+	@${ECHO_MSG} "${_PKGSRC_IN}> Replacing for ${PKGNAME}"
+        @${WARNING_MSG} "experimental target - DATA LOSS MAY OCCUR."
+
+su-replace: replace-pkg
+MAKEFLAGS.su-replace=	UPDATE_RUNNING=yes
+
+######################################################################
+### undo-replace (PUBLIC)
+######################################################################
+### undo-replace is a public target to undo the effects of the
+### "replace" target.  It will acquire elevated privileges just-in-time.
+###
+.PHONY: undo-replace su-undo-replace
+undo-replace: undo-replace-message su-target
+
+undo-replace-message:
+	@${ECHO_MSG} "${_PKGSRC_IN}> Undoing replacement for ${PKGNAME}"
+        @${WARNING_MSG} "experimental target - DATA LOSS MAY OCCUR."
+
+su-undo-replace: undo-replace-pkg
+MAKEFLAGS.su-undo-replace=	UPDATE_RUNNING=yes
+
+######################################################################
+### replace-pkg (PRIVATE, override)
+######################################################################
+### replace-pkg updates a package in-place on the system.  This should
+### be overridden per package system flavor.
+###
+.if !target(replace-pkg)
+replace-pkg:
+	@${DO_NADA}
+.endif
+
+######################################################################
+### undo-replace-pkg (PRIVATE, override)
+######################################################################
+### undo-replace-pkg undoes a "make replace".  This should be overridden
+### per package system flavor.
+###
+.if !target(undo-replace-pkg)
+undo-replace-pkg:
+	@${DO_NADA}
+.endif