summaryrefslogtreecommitdiff
path: root/mk/bulk/build
AgeCommit message (Collapse)AuthorFilesLines
2009-07-09With the most recent pkg_install, directories are removed when thehe1-1/+12
last file in a directory is removed. This might cause common and expected-to-be-existing directories in /usr/pkg to be removed, and a subsequent attempt at installing a file to the now non-existent directory will instead create a file with the name of the expected directory. This will create PLIST errors for the package in question, but also will cause the erroneously named file to not be removed on package removal. This can cause cascading bulk build failures for subsequent packages. To prevent this, after each package is done, check if some of the common top-level expected-to-be directories are now files, and remove them and emit an error message if so. I *think* I already fixed the single package which had the unfortunate problem of not declaring "include" as an installation directory, but this should prevent the problem from re-occurring in the future as well. Thanks to joerg@ for the hint for pulling in the value of PREFIX.
2008-06-13Make sure we pick up pax via USE_TOOLS.sketch1-3/+3
2008-06-08Commit two fixes which at least for me appear to be required whenhe1-3/+2
doing an old-style bulk build on 4.0 or older systems: - in mk/bulk/build, do "make update" instead of "make install + clean" when installing a new pkg_install - in pkgtools/pkg_install/Makefile, don't try to use our own executables (${WORKDIR}/pkg_add/pkg_add etc.) if they don't exist Discussed with joerg, and even though he's not entirely happy with the latter change, he didn't appear to have a better suggestion. This, and putting /usr/pkg/sbin earlier than /usr/sbin in the $PATH appears to be required to get an old-style bulk build going.
2008-01-04Replaced all references to pkglint with lintpkgsrc, which has its ownrillig1-5/+5
package since a few days.
2007-06-15The option --prepare only generates the bulk build database.rillig1-1/+16
2007-03-07A hint to the real problem is a nice addition to an error message.rillig1-2/+4
2007-02-23The post-build-only option also needs to load the variables.rillig1-1/+2
2007-02-21At the very beginning, the variables are extracted by running bmake inrillig1-54/+24
the pkglint directory. This part had been more time-consuming than necessary. Now it only needs one call to bmake instead of n.
2007-01-18pkgsrc bulk builds need much CPU time, but do they also need much stack?rillig1-3/+4
2006-12-15Remove trailing spaces.martti1-2/+2
2006-11-27- modularize functionality into a number of distinct functionsjschauma1-301/+357
- add the option "-p|--post-build" to allow you to only run the post-build portion of the bulk-build
2006-11-19nuke two Useless Use of Wc and let awk do the legwork. This saves usjschauma1-6/+8
two forks per package.
2006-02-11Renamed the ``restart'' variable to ``resume'', as that is what therillig1-13/+11
variable does.
2006-01-30Added the bulk build ID to the subject line of the report mail. It had beenrillig1-2/+2
${BUILDDATE} before, which has disappeared with the REPORTS_* change.
2006-01-21- Added a new top level file that saves the REPORT_BASEDIR from the firstrillig1-7/+20
call to the mk/bulk/build script. This is to allow restarting the bulk build without getting confused by the various timestamps that occur during a bulk build. It also fixes some problems where the text/plain report had ended up in a directory different from the other files. - Changed the default filename for the text/plain report from "pkgsrc-results-${BUILDDATE}.txt" to simply "report.txt", as the ${BUILDDATE} variable didn't give a hint to when the bulk build had been started, which is often more important than when the bulk build stopped. - Changed the way to specify the directory and the URL where the bulk build reports are made available. See http://mail-index.netbsd.org/tech-pkg/2005/12/05/0002.html for details. Specifying the directories using the FTPHOST and FTPURL variables will be possible for some months, after which it support for it will be dropped.
2005-12-05Shortened the code to extract pkgsrc variables by 30 lines, taking outrillig1-85/+49
much unnecessary redundancy from the code. Reordered the pkgsrc variables so that related variables appear adjacently in the output.
2005-11-30- Ignore the return value of the "unset" command. This would terminate therillig1-3/+3
program if the program would be run in "set -e" mode.
2005-11-28Instead of extracting the PKG_TOOLS_BIN make(1) variable, extractrillig1-10/+13
PKG_DELETE and PKG_INFO. This is more straigt-forward and adds the proper `-K' command line option to the calls, which in turn fixes bulk builds that don't use the default PKG_DBDIR.
2005-11-28Prefixed the messages with "build> " that I had forgotten in the lastrillig1-6/+6
commit.
2005-11-28Prefixed the status messages from this program with "build> ", torillig1-13/+13
distinguish them from the various other messages.
2005-11-28Print a message before creating the bulk build report, as this takesrillig1-1/+2
quite a long time. Currently the reader of the build log might assume that the installation of lang/perl5 takes so long, which is wrong.
2005-11-24Indentation fix to make the "case" statements look more uniform.rillig1-3/+3
2005-11-24Corrected the quoting of variables in clear cases.rillig1-12/+13
2005-11-23Per request, back out all the SKIP_AUDIT_PACKAGES changes.erh1-2/+2
bsd.pkg.mk:1.1758-1.1752 bsd.prefs.mk:1.210 bulk/build:1.79 defaults/mk.conf:1.93-1.92
2005-11-20- If the BULKFILESDIR differs from PKGSRCDIR, create it before trying torillig1-1/+16
create log files in it. - Fail early if the ORDERFILE does not exist. (As this program is not in "set -e" mode, it had continued without error checking, which resulted in _many_ error messages.)
2005-11-19Indentation fix.rillig1-3/+2
2005-11-19The toplevel database files are referenced in the report, so they must notrillig1-3/+1
be unlinked. Not even in a comment.
2005-11-19To check if the pkg_install tools are up to date, the bulk build scriptagc1-2/+2
cd's into pkgtools/pkglint and does a "make fetch". Since the audit-packages check was introduced to bsd.pkg.mk, this "make fetch" will fail with an error message which looks like: ===> do-fetch [pkglint-4.42.1] ===> Checking for vulnerabilities in pkglint-4.42.1 ===> check-vulnerable [pkglint-4.42.1] ===> *** The audit-packages package must be at least version 1.40 ===> check-vulnerable [pkglint-4.42.1] ===> *** Please install the security/audit-packages package and run ===> check-vulnerable [pkglint-4.42.1] ===> *** '/usr/pkg/sbin/download-vulnerability-list'. *** Error code 1 since there is no audit-packages package installed at this point (we are still doing the pre-installation checks). So call "make fetch" with an additional SKIP_AUDIT_PACKAGES=yes argument at this point.
2005-11-18Revert the change in 1.77, which does not work correctly.agc1-15/+11
However, this does not help people who are doing bulk builds in sandboxes via read-only nullfs mounts. A correct, long-term solution to this needs to be found.
2005-11-18If the bulk build is taking place in a sandbox, then typically read-onlyagc1-11/+15
nullfs mounts are used (on NetBSD). Only attempt to install new pkg_install tools if the bulk build is not taking place in a sandbox. This is a quick and dirty hack to allow sandboxed bulk builds (the majority of cases, I suspect) to operate properly once again.
2005-11-16Changes for enabling separate BULKFILESDIR and PKGSRCDIR.rillig1-6/+13
- The main .make file is stored in ${BULKFILESDIR} instead of ${PKGSRCDIR}. - The PKGLIST variable from the configuration file is passed to the bulk-cache command in the environment. There are very few variables left that have to be exported via export_config_vars(). Maybe that function can be avoided alltogether.
2005-11-11Removed the "BULK>" prefix from a message, as that prefix is reserved forrillig1-2/+2
bsd.bulk-pkg.mk.
2005-11-10Fixed the remaining instances of code relying on trailing white-space inrillig1-2/+2
the INDEXFILE.
2005-11-09Before extracting the pkgsrc variables (which takes some time), print arillig1-1/+3
line announcing that.
2005-11-08- Improved error handling.rillig1-8/+7
- Changed "foo; if [ $? -eg 0 ]; then" to "if foo; then" in three places.
2005-11-08- Removed the redundant check on USR_PKGSRC.rillig1-8/+3
- Removed the singleton variable pkg_install_dir.
2005-11-06Added basic checks to make sure the values from the configuration file arerillig1-1/+2
somewhat sane. If a check fails, the bulk build is terminated. Removed the conditional under which MAKECONF is exported, as it is not needed anymore.
2005-11-06Remove extra ${pkgsrc_dir} that slipped in.seb1-2/+2
2005-11-05- Replaced spaces with tabs in the "build --help" message.rillig1-38/+39
- Added file post-build-conf, which provides functions to print and export the config variables from build.conf. - All config variables are shown at the start of a bulk build. - Added a configuration variable MAKECONF with the obvious meaning.
2005-11-05- Condensed the code.rillig1-37/+18
- Sorted the command line options alphabetically in the parse loop.
2005-11-05Reformatted the --help output. Instead of many echo(1) commands, onerillig1-42/+43
cat(1) is used.
2005-11-05- Added a comment above each paragraph explaining what goes on there.rillig1-31/+74
- Removed the perl5_dir variable, as it has only been used in one place. - Added error checking for the installation of the prerequisite packages after the main build.
2005-11-05- Added an alias --resume for --restart, as "restart" sounds too much likerillig1-19/+13
"from the beginning". - Moved the code to get the value of BULK_PREREQ some lines up, as there are also 19 other variables using the same code. This saves one extra check if the pkglint directory exists. - Added BULK_PREREQ to the variables whose value is printed before starting the actual build. - When installing prerequisite packages, print a nice sentence instead of a simple "echo $pkgdir".
2005-11-05- Added a function post_filter_cmd() that can be overridden in bulk.conf.rillig1-33/+54
The default function prepends the well-known "date, progress, pkgdir" to each line. - Check that the environment variable USR_PKGSRC is given and is an absolute pathname. - Introduced variables for the various directories. Their names all end in _dir, and they contain absolute pathnames. Changed all instances of ${USR_PKGSRC}/pkgtools/pkglint to ${pkglint_dir}. Likewise for the other directories. - Added more error checking. The bulk build is terminated if the pkg_install package cannot be updated or the prerequisite packages cannot be installed.
2005-11-05- Added a shell function die() that is used to provide consistentlyrillig1-19/+18
formatted error messages in case of failure. It also helps shortening the code. - Replaced many uses of "exit 1" with die().
2005-11-05- Removed trailing white-space from the --help output.rillig1-39/+41
- All error messages go to stderr instead of stdout. - Added variable quoting. - Reindented some code for better readability. - Changed "Build Temporary Files" caption to "Some variables used in the bulk build", which is more appropriate.
2005-11-05Export the BULK_BUILD_CONF variable so that subprocesses can see it. Rewroterillig1-10/+12
the code a little.
2005-11-05Added a command line option --config that allows to specify a customrillig1-2/+10
build.conf file. The only way to do this before had been to set the (undocumented) environment variable BULK_BUILD_CONF. This is a conservative extension.
2005-11-02Microoptimization: awk has BRE pattern matching, so use it instead oftv1-2/+2
"grep | awk".
2005-09-21Make the directory before creating the pkgsrc-results text file in it.reed1-2/+2
(Maybe the BUILDDATE is not needed since by default the directory has the specific date stamp in it.)