summaryrefslogtreecommitdiff
path: root/mk/fetch/fetch.mk
AgeCommit message (Collapse)AuthorFilesLines
2011-02-10Disable certificate checks when using "curl". This fixes the build oftron1-2/+2
the "youtube-dl" package.
2011-02-08Add FETCH_USE_IPV4_ONLY option to force connecting to only IPv4 addresses.obache1-5/+12
2011-02-08Add `--location' option for curl to follow HTTP redirect.obache1-2/+2
2011-02-08Let curl to silent fail with server error.obache1-1/+2
fixes a part of PR#44528.
2011-01-22For curl, "=" is not required for longname options with value.obache1-3/+3
2011-01-18For PR #44396.reed1-2/+2
"FETCH_CMD is the program used to fetch files for FETCH_USING=custom."
2010-06-27Add FETCH_TIMEOUT option to limit the time connections may hang duringjoerg1-5/+13
fetch. Requested by spz@, OK by wiz@
2009-07-15Add depends-fetch and depends-checksum. They work likejoerg1-1/+11
show-depends-options, e.g. recurse. make fetch-list does not provide the same functionality as it doesn't deal well with broken mirror.
2009-03-17Replaced _PKG_SILENT and _PKG_DEBUG with RUN.rillig1-7/+7
2009-02-17Fix C&P error. Pointed out by OBATA Akio.joerg1-2/+2
2009-02-14Don't use += to append to an empty variable. If PASSIVE_FTP is notjoerg1-5/+3
defined, it would still leave the variable with a non-empty value and that would be passed down to the fetch command when running fetch-list.
2009-02-10Optionally request passive FTP for wget.joerg1-2/+2
2009-02-10Change the way the fetch process is organised as proposed on tech-pkg:joerg1-19/+26
- Introduce FETCH_USING, which specifies the mechanism to use; possible values are ftp, fetch, curl, wget, manual and custom. - Depend on the correct tool/program if not using manual or custom. - For manual, just use /usr/bin/false to bail out if a distfile is missing. - For custom, FETCH_CMD and related variables are used as before. - Default value is ftp.
2008-10-08Allow use of fetch as FETCH_CMD.joerg1-1/+6
2008-09-24Fix a typo in comment.obache1-2/+2
2008-09-07Add FETCH_*_ARGS templates for curl.obache1-1/+7
Should be solution for PR 39208.
2008-05-22Move digest dependency into checksum and fetch part.joerg1-25/+1
Make _ALL_FILES and in turn _CKSUMFILES available earlier. Use it to only depend on digest if _CKSUMFILES is non-empty. NO_CHECKSUM will now only skip the checksum and checksum-phase targets. FAILOVER_FETCH will independently check the sums on distfiles and depend on digest, fixing the remaining issues originally raised in PR 34914.
2008-04-03Move handling of pkg_install version into flavor. Add an explicitjoerg1-1/+2
phase pkg_install-depends before bootstrap-depends that just tries to install a new pkg_install if the current version is too old. Still keep it as bootstrap dependency for the bulk build code. For NetBSD, PKG_TOOLS_BIN has to be computed in shell code due to a make bug. OK: jlam@
2007-12-01Removed _INTERACTIVE_COOKIE, after asking on tech-pkg if anyone needsrillig1-2/+1
it. It had only been used to change the "nerrors" variable in bulk builds, but this variable is not used further anyway.
2007-12-01If a package defines FETCH_MESSAGE, show that instead of the genericrillig1-2/+2
instructions.
2007-09-06If DISTDIR points to a non-existing location and a package has neitherjoerg1-1/+6
PATCHFILES nor DISTFILES, make fetch would complain about an unbuildable dependency. Fix this by explicitly checking that _ALLFILES is not empty. Reported by Sean Boudreau.
2007-08-14Checksum checks only make sense when distinfo is also present, sojoerg1-3/+2
move this into the conditional.
2007-08-01Teach mirror-distfiles to use emul-fetch if EMUL_PLATFORMS is defined sojlam1-1/+3
that mirror sites will grab all of the distfiles, not just the ones that are for the native platform.
2007-07-14No need to pass "." as directory, it is default value.obache1-3/+1
2007-07-14Allow empty DIST_SUBDIR, need to fix PR 35494.obache1-3/+7
2007-06-12Define some variables for using FETCH_CMD=wget.obache1-1/+6
2006-12-29Silence some longstanding noise in mail/qmail, caused by the factschmonz1-1/+2
that its ${DISTFILES} is also a common ${PATCHFILES}, resulting in the file being listed twice in ${_ALLFILES}. This one-line change removes duplicate filenames from ${_ALLFILES}, mirroring an identical transformation to ${ALLFILES} that used to do the trick. (The fetch code, obviously, has since changed a bit.) What the noise looked like: $ cd pkgsrc/mail/qmail $ make clean PKG_OPTIONS="netqmail" make: "/home/build/trees/pkgsrc-current/mk/fetch/fetch.mk" line 133: warning: duplicate script for target "/home/build/trees/pkgsrc-current/distfiles/netqmail-1.05.tar.gz" ignored make: "/home/build/trees/pkgsrc-current/mk/fetch/fetch.mk" line 133: warning: using previous script for "/home/build/trees/pkgsrc-current/distfiles/netqmail-1.05.tar.gz" defined here ===> Cleaning for qmail-1.03nb12 $ Tested in a bulk build by joerg@.
2006-11-25set ORS to space for _MASTER_RAND_AWK, so that the output would be thejdolecek1-3/+3
in the format MASTER_SORT_REGEX expects when MASTER_SORT_RANDOM is off, in order for MASTER_SORT to actually work when MASTER_SORT_RANDOM=YES (default nowadays) XXX testing stuff before committing would surely be a good idea
2006-10-08pkg/34695: Static list of package master sites may (and often will)mishka1-7/+21
lead to overloads of very first distribution site. Moreover, if first site in the list is not available (often seen for sourceforge mirrors) you have to wait for timeout each time. To distribute load on master distribution sites and to make second problem not so annoying randomly intermix list of MASTER_SITES with MASTER_SORT_RANDOM feature. Any of MASTER_SORT and MASTER_SORT_REGEX can be applied later. The feature is turned ON by default and is disabled for PKG_DEVELOPERs or if MASTER_SORT_RANDOM=no.
2006-10-07Meta-packages that had DIST_SUBDIR set (for whatever reason) tried torillig1-5/+5
fetch files with an empty name, for example "geda/". The problem was that the expression ${LIST:S,^,${DIST_SUBDIR}/,} results in ${DIST_SUBDIR}/ when ${LIST} is empty. This is surprising but matches the documentation in the manual page, so this cannot be called a bug. The proper fix is to use ${LIST:@f@${DIST_SUBDIR}/${f}@} instead. Noticed by Don Woodstock on #netbsd-code.
2006-07-27Rename "SITES_* to "SITES.*" for file-specific lists of sites from whichjlam1-7/+4
to fetch the file. This completes the renaming described in revision 1.1799 of bsd.pkg.mk.
2006-07-27All of the other *SITES* variables are plural, so rename DYNAMIC_SITE_*jlam1-5/+5
to DYNAMIC_SITES_* for consistency.
2006-07-27Apply the "convention over configuration" principle:jlam1-3/+5
If ${FILESDIR}/getsite.sh exists, then use it to determine the fetch URL for each of the distfiles for the package. Otherwise, use SITE_<file> and MASTER_SITES, in order, to determine the URL for each distfile. If the script path differs from ${FILESDIR}/getsite.sh, then set DYNAMIC_SITE_SCRIPT to the full path to that script. Remove the need to set DYNAMIC_MASTER_SITES explicitly in the package Makefile for: graphics/ns-cult3d wm/sawfish-themes www/apache-tomcat55 www/jakarta-tomcat4 www/jakarta-tomcat5
2006-07-27Documented DIST_PATH and fixed two typos (${TARGET} is not ${.TARGET}).rillig1-14/+12
2006-07-19Only pass -v to the fetch script if PKG_VERBOSE is set.jlam1-2/+2
2006-07-19Restore original quoting from old fetch.mk for _ORDERED_SITES to resolvejlam1-3/+5
the problem with MASTER_SORT*. XXX The whole MASTER_SORT* thing should just be a standalone script.
2006-07-19Make some variables "private":jlam1-10/+10
MASTER_SORT_AWK SORT_SITES_CMD ORDERED_SITES These variables are purely internal to pkgsrc.
2006-07-19Work around incomprehensible quoting mess surrounding MASTER_SORT* byjlam1-2/+2
in do-fetch-file (formerly they were inserted by the old _FETCH_FILE inserting some quotes that are no longer added by the fetch command list macro). This makes fetching with MASTER_SORT* set work again.
2006-07-19PASSIVE_FETCH only affects "ftp", so move it to the ftp-related FETCH_*jlam1-8/+7
variables.
2006-07-18Convert _FETCH_MESSAGE to plain old FETCH_MESSAGE. Remove _FETCH_MESSAGEjlam1-4/+1
processing from mk/fetch/*.mk.
2006-07-18Overhaul fetch module:jlam1-337/+199
* All the smarts is now encapsulated in the "fetch" script. The fetch script understands how to use the distinfo file (if specified) to look up the size and checksums of the file to fetch and will use that information to verify checksums of the fetched files or resume transfers of interrupted fetches. * Move the default settings for FETCH_RESUME_ARGS and FETCH_OUTPUT_ARGS for "ftp" from mk/defaults/mk.conf into mk/fetch/fetch.mk. We rewrite it to avoid needing conditional statements. * Avoid spawning a new make(1) process just to mirror a distfile. * Split out fetch-list targets into a separate file fetch-list.mk. These targets should probably be moved into a standalone script. * Fix distclean target to properly remove partial downloads.
2006-07-17Remove duplicate files from ${ALLFILES}. This doesn't affect theschmonz1-1/+2
"fetch" target, but it does affect the "checksum" target's shell script, which errors out on the second occurrence of a file. The shell script should perhaps also be fixed, but it seems sensible regardless for ${ALLFILES} not to contain duplicate filenames. As a side effect, the file list is sorted. Regression found by building mail/qmail with the "qmail-netqmail" option, which adds to PATCHFILES a file that's already in ${DISTFILES}. Arguably this is gross, but it worked before, and now works again. Tested on my usual pkg_comp(8) build of 200+ packages, with an initially empty ${DISTDIR} and ${PACKAGES}. Thanks seb@ for the more idiomatic make(1) construction.
2006-07-15Split the variable BUILD_DEFS into those that are defined by packagesrillig1-2/+2
and those that are defined by the infrastructure (_BUILD_DEFS). This allows the build-defs-message target to be moved to the end of bsd.pkg.mk. Now it prints the correct result even in unprivileged builds, which had been wrong due to the order in which the files have been included. For example, ${UNPRIVILEGED_USER} was displayed as (not defined) although its value was defined, which could be checked with "bmake show-var". Tested with one package that _does_ define BUILD_DEFS and with one that doesn't. The behavior stays the same.
2006-07-13DIGEST is no longer defined, so use TOOLS_DIGEST instead.jlam1-2/+2
2006-07-13* Add a new stage "bootstrap-depends" that happens before all otherjlam1-3/+3
stages, and that installs dependencies listed in BOOTSTRAP_DEPENDS. The bootstrap-depends step works just like the normal depends step and honors the value of DEPENDS_TARGET. It's now possible to add dependencies solely to facilitate fetching the distfiles, e.g. BOOTSTRAP_DEPENDS+= curl-[0-9]*:../../www/curl * Teach the tools framework about ":bootstrap" as a tools modifier which indicates the tool should be added as a dependency via BOOTSTRAP_DEPENDS. * Add "digest" to the tools framework. * Use USE_TOOLS+=digest:bootstrap to force pkgsrc to install digest before anything else. Get rid of unused "uptodate-digest" target and related digest version-checking code. * Finish the refactoring work: split checksum-related code out of bsd.pkg.mk and into pkgsrc/mk/checksum and replace the "checksum" target command list with a script that does all the real work. * Make DIGEST_ALGORITHMS and PATCH_DIGEST_ALGORITHM into private variables by prepending them with an underscore. Also, rename _PATCH_DIGEST_ALGORITHM to _PATCH_DIGEST_ALGORITHMS and adjust the makepatchsum target to allow that variable to contain a list of algorithms, all of which are used when creating the patch checksums for ${DISTINFO_FILE}.
2006-07-05Make the check-vulnerable target more self-sufficient, by moving somejlam1-20/+2
of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that check-vulnerable can be used as a source target. Make check-vulnerable a source target for every phase of the build workflow, which ensures that it is always run if the user starts a new phase from the command line. Fix the cookie-generation targets so that they don't append, only overwrite to the cookie file. This works around potential problems due to recursive makes. Move the cookie checks so that they surround the corresponding phase target. The presence of the cookie should now inform the make process to avoid doing any processing of phases that occur before the phase corresponding to the cookie.
2006-06-20If FAILOVER_FETCH is defined, then we need the digest binary to alreadyjlam1-2/+2
be installed, so let uptodate-digest do that crazy thing it does.
2006-06-20Fix FAILOVER_FETCH usage to actually match documentation -- it'sjlam1-2/+2
supposed to be just defined/undefined, but was previously being used as non-empty/emtpy.
2006-06-06Improve the warning message if the vulnerabilities file is not found.jlam1-5/+5
2006-06-06Refactor "fetch" and "extract" code into correspondingly namedjlam1-0/+453
subdirectories of pkgsrc/mk. Move the following files around for locality: pkgsrc/mk/scripts/extract -> pkgsrc/mk/extract/extract pkgsrc/mk/bsd.sites.mk -> pkgsrc/mk/fetch/sites.mk Also get rid of the recursive make for the "fetch" and "extract" targets. This basically merges the "fetch" and "extract" phases into the "patch" phase. There is still much more work to do to simplify the fetch code, but this is a good start.
generated by cgit v1.2.3 (git 2.39.1) at 2025-03-30 01:00:41 +0000