summaryrefslogtreecommitdiff
path: root/www/crawl/distinfo
AgeCommit message (Collapse)AuthorFilesLines
2015-11-04Add SHA512 digests for distfiles for www categoryagc1-1/+2
Problems found locating distfiles: Package haskell-cgi: missing distfile haskell-cgi-20001206.tar.gz Package nginx: missing distfile array-var-nginx-module-0.04.tar.gz Package nginx: missing distfile encrypted-session-nginx-module-0.04.tar.gz Package nginx: missing distfile headers-more-nginx-module-0.261.tar.gz Package nginx: missing distfile nginx_http_push_module-0.692.tar.gz Package nginx: missing distfile set-misc-nginx-module-0.29.tar.gz Package nginx-devel: missing distfile echo-nginx-module-0.58.tar.gz Package nginx-devel: missing distfile form-input-nginx-module-0.11.tar.gz Package nginx-devel: missing distfile lua-nginx-module-0.9.16.tar.gz Package nginx-devel: missing distfile nginx_http_push_module-0.692.tar.gz Package nginx-devel: missing distfile set-misc-nginx-module-0.29.tar.gz Package php-owncloud: missing distfile owncloud-8.2.0.tar.bz2 Otherwise, existing SHA1 digests verified and found to be the same on the machine holding the existing distfiles (morden). All existing SHA1 digests retained for now as an audit trail.
2005-02-24Add RMD160 checksums.wiz1-1/+2
2005-01-18Initial import of crawl-0.4 into the NetBSD Packages Collection.peter1-0/+7
The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main features of crawl are: * Saves encountered images or other media types * Media selection based on regular expressions and size contraints * Resume previous crawl after graceful termination * Persistent database of visited URLs * Very small and efficient code * Asynchronous DNS lookups * Supports robots.txt