diff options
author | jmmv <jmmv> | 2003-04-17 11:10:41 +0000 |
---|---|---|
committer | jmmv <jmmv> | 2003-04-17 11:10:41 +0000 |
commit | 0f7e012cd276edd241f7b43a066c30d36b72a2c7 (patch) | |
tree | e2f8c5c6d5588071173868ba62b0f76d9595943d /pkgtools | |
parent | f689cbed046c870a3237088ac1a6a86411c1e294 (diff) | |
download | pkgsrc-0f7e012cd276edd241f7b43a066c30d36b72a2c7.tar.gz |
Initial import of dfdisk, version 1.0:
dfdisk is an utility that adds extra functionality to pkgsrc, allowing it
to fetch distfiles from multiple locations. It currently supports the
following methods: CD-ROM and Network. Others may be added in the future.
Diffstat (limited to 'pkgtools')
-rw-r--r-- | pkgtools/dfdisk/files/dfdisk.1 | 219 | ||||
-rw-r--r-- | pkgtools/dfdisk/files/dfdisk.conf | 18 | ||||
-rw-r--r-- | pkgtools/dfdisk/files/dfdisk.sh | 501 |
3 files changed, 738 insertions, 0 deletions
diff --git a/pkgtools/dfdisk/files/dfdisk.1 b/pkgtools/dfdisk/files/dfdisk.1 new file mode 100644 index 00000000000..7c333674fd8 --- /dev/null +++ b/pkgtools/dfdisk/files/dfdisk.1 @@ -0,0 +1,219 @@ +.\" $NetBSD: dfdisk.1,v 1.1.1.1 2003/04/17 11:10:41 jmmv Exp $ +.\" +.\" dfdisk - Fetch distfiles from multiple locations +.\" Copyright (c) 2003, Julio Merino <jmmv@netbsd.org> +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Neither the name of The NetBSD Foundation nor the names of its +.\" contributors may be used to endorse or promote products derived +.\" from this software without specific prior written permission. +.\" 3. Neither the name of author nor the names of its contributors may +.\" be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS +.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS +.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +.\" POSSIBILITY OF SUCH DAMAGE. +.\" +.Dd April 17, 2003 +.Dt DFDISK 1 +.Os +.Sh NAME +.Nm dfdisk +.Nd fetch distfiles from multiple locations +.Sh SYNOPSIS +.Nm +.Op Fl c Ar conf_file +.Ar target +.Op Ar target_args +.Sh DESCRIPTION +.Nm +is an utility that adds extra functionality to pkgsrc, allowing it to +fetch distfiles from multiple locations. +It currently supports the following methods: +.Bl -tag -width XXXXXXX +.It CD-ROM +The program manages a CD database where every disk is matched to a +series of distfiles. +Whenever a distfile is requested, the program will check if it is known +by the database and ask the user to place the needed disk in the drive. +.It Network +If a file is not found in the described database, the program fallbacks +to the network, fetching the file from there (the usual method in pkgsrc). +.El +.Pp +The following options are available: +.Bl -tag -width flag +.It Fl c +Path to the configuration file to use. +Overrides the default +.Pa @PKG_SYSCONFDIR@/dfdisk.conf . +.El +.Ss Configuration file details: dfdisk.conf +The configuration file is a simple shell script that sets some variable +values. +If the +.Fl c +flag is not given, +.Pa @PKG_SYSCONFDIR@/dfdisk.conf +is used as the default configuration file. +.Pp +The following variables ara available: +.Bl -tag -width XXXXXXXXX +.It Va CD_DIR +CD mount point. +Defaults to +.Pa /cdrom . +.It Va CD_MOUNT +Command to mount the CD media. +Defaults to +.Sq mount ${CD_DIR} . +.It Va CD_UMOUNT +Command to unmount the CD media. +Defaults to +.Sq umount ${CD_DIR} . +.It Va DBDIR +Database directory where all disk information is stored. +Defaults to +.Pa /var/db/dfdisk . +.It Va DISTDIR +Path to pkgsrc's distribution directory, where distfiles are stored +after downloading them. +Defaults to +.Pa /usr/pkgsrc/distfiles . +This +.Em must +match pkgsrc's +.Va DISTDIR +variable, as the program uses it to guess the +.Va DIST_SUBDIR +for some packages. +.It Va FTP_CMD +Command to fetch files from the network, if they were not found in the +disk database. +Defaults to +.Sq @FTP@ . +.El +.Ss Disk information file details: dfdisk.info +Every disk can contain a control file which is read by +.Nm +to automatically get information from it. +That file is called +.Pa dfdisk.info +and can be placed anywhere in the disk. +.Pp +This file can define the following variables: +.Bl -tag -width indent +.It Va dfdisk_name +Full name of disk. +Free form text field. +.It Va dfdisk_subdir +Relative path to where distfiles are stored in the CD. +.It Va dfdisk_type +Disk type. +The only value allowed for now is +.Sq cd . +This variable is not used for now, but must be there for future +compatibility. +.El +.Pp +WARNING: this file is a shell script and is directly sourced by +.Nm +to read its information. +Therefore it may contain malicious commands that will be executed +by this program whenever it is read. +If you have not authored the disk yourself, check the contents of +this file (if present) before proceeding. +.Ss Target description +.Nm +bases its behavior on the target given to it. +The following list details all work modes: +.Bl -tag -width indent +.It Ar add +Add the CD disk currently placed in the drive to the database. +If the disk contains a +.Pa dfdisk.info +file, it is automatically read; otherwise, all disk information is +requested to the user interactively. +.It Ar clean +Remove all local distfiles that are known to be stored in some disk. +This will leave all new files under +.Va DISTDIR , +so you can easily burn them to a new disk after creating a disk +information file with the +.Sq mkinfo +target. +.It Ar fetch Ar url +Fetch a distfile, given as an URL. +The program will search the database for the given file. +If it is found, the user will be required to put the requested media +on the drive. +If it is not found, +.Va FTP_CMD +will be used to fetch it from the network; this is why you need to pass +an URL to this target. +.It Ar mkinfo Op Ar info_file +Interactively create a +.Pa dfdisk.info +file. +It will be saved as +.Ar info_file +if given; if not, its name will be asked. +.It Ar remove +Interactively remove a disk from the database. +You will not want to use this target. +.El +.Sh EXAMPLES +To enable +.Nm +by default within pkgsrc, add the following lines to your +.Pa /etc/mk.conf +file: +.Bd -literal -offset indent +\&.if exists(@PREFIX@/bin/dfdisk) +FETCH_CMD= @PREFIX@/bin/dfdisk fetch +\&.endif +.Ed +.Pp +If you would like any user to be able to fetch files, you can use +.Xr sudo 8 +to allow them mount the CD drive. +To do it: +.Bd -literal -offset indent +CD_MOUNT="sudo mount /cdrom" +CD_UMOUNT="sudo umount /cdrom" +.Ed +.Pp +If you have more than one disk registered in your database and you +plan to leave your computer building stuff for a large period of time, +it is recommended that you fetch all required distfiles before starting +the build. +This way, it will not be interrupted waiting for any user action. +To do this, you can use pkgsrc's +.Sq fetch-list +target, which will call +.Nm +for each required file: +.Bd -literal -offset indent +$ cd category/package && make fetch-list | sh +.Ed +.Sh SEE ALSO +.Xr ftp 1 , +.Xr packages 7 , +.Xr sudo 8 +.Sh AUTHORS +.Nm +was written by +.An Julio Merino Aq jmmv@netbsd.org . diff --git a/pkgtools/dfdisk/files/dfdisk.conf b/pkgtools/dfdisk/files/dfdisk.conf new file mode 100644 index 00000000000..b5bb3081534 --- /dev/null +++ b/pkgtools/dfdisk/files/dfdisk.conf @@ -0,0 +1,18 @@ +# $NetBSD: dfdisk.conf,v 1.1.1.1 2003/04/17 11:11:42 jmmv Exp $ +# +# dfdisk.conf - system wide configuration file. See dfdisk(1) for more details. +# + +# Absolute path to database's directory. +#DBDIR="/var/db/dfdisk" + +# Path to pkgsrc's distdir. +#DISTDIR="/usr/pkgsrc/distfiles" + +# CD configuration. +#CD_DIR="/cdrom" +#CD_MOUNT="mount ${CD_DIR}" +#CD_UMOUNT="umount ${CD_DIR}" + +# Command to fetch files through the network. +#FTP_CMD="@FTP@" diff --git a/pkgtools/dfdisk/files/dfdisk.sh b/pkgtools/dfdisk/files/dfdisk.sh new file mode 100644 index 00000000000..fcdd2c65496 --- /dev/null +++ b/pkgtools/dfdisk/files/dfdisk.sh @@ -0,0 +1,501 @@ +#!@SH@ +# +# $NetBSD: dfdisk.sh,v 1.1.1.1 2003/04/17 11:12:52 jmmv Exp $ +# +# dfdisk - Fetch distfiles from multiple locations +# Copyright (c) 2003, Julio Merino <jmmv@netbsd.org> +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# 3. Neither the name of author nor the names of its contributors may +# be used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS +# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS +# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGE. +# + +ProgName="`basename $0`" +ProgVersion="@PKGVERSION@" +ConfFile="@PKG_SYSCONFDIR@/dfdisk.conf" +# dfdisk_* variables are also considered global + +# ------------------------------------------------------------------------ +# Miscellaneous functions +# ------------------------------------------------------------------------ + +err() { + local _code + + $CD_UMOUNT 2>/dev/null || true + _code=$1; shift + echo "$ProgName: $*" + exit $_code +} + +warn() { + echo "WARNING: $*" +} + +quotemeta() { + local qm_var qm_value + + qm_var="$1" + eval qm_value=\"\$$qm_var\" + + qm_char='-e s|\/|\\\/|g' + qm_char="$qm_char -e s/\./\\\./g" + qm_char="$qm_char -e s/\?/\\\?/g" + qm_char="$qm_char -e s/\+/\\\+/g" + qm_char="$qm_char -e s/\*/\\\*/g" + qm_char="$qm_char -e s/\\\"/\\\\\"/g" + + qm_value="`echo $qm_value | sed $qm_char`" + eval $qm_var=\"\$qm_value\" +} + +# ------------------------------------------------------------------------ +# CD routines +# ------------------------------------------------------------------------ + +cd_mount() { + $CD_MOUNT + if [ $? -ne 0 ]; then + err 1 "mount failed: $CD_MOUNT" + fi +} + +cd_umount() { + $CD_UMOUNT + if [ $? -ne 0 ]; then + err 1 "umount failed: $CD_UMOUNT" + fi +} + +cd_current_is() { + local _mounted _ret _cd _cds + + _mounted=yes + if [ -z "`mount | grep $CD_DIR`" ]; then + _mounted=no + $CD_MOUNT 2>/dev/null + if [ $? -ne 0 ]; then + return 1 + fi + fi + + _ret=1 + _cds="`cd $DBDIR; ls`" + for _cd in $_cds; do + _stamp="`md5 $CD_DIR | cut -d ' ' -f 4`" + if [ -n "`grep $_stamp $DBDIR/$_cd/info`" ]; then + _ret=0 + break + fi + done + + if [ "$_mounted" = "no" ]; then + $CD_UMOUNT 2>/dev/null + fi + + return $_ret +} + +# ------------------------------------------------------------------------ +# Add target +# ------------------------------------------------------------------------ + +do_add() { + local _info _stamp _dir + + if [ ! -d "$DBDIR" ]; then + mkdir -p "$DBDIR" >/dev/null 2>&1 + if [ $? -ne 0 ]; then + err 1 "cannot create \`$DBDIR'; permission denied?" + fi + elif [ ! -w "$DBDIR" ]; then + err 1 "cannot write to \`$DBDIR'; permission denied?" + fi + + cd_mount + _stamp="`md5 $CD_DIR | cut -d ' ' -f 4`" + echo "Media ID: $_stamp" + + _dir="$DBDIR/$_stamp" + if [ -d "$_dir" ]; then + cd_umount + err 1 "this CD is already registered" + fi + + echo "Scanning CD (searching a dfdisk.info file)..." + _info="`find $CD_DIR -name dfdisk.info -print`" + + # Fill in the CD info file + if [ -n "$_info" ]; then + echo "Found, using $_info:" + . "$_info" + echo "- Disk name: $dfdisk_name" + echo "- Path to distfiles: $dfdisk_subdir" + mkdir -p $_dir + cp -f "$_info" $_dir/info + chmod 644 $_dir/info + else + echo "No dfdisk.info file found; enter data manually." + ask_info + if [ ! -d "$CD_DIR/$dfdisk_subdir" ]; then + err 1 "the specified directory cannot be found" + fi + mkdir -p $_dir + write_info $_dir/info + fi + + # Store stamp in local control file + echo >> $_dir/info + echo "# Appended by @PKGBASE@ @PKGVERSION@" >> $_dir/info + echo "# `date`" >> $_dir/info + echo "dfdisk_stamp=$_stamp" >> $_dir/info + + # Generate file list + echo "Generating file list..." + ( cd $CD_DIR && find $dfdisk_subdir -type f -print > $_dir/contents ) + chmod 644 $_dir/contents + cd_umount +} + +# ------------------------------------------------------------------------ +# Clean target +# ------------------------------------------------------------------------ + +do_clean() { + local _cds _cd _files _file _f _found + + if [ -d $DBDIR ]; then + _cds="`cd $DBDIR; ls`" + if [ -z "$_cds" ]; then + echo "No known disks; nothing to be done." + else + echo "Removing files..." + _files="`find $DISTDIR -type f -print`" + for _file in $_files; do + _f="`echo $_file | sed -e s,$DISTDIR,,`" + quotemeta _f + _found="" + for _cd in $_cds; do + if [ -n "`egrep $_f\$ $DBDIR/$_cd/contents`" ]; then + _found=yes + break + fi + done + if [ -n "$_found" ]; then + rm -f $_file + fi + done + echo "Removing empty directories..." + rmdir -p `find $DISTDIR -type d -print` 2>/dev/null || true + fi + else + echo "No known disks; nothing to be done." + fi +} + +# ------------------------------------------------------------------------ +# Fetch target +# ------------------------------------------------------------------------ + +do_fetch() { + local _cds _cd _qdn _url _subdir _distname _found _res _key + + _url="$1" + + if [ -z "`pwd | grep $DISTDIR`" ]; then + warn "you are not inside the distdir tree!" + echo + _subdir="" + else + _subdir="`pwd | sed -e s,$DISTDIR,,`/" + fi + + set -- `echo $_url | tr '/' ' '` + while [ $# -gt 1 ]; do shift; done + _distname="$1" + + if [ ! -d $DBDIR ]; then + echo "No disks registered in the database." + fetch_ftp $_url + return + fi + + _cds="`cd $DBDIR; ls`" + _found="" + _qdn="$_distname" + quotemeta _qdn + for _cd in $_cds; do + _res=`egrep "$_subdir$_qdn\$" $DBDIR/$_cd/contents` + if [ -n "$_res" ]; then + . $DBDIR/$_cd/info + _found="$_cd"; break + fi + done + + if [ -z "$_found" ]; then + echo "$_distname not found in the database." + fetch_ftp $_url + else + . "$DBDIR/$_found/info" + echo "$_distname found in the database." + + while ! cd_current_is $dfdisk_stamp; do + echo "Wrong disk in drive (or no disk). You can:" + echo "- Insert the CD given below and press [RETURN]" + echo " $dfdisk_name" + echo "- Type \`1' and press [RETURN] to refetch the file from the network" + echo " $_url" + printf "Your choice> " + read _key + if [ "$_key" = "1" ]; then + fetch_ftp $_url + return + fi + # The default action (fetch from CD) is handled outside the + # while loop. + done + + echo "Copying `basename $1` to `pwd`" + cd_mount + cp "$CD_DIR/$dfdisk_subdir/$_subdir/$_distname" . + cd_umount + fi +} + +fetch_ftp() { + echo "Using \`$FTPCMD' to fetch $1" + $FTPCMD $1 +} + +# ------------------------------------------------------------------------- +# Mkinfo target +# ------------------------------------------------------------------------- + +do_mkinfo() { + local _file + + _file="$1" + + ask_info + + if [ -z "$_file" ]; then + printf "Enter file name [$DISTDIR/dfdisk.info]: " + read _file + if [ -z "$_file" ]; then + _file="$DISTDIR/dfdisk.info" + fi + fi + + write_info $_file + echo "File saved as \`$_file'" +} + +write_info() { + cat > $1 <<EOF +# +# File automatically generated by @PKGBASE@ @PKGVERSION@ +# `date` +# + +dfdisk_name="$dfdisk_name" +dfdisk_subdir="$dfdisk_subdir" +dfdisk_type="cd" +EOF +} + +ask_info() { + dfdisk_name="" + printf "Enter disk name [pkgsrc distfiles `date +%Y%m%d`]: " + read dfdisk_name + if [ -z "$dfdisk_name" ]; then + dfdisk_name="pkgsrc distfiles `date +%Y%m%d`" + fi + + dfdisk_subdir="" + while [ -z "$dfdisk_subdir" ]; do + printf "Enter relative path to distfiles: " + read dfdisk_subdir + done +} + +# ------------------------------------------------------------------------ +# Remove target +# ------------------------------------------------------------------------ + +do_remove() { + local _cds _cd _count _ans + + if [ ! -d "$DBDIR" ]; then + echo "No known disks; nothing to be done." + return 0 + fi + + _cds="`cd $DBDIR; ls`" + if [ -z "$_cds" ]; then + echo "No known disks; nothing to be done." + return 0 + fi + + if [ ! -w "$DBDIR" ]; then + err 1 "cannot write to \`$DBDIR'; permission denied?" + fi + + # Print menu + echo "List of disks in the database:" + echo "0 - Exit" + _count=1 + for _cd in $_cds; do + . $DBDIR/$_cd/info + echo "$_count - $dfdisk_name" + _count=$(($_count + 1)) + done + + # Wait for an answer + _ans="" + while [ -z "$_ans" ]; do + printf "Remove> " + read _ans + done + + # Parse answer + if [ "$_ans" -ne 0 ]; then + _count=1 + for _cd in $_cds; do + if [ "$_count" -eq "$_ans" ]; then + . $DBDIR/$_cd/info + rm -f $DBDIR/$_cd/contents + rm -f $DBDIR/$_cd/info + rmdir $DBDIR/$_cd + echo "\`$dfdisk_name' removed successfully" + break + fi + _count=$(($_count + 1)) + done + fi +} + +# ------------------------------------------------------------------------ +# Main program +# ------------------------------------------------------------------------ + +usage() { + echo "usage: $ProgName [-c conf_file] target [target_args]" + echo + echo "Available targets:" + echo " add Add a disk to the database interactively." + echo " clean Remove local distfiles known to be in disks." + echo " fetch url Fetch a distfile from a disk or the network." + echo " mkinfo [file] Interactively create a dfdisk.info file." + echo " remove Remove a disk from the database interactively." + echo + echo "See dfdisk(1) for more information." + exit $1 +} + +show_banner() { + echo "@PKGBASE@ $ProgVersion ==> $Target" +} + +silent_umount() { + $CD_UMOUNT 2>/dev/null || true +} + +args=`getopt c: $*` +if [ $? -ne 0 ]; then + usage 2 +fi +set -- $args +while [ $# -gt 0 ]; do + case "$1" in + -c) + ConfFile="$2"; shift + ;; + --) + shift; break + ;; + esac + shift +done + +# Read configuration file +if [ -f "$ConfFile" ]; then + . $ConfFile +fi + +# Set configuration defaults +: ${DBDIR:=/var/db/dfdisk} +: ${CD_DIR:=/cdrom} +: ${CD_MOUNT:="mount ${CD_DIR}"} +: ${CD_UMOUNT:="umount ${CD_DIR}"} +: ${DISTDIR:=/usr/pkgsrc/distfiles} +: ${FTPCMD:="@FTP@"} + +if [ $# -eq 0 ]; then + echo "$ProgName: no target specified" + usage 2 +fi + +Target="$1"; shift +TargetArgs="$*" + +trap "silent_umount ; echo ; exit 1" INT QUIT + +case "$Target" in + add) + show_banner + do_add + ;; + clean) + show_banner + do_clean + ;; + fetch) + if [ -z "$TargetArgs" ]; then + usage 2 + fi + + while [ $# -gt 0 -a -z "`echo $1 | egrep '^(ftp|http)://'`" ]; do + shift + done + if [ $# -eq 0 ]; then + err 1 "the \`fetch' target expects an url" + fi + show_banner + do_fetch $1 + ;; + mkinfo) + show_banner + do_mkinfo $1 + ;; + remove) + show_banner + do_remove + ;; + *) + echo "$ProgName: unknown target \`$Target'" + usage 2 + ;; +esac + +exit |