diff options
Diffstat (limited to 'pkgtools/R2pkg')
| -rw-r--r-- | pkgtools/R2pkg/Makefile | 4 | ||||
| -rw-r--r-- | pkgtools/R2pkg/files/R2pkg.8 | 206 | ||||
| -rwxr-xr-x | pkgtools/R2pkg/files/R2pkg.sh | 1451 | ||||
| -rw-r--r-- | pkgtools/R2pkg/files/RELEASE | 16 |
4 files changed, 1405 insertions, 272 deletions
diff --git a/pkgtools/R2pkg/Makefile b/pkgtools/R2pkg/Makefile index c4d229036d7..010fae6c3a8 100644 --- a/pkgtools/R2pkg/Makefile +++ b/pkgtools/R2pkg/Makefile @@ -1,7 +1,7 @@ -# $NetBSD: Makefile,v 1.3 2019/06/24 13:46:04 brook Exp $ +# $NetBSD: Makefile,v 1.4 2019/08/01 13:11:08 brook Exp $ # -VERS= 0.4 +VERS= 0.5 PKGNAME= R2pkg-${VERS} CATEGORIES= pkgtools diff --git a/pkgtools/R2pkg/files/R2pkg.8 b/pkgtools/R2pkg/files/R2pkg.8 index c882a5e5c15..0e300d69674 100644 --- a/pkgtools/R2pkg/files/R2pkg.8 +++ b/pkgtools/R2pkg/files/R2pkg.8 @@ -1,4 +1,4 @@ -.\" $NetBSD: R2pkg.8,v 1.5 2019/06/24 13:46:04 brook Exp $ +.\" $NetBSD: R2pkg.8,v 1.6 2019/08/01 13:11:08 brook Exp $ .\" .\" Copyright (c) 2014,2015,2016,2017,2018,2019 .\" Brook Milligan. All rights reserved. @@ -36,61 +36,97 @@ .Nd create a pkgsrc package for an R package .Sh SYNOPSIS .Nm -.Op Fl DehqruVv +.Op Fl cDehqruVv .Op Fl E Ar editor .Op Fl M Ar maintainer_email -.Op Fl R Ar dependency_file .Op Ar package .Sh DESCRIPTION .Nm -takes the name of an R +creates or updates a +.Xr pkgsrc 7 +package for the corresponding R +.Ar package . +The +.Xr pkgsrc 7 +package will be created in the current directory, +which by convention should be named +.Pa R-package . +If the user does not supply the .Ar package -that should be turned into a +on the command line, then +.Nm +will use the basename of the current directory +as the name of the R package after stripping the leading +.Pa R- . +.Pp +.Nm +tries to make as complete a .Xr pkgsrc 7 -package in the current directory. -The generated package includes a -.Pa Makefile -and a mostly complete set of files to get the package going. -However, +package as possible, and in the case of updating tries to maintain +as much of the original as possible. +Nevertheless, the files should be reviewed and perhaps adjusted manually after .Nm has finished its job. To help with this, the original versions of files created by .Nm -are preserved for reference. +are preserved for reference, but should be removed before commiting the +.Xr pkgsrc 7 +package. .Pp .Nm -is intended to help the process of creating a package but is not +is intended to help the process of creating or updating a package but is not intended to fly on autopilot. .Pp The -.Pa Makefile -contains all of the variables required for an R package, with -information being taken from the package's +.Pa DESCR +file is populated with information from the package's .Pa DESCRIPTION file on CRAN, which is automatically fetched by .Nm . Likewise, the -.Pa DESCR -file is populated with information from the same file. -After .Pa Makefile -and +contains all of the variables required for an R package, with +information being taken from the same +.Pa DESCRIPTION +file. +In the case of updating, a file +.Pq Pa DESCR.new +containing the new description is created, and +as much as possible of the original +.Pa Makefile +is retained. +After .Pa DESCR +and +.Pa Makefile are completed, .Nm -fetches the package and computes its checksum via the +fetches the distfile and computes its checksum via the .Ic makesum -target. -.Pp -If the user does not supply the -.Ar package -on the command line, then -.Nm -will prompt for it. +and +.Ic makepatchsum +targets. +Finally, a +.Pa buildlink3.mk +file may be created if necessary. .Pp The following options are available: .Bl -tag -width indent +.It Fl c +Create the +.Xr pkgsrc 7 +package +.Po +and any dependencies if the +.Fl r +option is given +.Pc . +This is the default if neither +.Fl c +nor +.Fl u +is given. .It Fl D Write the package's description into .Pa DESCRIPTION . @@ -103,55 +139,72 @@ Use .Ar editor instead of the user's default editor. .It Fl e -Do not edit Makefile and DESCR. +Do not present the +.Pa Makefile +and +.Pa DESCR +files for editing. .It Fl h Produce a short help message. .It Fl M Ar maintainer_email Set the maintainer email address for any newly created packages. +By default, the email address will be +.Aq pkgsrc-users@NetBSD.org . .It Fl q Do not produce status messages along the way. -.It Fl R Ar dependency_file -Process dependency packages recursively using -.Pa dependency_file -to record dependency information. -This option is intended for -internal use only to implement recursion over dependencies. -Instead, the -.Fl r -option should be used to select recursion. .It Fl r Process dependency packages recursively. +Dependency packages that do not exist will be created in the +.Pa pkgsrc/wip +directory, and processed by +.Nm +recursively. +Dependency packages that do exist will only be processed when updating. A .Pa depends file is created that lists the dependencies in .Xr tsort 1 -order. -This is likely the best order for testing the resulting +order, which is likely the best for testing the resulting dependency packages. In particular, a useful strategy for creating +.Po +or updating with the +.Fl u +option +.Pc packages recursively is to (i) run .Nm with the .Fl r -option, (ii) fix all the leaf packages, (iii) rerun +option, +(ii) move any newly created packages from +.Pa pkgsrc/wip +into appropriate categories, +(iii) fix each package as needed, and +(iv) rerun .Nm -with the +with both the .Fl r -option, and (iv) continue until all packages are created. -Repeating runs of -.Nm -once the leaf dependencies are moved into the correct categories will -allow the program to capture the correct dependencies in each -.Pa Makefile . +and +.Fl u +options. +The second +.Pq and subsequent +run(s) will correct each package's +.Pa Makefile +with the correct category information and the correct dependency directories. .It Fl u Update the .Xr pkgsrc 7 -package. -.Nm -must be run in an R package directory, which is assumed to be named as -.Pa R-package . -The name of the package to be updated will be taken from the directory -name and need not be given on the command line. +package +.Po +and any dependencies if the +.Fl r +option is given +.Pc . +This has no effect if a package +.Pq including a dependency +is being newly created. .It Fl V Print the version. .It Fl v @@ -178,29 +231,36 @@ Use the following commands to create a .Xr pkgsrc 7 package for the R package .Ar foo : -.Dl cd pkgsrc/math +.Dl cd pkgsrc/category .Dl mkdir R-foo .Dl cd R-foo -.Dl R2pkg foo -Remember to test the resulting package and remove any extra files -created by -.Nm . +.Dl R2pkg -c foo Adding .Fl r to the command line will also recursively create all necessary dependencies in the -.Pa wip -directory. +.Pa pkgsrc/wip +directory if they do not already exist. .Pp Use the following commands to update a .Xr pkgsrc 7 package (and dependencies) for the R package .Ar foo : -.Dl cd pkgsrc/math/R-foo -.Dl R2pkg -r -u -Packages built for any new dependencies will be added to the -.Pa wip +.Dl cd pkgsrc/category/R-foo +.Dl R2pkg -u -r +Packages created for any new dependencies will be added to the +.Pa pkgsrc/wip directory. +.Pp +Remember to test the resulting package(s) and remove any extra files +created by +.Nm . +If new packages are created in the +.Pa pkgsrc/wip +directory, they should be moved to appropriate categories +and +.Nm +should be rerun to update categories and paths. .Sh SEE ALSO .Xr pkgsrc 7 .Sh AUTHORS @@ -212,14 +272,16 @@ was written by does not create a finished package; instead, it facilitates the task. However, the resulting files must be reviewed and possibly edited by hand. -Although dependencies are included in the generated -.Pa Makefile , -they are not fully analyzed and likely have incorrect category -information. Packages created recursively for dependencies are placed -in a newly created, nonstandard -.Pa pkgsrc/R -category directory and must be moved to appropriate category -directories within +in the +.Pa pkgsrc/wip +directory; +they must be moved to appropriate category directories within .Xr pkgsrc 7 and checked for correctness. +Therefore, although dependencies are included in the generated +.Pa Makefile , +they will have incorrect category information +unless moved to an appopriate category and +.Nm +rerun. diff --git a/pkgtools/R2pkg/files/R2pkg.sh b/pkgtools/R2pkg/files/R2pkg.sh index ab3fa404067..c39823e233c 100755 --- a/pkgtools/R2pkg/files/R2pkg.sh +++ b/pkgtools/R2pkg/files/R2pkg.sh @@ -1,5 +1,5 @@ #!/bin/sh -# $NetBSD: R2pkg.sh,v 1.4 2019/06/24 13:46:04 brook Exp $ +# $NetBSD: R2pkg.sh,v 1.5 2019/08/01 13:11:08 brook Exp $ # # Copyright (c) 2014,2015,2016,2017,2018,2019 # Brook Milligan. All rights reserved. @@ -36,7 +36,7 @@ VERS="@VERS@" R2PKG=${0} -USAGE="${NAME} [-DVehqrv] [-E editor] [-M maintainer] [-R dependency_file] [package] -- create an R package for pkgsrc" +USAGE="${NAME} [-cDehqruVv] [-E editor] [-M maintainer] [package] -- create an R package for pkgsrc" : ${CRAN_URL:=ftp://cran.r-project.org} : ${PKGEDITOR:=${EDITOR:=vi}} @@ -44,61 +44,65 @@ USAGE="${NAME} [-DVehqrv] [-E editor] [-M maintainer] [-R dependency_file] [pack # Substituted by pkgsrc at pre-configure time. MAKE=@MAKE@ EDIT=1 +LEVEL=0 MAINTAINER_EMAIL=pkgsrc-users@NetBSD.org +PID=$$ QUIET=0 RECURSIVE=0 -TOP_LEVEL=1 UPDATE=0 VERBOSE=0 DESCRIPTION_CONNECTION=connection -while getopts DE:M:R:Vehqruv f +while getopts cDehqruVvE:M:L:P: f do - case $f in + case ${f} in + # options without arguments + c) UPDATE=0; ARGS="${ARGS} -c";; D) DESCRIPTION=yes; DESCRIPTION_CONNECTION="'DESCRIPTION'"; ARGS="${ARGS} -D";; - E) PKGEDITOR=${OPTARG}; ARGS="${ARGS} -E ${PKGEDITOR}";; - M) MAINTAINER_EMAIL=${OPTARG}; ARGS="${ARGS} -M ${MAINTAINER_EMAIL}";; - R) DEPENDENCY_LIST=${OPTARG}; RECURSIVE=1; TOP_LEVEL=0; ARGS="${ARGS} -R ${DEPENDENCY_LIST}";; - V) echo "${NAME} v${VERS}"; exit 0;; e) EDIT=0; ARGS="${ARGS} -e";; h) echo ${USAGE}; exit 0;; q) QUIET=1; ARGS="${ARGS} -q";; r) RECURSIVE=1; RECURSIVE_MESSAGE=1; ARGS="${ARGS} -r";; u) UPDATE=1; ARGS="${ARGS} -u";; + V) echo "${NAME} v${VERS}"; exit 0;; v) VERBOSE=$((${VERBOSE}+1)); ARGS="${ARGS} -v";; + # options taking arguments + E) PKGEDITOR=${OPTARG}; ARGS="${ARGS} -E ${PKGEDITOR}";; + M) MAINTAINER_EMAIL=${OPTARG}; ARGS="${ARGS} -M ${MAINTAINER_EMAIL}";; + # options for recursion; only for internal use + L) LEVEL=${OPTARG};; + P) PID=${OPTARG};; + # unknown options \?) echo ${USAGE}; exit 1;; esac done shift `expr ${OPTIND} - 1` -if [ ${UPDATE} -eq 1 ]; then +# Update ${ARGS} for recursive call +ARGS="${ARGS} -L $((${LEVEL}+1)) -P ${PID}" + +if [ ${#} -eq 0 ]; then RPKG=$(echo $(basename $(pwd)) | sed -e 's/^R-//'); -elif [ ${#} -eq 0 ]; then - read -p "package: " RPKG TAIL - if [ "X${TAIL}" != "X" ]; then - echo "Error: multiple package names given." - echo ${USAGE} - exit 1 - fi elif [ ${#} -eq 1 ]; then RPKG=${1} else - echo ${USAGE} + echo "Error: multiple package names given." + echo "${USAGE}" exit 1 fi R_FILE=${TMPDIR}/R2pkg.$$.R -if [ ${TOP_LEVEL} -eq 1 ]; then - BANNER_MSG="===> Creating R package: R-${RPKG}" +if [ ${UPDATE} -eq 1 -a -r Makefile ]; then + BANNER_MSG="[ ${LEVEL} ] ===> Updating R package R-${RPKG} in $(pwd)" else - BANNER_MSG="===> Creating dependency package: R-${RPKG}" -fi -if [ "X${DEPENDENCY_LIST}" = "X" ]; then - DEPENDENCY_LIST=${TMPDIR}/R2pkg.depends.$$ - ARGS="${ARGS} -R ${DEPENDENCY_LIST}" + BANNER_MSG="[ ${LEVEL} ] ===> Creating R package R-${RPKG} in $(pwd)" fi + +PACKAGES_LIST=${TMPDIR}/R2pkg.packages.${PID} +DEPENDENCY_LIST=${TMPDIR}/R2pkg.depends.${PID} + if [ ${QUIET} -eq 1 ]; then STDOUT_MAKESUM=">/dev/null" STDOUT_EXTRACT=">/dev/null" @@ -107,7 +111,7 @@ if [ ${QUIET} -eq 1 ]; then ECHO_DONE=":" ECHO_FETCH=":" ECHO_EXTRACT=":" - if [ ${TOP_LEVEL} -eq 0 ]; then + if [ ${LEVEL} -ne 0 ]; then ECHO=":" fi elif [ ${VERBOSE} -eq 0 ]; then @@ -149,11 +153,28 @@ check_for_R () fi } -preserve_original_files () +check_for_no_recursion () { - [ -f DESCR ] && mv DESCR DESCR.orig + touch ${PACKAGES_LIST} + grep -E -q -e "${RPKG}" ${PACKAGES_LIST} \ + && echo "ERROR: circular dependency" + echo "${RPKG}" >> ${PACKAGES_LIST} +} + +preserve_original_content () +{ + [ -f Makefile ] && grep -e "CATEGORIES=" Makefile > CATEGORIES + [ -f Makefile ] && grep -e "COMMENT=" Makefile > COMMENT + [ -f Makefile ] && grep -e "LICENSE=" Makefile > LICENSE [ -f Makefile ] && grep -e "MAINTAINER=" Makefile > MAINTAINER + [ -f Makefile ] && grep -e "USE_LANGUAGES" Makefile > USE_LANGUAGES + [ -f Makefile ] && grep -e "USE_TOOLS" Makefile > USE_TOOLS + [ -f Makefile ] && grep -e "DEPENDS" Makefile > DEPENDS + [ -f Makefile ] && grep -e "buildlink3.mk" Makefile > BUILDLINK3.MK + + [ -f DESCR ] && mv DESCR DESCR.orig [ -f Makefile ] && mv Makefile Makefile.orig + [ -f buildlink3.mk ] && mv buildlink3.mk buildlink3.mk.orig [ -f distinfo ] && mv distinfo distinfo.orig } @@ -161,6 +182,15 @@ make_package () { R_CMD="Rscript --no-save ${R_FILE}" cat << EOF > ${R_FILE} +R_version <- function() +{ + info <- R.Version() + version <- paste0(info[['major']],'.',info[['minor']]) + version +} + +set.locale <- function() { invisible(Sys.setlocale('LC_ALL','C')) } + trim.space <- function(s) gsub('[[:space:]]','',s) trim.blank <- function(s) gsub('[[:blank:]]','',s) one.space <- function(s) gsub('[[:blank:]]+',' ',s) @@ -168,21 +198,84 @@ one.line <- function(s) gsub('\n',' ',s) pkg.vers <- function(s) gsub('_','.',s) field <- function(key,value) paste(key,'=\t',value,sep='') +base.packages <- c( + 'MASS', + 'Matrix', + 'R', + 'Rcpp', + 'boot', + 'class', + 'cluster', + 'codetools', + 'foreign', + 'grDevices', + 'graphics', + 'grid', + 'lattice', + 'methods', + 'mgcv', + 'nlme', + 'nnet', + 'parallel', + 'rpart', + 'splines', + 'stats', + 'survival', + 'tools', + 'utils') + licenses <- list() -licenses[['ACM']] <- 'acm-license' -licenses[['APACHE']] <- 'apache-2.0' -licenses[['ARTISTIC']] <- 'artistic-2.0' -licenses[['BSD-2']] <- '2-clause-bsd' -licenses[['GPL-2']] <- 'gnu-gpl-v2' -licenses[['GPL-3']] <- 'gnu-gpl-v3' -licenses[['GPL (>= 2)']] <- 'gnu-gpl-v2' -licenses[['GPL-2 | GPL-3']] <- 'gnu-gpl-v2 OR gnu-gpl-v3' -licenses[['LGPL-2']] <- 'gnu-lgpl-v2' -licenses[['LGPL-2.1']] <- 'gnu-lgpl-v2.1' -licenses[['LGPL (>= 2)']] <- 'gnu-lgpl-v2' -licenses[['LUCENT']] <- 'lucent' -licenses[['MIT']] <- 'mit' -licenses[['POSTGRESQL']] <- 'postgresql-license' +licenses[['ACM']] <- 'acm-license' +licenses[['ACM | file LICENSE']] <- 'acm-license # OR file LICENSE' +licenses[['APACHE']] <- 'apache-1.1 OR apache-2.0' +licenses[['Apache License 2.0']] <- 'apache-2.0' +licenses[['Apache License (== 2.0)']] <- 'apache-2.0' +licenses[['Apache License (== 2.0) | file LICENSE']] <- 'apache-2.0 # OR file LICENSE' +licenses[['ARTISTIC']] <- 'artistic OR artistic-2.0' +licenses[['Artistic-2.0']] <- 'artistic-2.0' +licenses[['BSD']] <- '2-clause-bsd OR modified-bsd OR original-bsd' +licenses[['BSD-2']] <- '2-clause-bsd' +licenses[['BSD_2_clause + file LICENSE']] <- '2-clause-bsd # + file LICENSE' +licenses[['BSD 3 clause']] <- 'modified-bsd' +licenses[['BSD 3 clause + file LICENSE']] <- 'modified-bsd # + file LICENSE' +licenses[['BSD_3_clause + file LICENSE']] <- 'modified-bsd # + file LICENSE' +licenses[['BSL-1.0']] <- 'boost-license' +licenses[['CC0']] <- 'cc0-1.0-universal' +licenses[['GPL']] <- 'gnu-gpl-v1 OR gnu-gpl-v2 OR gnu-gpl-v3' +licenses[['GPL-1']] <- 'gnu-gpl-v1' +licenses[['GPL-2']] <- 'gnu-gpl-v2' +licenses[['GPL-2 | file LICENSE']] <- 'gnu-gpl-v2 # OR file LICENSE' +licenses[['GPL-3']] <- 'gnu-gpl-v3' +licenses[['GPL-2 | GPL-3']] <- 'gnu-gpl-v2 OR gnu-gpl-v3' +licenses[['GPL (>= 2)']] <- 'gnu-gpl-v2 OR gnu-gpl-v3' +licenses[['GPL (>= 2.0)']] <- 'gnu-gpl-v2 OR gnu-gpl-v3' +licenses[['GPL (>= 2) | file LICENSE']] <- 'gnu-gpl-v2 OR gnu-gpl-v3 # OR file LICENSE' +licenses[['GPL (>= 3)']] <- 'gnu-gpl-v3' +licenses[['LGPL']] <- 'gnu-lgpl-v2 OR gnu-lgpl-v2.1 OR gnu-lgpl-v3' +licenses[['LGPL-2']] <- 'gnu-lgpl-v2' +licenses[['LGPL-2.1']] <- 'gnu-lgpl-v2.1' +licenses[['LGPL-3']] <- 'gnu-lgpl-v3' +licenses[['LGPL-2 | LGPL-3']] <- 'gnu-lgpl-v2 OR gnu-lgpl-v3' +licenses[['LGPL (>= 2)']] <- 'gnu-lgpl-v2 OR gnu-lgpl-v2.1 OR gnu-lgpl-v3' +licenses[['LUCENT']] <- 'lucent' +licenses[['Lucent Public License']] <- 'lucent' +licenses[['MIT']] <- 'mit' +licenses[['MIT + file LICENSE']] <- 'mit # + file LICENSE' +licenses[['MIT + file LICENSE | Unlimited']] <- 'mit # + file LICENSE OR unlimited' +licenses[['MPL-1.0']] <- 'mpl-1.0' +licenses[['MPL-1.1']] <- 'mpl-1.1' +licenses[['MPL-2.0']] <- 'mpl-2.0' +licenses[['MPL-2.0 | file LICENSE']] <- 'mpl-2.0 # OR file LICENSE' +licenses[['POSTGRESQL']] <- 'postgresql-license' + +adjacent.duplicates <- function(x) +{ + a <- x[-length(x)] + b <- x[-1] + dups <- a == b + dups <- c(FALSE,dups) + dups +} paste2 <- function(s1,s2) { @@ -192,6 +285,278 @@ paste2 <- function(s1,s2) if (!is.na(s1) && !is.na(s2)) return (paste(s1,s2)) } +end.paragraph <- function(l,l1=l,l2=list()) +{ + if (length(l1) > 0 || length(l2) > 0) + l <- append(l,'') + l +} + +as.sorted.list <- function(df) +{ + l <- list() + df <- df[!duplicated(df),] + if (nrow(df) > 0) + { + key <- as.vector(df[,1]) + value <- as.vector(df[,2]) + key <- order(key,value) + l <- as.list(value[key]) + } + l +} + +read.file.as.dataframe <- function(filename) +{ + # message('===> read.file.as.dataframe(',filename,')') + contents <- as.list(readLines(filename)) + df <- data.frame() + for (line in contents) + { + # str(line) + df <- rbind(df,data.frame(line=line,stringsAsFactors=FALSE)) + } + df +} + +categorize.key_value <- function(df,line='line') +{ + re.skip_blank <- '[[:blank:]]*' + re.blank <- '[[:blank:]]+' + re.anything <- '.*' + + re.key <- '[^+=[:blank:]]+' + re.operator <- '[+=]+' + re.delimiter <- re.skip_blank + re.value <- re.anything + re.optional_TODO <- '(#[[:blank:]]*TODO[[:blank:]]*:[[:blank:]]*)*' + + re.match_key_value_line <- paste0('^', + re.skip_blank, + re.optional_TODO, + re.key, + re.skip_blank, + re.operator, + re.delimiter, + re.value, + '\$') + + re.match_key <- paste0('^', + re.skip_blank, + re.optional_TODO, + '(',re.key,')', + re.skip_blank, + re.operator, + re.delimiter, + re.value, + '\$') + + df\$key_value <- grepl(re.match_key_value_line,df[,line]) + df\$key <- sub(re.match_key,'\\\2',df[,line]) + df\$key[!df\$key_value] <- NA + df +} + +categorize.depends <- function(df,line='line') +{ + df\$depends <- df\$key_value & df\$key == 'DEPENDS' + df\$category[df\$depends] <- unlist(sapply(strsplit(df[df\$depends,line],'/',fixed=TRUE),'[',3)) + df +} + +categorize.buildlink <- function(df,line='line') +{ + df\$buildlink3.mk <- grepl('buildlink3.mk',df[,line]) + df\$category[df\$buildlink3.mk] <- unlist(sapply(strsplit(df[df\$buildlink3.mk,line],'/',fixed=TRUE),'[',3)) + df +} + +fix.continued.lines <- function(df,line='line') +{ + if (nrow(df) > 1) + { + continued <- grepl('\\\\\\\\$',df[,line]) + continued_key_value <- df\$key_value & continued + if (FALSE %in% df[continued,'key_value']) + { + message('[ ${LEVEL} ] WARNING: unhandled continued line(s)') + } + for (i in 1:(length(continued_key_value)-1)) + { + next_line <- i + 1 + if (continued_key_value[i]) + { + df[i,line] <- sub('[[:blank:]]*\\\\\\\\$','',df[i,line]) + df\$key_value[next_line] <- TRUE + df\$key[next_line] <- df\$key[i] + df[next_line,line] <- paste0(df\$key[next_line],'+=',df[next_line,line]) + } + } + } + df +} + +read.Makefile.as.dataframe <- function() +{ + # message('===> read.Makefile.as.dataframe():') + + re.skip_blank <- '[[:blank:]]*' + re.blank <- '[[:blank:]]+' + re.anything <- '.*' + + re.key <- '[^+=[:blank:]]+' + re.operator <- '[+=]+' + re.delimiter <- re.skip_blank + re.value <- re.anything + re.optional_TODO <- '(#[[:blank:]]*TODO[[:blank:]]*:[[:blank:]]*)*' + + re.match_operator <- paste0('^', + re.skip_blank, + re.optional_TODO, + re.key, + re.skip_blank, + '(',re.operator,')', + re.delimiter, + re.value, + '\$') + re.match_delimiter <- paste0('^', + re.skip_blank, + re.optional_TODO, + re.key, + re.skip_blank, + re.operator, + '(',re.delimiter,')', + re.value, + '\$') + re.match_value <- paste0('^', + re.skip_blank, + re.optional_TODO, + re.key, + re.skip_blank, + re.operator, + re.delimiter, + '(',re.value,')', + '\$') + re.match_optional_TODO <- paste0('^', + re.skip_blank, + '(',re.optional_TODO,')', + re.key, + re.skip_blank, + re.operator, + re.delimiter, + re.value, + '\$') + + df <- read.file.as.dataframe('Makefile.orig') + + df\$order <- 1:nrow(df) + df\$category <- NA + + df <- categorize.key_value(df) + df <- fix.continued.lines(df) + df <- categorize.depends(df) + df <- categorize.buildlink(df) + + df\$operator <- sub(re.match_operator,'\\\2',df\$line) + df\$delimiter <- sub(re.match_delimiter,'\\\2',df\$line) + df\$old_value <- sub(re.match_value,'\\\2',df\$line) + df\$old_todo <- sub(re.match_optional_TODO,'\\\1',df\$line) + + df\$operator[!df\$key_value] <- NA + df\$delimiter[!df\$key_value] <- NA + df\$old_value[!df\$key_value] <- NA + df\$old_todo[!df\$key_value] <- NA + + df +} + +read.file.as.list <- function(filename) +{ + result <- list() + info <- file.info(filename) + size <- info[filename,'size'] + if (!is.na(size) && size > 0) + { + contents <- readLines(filename) + result <- as.list(contents) + } + result +} + +read.file.as.value <- function(filename) +{ + value <- '' + l <- read.file.as.list(filename) + if (length(l) == 1) + { + line <- l[[1]] + fields <- strsplit(line,'[[:blank:]]+') + value <- fields[[1]][2] + } + value +} + +read.file.as.values <- function(filename) +{ + message('===> read.file.as.values(',filename,'):') + values <- list() + l <- read.file.as.list(filename) + print(l) + for (line in l) + { + # fields <- strsplit(line,'[[:blank:]]+') + # value <- fields[[1]][2] + } + print(values) + values +} + +simplify.whitespace <- function(s) { gsub('[[:blank:]]+',' ',s) } +remove.punctuation <- function(s) +{ + punctuation <- '[,-]' + gsub(punctuation,'',s) +} +remove.quotes <- function(s) +{ + quotes <- '[\'\`"]' + gsub(quotes,'',s) +} +remove.articles <- function(s) +{ + pattern <- '^([[:blank:]]*)An* |([[:blank:]]+)[Aa]n*[[:blank:]]+' + result <- gsub(pattern,'\\\1',s) + result +} + +case.insensitive.equals <- function(s1,s2) +{ + s1.lower <- tolower(simplify.whitespace(s1)) + s2.lower <- tolower(simplify.whitespace(s2)) + result <- s1.lower == s2.lower + result +} + +weakly.equals <- function(s1,s2) +{ + result <- case.insensitive.equals(remove.articles(remove.quotes(remove.punctuation(s1))), + remove.articles(remove.quotes(remove.punctuation(s2)))) + result +} + +new.field.if.different <- function(filename,s) +{ + field <- field(filename,one.line(s)) + field.list <- read.file.as.list(filename) + if (length(field.list) == 1) + { + f <- field.list[[1]] + if (case.insensitive.equals(f,field)) + field <- f + } + field +} + todo.license <- function(s) { if (is.null(licenses[[s]])) @@ -209,30 +574,108 @@ pkgsrc.license <- function(s) license } -package <- function(s) field('R_PKGNAME',one.line(s)) -version <- function(s) field('R_PKGVER',one.line(s)) -comment <- function(s) field('COMMENT',one.line(s)) -license <- function(s) field(todo.license(s),pkgsrc.license(s)) +package <- function(s) one.line(s) +version <- function(s) one.line(s) +comment <- function(s) one.line(s) +use.tools <- function(s) read.file.as.list(s) + +license <- function(s) +{ + license <- pkgsrc.license(s) + old.license <- read.file.as.value('LICENSE') + if (old.license != '' && old.license != license) + license <- paste0(license,' # previously: ',old.license) + license +} + maintainer <- function(email) - { - if (file.exists('MAINTAINER')) - { - x <- scan('MAINTAINER','character',quiet=TRUE) - if (length(x) == 2) - email = x[2] - else - message('WARNING: previous MAINTAINER is ignored') - } - email <- paste0('MAINTAINER= ',email) - email - } - -categories <- function() paste('CATEGORIES=',paste(basename(dirname(getwd())),'R'),sep=' ') +{ + MAINTAINER <- read.file.as.value('MAINTAINER') + if (MAINTAINER == '') + MAINTAINER <- email + MAINTAINER +} + +make.sed.command <- function(key,value) +{ + address <- paste0('/^[[:blank:]]*',key,'/') + match <- paste0('(',key,'[[:blank:]]*=[[:blank:]]*).*$') + replacement <- paste0('\\\1',value) + command <- paste0(' -e "',address,'s/',match,'/',replacement,'/"') + command +} + +sed.categories <- function(categories) make.sed.command('CATEGORIES',categories) +sed.comment <- function(comment) +{ + old.comment <- read.file.as.value('COMMENT') + if (weakly.equals(old.comment,comment)) + comment <- old.comment + make.sed.command('COMMENT',comment) +} +sed.maintainer <- function(email) +{ + make.sed.command('MAINTAINER',maintainer(email)) +} +sed.license <- function(license) +{ + make.sed.command('LICENSE',license) +} +sed.r_pkgver <- function(r_pkgver) make.sed.command('R_PKGVER',r_pkgver) + +buildlink3.mk <- function(s1,s2) +{ + BUILDLINK3.MK <- data.frame() + buildlink3.mk.list <- read.file.as.list('BUILDLINK3.MK') + for (line in buildlink3.mk.list) + { + fields <- strsplit(line[[1]],'/',fixed=TRUE) + key <- fields[[1]][3] + value <- line + BUILDLINK3.MK <- rbind(BUILDLINK3.MK,data.frame(key=key,value=value)) + } + if (find.Rcpp(s1,s2)) + { + buildlink3.line <- '.include "../../devel/R-Rcpp/buildlink3.mk"' + key <- 'devel' + value <- buildlink3.line + BUILDLINK3.MK <- rbind(BUILDLINK3.MK,data.frame(key=key,value=value)) + } + BUILDLINK3.MK +} + +makefile.field <- function(key,value) +{ + # message('===> makefile.field(',key,',',value,'):') + field <- paste0(key,'= ',value) + # print(field) + field +} + +makefile.fields <- function(key,values) +{ + # message('===> makefile.fields():') + fields <- list() + for (l in values) + { + value <- unlist(l) + # message('===> value=',value,' ',length(value),' ',value == '') + # print(value) + if (value != '') + fields <- append(fields,makefile.field(key,list(value))) + else + fields <- append(fields,list('')) + # print(fields) + } + # print(fields) + fields +} + +categories <- function() paste(basename(dirname(getwd())),'R') description <- function(s) strwrap(s,width=71) filter.imports <- function(s) { - base.packages <- c('R','MASS','Matrix','Rcpp','cluster','grDevices','graphics','grid','foreign','lattice','nlme','methods','nnet','parallel','rpart','stats','survival','tools','utils') for (pkg in base.packages) { re.pkg <- paste('^',pkg,sep='') @@ -241,6 +684,13 @@ filter.imports <- function(s) s } +find.Rcpp <- function(s1,s2) +{ + s <- paste(s1,s2) + Rcpp <- grepl('Rcpp',s) + Rcpp +} + make.imports <- function(s1,s2) { s <- paste2(s1,s2) @@ -254,112 +704,221 @@ make.imports <- function(s1,s2) make.dependency <- function(s) { s <- gsub('\\\\)','',s) + s <- gsub('-','.',s) s <- unlist(strsplit(s,'\\\\(')) s } -depends <- function(s1,s2) +depends <- function(dependency) dependency[1] + +depends.pkg <- function(dependency) +{ + # XXX message('===> ',depends(dependency)) + result <- Sys.glob(paste0('../../*/R-',depends(dependency))) + result +} + +new.depends.pkg <- function(dependency) +{ + result <- Sys.glob(paste0('../../wip/R-',depends(dependency))) + result +} + +depends.pkg.fullname <- function(dependency,index=1) +{ + result <- system(paste('cd',depends.pkg(dependency)[index],'&& bmake show-var VARNAME=PKGNAME'),intern=TRUE) + result +} + +depends.pkg.name <- function(dependency,index=1) +{ + result <- sub('^(.*)-([^-]*)$','\\\\1',depends.pkg.fullname(dependency,index)) + result +} + +depends.pkg.vers <- function(dependency,index=1) +{ + result <- sub('^(.*)-([^-]*)$','\\\\2',depends.pkg.fullname(dependency,index)) + result +} + +depends.vers <- function(dependency,index=1) +{ + if (length(dependency) == 2) + result <- dependency[2] + else + result <- paste0('>=',depends.pkg.vers(dependency,index)) + result <- trim.space(result) + result +} + +depends.vers.2 <- function(dependency) +{ + result <- ifelse(length(dependency) == 2, dependency[2], '>=???') + result <- trim.space(result) + result +} + +depends.dir <- function(dependency,index=1) +{ + fields <- strsplit(depends.pkg(dependency)[index],'/',fixed=TRUE) + result <- fields[[1]][3] + result +} + +depends.line <- function(dependency,index=1) +{ + result <- paste0('DEPENDS+=\tR-',depends(dependency),depends.vers(dependency,index),':',depends.pkg(dependency)[index]) + result +} + +depends.line.2 <- function(dependency) +{ + result <- paste0('DEPENDS+=\tR-',depends,depends.vers.2(dependency),':../../???/R-',depends) + result <- paste0(result,' # XXX - found') + for (pkg in depends.pkg(dependency)) + result <- paste(result,pkg) + result +} + +buildlink3.file <- function(dependency,index=1) +{ + result <- paste0(depends.pkg(dependency)[index],'/buildlink3.mk') + result +} + +buildlink3.line <- function(dependency,index=1) +{ + result <- paste0('.include "',buildlink3.file(dependency,index),'"') + result +} + +dependency.dir <- function(dependency) +{ + result <- paste0('../../wip/R-',depends(dependency)) + result +} + +message.wip.dependency <- function(dependency,index=1) +{ + dir <- depends.dir(dependency,index) + dir.in.wip <- grepl('wip',dir) + wd.in.wip <- grepl('/wip/',getwd()) + if (dir.in.wip && !wd.in.wip) + message('[ ${LEVEL} ] WARNING: R-${RPKG} should not depend on a wip package: ',depends.pkg(dependency)[index]) +} + +message.too.many.dependencies <- function(dependency) +{ + msg <- paste0('[ ${LEVEL} ] WARNING: too many dependencies found for ',depends(dependency),':') + for (pkg in depends.pkg(dependency)) + msg <- paste(msg,pkg) + msg +} + +update.dependency <- function(dependency,index=1) +{ + message('[ ${LEVEL} ] WARNING: updating dependency for ',depends(dependency),': ',depends.pkg(dependency)[index]) + grep <- paste0('grep -E -q -e "',depends(dependency),'" ${PACKAGES_LIST}') + command <- paste0(grep,' || (cd ',depends.pkg(dependency)[index],' && ${R2PKG} ${ARGS} ',depends(dependency),')') + error <- system(command) + if (error != 0) + message('[ ${LEVEL} ] WARNING: error updating dependency for ',depends(dependency)) +} + +make.depends <- function(s1,s2) { imports <- make.imports(s1,s2) + # XXX message('===> imports:') + # XXX print(imports) DEPENDS <- data.frame() - BUILDLINK3.MK <- data.frame() + BUILDLINK3.MK <- buildlink3.mk(s1,s2) if (length(imports) > 0) { for (i in 1:length(imports)) { dependency <- make.dependency(imports[i]) - depends <- dependency[1] - depends.pkg <- Sys.glob(paste('../../*/R-',depends,sep='')) - if (length(depends.pkg) == 0) # a dependency cannot be found + # XXX message('[ ',${LEVEL},' ] ===> ',i,' / ',length(imports),': ',depends(dependency)) + if (length(depends.pkg(dependency)) == 0) # a dependency cannot be found { - message('WARNING: creating the dependency',depends) + message('[ ${LEVEL} ] 0 dependencies match ',dependency) if (${RECURSIVE}) - { - dependency.dir <- paste('../../wip/R-',depends,sep='') - dir.create(path=dependency.dir,recursive=TRUE) - error <- system(paste('(cd',dependency.dir,'&& ${R2PKG} ${ARGS}',depends,')')) - if (error != 0) - file.remove(dependency.dir) - } + { + dir.create(path=dependency.dir(dependency),recursive=TRUE) + update.dependency(dependency) + } + else + message('[ ${LEVEL} ] WARNING: dependency needed for ',depends(dependency)) } - depends.pkg <- Sys.glob(paste('../../*/R-',depends,sep='')) - if (length(depends.pkg) == 0) # no dependency was created - message('WARNING: the dependency',depends,'does not exist') - else if (length(depends.pkg) == 1) # a unique dependency found + else if (length(depends.pkg(dependency)) == 1) # a unique dependency found { - fields <- strsplit(depends.pkg,'/',fixed=TRUE) - depends.dir <- fields[[1]][3] - buildlink3.mk <- paste(depends.pkg,'/buildlink3.mk',sep='') - if (file.exists(buildlink3.mk)) - { - buildlink3.line <- paste('.include "',buildlink3.mk,'"',sep='') - BUILDLINK3.MK <- rbind(BUILDLINK3.MK,data.frame(key=depends.dir,value=buildlink3.line)) - } + message('[ ${LEVEL} ] 1 dependency matches ',dependency,': ',depends.pkg(dependency)) + message.wip.dependency(dependency) + if (${RECURSIVE} && ${UPDATE}) + update.dependency(dependency) + if (file.exists(buildlink3.file(dependency))) + BUILDLINK3.MK <- rbind(BUILDLINK3.MK,data.frame(key=depends.dir(dependency),value=buildlink3.line(dependency))) else - { - depends.pkg.fullname <- system(paste('cd',depends.pkg,'&& bmake show-var VARNAME=PKGNAME'),intern=TRUE) - depends.pkg.name <- sub('^(.*)-([^-]*)$','\\\\1',depends.pkg.fullname) - depends.pkg.vers <- sub('^(.*)-([^-]*)$','\\\\2',depends.pkg.fullname) - if (length(dependency) == 2) - depends.vers <- dependency[2] - else - depends.vers <- paste('>=',depends.pkg.vers,sep='') - depends.line <- paste('DEPENDS+=\tR-',depends,depends.vers,':',depends.pkg,sep='') - depends.line <- paste(depends.line,' # XXX - found ',depends.pkg.fullname,' (',depends.pkg,')',sep='') - DEPENDS <- rbind(DEPENDS,data.frame(key=depends.dir,value=depends.line)) - } + DEPENDS <- rbind(DEPENDS,data.frame(key=depends.dir(dependency),value=depends.line(dependency))) } - else # more than 1 dependency found + else if (length(depends.pkg(dependency)) == 2) # two dependencies found + { + d <- depends.pkg(dependency) + index <- grep('/wip/',d,invert=TRUE) + message('[ ${LEVEL} ] 2 dependencies match ',dependency,':',paste(' ',depends.pkg(dependency))) + # message('===> depends(dependency): ',depends(dependency)) + # message('===> depends.pkg(dependency):',paste(' ',d)) + # message('===> index: ',index) + # message('===> buildlinke.line(): ',buildlink3.line(dependency,index)) + if (length(index) == 1) # a unique, non-wip, dependency found + { + message('[ ${LEVEL} ] choosing unique non-wip dependency for ',dependency,': ',depends.pkg(dependency)[index]) + if (${RECURSIVE} && ${UPDATE}) + update.dependency(dependency,index) + if (file.exists(buildlink3.file(dependency,index))) + BUILDLINK3.MK <- rbind(BUILDLINK3.MK,data.frame(key=depends.dir(dependency,index),value=buildlink3.line(dependency,index))) + else + DEPENDS <- rbind(DEPENDS,data.frame(key=depends.dir(dependency,index),value=depends.line(dependency,index))) + } + else + { + message('[ ${LEVEL} ] no unique non-wip dependency matches') + message(message.too.many.dependencies(dependency)) + DEPENDS <- rbind(DEPENDS,data.frame(key='???',value=depends.line.2(dependency))) + } + } + else # more than 2 dependencies found { - msg <- paste('WARNING: too many dependencies found for ',depends,':',sep='') - for (pkg in depends.pkg) - msg <- paste(msg,' ',pkg,sep='') - message(msg) - depends.vers <- ifelse(length(dependency) == 2, dependency[2], '>=???') - depends.vers <- trim.space(depends.vers) - depends.line <- paste('DEPENDS+=\tR-',depends,depends.vers,':../../???/R-',depends,sep='') - depends.line <- paste(depends.line,' # XXX - found',sep='') - for (pkg in depends.pkg) - depends.line <- paste(depends.line,' ',pkg,sep='') - DEPENDS <- rbind(DEPENDS,data.frame(key='???',value=depends.line)) + message('[ ${LEVEL} ] ',length(depends.pkg(dependency)),' dependencies match ',dependency,':',paste(' ',depends.pkg(dependency))) + message(message.too.many.dependencies(dependency)) + DEPENDS <- rbind(DEPENDS,data.frame(key='???',value=depends.line.2(dependency))) } - new.depends.pkg <- Sys.glob(paste('../../wip/R-',depends,sep='')) - if (length(new.depends.pkg) > 0) - system(paste('echo',depends,'${RPKG} >> ${DEPENDENCY_LIST}')) + if (length(new.depends.pkg(dependency)) > 0) + system(paste('echo',depends(dependency),'${RPKG} >> ${DEPENDENCY_LIST}')) } } - if (nrow(DEPENDS) > 0) - { - key <- as.vector(DEPENDS[,1]) - value <- as.vector(DEPENDS[,2]) - key <- order(key,value) - DEPENDS <- as.list(value[key]) - if (length(DEPENDS) > 0) - DEPENDS <- append(DEPENDS,'') - } - else - DEPENDS = list() - if (nrow(BUILDLINK3.MK) > 0) - { - key <- as.vector(BUILDLINK3.MK[,1]) - value <- as.vector(BUILDLINK3.MK[,2]) - key <- order(key,value) - BUILDLINK3.MK <- as.list(value[key]) - } - else - BUILDLINK3.MK <- list() - list(DEPENDS,BUILDLINK3.MK) + DEPENDS <- as.sorted.list(DEPENDS) + DEPENDS <- end.paragraph(DEPENDS) + BUILDLINK3.MK <- as.sorted.list(BUILDLINK3.MK) + result <- list(DEPENDS,BUILDLINK3.MK) + result } use.languages <- function(s1,s2) { +# message('===> use.languages(',s1,',',s2,'):') +# USE_LANGUAGES <- read.file.as.values('USE_LANGUAGES') +# if (length(USE_LANGUAGES) == 0) +# { +# if (find.Rcpp(s1,s2)) +# USE_LANGUAGES <- append(USE_LANGUAGES,list('USE_LANGUAGES+= c c++')) +# } USE_LANGUAGES <- list() - s <- paste(s1,s2) - Rcpp <- grepl('Rcpp',s) - if (Rcpp) - USE_LANGUAGES <- append(USE_LANGUAGES,list('USE_LANGUAGES+= c c++')) - if (length(USE_LANGUAGES) > 0) - USE_LANGUAGES <- append(USE_LANGUAGES,'') + if (find.Rcpp(s1,s2)) + USE_LANGUAGES <- append(USE_LANGUAGES,list('c c++')) + if (length(USE_LANGUAGES) == 0) + USE_LANGUAGES <- '# none' + USE_LANGUAGES <- end.paragraph(USE_LANGUAGES) USE_LANGUAGES } @@ -369,53 +928,437 @@ copy.description <- function(connection) writeLines(description,con='DESCRIPTION') } +write.Makefile <- function(metadata) +{ + RCSID <- '# \$NetBSD\$' + CATEGORIES <- makefile.field('CATEGORIES',categories()) + MAINTAINER <- makefile.field('MAINTAINER',maintainer('${MAINTAINER_EMAIL}')) + HOMEPAGE <- makefile.field('HOMEPAGE','\${R_HOMEPAGE_BASE}/${RPKG}/') + COMMENT <- makefile.field('COMMENT',comment(metadata[3])) + LICENSE <- makefile.field('LICENSE',license(metadata[5])) + R_PKGNAME <- makefile.field('R_PKGNAME',package(metadata[1])) + R_PKGVER <- makefile.field('R_PKGVER',version(metadata[2])) + USE_LANGUAGES <- makefile.fields('USE_LANGUAGES',use.languages(metadata[6],metadata[7])) + DEPENDENCIES <- make.depends(metadata[6],metadata[7]) + DEPENDS <- DEPENDENCIES[1] + BUILDLINK3.MK <- DEPENDENCIES[2] + INCLUDE.R <- '.include "../../math/R/Makefile.extension"' + INCLUDE.PKG <- '.include "../../mk/bsd.pkg.mk"' + + Makefile <- list() + Makefile <- append(Makefile,RCSID) + Makefile <- append(Makefile,'') + Makefile <- append(Makefile,CATEGORIES) + Makefile <- append(Makefile,'') + Makefile <- append(Makefile,MAINTAINER) + Makefile <- append(Makefile,HOMEPAGE) + Makefile <- append(Makefile,COMMENT) + Makefile <- append(Makefile,LICENSE) + Makefile <- append(Makefile,'') + Makefile <- append(Makefile,R_PKGNAME) + Makefile <- append(Makefile,R_PKGVER) + Makefile <- append(Makefile,'') + Makefile <- append(Makefile,DEPENDS) + Makefile <- append(Makefile,USE_LANGUAGES) + Makefile <- append(Makefile,INCLUDE.R) + Makefile <- append(Makefile,BUILDLINK3.MK) + Makefile <- append(Makefile,INCLUDE.PKG) + Makefile <- paste(unlist(Makefile),collapse='\n') + + write(Makefile,'Makefile') +} + +construct.line <- function(df,key,value) +{ + key <- df[df\$key==key,'key'] + operator <- df[df\$key==key,'operator'] + delimiter <- df[df\$key==key,'delimiter'] + value <- df[df\$key==key,value] + df\$new_line[df\$key==key] <- paste0(key,operator,delimiter,value) + df +} + +element <- function(df,key,value,quiet=FALSE) +{ + key.index <- match(key,df\$key,0) + if (key.index != 0 && df\$key_value[key.index]) + result <- df[key.index,value] + else + { + result <- '???' + if (!quiet) + { + if (key.index == 0) + message('[ ${LEVEL} ] WARNING: ',key,' not found') + else + message('[ ${LEVEL} ] WARNING: ',key,' is not a key-value field') + } + } + result +} + +make.categories <- function(df) +{ + # message('===> make.categories():') + directory <- basename(dirname(getwd())) + categories <- unlist(element(df,'CATEGORIES','old_value')) + categories <- unlist(strsplit(categories,'[[:blank:]]+')) + categories <- c(directory,categories,'R') + if (directory != 'wip') + categories <- categories[ categories != 'wip' ] + categories <- categories[!duplicated(categories)] + categories <- paste(categories,collapse=' ') + categories +} + +make.maintainer <- function(df) +{ + old.maintainer <- element(df,'MAINTAINER','old_value') + new.maintainer <- element(df,'MAINTAINER','new_value') + maintainer <- ifelse(old.maintainer == '',new.maintainer,old.maintainer) + maintainer +} + +make.comment <- function(df) +{ + old.comment <- element(df,'COMMENT','old_value') + new.comment <- element(df,'COMMENT','new_value') + comment <- old.comment + if (!weakly.equals(old.comment,new.comment)) + comment <- paste0(comment,' # updated to: ',new.comment) + comment +} + +make.new_license <- function(df,license) +{ + new_license <- licenses[[license]] + if (is.null(new_license)) + new_license <- license + df\$new_value[df\$key == 'LICENSE'] <- new_license + df +} + +license.marked.todo <- function(todo) { todo != '' } +license.in.pkgsrc <- function(license) { license %in% sapply(licenses,'[',1) } + +make.license <- function(df) +{ + # message('===> make.license():') + old_license <- element(df,'LICENSE','old_value') + old_todo <- element(df,'LICENSE','old_todo') + new_license <- element(df,'LICENSE','new_value') + + if (license.in.pkgsrc(old_license) && license.in.pkgsrc(new_license)) + { + if (case.insensitive.equals(old_license,new_license)) + { + license <- old_license + todo <- old_todo + } + else + { + license <- paste0(new_license,' # previously: ',old_license) + todo <- old_todo + } + } + else if (license.in.pkgsrc(old_license) && !license.in.pkgsrc(new_license)) + { + license <- paste0(old_license,' # updated to: ',new_license) + todo <- '# TODO: ' + } + else if (!license.in.pkgsrc(old_license) && license.in.pkgsrc(new_license)) + { + license <- paste0(new_license,' # previously: ',old_license) + todo <- '' + } + else if (!license.in.pkgsrc(old_license) && !license.in.pkgsrc(new_license)) + { + license <- paste0(new_license,' # previously: ',old_license) + todo <- '# TODO: ' + } + + df\$value[df\$key == 'LICENSE'] <- license + df\$todo[df\$key == 'LICENSE'] <- todo + + df +} + +make.r_pkgver <- function(df) element(df,'R_PKGVER','new_value') + +find.order <- function(df,key,field) +{ + x <- df[,key] + value <- match(TRUE,x) + if (!is.na(value)) + value <- df[value,field] + value +} + +write.makefile <- function(lines) write(lines,'Makefile') + +update.Makefile.with.metadata <- function(df,metadata) +{ + # message('===> update.Makefile.with.metadata():') + + df\$new_value <- NA + + df <- make.new_license(df,metadata[5]) + + df\$new_value[df\$key == 'CATEGORIES'] <- categories() + df\$new_value[df\$key == 'MAINTAINER'] <- '${MAINTAINER_EMAIL}' + df\$new_value[df\$key == 'COMMENT'] <- one.line(metadata[3]) + df\$new_value[df\$key == 'R_PKGVER'] <- version(metadata[2]) + + # str(df) + # print(df) + df +} + +update.Makefile.with.new.values <- function(df) +{ + # message('===> update.Makefile.with.new.values():') + df\$value <- NA + df\$todo <- '' + df <- make.license(df) + df\$value[df\$key == 'CATEGORIES'] <- make.categories(df) + df\$value[df\$key == 'MAINTAINER'] <- make.maintainer(df) + df\$value[df\$key == 'COMMENT'] <- make.comment(df) + df\$value[df\$key == 'R_PKGVER'] <- make.r_pkgver(df) + + # str(df) + # print(df) + df +} + +update.Makefile.with.new.line <- function(df) +{ + # message('===> update.Makefile.with.new.line():') + df\$new_line <- NA + + construct_key_value <- df\$key_value & !is.na(df\$value) + df\$new_line[construct_key_value] <- + paste0(df\$todo[construct_key_value], + df\$key[construct_key_value], + df\$operator[construct_key_value], + df\$delimiter[construct_key_value], + df\$value[construct_key_value]) + + copy_line <- !df\$key_value | !construct_key_value + df\$new_line[copy_line] <- df\$line[copy_line] + + # str(df) + # print(df) + df +} + +annotate.distname.in.Makefile <- function(df) +{ + match <- grepl('^[[:blank:]]*DISTNAME',df\$new_line) + line <- df\$new_line[match] + value <- sub('^[[:blank:]]*DISTNAME[[:blank:]]*=[[:blank:]]*','',line) + pkgname <- sub('_.+$','',value) + pkgver <- sub('^.+_','',value) + PKGNAME <- paste0('R_PKGNAME=',pkgname) + PKGVER <- paste0('R_PKGVER=',pkgver) + comment <- paste0(' # XXX -- replace this line with ',PKGNAME,' and ',PKGVER,' as third stanza (and rerun R2pkg)') + df\$new_line[match] <- paste0(line,comment) + df +} + +annotate.Makefile <- function(df) +{ + df <- annotate.distname.in.Makefile(df) + df +} + +remove.master.sites.from.Makefile <- function(df) +{ + match <- grepl('^[[:blank:]]*MASTER_SITES',df\$new_line) + df <- df[!match,] + df +} + +remove.buildlink.abi.depends.from.Makefile <- function(df) +{ + match <- grepl('^[[:blank:]]*BUILDLINK_ABI_DEPENDS',df\$new_line) + df <- df[!match,] + df +} + +remove.buildlink.api.depends.from.Makefile <- function(df) +{ + match <- grepl('^[[:blank:]]*BUILDLINK_API_DEPENDS',df\$new_line) + df <- df[!match,] + df +} + +remove.lines.from.Makefile <- function(df) +{ + df <- remove.master.sites.from.Makefile(df) + df <- remove.buildlink.abi.depends.from.Makefile(df) + df <- remove.buildlink.api.depends.from.Makefile(df) + df +} + +conflicts <- function(pkg) +{ + conflict <- paste0('R>=',R_version()) + conflicts <- list() + if (pkg %in% base.packages) + { + conflicts <- append(conflicts,makefile.field('CONFLICTS',conflict)) + conflicts <- end.paragraph(conflicts) + } + conflicts +} + +conflicts.order <- function(df) +{ + order <- element(df,'R_PKGNAME','order') + order +} + +make.df.conflicts <- function(df,metadata) +{ + df.conflicts <- data.frame() + conflicts.exist <- element(df,'CONFLICTS','old_value',quiet=TRUE) != '???' + if (!conflicts.exist) + { + c <- conflicts(metadata[1]) + order <- conflicts.order(df) + order <- order + 2.5 + i <- 0 + for (conflict in c) + { + i <- i + 1 + category <- as.character(i) + depends <- FALSE + buildlink3.mk <- FALSE + x <- data.frame(new_line=conflict,order=order,category=category,depends=depends,buildlink3.mk=buildlink3.mk) + df.conflicts <- rbind(df.conflicts,x) + } + } + df.conflicts +} + +make.df.depends <- function(df,DEPENDS) +{ + # message('===> make.df.depends():') + # str(df) + # print(df) + df.depends <- data.frame() + if (TRUE %in% df\$depends) + df.depends <- data.frame(new_line=df[df\$depends,'line'],stringsAsFactors=FALSE) + for (line in DEPENDS) + df.depends <- rbind(df.depends,data.frame(new_line=line,stringsAsFactors=FALSE)) + if (nrow(df.depends) > 0) + { + df.depends\$category <- NA + df.depends\$buildlink3.mk <- FALSE + df.depends <- categorize.key_value(df.depends,'new_line') + df.depends <- categorize.depends(df.depends,'new_line') + df.depends\$key_value <- NULL + df.depends\$key <- NULL + df.depends <- df.depends[!duplicated(df.depends),] + df.depends\$order <- find.order(df,'depends','order') + } + # message('===> df.depends:') + # str(df.depends) + # print(df.depends) + df.depends +} + +make.df.buildlink3 <- function(df,BUILDLINK3.MK) +{ + # message('===> make.df.buildlink3():') + df.buildlink3.mk <- data.frame() + if (TRUE %in% df\$buildlink3.mk) + df.buildlink3.mk <- data.frame(new_line=df[df\$buildlink3.mk,'line'],stringsAsFactors=FALSE) + for (line in BUILDLINK3.MK) + df.buildlink3.mk <- rbind(df.buildlink3.mk,data.frame(new_line=line,stringsAsFactors=FALSE)) + if (nrow(df.buildlink3.mk) > 0) + { + df.buildlink3.mk\$category <- NA + df.buildlink3.mk\$depends <- FALSE + df.buildlink3.mk <- categorize.buildlink(df.buildlink3.mk,'new_line') + df.buildlink3.mk <- df.buildlink3.mk[!duplicated(df.buildlink3.mk),] + df.buildlink3.mk\$order <- find.order(df,'buildlink3.mk','order') + } + # str(df.buildlink3.mk) + # print(df.buildlink3.mk) + df.buildlink3.mk +} + +make.df.makefile <- function(df,df.conflicts,df.depends,df.buildlink3.mk) +{ + # message('===> make.df.makefile():') + # message('===> df:') + # str(df) + # print(df) + fields <- c('new_line','order','category','depends','buildlink3.mk') + df.makefile <- df[!df\$depends & !df\$buildlink3.mk,fields] + df.makefile <- rbind(df.makefile,df.conflicts,df.depends,df.buildlink3.mk) + df.makefile <- df.makefile[order(df.makefile\$order,df.makefile\$category,df.makefile\$new_line),] + df.makefile <- df.makefile[!adjacent.duplicates(df.makefile\$new_line),] + df.makefile +} + +update.Makefile <- function(metadata) +{ + DEPENDENCIES <- make.depends(metadata[6],metadata[7]) + DEPENDS <- DEPENDENCIES[[1]] + BUILDLINK3.MK <- DEPENDENCIES[[2]] + # message('===> DEPENDS:') + # str(DEPENDS) + # print(DEPENDS) + # message('===> BUILDLINK3.MK:') + # str(BUILDLINK3.MK) + # print(BUILDLINK3.MK) + + # message('===> df:') + df <- read.Makefile.as.dataframe() + df <- update.Makefile.with.metadata(df,metadata) + df <- update.Makefile.with.new.values(df) + df <- update.Makefile.with.new.line(df) + df <- annotate.Makefile(df) + df <- remove.lines.from.Makefile(df) + + df.conflicts <- make.df.conflicts(df,metadata) + df.depends <- make.df.depends(df,DEPENDS) + df.buildlink3 <- make.df.buildlink3(df,BUILDLINK3.MK) + df.makefile <- make.df.makefile(df,df.conflicts,df.depends,df.buildlink3) + + write.makefile(df.makefile[,'new_line']) +} + +create.Makefile <- function(metadata) +{ + if (${UPDATE} == 1 && file.exists('Makefile.orig')) + update.Makefile(metadata) + else + write.Makefile(metadata) +} + +create.DESCR <- function(metadata) +{ + DESCR <- description(metadata[4]) + write(DESCR,'DESCR') +} + +set.locale() + error <- download.file(url='${RPKG_DESCRIPTION_URL}',destfile='DESCRIPTION',quiet=${QUIET_CURL},method='curl') if (error) - quit(status=error) + { + message('ERROR: Downloading the DESCRIPTION file for ${RPKG} failed;') + message(' perhaps the package no longer exists?') + quit(save='no',status=error) + } metadata <- read.dcf(file='DESCRIPTION', fields=c('Package','Version','Title','Description','License','Imports','Depends')) -CVS <- '# \$NetBSD\$' -CATEGORIES <- categories() -MASTER.SITES <- 'MASTER_SITES= \${MASTER_SITE_R_CRAN:=contrib/}' -MAINTAINER <- maintainer('${MAINTAINER_EMAIL}') -HOMEPAGE <- 'HOMEPAGE= \${R_HOMEPAGE_BASE}/${RPKG}/' -COMMENT <- comment(metadata[3]) -LICENSE <- license(metadata[5]) -R_PKGNAME <- package(metadata[1]) -R_PKGVER <- version(metadata[2]) -USE_LANGUAGES <- use.languages(metadata[6],metadata[7]) -DEPENDENCIES <- depends(metadata[6],metadata[7]) -DEPENDS <- DEPENDENCIES[1] -BUILDLINK3.MK <- DEPENDENCIES[2] -INCLUDE.R <- '.include "../../math/R/Makefile.extension"' -INCLUDE.PKG <- '.include "../../mk/bsd.pkg.mk"' - -DESCR <- description(metadata[4]) - -Makefile <- list() -Makefile <- append(Makefile,CVS) -Makefile <- append(Makefile,'') -Makefile <- append(Makefile,CATEGORIES) -Makefile <- append(Makefile,MASTER.SITES) -Makefile <- append(Makefile,'') -Makefile <- append(Makefile,MAINTAINER) -Makefile <- append(Makefile,HOMEPAGE) -Makefile <- append(Makefile,COMMENT) -Makefile <- append(Makefile,LICENSE) -Makefile <- append(Makefile,'') -Makefile <- append(Makefile,R_PKGNAME) -Makefile <- append(Makefile,R_PKGVER) -Makefile <- append(Makefile,'') -Makefile <- append(Makefile,USE_LANGUAGES) -Makefile <- append(Makefile,DEPENDS) -Makefile <- append(Makefile,INCLUDE.R) -Makefile <- append(Makefile,BUILDLINK3.MK) -Makefile <- append(Makefile,INCLUDE.PKG) -Makefile <- paste(unlist(Makefile),collapse='\n') - -write(Makefile,'Makefile') -write(DESCR,'DESCR') +create.Makefile(metadata) +create.DESCR(metadata) EOF eval ${R_CMD} retval=${?} @@ -444,35 +1387,139 @@ create_distinfo () ${ECHO_FETCH} "==> Fetching R-${RPKG} ..." MAKE_CMD="${MAKE} makesum ${STDOUT_MAKESUM}" eval ${MAKE_CMD} - return ${?} + error=${?} + if [ ${error} -eq 0 ]; then + MAKE_CMD="${MAKE} makepatchsum ${STDOUT_MAKESUM}" + eval ${MAKE_CMD} + error=${?} + fi + return ${error} +} + +create_buildlink3_mk () +{ + if [ -f buildlink3.mk.orig ]; then + PKGVERSION=$(${MAKE} show-var VARNAME=PKGVERSION) + sed -E -e "/BUILDLINK_API_DEPENDS\./s/[[:digit:].]+$/${PKGVERSION}/" \ + buildlink3.mk.orig > buildlink3.mk + fi } extract () { - ${ECHO_EXTRACT} "==> Extracting R-${RPKG} ..." - MAKE_CMD="${MAKE} extract ${STDOUT_EXTRACT}" + ${ECHO_EXTRACT} "[ ${LEVEL} ] Extracting R-${RPKG} ..." + MAKE_CMD="env SKIP_DEPENDS=yes ${MAKE} clean extract ${STDOUT_EXTRACT}" eval ${MAKE_CMD} } -cleanup () +check_license () { - [ "X${DESCRIPTION}" != "X" ] || rm -f DESCRIPTION - if [ -f DESCR.orig ] && cmp -s DESCR.orig DESCR; then + rm -f LICENSE + # echo '===> LICENSE files:' + if [ -f work/*/LICENSE ]; then + grep -v "^YEAR: " work/*/LICENSE \ + | grep -v "^COPYRIGHT HOLDER: " \ + | grep -v "^ORGANIZATION: " \ + > LICENSE + if [ -s LICENSE ]; then + # ninka -d LICENSE + cp work/*/LICENSE . + /bin/echo -n "[ ${LEVEL} ] Current license: " + grep LICENSE Makefile + echo "[ ${LEVEL} ] Please check it against the following:" + cat LICENSE + else + rm LICENSE + sed -E -e 's/[[:blank:]]+#[[:blank:]]+\+ file LICENSE[[:blank:]]+.*$//' Makefile > Makefile.$$ \ + && mv Makefile.$$ Makefile + grep -q "file LICENSE" Makefile && echo "[ ${LEVEL} ] 'file LICENSE' in Makefile but no relevant license information" + fi + fi +} + +check_copying () +{ + if [ -f work/*/COPYING ]; then + cp work/*/COPYING . + fi +} + +cleanup_DESCR () +{ + if [ -f DESCR -a -f DESCR.orig ]; then + if diff --ignore-case --ignore-all-space --ignore-blank-lines DESCR.orig DESCR > /dev/null; then + mv DESCR.orig DESCR + else + mv DESCR DESCR.new + mv DESCR.orig DESCR + fi + elif [ -f DESCR.orig ]; then mv DESCR.orig DESCR fi - if [ -f Makefile.orig ] && cmp -s Makefile.orig Makefile; then +} + +cleanup_Makefile () +{ + if [ -f Makefile -a -f Makefile.orig ]; then + diff --ignore-case --ignore-all-space --ignore-blank-lines Makefile.orig Makefile > /dev/null \ + && mv Makefile.orig Makefile + elif [ -f Makefile.orig ]; then mv Makefile.orig Makefile + else + echo "[ ${LEVEL} ] $(pwd): neither Makefile nor Makefile.orig" fi - if [ -f distinfo.orig ] && cmp -s distinfo.orig distinfo; then +} + +cleanup_buildlink3 () +{ + if [ buildlink3.mk -a -f buildlink3.mk.orig ]; then + diff --ignore-case --ignore-all-space --ignore-blank-lines buildlink3.mk.orig buildlink3.mk > /dev/null \ + && mv buildlink3.mk.orig buildlink3.mk + elif [ -f buildlink3.mk.orig ]; then + mv buildlink3.mk.orig buildlink3.mk + fi +} + +cleanup_distinfo () +{ + if [ -f distinfo -a -f distinfo.orig ]; then + tail +2 distinfo.orig > ${TMPDIR}/distinfo.orig.${PID} + tail +2 distinfo > ${TMPDIR}/distinfo.${PID} + cmp -s ${TMPDIR}/distinfo.orig.${PID} ${TMPDIR}/distinfo.${PID} \ + && mv distinfo.orig distinfo + rm -f ${TMPDIR}/distinfo.orig.${PID} ${TMPDIR}/distinfo.${PID} + elif [ -f distinfo.orig ]; then mv distinfo.orig distinfo fi +} + +cleanup_misc_files () +{ + [ "X${DESCRIPTION}" != "X" ] || rm -f DESCRIPTION rm -f ${R_FILE} + rm -f CATEGORIES + rm -f COMMENT rm -f MAINTAINER + rm -f USE_LANGUAGES + rm -f USE_TOOLS + rm -f DEPENDS + rm -f BUILDLINK3.MK + [ ${LEVEL} -eq 0 ] && rm -f ${PACKAGES_LIST} + [ ${LEVEL} -eq 0 ] && rm -f ${DEPENDENCY_LIST} +} + +cleanup () +{ + cleanup_DESCR + cleanup_Makefile + cleanup_buildlink3 + cleanup_distinfo + cleanup_misc_files } messages () { - if [ ${QUIET} -eq 0 -a ${TOP_LEVEL} -ne 0 ]; then + if [ ${QUIET} -eq 0 -a ${LEVEL} -eq 0 ]; then cat << EOF Please do not forget the following: @@ -482,6 +1529,8 @@ Please do not forget the following: o verify the LICENSE. o verify the DEPENDS, especially the categories. EOF + [ -f buildlink3.mk ] && echo "- check buildlink3.mk" + [ "X${DESCRIPTION}" != "X" ] && echo "- remove DESCRIPTION." if [ ${RECURSIVE} -ne 0 ]; then cat << EOF @@ -497,7 +1546,6 @@ EOF tsort ${DEPENDENCY_LIST} > depends echo "- It may be useful to test these packages in the following order:" awk 'BEGIN{printf(" ")} {printf(" R-%s",$0)}' depends && echo - [ ${TOP_LEVEL} -eq 0 ] || rm -f ${DEPENDENCY_LIST} fi fi fi @@ -505,17 +1553,26 @@ EOF ${ECHO_BANNER} "${BANNER_MSG} ..." check_for_R -preserve_original_files +check_for_no_recursion +preserve_original_content make_package error=${?} if [ ${error} -eq 0 ]; then edit_Makefile + error=${?}; [ ${error} -eq 0 ] || exit ${error} edit_DESCR - create_distinfo error=${?}; [ ${error} -eq 0 ] || exit ${error} - # extract + create_distinfo + create_buildlink3_mk + extract + check_license + check_copying fi -cleanup messages -${ECHO_DONE} "${BANNER_MSG}: done" +cleanup +if [ ${error} -eq 0 ]; then + ${ECHO_DONE} "${BANNER_MSG}: completed successfully" +else + ${ECHO_DONE} "${BANNER_MSG}: FAILED" +fi exit ${error} diff --git a/pkgtools/R2pkg/files/RELEASE b/pkgtools/R2pkg/files/RELEASE index 366472af4d8..44e38f74a77 100644 --- a/pkgtools/R2pkg/files/RELEASE +++ b/pkgtools/R2pkg/files/RELEASE @@ -1,4 +1,4 @@ -$NetBSD: RELEASE,v 1.1 2019/06/24 13:46:04 brook Exp $ +$NetBSD: RELEASE,v 1.2 2019/08/01 13:11:08 brook Exp $ RELEASE ======= @@ -28,3 +28,17 @@ R2pkg v.0.4 (2019-06-24) * Use buildlink3.mk files for dependencies when available. +R2pkg v.0.5 (2019-07-31) +------------------------ + +* Improve support for updating packages. + +* Add explicit option to create packages (-c). + +* Relax comparisons for COMMENTS to allow case and some punctuation + differences. + +* Expand the list of licenses and base R packages checked for. + +* Improve support for recursion. + |