summaryrefslogtreecommitdiff
path: root/parallel
AgeCommit message (Collapse)AuthorFilesLines
2018-04-22slurm: Rename to slurm-wlm to resolve conflict with net/slurmbacon3-13/+6
ok wiz@
2018-04-16Recursive bump for new fribidi dependency in pango.wiz1-2/+2
2018-03-25parallel/slurm: Update to 17.11.5bacon33-309/+446
Changes since 2.6.4: Adds additional capabilities such as SQL accounting and job profiling Change maintainer to bacon@NetBSD.org Install example Linux init scripts
2018-03-12Recursive bumps for fontconfig and libzip dependency changes.wiz1-2/+2
2018-02-25pvm3: follow redirectwiz1-2/+2
2018-02-25p5-Parallel-Pvm: follow redirectwiz1-3/+2
2018-02-02py-billiard: Set _XOPEN_SOURCE correctly.jperkin2-1/+24
2018-01-07Fix indentation in buildlink3.mk files.rillig1-3/+3
The actual fix as been done by "pkglint -F */*/buildlink3.mk", and was reviewed manually. There are some .include lines that still are indented with zero spaces although the surrounding .if is indented. This is existing practice.
2018-01-04Added CONFLICTS; removed unused linesadam1-8/+2
2018-01-04openmpi: updated to 1.10.7adam5-66/+53
1.10.7: - Fix bug in TCP BTL that impacted performance on 10GbE (and faster) networks by not adjusting the TCP send/recv buffer sizes and using system default values - Add missing MPI_AINT_ADD and MPI_AINT_DIFF function delcarations in mpif.h - Fixed time reported by MPI_WTIME; it was previously reported as dependent upon the CPU frequency. - Fix platform detection on FreeBSD - Fix a bug in the handling of MPI_TYPE_CREATE_DARRAY in MPI_(R)(GET_)ACCUMULATE - Fix openib memory registration limit calculation - Add missing MPI_T_PVAR_SESSION_NULL in mpi.h - Fix "make distcheck" when using external hwloc and/or libevent packages - Add latest ConnectX-5 vendor part id to OpenIB device params - Fix race condition in the UCX PML - Fix signal handling for rsh launcher - Fix Fortran compilation errors by removing MPI_SIZEOF in the Fortran interfaces when the compiler does not support it - Fixes for the pre-ignore-TKR "mpi" Fortran module implementation (i.e., for older Fortran compilers -- these problems did not exist in the "mpi" module implementation for modern Fortran compilers): - Add PMPI_* interfaces - Fix typo in MPI_FILE_WRITE_AT_ALL_BEGIN interface name - Fix typo in MPI_FILE_READ_ORDERED_BEGIN interface name - Fixed the type of MPI_DISPLACEMENT_CURRENT in all Fortran interfaces to be an INTEGER(KIND=MPI_OFFSET_KIND). - Fixed typos in MPI_INFO_GET_* man pages. Thanks to Nicolas Joly for the patch - Fix typo bugs in wrapper compiler script
2018-01-01Sort PLIST files.rillig3-28/+28
Unsorted entries in PLIST files have generated a pkglint warning for at least 12 years. Somewhat more recently, pkglint has learned to sort PLIST files automatically. Since pkglint 5.4.23, the sorting is only done in obvious, simple cases. These have been applied by running: pkglint -Cnone,PLIST -Wnone,plist-sort -r -F
2018-01-01Replaced $(ROUND) with ${CURLY} variable references.rillig4-20/+20
This has been a pkglint warning for several years now, and pkglint can even fix it automatically. And it did for this commit. Only in lang/mercury, two passes of autofixing were necessary because there were nested variables.
2017-09-16Reset maintainerwiz1-2/+2
2017-09-06Follow some redirects.wiz1-2/+2
2017-08-16Follow some http redirects.wiz1-2/+2
2017-07-30Switch github HOMEPAGEs to https.wiz1-2/+2
2017-07-173.5.0.3:adam2-8/+10
- Adds Process._authkey alias to .authkey for 2.7 compat. - Remove superfluous else clause from max_memory_per_child_check. - Document and test all supported Python versions. - Extend 'Process' to be compatible with < Py3.5. - Use a properly initialized logger in pool.py error logging. - _trywaitkill can now kill a whole process group if the worker process declares itself as a group leader. - Fix cpython issue 14881 (See http://bugs.python.org/issue14881). - Fix for a crash on windows. - Fix messaging in case of worker exceeds max memory.
2017-06-30Correct ganglia-monitor-core build failure on NetBSD 8.99.1kamil2-7/+15
Address <sys/user.h> removal fallout
2017-06-20PLIST fix for non-f90 buildsadam2-7/+5
2017-06-05Recursive revbump from lang/perl5 5.26.0ryoon1-2/+2
2017-04-29Changes in 3.2:adam12-159/+562
* Added support for MPI-3.1 features including nonblocking collective I/O, address manipulation routines, thread-safety for MPI initialization, pre-init functionality, and new MPI_T routines to look up variables by name. * Fortran 2008 bindings are enabled by default and fully supported. * Added support for the Mellanox MXM InfiniBand interface. (thanks to Mellanox for the code contribution). * Added support for the Mellanox HCOLL interface for collectives. (thanks to Mellanox for the code contribution). * Significant stability improvements to the MPICH/portals4 implementation. * Completely revamped RMA infrastructure including several scalability improvements, performance improvements, and bug fixes. * Added experimental support for Open Fabrics Interfaces (OFI) version 1.0.0. https://github.com/ofiwg/libfabric (thanks to Intel for code contribution) * The Myrinet MX network module, which had a life cyle from 1.1 till 3.1.2, has now been deleted. * Several other minor bug fixes, memory leak fixes, and code cleanup.
2017-04-28Version 1.11.7adam3-9/+10
-------------- * Fix hwloc-bind --membind for CPU-less NUMA nodes (again). Thanks to Gilles Gouaillardet for reporting the issue. * Fix a memory leak on IBM S/390 platforms running Linux. * Fix a memory leak when forcing the x86 backend first on amd64/topoext platforms running Linux. * Command-line tools now support "hbm" instead "numanode" for filtering only high-bandwidth memory nodes when selecting locations. + hwloc-bind also support --hbm and --no-hbm for filtering only or no HBM nodes. * Add --children and --descendants to hwloc-info for listing object children or object descendants of a specific type. * Add --no-index, --index, --no-attrs, --attrs to disable/enable display of index numbers or attributes in the graphical lstopo output. * Try to gather hwloc-dump-hwdata output from all possible locations in hwloc-gather-topology. * Updates to the documentation of locations in hwloc(7) and command-line tools manpages.
2017-04-09Changes 3.5.0.2:adam3-48/+42
- max_memory_per_child was measured in kilobytes on Linux, but bytes on *BSD/MacOS, it's now always kilobytes. - Windows: Adds support for max_memory_per_child, but requires the ``psutil`` package to be installed. - Fixed bug in ForkingPickler.loadbuf, where it tried to pass a BytesIO instance directly to ``pickle.loads`` on Python 2.7.
2017-03-23Extend SHA512 checksums to various files I have on my local distfilejoerg1-1/+2
mirror.
2017-02-12Recursive revbump from fonts/harfbuzzryoon1-2/+2
2017-02-06Recursive bump for harfbuzz's new graphite2 dependency.wiz1-1/+2
2017-02-06Fix PLIST for linux.markd1-3/+14
2017-02-06fix linux build and PLISTmarkd2-3/+6
2017-01-19Convert all occurrences (353 by my count) ofagc2-7/+7
MASTER_SITES= site1 \ site2 style continuation lines to be simple repeated MASTER_SITES+= site1 MASTER_SITES+= site2 lines. As previewed on tech-pkg. With thanks to rillig for fixing pkglint accordingly.
2016-12-25PLIST catchup for recent update.markd1-1/+31
2016-12-19Revert nonsensical changes.asau2-15/+15
2016-12-17Fixed pkglint warnings.rillig2-15/+15
2016-10-23Update to OpenMPI 1.10.4asau6-52/+93
Prompted by Nicolas Joly in private mail. 1.10.4 - 01 Sept 2016 ------ - Fix assembler support for MIPS - Improve memory handling for temp buffers in collectives - Fix [all]reduce with non-zero lower bound datatypes Thanks Hristo Iliev for the report - Fix non-standard ddt handling. Thanks Yuki Matsumoto for the report - Various libnbc fixes. Thanks Yuki Matsumoto for the report - Fix typos in request RMA bindings for Fortran. Thanks to @alazzaro and @vondele for the assist - Various bug fixes and enhancements to collective support - Fix predefined types mapping in hcoll - Revive the coll/sync component to resolve unexpected message issues during tight loops across collectives - Fix typo in wrapper compiler for Fortran static builds 1.10.3 - 15 June 2016 ------ - Fix zero-length datatypes. Thanks to Wei-keng Liao for reporting the issue. - Minor manpage cleanups - Implement atomic support in OSHMEM/UCX - Fix support of MPI_COMBINER_RESIZED. Thanks to James Ramsey for the report - Fix computation of #cpus when --use-hwthread-cpus is used - Add entry points for Allgatherv, iAllgatherv, Reduce, and iReduce for the HCOLL library - Fix an HCOLL integration bug that could signal completion of request while still being worked - Fix computation of cores when SMT is enabled. Thanks to Ben Menadue for the report - Various USNIC fixes - Create a datafile in the per-proc directory in order to make it unique per communicator. Thanks to Peter Wind for the report - Fix zero-size malloc in one-sided pt-to-pt code. Thanks to Lisandro Dalcin for the report - Fix MPI_Get_address when passed MPI_BOTTOM to not return an error. Thanks to Lisandro Dalcin for the report - Fix MPI_TYPE_SET_ATTR with NULL value. Thanks to Lisandro Dalcin for the report - Fix various Fortran08 binding issues - Fix memchecker no-data case. Thanks to Clinton Stimpson for the report - Fix CUDA support under OS-X - Fix various OFI/MTL integration issues - Add MPI_T man pages - Fix one-sided pt-to-pt issue by preventing communication from happening before a target enters a fence, even in the no-precede case - Fix a bug that disabled Totalview for MPMD use-case - Correctly support MPI_UNWEIGHTED in topo-graph-neighbors. Thanks to Jun Kudo for the report - Fix singleton operations under SLURM when PMI2 is enabled - Do not use MPI_IN_PLACE in neighborhood collectives for non-blocking collectives (libnbc). Thanks to Jun Kudo for the report - Silence autogen deprecation warnings for newer versions of Perl - Do not return MPI_ERR_PENDING from collectives - Use type int* for MPI_WIN_DISP_UNIT, MPI_WIN_CREATE_FLAVOR, and MPI_WIN_MODEL. Thanks to Alastair McKinstry for the report - Fix register_datarep stub function in IO/OMPIO. Thanks to Eric Chamberland for the report - Fix a bus error on MPI_WIN_[POST,START] in the shared memory one-sided component - Add several missing MPI_WIN_FLAVOR constants to the Fortran support - Enable connecting processes from different subnets using the openib BTL - Fix bug in basic/barrier algorithm in OSHMEM - Correct process binding for the --map-by node case - Include support for subnet-to-subnet routing over InfiniBand networks - Fix usnic resource check - AUTHORS: Fix an errant reference to Subversion IDs - Fix affinity for MPMD jobs running under LSF - Fix many Fortran binding bugs - Fix `MPI_IN_PLACE`-related bugs - Fix PSM/PSM2 support for singleton operations - Ensure MPI transports continue to progress during RTE barriers - Update HWLOC to 1.9.1 end-of-series - Fix a bug in the Java command line parser when the -Djava.library.path options was given by the user - Update the MTL/OFI provider selection behavior - Add support for clock_gettime on Linux. - Correctly detect and configure for Solaris Studio 12.5 beta compilers - Correctly compute #slots when -host is used for MPMD case - Fix a bug in the hcoll collectives due to an uninitialized field - Do not set a binding policy when oversubscribing a node - Fix hang in intercommunicator operations when oversubscribed - Speed up process termination during MPI_Abort - Disable backtrace support by default in the PSM/PSM2 libraries to prevent unintentional conflicting behavior. 1.10.2: 26 Jan 2016 ------------------- ********************************************************************** * OSHMEM is now 1.2 compliant ********************************************************************** - Fix NBC_Copy for legitimate zero-size messages - Fix multiple bugs in OSHMEM - Correctly handle mpirun --host <user>@<ip-address> - Centralize two MCA params to avoid duplication between OMPI and OSHMEM layers: opal_abort_delay and opal_abort_print_stack - Add support for Fujitsu compilers - Add UCX support for OMPI and OSHMEM - Correctly handle oversubscription when not given directives to permit it. Thanks to @ammore1 for reporting it - Fix rpm spec file to not include the /usr directory - Add Intel HFI1 default parameters for the openib BTL - Resolve symbol conflicts in the PSM2 library - Add ability to empty the rgpusm cache when full if requested - Fix another libtool bug when -L requires a space between it and the path. Thanks to Eric Schnetter for the patch. - Add support for OSHMEM v1.2 APIs - Improve efficiency of oshmem_preconnect_all algorithm - Fix bug in buffered sends support - Fix double free in edge case of mpirun. Thanks to @jsharpe for the patch - Multiple one-sided support fixes - Fix integer overflow in the tuned "reduce" collective when using buffers larger than INT_MAX in size - Fix parse of user environment variables in mpirun. Thanks to Stefano Garzarella for the patch - Performance improvements in PSM2 support - Fix NBS iBarrier for inter-communicators - Fix bug in vader BTL during finalize - Improved configure support for Fortran compilers - Fix rank_file mapper to support default --slot-set. Thanks to Matt Thompson for reporting it - Update MPI_Testsome man page. Thanks to Eric Schnetter for the suggestion - Fix missing resize of the returned type for subarray and darray types. Thanks to Keith Bennett and Dan Garmann for reporting it - Fix Java support on OSX 10.11. Thanks to Alexander Daryin for reporting the problem - Fix some compilation issues on Solaris 11.2. Thanks to Paul Hargrove for his continued help in such areas
2016-10-21Update to hwloc 1.11.4.asau3-9/+10
Version 1.11.4 -------------- * Add MemoryMode and ClusterMode attributes in the Machine object on KNL. Add doc/examples/get-knl-modes.c for an example of retrieving them. Thanks to Grzegorz Andrejczuk. * Fix Linux build with -m32 with respect to libudev. Thanks to Paul Hargrove for reporting the issue. * Fix build with Visual Studio 2015, thanks to Eloi Gaudry for reporting the issue and providing the patch. * Don't forget to display OS device children in the graphical lstopo. * Fix a memory leak on Solaris, thanks to Bryon Gloden for the patch. * Properly handle realloc() failures, thanks to Bryon Gloden for reporting the issue. * Fix lstopo crash in ascii/fig/windows outputs when some objects have a lstopoStyle info attribute. Version 1.11.3 -------------- * Bug fixes + Fix a memory leak on Linux S/390 hosts with books. + Fix /proc/mounts parsing on Linux by using mntent.h. Thanks to Nathan Hjelm for reporting the issue. + Fix a x86 infinite loop on VMware due to the x2APIC feature being advertised without actually being fully supported. Thanks to Jianjun Wen for reporting the problem and testing the patch. + Fix the return value of hwloc_alloc() on mmap() failure. Thanks to Hugo Brunie for reporting the issue. + Fix the return value of command-line tools in some error cases. + Do not break individual thread bindings during x86 backend discovery in a multithreaded process. Thanks to Farouk Mansouri for the report. + Fix hwloc-bind --membind for CPU-less NUMA nodes. + Fix some corner cases in the XML export/import of application userdata. * API Improvements + Add HWLOC_MEMBIND_BYNODESET flag so that membind() functions accept either cpusets or nodesets. + Add hwloc_get_area_memlocation() to check where pages are actually allocated. Only implemented on Linux for now. - There's no _nodeset() variant, but the new flag HWLOC_MEMBIND_BYNODESET is supported. + Make hwloc_obj_type_sscanf() parse back everything that may be outputted by hwloc_obj_type_snprintf(). * Detection Improvements + Allow the x86 backend to add missing cache levels, so that it completes what the Solaris backend lacks. Thanks to Ryan Zezeski for reporting the issue. + Do not filter-out FibreChannel PCI adapters by default anymore. Thanks to Matt Muggeridge for the report. + Add support for CUDA compute capability 6.x. * Tools + Add --support to hwloc-info to list supported features, just like with hwloc_topology_get_support(). - Also add --objects and --topology to explicitly switch between the default modes. + Add --tid to let hwloc-bind operate on individual threads on Linux. + Add --nodeset to let hwloc-bind report memory binding as NUMA node sets. + hwloc-annotate and lstopo don't drop application userdata from XMLs anymore. - Add --cu to hwloc-annotate to drop these application userdata. + Make the hwloc-dump-hwdata dump directory configurable through configure options such as --runstatedir or --localstatedir. * Misc Improvements + Add systemd service template contrib/systemd/hwloc-dump-hwdata.service for launching hwloc-dump-hwdata at boot on Linux. Thanks to Grzegorz Andrejczuk. + Add HWLOC_PLUGINS_BLACKLIST environment variable to prevent some plugins from being loaded. Thanks to Alexandre Denis for the suggestion. + Small improvements for various Windows build systems, thanks to Jonathan L Peyton and Marco Atzeri. Version 1.11.2 -------------- * Improve support for Intel Knights Landing Xeon Phi on Linux: + Group local NUMA nodes of normal memory (DDR) and high-bandwidth memory (MCDRAM) together through "Cluster" groups so that the local MCDRAM is easy to find. - See "How do I find the local MCDRAM NUMA node on Intel Knights Landing Xeon Phi?" in the documentation. - For uniformity across all KNL configurations, always have a NUMA node object even if the host is UMA. + Fix the detection of the memory-side cache: - Add the hwloc-dump-hwdata superuser utility to dump SMBIOS information into /var/run/hwloc/ as root during boot, and load this dumped information from the hwloc library at runtime. - See "Why do I need hwloc-dump-hwdata for caches on Intel Knights Landing Xeon Phi?" in the documentation. Thanks to Grzegorz Andrejczuk for the patches and for the help. * The x86 and linux backends may now be combined for discovering CPUs through x86 CPUID and memory from the Linux kernel. This is useful for working around buggy CPU information reported by Linux (for instance the AMD Bulldozer/Piledriver bug below). Combination is enabled by passing HWLOC_COMPONENTS=x86 in the environment. * Fix L3 cache sharing on AMD Opteron 63xx (Piledriver) and 62xx (Bulldozer) in the x86 backend. Thanks to many users who helped. * Fix the overzealous L3 cache sharing fix added to the x86 backend in 1.11.1 for AMD Opteron 61xx (Magny-Cours) processors. * The x86 backend may now add the info attribute Inclusive=0 or 1 to caches it discovers, or to caches discovered by other backends earlier. Thanks to Guillaume Beauchamp for the patch. * Fix the management on alloc_membind() allocation failures on AIX, HP-UX and OSF/Tru64. * Fix spurious failures to load with ENOMEM on AIX in case of Misc objects below PUs. * lstopo improvements in X11 and Windows graphical mode: + Add + - f 1 shortcuts to manually zoom-in, zoom-out, reset the scale, or fit the entire window. + Display all keyboard shortcuts in the console. * Debug messages may be disabled at runtime by passing HWLOC_DEBUG_VERBOSE=0 in the environment when --enable-debug was passed to configure. * Add a FAQ entry "What are these Group objects in my topology?".
2016-10-03Call sun-iss sissl-1.1 to match OSI.wiz1-3/+2
2016-09-19Update parallel/pdsh to 2.31.fhajny2-11/+11
* Changes in pdsh-2.31 (2013-11-07) =================================== -- Fix issue 56: slurm: Allow mixed use of -P, -w and -j options. -- Fix issue 59: pdsh very slow when using a few thousand hosts and genders. -- testsuite: Expanded tests for genders module (Pythagoras Watson) * Changes in pdsh-2.30 (2013-03-02) =================================== -- Fix issue 55: genders -X option removes more hosts than expected. (This was a generic fix for hostname matching, so it probably affected -x and other options as well.) -- testsuite: Add test for issue 55.
2016-09-13Update parallel/parallel to 20160822fhajny4-62/+35
Upstream changes: 20160822 - Bug fixes and man page updates. 20160722 - env_parallel is now ready for wider testing. It is still beta quality. - env_parallel is heavily modified for all shells and testing has been increased. - Selectively choosing what to export using --env now works for env_parallel (bash, csh, fish, ksh, pdksh, tcsh, zsh). - --round-robin now gives more work to a job that processes faster instead of same amount to all jobs. - --pipepart works on block devices on GNU/Linux. - Bug fixes and man page updates. 20160622 - $PATH can now be exported using --env PATH. Useful if GNU Parallel is not in your path on remote machines. - If --block is left out, --pipepart will use a block size that will result in 10 jobs per jobslot. - Bug fixes and man page updates. 20160522 - niceload --net pauses the program if the internet connection is overloaded. - Bug fixes and man page updates. 20160422 - :::+ and ::::+ work like ::: and :::: but links this input source to the previous input source in a --xapply fashion. Contrary to --xapply values do not wrap: The shortest input source determines the length. - --line-buffer --keep-order now outputs continously from the oldest job still running. This is more what you would expect than the earlier behaviour where --keep-order had no effect with --line-buffer. - env_parallel supports tcsh, csh, pdksh. In fish it now supports arrays. In csh/tcsh it now supports variables, aliases, and arrays with no special chars. In pdksh it supports aliases, functions, variables, and arrays. - Function exporting on Mac OS X works around old Bash version. - Better CPU detection on OpenIndiana. - Bug fixes and man page updates. 20160322 - env_parallel is a function that exports the environment (functions, aliases, variables, and arrays) to GNU Parallel. Run 'man env_parallel' for details. - niceload --prg now searches for substrings if no process with the name is found. - Bug fixes and man page updates. 20160222 - Bug fixes and man page updates. 20160122 - --sql DBURL uses DBURL as storage for jobs and output. It does not run any jobs so it requires at least one --sqlworker. DBURL must point to a table. - --sqlworker DBURL gets jobs from DBURL and stores the result back to DBURL. - --sqlandworker is a shorthand for --sql and --sqlworker. - --sqlworker requires the output of a single job to fit in memory. - --results now also saves a file called 'seq' containing the sequence number. - If $PARALLEL_ENV is a file, then that file will be read into $PARALLEL_ENV. - man parallel_tutorial has been given an overhaul. - --workdir now accepts replacementstrings. - Bug fixes and man page updates. 20151222 - --transfer is now an alias for --transferfile {}. - --transferfile works like --transfer, but takes an argument like --return. This makes it possible to combine transferring files with multiple input sources: parallel -S server --tf {1} wc {2} {1} ::: * ::: -l -w -c - total_jobs() can now be used in {= =}: parallel echo job {#} of '{= $_=total_jobs() =}' ::: {1..50} - Bug fixes and man page updates. 20151122 - Bug fixes and man page updates. 20151022 - --plus makes it possible to use {##} as a shorthand for {= $_=$Global::JobQueue->total_jobs() =} which gives the the number of jobs to run in total. - {= $_=$Global::JobQueue->total_jobs() =} is incompatible with -X, -m, and --xargs. - GNU Parallel is now mostly compatible with lsh (http://www.lysator.liu.se/~nisse/lsh/) and somewhat compatible with autossh (http://www.harding.motd.ca/autossh/). - --workdir ... now also works when run locally. - Bug fixes and man page updates. 20150922 - Bug fixes and man page updates. 20150822 - If $job->skip() is called in {= =} the job will not be run. - @arg can now be used in {= =}: parallel echo '{= $arg[2] < 5 and $_="j" =}' ::: 1 2 3 ::: 4 5 6 - Bug fixes and man page updates. 20150722 - If a record fits in a block, --block only reads upto the size of --block. If not: --block reads a full --block more. - Due to widely spread distributions still using --tollef even after a year of being retired following a year of being obsolete, --tollef now gives an error unless used with --gnu. - --nice now uses a perl wrapper instead of the nice command. - Bug fixes and man page updates. 20150622 - --halt has been rewritten completely. You can now combine percentages with success or fail. See the man page. - Exit values 102..254 have been removed. 101 means more than 100 jobs failed. - Killing through --timeout, --memfree, or --halt is now done as a process group. - --termseq determines which signals are sent when a job is killed. - An empty argument would previously cause no string to be inserted. This is now changed to '' being inserted, thus prepending a space to the output of: parallel echo {} b ::: '' - $PARALLEL_ENV can now be set to an environment prepending the command. Used in env_parallel as mentioned in the manpage. - --retry-failed will retry all failed jobs in a joblog. It will ignore any command given. - --ssh and $PARALLEL_SSH can be used to set the command used for ssh. The command is assumed to behave the same way as ssh. - --fifo now works in csh, too. - Q(...) can be used in {= =} to shell quote a string. - Bug fixes and man page updates. 20150522 - Security: The security issue for --sshlogin + --fifo/--cat has been fixed. Thereby all issues with http://lists.gnu.org/archive/html/parallel/2015-04/msg00045.html have been fixed. - Security: After further security analysis the issue fixed in 20150422 also fixed the problem for --tmux. - Bug fixes and man page updates. 20150422 New in this release: - Security fix. An attacker on the local system could make you overwrite one of your own files with a single byte. The problem exist when you use --compress or --tmux or --pipe or --cat or --fifo. The attacker must figure out the randomly chosen file name and create a symlink within a time window of 15 ms. - --shuf will generate all jobs, and shuffle them before running them. This is useful to get a quick preview of the results before running the full batch. - Bug fixes and man page updates. 20150322 New in this release: - --number-of-cores respects 'taskset' on GNU/Linux. - --joblog --pipe gives the data send and received in the log. - Bug fixes and man page updates. 20150222 - --tmux has gotten a major overhaul. - Bug fixes and man page updates. 20150122 - Remote jobs now send stderr (standard error) to stderr (standard error) instead of stdout (standard output). - Remote execution command is now packed using base64 encoded bzip2. This means that big environments (app. 100 KB) can be transferred. This change makes remote execution alpha quality. - --semaphoretimeout now takes a negative value. This means give up if the semaphore is not gotten within the timeout. - --halt -1 and -2 now means halt if a job succeeds (so the opposite of 1 and 2). - --no-keep-order will reverse --keep-order. - Bash's second fix of shellshock caused --env to break again when exporting functions. This has been fixed again. - A semibig refactoring of big functions. All non-trivial functions are now less than 100 lines. The refactoring makes this release beta quality. - A description of the design decisions for GNU Parallel can be found in 'man parallel_design'. - A bug fix in replacement strings caused rewrite of the replacement function. This makes use of replacement strings alpha quality. - Bug fixes and man page updates. 20141122 - Remote systems can be divided into hostgroups (e.g. web and db) by prepending '@groupname/' to the sshlogin. Multiple groups can be given by separating groups with '+'. E.g. @web/www1 @web+db/www2 @db/mariadb - Remote execution can be restricted to servers that are part of one or more groups by '@groupname' as an sshlogin. Multiple groups can be given by separating groups with '+'. E.g. -S @web or -S @db+web - With --hostgroup you can restrict arguments to certain hostgroups by appending '@groupname' to the argument. Multiple groups can be given by separating groups with '+'. E.g. my_web_arg@web db-or-web-arg@db+web db-only-arg@db Thanks to Michel Courtine for developing a prototype for this. - Bug fixes and man page updates. 20141022 - --env was changed to deal with Bash's new way of exporting a function. - Bug fixes and man page updates. 20140922 - If the file give as --sshloginfile is changed it will be re-read when a job finishes though at most once per second. This makes it possible to add and remove hosts while running. 20140822 - GNU Parallel now uses the same shell it was started from as the command shell for local jobs. So if GNU Parallel is started from tcsh it will use tcsh as its shell even if the login $SHELL is different. For remote jobs the login $SHELL will be used. - The whole current environment in bash can be copied by using a shell wrapper function (Search manual for env_parallel). - --plus adds the replacement strings {+/} {+.} {+..} {+...} {..} {...} {/..} {/...}. The idea being that '+foo' matches the opposite of 'foo' and {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} = {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...} - GNU Parallel now deals correctly with the combination rsync-3.1.X-client and rsync-2.5.7-server - Bug fixes and man page updates. 20140722 - {= perl expression =} can be used as replacement string. The expression should modify $_. E.g. {= s/\.gz$// =} to remove .gz from the string. This makes replacement strings extremely flexible. - Positional perl expressions (similar to {2}) are given as {=2 perl expression=} where 2 is the position. - One small backwards incompatibility: {1}_{2} will replace {2} with the empty string if there is only one argument. Previously {2} would have been left untouched. - Replacement strings can be defined using --rpl. E.g. parallel --rpl '{.gz} s/\.gz$//' echo {.gz} ::: *.gz - The parenthesis around {= perl expression =} can be changed with --parens. - --tmux will direct the output to a tmux session instead of files. Each running jobs will be in its own window. - --halt 10% will stop spawning new jobs if 10% failed so far. - Bug fixes and man page updates. 20140622 - --shebang and --shebang-wrap now works on FreeBSD. - --pipepart works with --header, --regexp, --cat and --fifo. - ./configure --disable-documentation will not build documentation. - {%} works as job slot. - Bug fixes and man page updates. 20140522 - Use --semaphoretimeout to ignore the semaphore lock after a while. - {%} introduced as job slot replacement string. It has known bugs. - --slotreplace changes slot replacement string. - Bug fixes and man page updates. 20140422 - --pipepart is a highly efficient alternative to --pipe if the input is a real file and not a pipe. - If using --cat or --fifo with --pipe the {} in the command will be replaced with the name of a physical file and a fifo respectively containing the block from --pipe. Useful for commands that cannot read from standard input (stdin). - --controlmaster has gotten an overhaul and is no longer experimental. - --env is now copied when determining CPUs on remote system. Useful for copying $PATH if parallel is not in the normal path. - --results now chops the argument if the argument is longer than the allowed path length. - Build now survives if pod2* are not installed. - Bug fixes and man page updates. 20140322 - Bug fixes and man page updates. 20140222 - --tollef has been retired. - --compress has be redesigned due to bugs. - Format of --eta and --joblog has changed slightly. - Bug fixes and man page updates. 20140122 - Bug fixes and man page updates. 20131222 - GNU Parallel now has a motto: For people who live life in the parallel lane. - Detect if the buffer dir $TMPDIR runs out of space. This changes a central piece of code, making this release alpha quality. - --keep-order --(n)onall will sort according to the sshlogin. - Detect the number of CPUs on DEC Tru64. - GNU sql: --list-tables for sqlite3. - Bug fixes and man page updates. 20131122 - A citation notice is printed on stderr only if stderr is a terminal, the user has not specified --no-notice and the user has not run --bibtex once. This makes the release alpha quality. - --compress will compress temporary files. If the output is big and very compressible this will take up less disk space in $TMPDIR and possibly be faster due to less disk I/O. - --compress-program comtrols which program to use for compressing temporary files. - --bar show progress as a progress bar compatible with zenity. - --resume can now be used with --result: Jobs already run will be skipped. - --transfer and --basefile support paths relative to the --workdir by inserting /./ into the path. - Bug fixes and man page updates. 20131022 - --transfer files with /./ in the path will copy the files relative to the --workdir. - The maximal command length is now cached in a file halfing the startup time. - Bug fixes and man page updates. 20130922 - PDF-files of documentation is now included. - Bug fixes and man page updates. 20130822 - A tutorial showing much of GNU Parallel's functionality. Spend an hour walking through the tutorial. Your commandline will love you for it. man parallel_tutorial - --line-buffer will buffer output on line basis. --group keeps the output together for a whole job. --ungroup allows output to mixup with half a line coming from one job and half a line coming from another job. --line-buffer fits between these two; it prints a full line, but will allow for mixing lines of different jobs. - --record-env records all variables and functions set. Useful to record the normal environment before using --env _. - --env _ exports all functions and variables to the remote server except for the ones recorded. - New signing key. Due to recommendations from NIST http://www.keylength.com/en/4/ the signing key was changed from 1024D/ID:FFFFFFF1 to 9888R/ID:88888888. - Bug fixes and man page updates. 20130722 - --round-robin with --pipe will write all blocks to already running jobs. - --env can now transfer Bash function for remote execution. That is pretty cool! - Quite a few bug fixes and man page updates. 20130622 - --xapply now recycles arguments if an input source has more arguments than others. - The sleep time between jobs is now both increased and decreased exponentially. - 10 seconds installation check the signature using GnuPG if GnuPG is installed. - Bug fixes and man page updates. 20130522 - --ctrlc is now default if --pipe is not used; thus making it possible to kill remotely started jobs simply by pressing Ctrl-C. - --timeout 200% now means kill jobs that take more than twice the time of the median run time of a job. - Jobs are now distributed round robin when having mulitiple --sshlogin instead of filling up one --sshlogin at a time. - niceload: darwin version of --io --mem --swap. Thanks to Anders F Bjorklund. - Bug fixes and man page updates. 20130422 - 10 seconds installation: wget -O - pi.dk/3 | sh - HPUX CPU/core detection. Thanks to Javier Tarradas. - CPU detection for HURD, IRIX, SCO OpenServer and (old) HPUX. - --ctrlc will send SIGINT to tasks running on remote computers and thus killing them. - --load now uses 'ps' to see immediately see the number of running processes instead of 'uptime' thus making --load react much faster. - Testing on Centos 3.9 and FreeBSD 7 revealed a lot of compability bugs. Some of these required quite extensive changes making this release beta quality. - --retries works with --onall. - The new --load computation now works on FreeBSD 7. - --nice works under tcsh. - Bug fixes and man page updates. 20130222 - --resume works with --pipe. - --resume-failed will go through --joblog, redo the failed jobs and then continue like --resume. - Negative positional arguments count from the end: {-1} means the last argument, {-2} the second to last. - NetBSD CPU detection. - --blocksize increases exponentially if it smaller than a full record. - Processing n-line records (--pipe -L n) is now much faster. - --tollef is obsoleted after discussion on the mailing list. - Bug fixes and man page updates.
2016-07-09Bump PKGREVISION for perl-5.24.0 for everything mentioning perl.wiz7-14/+14
2016-06-19Remove INSTALLATION_PREFIX; no change to package.wiz1-2/+1
2016-06-08Bump PKGREVISION for perl-5.24.wiz1-2/+2
2016-06-08Switch to MASTER_SITES_PYPI.wiz1-2/+2
2016-06-08Remove the stability entity, it has no meaning outside of an official context.jperkin1-1/+0
2016-06-08Change the service_bundle name to "export" to reduce diffs between thejperkin1-1/+1
original manifest.xml file and the output from "svccfg export".
2016-06-01Include correct directory and rpath for finding bdb.joerg2-1/+15
2016-03-05Bump PKGREVISION for security/openssl ABI bump.jperkin3-5/+6
2016-02-28Let the compiler decide whether or not DT_RPATH vs DT_RUNPATH should bejoerg4-2/+147
used, don't force the crap in places that definitely don't know the right answer. Bump revision.
2016-02-16Update parallel/ganglia-monitor-core to 3.7.2. Add SMF support.fhajny17-72/+379
Upstream changelog follows. 3.7.2 * Fix for a memory leak if override_hostname or override_ip are used 3.7.1 * Hash table in gmetad has been reworked to support much higher metric counts and larger number of metrics * A number of GMond python modules have been rewritten and enhanced 3.7.0 * Add support for sending Ganglia metrics to Riemann * Performance improvements for gmetad * Export internal gmetad metrics 3.6.0 * Improvements - Add ability to gzip XML output in gmond - Forward Graphite metrics over UDP for increased scalability - Configurable graphite prefix - Memcached publish support - export metrics into memcached - Improve metric summarization performance under heavy load * Bug fixes - Reduce CPU utilization when running in deaf mode 3.5.0 * gmond uses a separate thread to service gmetad requests. This should improve performance * Updates to a number of core Python collection modules 3.4.0 * web - now distributed as a separate source tarball, web/ tree is not present in the ganglia-3.4.0 tarball * gmond: - setgid() and initgroups() are invoked when setuid() is invoked - gmond no longer runs in root group (there may be side-effects from this change) - new UDP RX channel parameter `buffer' to set the socket buffer size 3.3.7 * gmond - BUG100: fails to start in Solaris zones - fails to start when no address on the network interface (added retry_bind parameter) - BUG321: fails to start when Solaris CPU in state FAILED (seg fault) 3.3.6 * gmond - BUG327: memory leak when receive channel is not configured or not hearing any data 3.3.5 * General - Return to the use of autotools (make dist) for building the release - Release with a valid version number in configure.in * Gmetad - Fix for gmetad buffer overflow with metric value of DOUBLE_MAX * Gmond python modules - multi traffic module has been replaced with multi interface which correctly reports all interfaces * Gmond - Fix for issues/11 - Expose some useful functions in the public C API to assist module developers 3.3.1 * Web - Graph Zooming off when Ganglia Web server and web browser in different time zones - Metrics chooser in cluster view now provides metric drop down in addition to autocomplete - Ability to specify default size for graphs in Views Gmetad - You can now disable RRD writing - useful if you are using Ganglia only to feed Graphite - Export to graphite now handles spaces in metrics, groups - Minor code clean up and fixes * Gmond - Diskstat python module incorrect IO time reporting has been fixed. 3.3.0 * Web UI shipped with Ganglia has been upgraded to the Ganglia Web 2.0 interface. Changes to web UI from Gweb 2.2.0 are as follows - Additional Nagios integration scripts - Contributed by Etsy - stacked graph in the cluster view - Ability to hide/show events on individual graphs or whole pages - Numerous bug fixes and small enhancements * metric names are sanitized to avoid breaking gmetad/UI when characters such as <, > are supplied. Those are now replaced with underscores * gmetad supports sending metrics to graphite. Uncomment proper section of gmetad.conf to enable * Default RRD resolution changed. It affects only newly created RRDs. To change to old behavior please modify the RRA field in gmetad.conf * Number of Python modules from Ganglia gmond_python_modules repository have been included such as disk statistics, individual interface statistics, etc. Most of the modules are disabled. To enable rename the files in /etc/ganglia/conf.d. * sFlow has been enhanced to support Virtual machine pool/Java VM/Memcache/Apache/NGINX/Tomcat and node.js metrics. You can read more at sFlow blog 3.2.0 * sFlow support - more and even more * hostname/ip override - you can specify an arbitrary host name and IP to be shown in the UI. * FreeBSD patches * Python module improvements * Bugfixes and improvements over 3.1.7 3.1.7: * Fix build support for RHEL5/issue with commas in CFLAGS * multicpu module: show CPU utilization as a value between 0-100% for each core 3.1.3 - 3.1.6: * Merge commit 1966 from trunk to fix "contrib/removespikes.pl" * Bootstrapping with Debian 5.0 (lenny) versions of autotools for this and future releases. * Require user to explicitly specify sysconfdir when building from source, due to the fact that the old behavior was not consistent with the documented behavior. * Configuration files and scripts are now created during the install phase rather than during configure. This allows values such as @sysconfdir@ to be used in the template configuration files. * Abolish the use of release names - only release numbers will be used to distinguish versions in future * libmetrics: workaround system header conflict in DFBSD >= 2.4 (BUG245) * Use PCRE regex matching to configure metrics using the name_match directive * rrdcached support * gmetad now uses apr and the sleep intervals between polls are randomized in a way that supports shorter polling intervals * FreeBSD support: fixes for crashes and disk statistics (BUG153) * Further tweaks to Solaris build support (remove C99 hack) * Eliminate conflict with ncpus symbol name on older Solaris * AIX support: determine if the host is a virtual server (BUG226) * AIX support: setting linker flags (BUG227), add -lm * AIX support: tweaks for AIX >= v6.1 * AIX support: revised init scripts for gmond and gmetad * Check for Python.h explicitly * Include the necessary Python files in the distribution tarball, regardless of how BUILD_PYTHON is set (r2215). * Remove references to GNU toolchain in documentation * Fortify write_data_to_rrd against overflows * Web interface: minor formatting changes * mcast_if implementation tweaked so that the send channel will be bound to the IP of the outgoing interface * Documentation updates relating to the options for multihomed hosts, particularly bind, bind_hostname and mcast_if * gmond: Limit the use of APR_POLLSET_THREADSAFE to Linux >= 2.6 * gmond: improve/revert setuid behavior in configure script * gmond: Fix the allow_extra_data configuration directive (BUG199) * gmond: Ensure that a complete XML dump is delivered before closing the send socket. Submitted by: Jerry <adawzq 126 com> * gmond: add bind and bind_hostname parameters for udp_send_channel() * gmetad: BUG232: eliminate case-sensitive hostname bug, user can choose to maintain legacy behavior though. * gmond: BUG237: revise fix for segfault on Solaris where first CPU not in slot 0 * gmond: support for HUP signal on platforms with execve * gmond: status module: return gmond version info as string metrics * gmond: Check return status of apr_pollset_create. Use APR_POLLSET_THREADSAFE on Linux. * build: various configure options: Solaris 8 with Sun Studio 11 support, extra modules for static linking, default setuid, release number, build multicpu and status during static builds, support for SYSCONFDIR (BUG16) * RPM: include status module, allow packager to supply own gmond.conf * build: Look in lib64 rather than lib for apr, confuse and expat on x86_64
2015-12-29Fix missing/broken rcsids.dholland3-3/+11
2015-12-25-export-dynamic is a linker option, mark it accordingly. Fix non-catmanjoerg4-11/+54
build as root. Fix rpath path of spooledit.
2015-12-13Add a java option.markd5-13/+276
Fix PLIST for the sge option. Fix PLIST for linux.