summaryrefslogtreecommitdiff
path: root/parallel
AgeCommit message (Collapse)AuthorFilesLines
2017-09-16Reset maintainerwiz1-2/+2
2017-09-06Follow some redirects.wiz1-2/+2
2017-08-16Follow some http redirects.wiz1-2/+2
2017-07-30Switch github HOMEPAGEs to https.wiz1-2/+2
2017-07-173.5.0.3:adam2-8/+10
- Adds Process._authkey alias to .authkey for 2.7 compat. - Remove superfluous else clause from max_memory_per_child_check. - Document and test all supported Python versions. - Extend 'Process' to be compatible with < Py3.5. - Use a properly initialized logger in pool.py error logging. - _trywaitkill can now kill a whole process group if the worker process declares itself as a group leader. - Fix cpython issue 14881 (See http://bugs.python.org/issue14881). - Fix for a crash on windows. - Fix messaging in case of worker exceeds max memory.
2017-06-30Correct ganglia-monitor-core build failure on NetBSD 8.99.1kamil2-7/+15
Address <sys/user.h> removal fallout
2017-06-20PLIST fix for non-f90 buildsadam2-7/+5
2017-06-05Recursive revbump from lang/perl5 5.26.0ryoon1-2/+2
2017-04-29Changes in 3.2:adam12-159/+562
* Added support for MPI-3.1 features including nonblocking collective I/O, address manipulation routines, thread-safety for MPI initialization, pre-init functionality, and new MPI_T routines to look up variables by name. * Fortran 2008 bindings are enabled by default and fully supported. * Added support for the Mellanox MXM InfiniBand interface. (thanks to Mellanox for the code contribution). * Added support for the Mellanox HCOLL interface for collectives. (thanks to Mellanox for the code contribution). * Significant stability improvements to the MPICH/portals4 implementation. * Completely revamped RMA infrastructure including several scalability improvements, performance improvements, and bug fixes. * Added experimental support for Open Fabrics Interfaces (OFI) version 1.0.0. https://github.com/ofiwg/libfabric (thanks to Intel for code contribution) * The Myrinet MX network module, which had a life cyle from 1.1 till 3.1.2, has now been deleted. * Several other minor bug fixes, memory leak fixes, and code cleanup.
2017-04-28Version 1.11.7adam3-9/+10
-------------- * Fix hwloc-bind --membind for CPU-less NUMA nodes (again). Thanks to Gilles Gouaillardet for reporting the issue. * Fix a memory leak on IBM S/390 platforms running Linux. * Fix a memory leak when forcing the x86 backend first on amd64/topoext platforms running Linux. * Command-line tools now support "hbm" instead "numanode" for filtering only high-bandwidth memory nodes when selecting locations. + hwloc-bind also support --hbm and --no-hbm for filtering only or no HBM nodes. * Add --children and --descendants to hwloc-info for listing object children or object descendants of a specific type. * Add --no-index, --index, --no-attrs, --attrs to disable/enable display of index numbers or attributes in the graphical lstopo output. * Try to gather hwloc-dump-hwdata output from all possible locations in hwloc-gather-topology. * Updates to the documentation of locations in hwloc(7) and command-line tools manpages.
2017-04-09Changes 3.5.0.2:adam3-48/+42
- max_memory_per_child was measured in kilobytes on Linux, but bytes on *BSD/MacOS, it's now always kilobytes. - Windows: Adds support for max_memory_per_child, but requires the ``psutil`` package to be installed. - Fixed bug in ForkingPickler.loadbuf, where it tried to pass a BytesIO instance directly to ``pickle.loads`` on Python 2.7.
2017-03-23Extend SHA512 checksums to various files I have on my local distfilejoerg1-1/+2
mirror.
2017-02-12Recursive revbump from fonts/harfbuzzryoon1-2/+2
2017-02-06Recursive bump for harfbuzz's new graphite2 dependency.wiz1-1/+2
2017-02-06Fix PLIST for linux.markd1-3/+14
2017-02-06fix linux build and PLISTmarkd2-3/+6
2017-01-19Convert all occurrences (353 by my count) ofagc2-7/+7
MASTER_SITES= site1 \ site2 style continuation lines to be simple repeated MASTER_SITES+= site1 MASTER_SITES+= site2 lines. As previewed on tech-pkg. With thanks to rillig for fixing pkglint accordingly.
2016-12-25PLIST catchup for recent update.markd1-1/+31
2016-12-19Revert nonsensical changes.asau2-15/+15
2016-12-17Fixed pkglint warnings.rillig2-15/+15
2016-10-23Update to OpenMPI 1.10.4asau6-52/+93
Prompted by Nicolas Joly in private mail. 1.10.4 - 01 Sept 2016 ------ - Fix assembler support for MIPS - Improve memory handling for temp buffers in collectives - Fix [all]reduce with non-zero lower bound datatypes Thanks Hristo Iliev for the report - Fix non-standard ddt handling. Thanks Yuki Matsumoto for the report - Various libnbc fixes. Thanks Yuki Matsumoto for the report - Fix typos in request RMA bindings for Fortran. Thanks to @alazzaro and @vondele for the assist - Various bug fixes and enhancements to collective support - Fix predefined types mapping in hcoll - Revive the coll/sync component to resolve unexpected message issues during tight loops across collectives - Fix typo in wrapper compiler for Fortran static builds 1.10.3 - 15 June 2016 ------ - Fix zero-length datatypes. Thanks to Wei-keng Liao for reporting the issue. - Minor manpage cleanups - Implement atomic support in OSHMEM/UCX - Fix support of MPI_COMBINER_RESIZED. Thanks to James Ramsey for the report - Fix computation of #cpus when --use-hwthread-cpus is used - Add entry points for Allgatherv, iAllgatherv, Reduce, and iReduce for the HCOLL library - Fix an HCOLL integration bug that could signal completion of request while still being worked - Fix computation of cores when SMT is enabled. Thanks to Ben Menadue for the report - Various USNIC fixes - Create a datafile in the per-proc directory in order to make it unique per communicator. Thanks to Peter Wind for the report - Fix zero-size malloc in one-sided pt-to-pt code. Thanks to Lisandro Dalcin for the report - Fix MPI_Get_address when passed MPI_BOTTOM to not return an error. Thanks to Lisandro Dalcin for the report - Fix MPI_TYPE_SET_ATTR with NULL value. Thanks to Lisandro Dalcin for the report - Fix various Fortran08 binding issues - Fix memchecker no-data case. Thanks to Clinton Stimpson for the report - Fix CUDA support under OS-X - Fix various OFI/MTL integration issues - Add MPI_T man pages - Fix one-sided pt-to-pt issue by preventing communication from happening before a target enters a fence, even in the no-precede case - Fix a bug that disabled Totalview for MPMD use-case - Correctly support MPI_UNWEIGHTED in topo-graph-neighbors. Thanks to Jun Kudo for the report - Fix singleton operations under SLURM when PMI2 is enabled - Do not use MPI_IN_PLACE in neighborhood collectives for non-blocking collectives (libnbc). Thanks to Jun Kudo for the report - Silence autogen deprecation warnings for newer versions of Perl - Do not return MPI_ERR_PENDING from collectives - Use type int* for MPI_WIN_DISP_UNIT, MPI_WIN_CREATE_FLAVOR, and MPI_WIN_MODEL. Thanks to Alastair McKinstry for the report - Fix register_datarep stub function in IO/OMPIO. Thanks to Eric Chamberland for the report - Fix a bus error on MPI_WIN_[POST,START] in the shared memory one-sided component - Add several missing MPI_WIN_FLAVOR constants to the Fortran support - Enable connecting processes from different subnets using the openib BTL - Fix bug in basic/barrier algorithm in OSHMEM - Correct process binding for the --map-by node case - Include support for subnet-to-subnet routing over InfiniBand networks - Fix usnic resource check - AUTHORS: Fix an errant reference to Subversion IDs - Fix affinity for MPMD jobs running under LSF - Fix many Fortran binding bugs - Fix `MPI_IN_PLACE`-related bugs - Fix PSM/PSM2 support for singleton operations - Ensure MPI transports continue to progress during RTE barriers - Update HWLOC to 1.9.1 end-of-series - Fix a bug in the Java command line parser when the -Djava.library.path options was given by the user - Update the MTL/OFI provider selection behavior - Add support for clock_gettime on Linux. - Correctly detect and configure for Solaris Studio 12.5 beta compilers - Correctly compute #slots when -host is used for MPMD case - Fix a bug in the hcoll collectives due to an uninitialized field - Do not set a binding policy when oversubscribing a node - Fix hang in intercommunicator operations when oversubscribed - Speed up process termination during MPI_Abort - Disable backtrace support by default in the PSM/PSM2 libraries to prevent unintentional conflicting behavior. 1.10.2: 26 Jan 2016 ------------------- ********************************************************************** * OSHMEM is now 1.2 compliant ********************************************************************** - Fix NBC_Copy for legitimate zero-size messages - Fix multiple bugs in OSHMEM - Correctly handle mpirun --host <user>@<ip-address> - Centralize two MCA params to avoid duplication between OMPI and OSHMEM layers: opal_abort_delay and opal_abort_print_stack - Add support for Fujitsu compilers - Add UCX support for OMPI and OSHMEM - Correctly handle oversubscription when not given directives to permit it. Thanks to @ammore1 for reporting it - Fix rpm spec file to not include the /usr directory - Add Intel HFI1 default parameters for the openib BTL - Resolve symbol conflicts in the PSM2 library - Add ability to empty the rgpusm cache when full if requested - Fix another libtool bug when -L requires a space between it and the path. Thanks to Eric Schnetter for the patch. - Add support for OSHMEM v1.2 APIs - Improve efficiency of oshmem_preconnect_all algorithm - Fix bug in buffered sends support - Fix double free in edge case of mpirun. Thanks to @jsharpe for the patch - Multiple one-sided support fixes - Fix integer overflow in the tuned "reduce" collective when using buffers larger than INT_MAX in size - Fix parse of user environment variables in mpirun. Thanks to Stefano Garzarella for the patch - Performance improvements in PSM2 support - Fix NBS iBarrier for inter-communicators - Fix bug in vader BTL during finalize - Improved configure support for Fortran compilers - Fix rank_file mapper to support default --slot-set. Thanks to Matt Thompson for reporting it - Update MPI_Testsome man page. Thanks to Eric Schnetter for the suggestion - Fix missing resize of the returned type for subarray and darray types. Thanks to Keith Bennett and Dan Garmann for reporting it - Fix Java support on OSX 10.11. Thanks to Alexander Daryin for reporting the problem - Fix some compilation issues on Solaris 11.2. Thanks to Paul Hargrove for his continued help in such areas
2016-10-21Update to hwloc 1.11.4.asau3-9/+10
Version 1.11.4 -------------- * Add MemoryMode and ClusterMode attributes in the Machine object on KNL. Add doc/examples/get-knl-modes.c for an example of retrieving them. Thanks to Grzegorz Andrejczuk. * Fix Linux build with -m32 with respect to libudev. Thanks to Paul Hargrove for reporting the issue. * Fix build with Visual Studio 2015, thanks to Eloi Gaudry for reporting the issue and providing the patch. * Don't forget to display OS device children in the graphical lstopo. * Fix a memory leak on Solaris, thanks to Bryon Gloden for the patch. * Properly handle realloc() failures, thanks to Bryon Gloden for reporting the issue. * Fix lstopo crash in ascii/fig/windows outputs when some objects have a lstopoStyle info attribute. Version 1.11.3 -------------- * Bug fixes + Fix a memory leak on Linux S/390 hosts with books. + Fix /proc/mounts parsing on Linux by using mntent.h. Thanks to Nathan Hjelm for reporting the issue. + Fix a x86 infinite loop on VMware due to the x2APIC feature being advertised without actually being fully supported. Thanks to Jianjun Wen for reporting the problem and testing the patch. + Fix the return value of hwloc_alloc() on mmap() failure. Thanks to Hugo Brunie for reporting the issue. + Fix the return value of command-line tools in some error cases. + Do not break individual thread bindings during x86 backend discovery in a multithreaded process. Thanks to Farouk Mansouri for the report. + Fix hwloc-bind --membind for CPU-less NUMA nodes. + Fix some corner cases in the XML export/import of application userdata. * API Improvements + Add HWLOC_MEMBIND_BYNODESET flag so that membind() functions accept either cpusets or nodesets. + Add hwloc_get_area_memlocation() to check where pages are actually allocated. Only implemented on Linux for now. - There's no _nodeset() variant, but the new flag HWLOC_MEMBIND_BYNODESET is supported. + Make hwloc_obj_type_sscanf() parse back everything that may be outputted by hwloc_obj_type_snprintf(). * Detection Improvements + Allow the x86 backend to add missing cache levels, so that it completes what the Solaris backend lacks. Thanks to Ryan Zezeski for reporting the issue. + Do not filter-out FibreChannel PCI adapters by default anymore. Thanks to Matt Muggeridge for the report. + Add support for CUDA compute capability 6.x. * Tools + Add --support to hwloc-info to list supported features, just like with hwloc_topology_get_support(). - Also add --objects and --topology to explicitly switch between the default modes. + Add --tid to let hwloc-bind operate on individual threads on Linux. + Add --nodeset to let hwloc-bind report memory binding as NUMA node sets. + hwloc-annotate and lstopo don't drop application userdata from XMLs anymore. - Add --cu to hwloc-annotate to drop these application userdata. + Make the hwloc-dump-hwdata dump directory configurable through configure options such as --runstatedir or --localstatedir. * Misc Improvements + Add systemd service template contrib/systemd/hwloc-dump-hwdata.service for launching hwloc-dump-hwdata at boot on Linux. Thanks to Grzegorz Andrejczuk. + Add HWLOC_PLUGINS_BLACKLIST environment variable to prevent some plugins from being loaded. Thanks to Alexandre Denis for the suggestion. + Small improvements for various Windows build systems, thanks to Jonathan L Peyton and Marco Atzeri. Version 1.11.2 -------------- * Improve support for Intel Knights Landing Xeon Phi on Linux: + Group local NUMA nodes of normal memory (DDR) and high-bandwidth memory (MCDRAM) together through "Cluster" groups so that the local MCDRAM is easy to find. - See "How do I find the local MCDRAM NUMA node on Intel Knights Landing Xeon Phi?" in the documentation. - For uniformity across all KNL configurations, always have a NUMA node object even if the host is UMA. + Fix the detection of the memory-side cache: - Add the hwloc-dump-hwdata superuser utility to dump SMBIOS information into /var/run/hwloc/ as root during boot, and load this dumped information from the hwloc library at runtime. - See "Why do I need hwloc-dump-hwdata for caches on Intel Knights Landing Xeon Phi?" in the documentation. Thanks to Grzegorz Andrejczuk for the patches and for the help. * The x86 and linux backends may now be combined for discovering CPUs through x86 CPUID and memory from the Linux kernel. This is useful for working around buggy CPU information reported by Linux (for instance the AMD Bulldozer/Piledriver bug below). Combination is enabled by passing HWLOC_COMPONENTS=x86 in the environment. * Fix L3 cache sharing on AMD Opteron 63xx (Piledriver) and 62xx (Bulldozer) in the x86 backend. Thanks to many users who helped. * Fix the overzealous L3 cache sharing fix added to the x86 backend in 1.11.1 for AMD Opteron 61xx (Magny-Cours) processors. * The x86 backend may now add the info attribute Inclusive=0 or 1 to caches it discovers, or to caches discovered by other backends earlier. Thanks to Guillaume Beauchamp for the patch. * Fix the management on alloc_membind() allocation failures on AIX, HP-UX and OSF/Tru64. * Fix spurious failures to load with ENOMEM on AIX in case of Misc objects below PUs. * lstopo improvements in X11 and Windows graphical mode: + Add + - f 1 shortcuts to manually zoom-in, zoom-out, reset the scale, or fit the entire window. + Display all keyboard shortcuts in the console. * Debug messages may be disabled at runtime by passing HWLOC_DEBUG_VERBOSE=0 in the environment when --enable-debug was passed to configure. * Add a FAQ entry "What are these Group objects in my topology?".
2016-10-03Call sun-iss sissl-1.1 to match OSI.wiz1-3/+2
2016-09-19Update parallel/pdsh to 2.31.fhajny2-11/+11
* Changes in pdsh-2.31 (2013-11-07) =================================== -- Fix issue 56: slurm: Allow mixed use of -P, -w and -j options. -- Fix issue 59: pdsh very slow when using a few thousand hosts and genders. -- testsuite: Expanded tests for genders module (Pythagoras Watson) * Changes in pdsh-2.30 (2013-03-02) =================================== -- Fix issue 55: genders -X option removes more hosts than expected. (This was a generic fix for hostname matching, so it probably affected -x and other options as well.) -- testsuite: Add test for issue 55.
2016-09-13Update parallel/parallel to 20160822fhajny4-62/+35
Upstream changes: 20160822 - Bug fixes and man page updates. 20160722 - env_parallel is now ready for wider testing. It is still beta quality. - env_parallel is heavily modified for all shells and testing has been increased. - Selectively choosing what to export using --env now works for env_parallel (bash, csh, fish, ksh, pdksh, tcsh, zsh). - --round-robin now gives more work to a job that processes faster instead of same amount to all jobs. - --pipepart works on block devices on GNU/Linux. - Bug fixes and man page updates. 20160622 - $PATH can now be exported using --env PATH. Useful if GNU Parallel is not in your path on remote machines. - If --block is left out, --pipepart will use a block size that will result in 10 jobs per jobslot. - Bug fixes and man page updates. 20160522 - niceload --net pauses the program if the internet connection is overloaded. - Bug fixes and man page updates. 20160422 - :::+ and ::::+ work like ::: and :::: but links this input source to the previous input source in a --xapply fashion. Contrary to --xapply values do not wrap: The shortest input source determines the length. - --line-buffer --keep-order now outputs continously from the oldest job still running. This is more what you would expect than the earlier behaviour where --keep-order had no effect with --line-buffer. - env_parallel supports tcsh, csh, pdksh. In fish it now supports arrays. In csh/tcsh it now supports variables, aliases, and arrays with no special chars. In pdksh it supports aliases, functions, variables, and arrays. - Function exporting on Mac OS X works around old Bash version. - Better CPU detection on OpenIndiana. - Bug fixes and man page updates. 20160322 - env_parallel is a function that exports the environment (functions, aliases, variables, and arrays) to GNU Parallel. Run 'man env_parallel' for details. - niceload --prg now searches for substrings if no process with the name is found. - Bug fixes and man page updates. 20160222 - Bug fixes and man page updates. 20160122 - --sql DBURL uses DBURL as storage for jobs and output. It does not run any jobs so it requires at least one --sqlworker. DBURL must point to a table. - --sqlworker DBURL gets jobs from DBURL and stores the result back to DBURL. - --sqlandworker is a shorthand for --sql and --sqlworker. - --sqlworker requires the output of a single job to fit in memory. - --results now also saves a file called 'seq' containing the sequence number. - If $PARALLEL_ENV is a file, then that file will be read into $PARALLEL_ENV. - man parallel_tutorial has been given an overhaul. - --workdir now accepts replacementstrings. - Bug fixes and man page updates. 20151222 - --transfer is now an alias for --transferfile {}. - --transferfile works like --transfer, but takes an argument like --return. This makes it possible to combine transferring files with multiple input sources: parallel -S server --tf {1} wc {2} {1} ::: * ::: -l -w -c - total_jobs() can now be used in {= =}: parallel echo job {#} of '{= $_=total_jobs() =}' ::: {1..50} - Bug fixes and man page updates. 20151122 - Bug fixes and man page updates. 20151022 - --plus makes it possible to use {##} as a shorthand for {= $_=$Global::JobQueue->total_jobs() =} which gives the the number of jobs to run in total. - {= $_=$Global::JobQueue->total_jobs() =} is incompatible with -X, -m, and --xargs. - GNU Parallel is now mostly compatible with lsh (http://www.lysator.liu.se/~nisse/lsh/) and somewhat compatible with autossh (http://www.harding.motd.ca/autossh/). - --workdir ... now also works when run locally. - Bug fixes and man page updates. 20150922 - Bug fixes and man page updates. 20150822 - If $job->skip() is called in {= =} the job will not be run. - @arg can now be used in {= =}: parallel echo '{= $arg[2] < 5 and $_="j" =}' ::: 1 2 3 ::: 4 5 6 - Bug fixes and man page updates. 20150722 - If a record fits in a block, --block only reads upto the size of --block. If not: --block reads a full --block more. - Due to widely spread distributions still using --tollef even after a year of being retired following a year of being obsolete, --tollef now gives an error unless used with --gnu. - --nice now uses a perl wrapper instead of the nice command. - Bug fixes and man page updates. 20150622 - --halt has been rewritten completely. You can now combine percentages with success or fail. See the man page. - Exit values 102..254 have been removed. 101 means more than 100 jobs failed. - Killing through --timeout, --memfree, or --halt is now done as a process group. - --termseq determines which signals are sent when a job is killed. - An empty argument would previously cause no string to be inserted. This is now changed to '' being inserted, thus prepending a space to the output of: parallel echo {} b ::: '' - $PARALLEL_ENV can now be set to an environment prepending the command. Used in env_parallel as mentioned in the manpage. - --retry-failed will retry all failed jobs in a joblog. It will ignore any command given. - --ssh and $PARALLEL_SSH can be used to set the command used for ssh. The command is assumed to behave the same way as ssh. - --fifo now works in csh, too. - Q(...) can be used in {= =} to shell quote a string. - Bug fixes and man page updates. 20150522 - Security: The security issue for --sshlogin + --fifo/--cat has been fixed. Thereby all issues with http://lists.gnu.org/archive/html/parallel/2015-04/msg00045.html have been fixed. - Security: After further security analysis the issue fixed in 20150422 also fixed the problem for --tmux. - Bug fixes and man page updates. 20150422 New in this release: - Security fix. An attacker on the local system could make you overwrite one of your own files with a single byte. The problem exist when you use --compress or --tmux or --pipe or --cat or --fifo. The attacker must figure out the randomly chosen file name and create a symlink within a time window of 15 ms. - --shuf will generate all jobs, and shuffle them before running them. This is useful to get a quick preview of the results before running the full batch. - Bug fixes and man page updates. 20150322 New in this release: - --number-of-cores respects 'taskset' on GNU/Linux. - --joblog --pipe gives the data send and received in the log. - Bug fixes and man page updates. 20150222 - --tmux has gotten a major overhaul. - Bug fixes and man page updates. 20150122 - Remote jobs now send stderr (standard error) to stderr (standard error) instead of stdout (standard output). - Remote execution command is now packed using base64 encoded bzip2. This means that big environments (app. 100 KB) can be transferred. This change makes remote execution alpha quality. - --semaphoretimeout now takes a negative value. This means give up if the semaphore is not gotten within the timeout. - --halt -1 and -2 now means halt if a job succeeds (so the opposite of 1 and 2). - --no-keep-order will reverse --keep-order. - Bash's second fix of shellshock caused --env to break again when exporting functions. This has been fixed again. - A semibig refactoring of big functions. All non-trivial functions are now less than 100 lines. The refactoring makes this release beta quality. - A description of the design decisions for GNU Parallel can be found in 'man parallel_design'. - A bug fix in replacement strings caused rewrite of the replacement function. This makes use of replacement strings alpha quality. - Bug fixes and man page updates. 20141122 - Remote systems can be divided into hostgroups (e.g. web and db) by prepending '@groupname/' to the sshlogin. Multiple groups can be given by separating groups with '+'. E.g. @web/www1 @web+db/www2 @db/mariadb - Remote execution can be restricted to servers that are part of one or more groups by '@groupname' as an sshlogin. Multiple groups can be given by separating groups with '+'. E.g. -S @web or -S @db+web - With --hostgroup you can restrict arguments to certain hostgroups by appending '@groupname' to the argument. Multiple groups can be given by separating groups with '+'. E.g. my_web_arg@web db-or-web-arg@db+web db-only-arg@db Thanks to Michel Courtine for developing a prototype for this. - Bug fixes and man page updates. 20141022 - --env was changed to deal with Bash's new way of exporting a function. - Bug fixes and man page updates. 20140922 - If the file give as --sshloginfile is changed it will be re-read when a job finishes though at most once per second. This makes it possible to add and remove hosts while running. 20140822 - GNU Parallel now uses the same shell it was started from as the command shell for local jobs. So if GNU Parallel is started from tcsh it will use tcsh as its shell even if the login $SHELL is different. For remote jobs the login $SHELL will be used. - The whole current environment in bash can be copied by using a shell wrapper function (Search manual for env_parallel). - --plus adds the replacement strings {+/} {+.} {+..} {+...} {..} {...} {/..} {/...}. The idea being that '+foo' matches the opposite of 'foo' and {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} = {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...} - GNU Parallel now deals correctly with the combination rsync-3.1.X-client and rsync-2.5.7-server - Bug fixes and man page updates. 20140722 - {= perl expression =} can be used as replacement string. The expression should modify $_. E.g. {= s/\.gz$// =} to remove .gz from the string. This makes replacement strings extremely flexible. - Positional perl expressions (similar to {2}) are given as {=2 perl expression=} where 2 is the position. - One small backwards incompatibility: {1}_{2} will replace {2} with the empty string if there is only one argument. Previously {2} would have been left untouched. - Replacement strings can be defined using --rpl. E.g. parallel --rpl '{.gz} s/\.gz$//' echo {.gz} ::: *.gz - The parenthesis around {= perl expression =} can be changed with --parens. - --tmux will direct the output to a tmux session instead of files. Each running jobs will be in its own window. - --halt 10% will stop spawning new jobs if 10% failed so far. - Bug fixes and man page updates. 20140622 - --shebang and --shebang-wrap now works on FreeBSD. - --pipepart works with --header, --regexp, --cat and --fifo. - ./configure --disable-documentation will not build documentation. - {%} works as job slot. - Bug fixes and man page updates. 20140522 - Use --semaphoretimeout to ignore the semaphore lock after a while. - {%} introduced as job slot replacement string. It has known bugs. - --slotreplace changes slot replacement string. - Bug fixes and man page updates. 20140422 - --pipepart is a highly efficient alternative to --pipe if the input is a real file and not a pipe. - If using --cat or --fifo with --pipe the {} in the command will be replaced with the name of a physical file and a fifo respectively containing the block from --pipe. Useful for commands that cannot read from standard input (stdin). - --controlmaster has gotten an overhaul and is no longer experimental. - --env is now copied when determining CPUs on remote system. Useful for copying $PATH if parallel is not in the normal path. - --results now chops the argument if the argument is longer than the allowed path length. - Build now survives if pod2* are not installed. - Bug fixes and man page updates. 20140322 - Bug fixes and man page updates. 20140222 - --tollef has been retired. - --compress has be redesigned due to bugs. - Format of --eta and --joblog has changed slightly. - Bug fixes and man page updates. 20140122 - Bug fixes and man page updates. 20131222 - GNU Parallel now has a motto: For people who live life in the parallel lane. - Detect if the buffer dir $TMPDIR runs out of space. This changes a central piece of code, making this release alpha quality. - --keep-order --(n)onall will sort according to the sshlogin. - Detect the number of CPUs on DEC Tru64. - GNU sql: --list-tables for sqlite3. - Bug fixes and man page updates. 20131122 - A citation notice is printed on stderr only if stderr is a terminal, the user has not specified --no-notice and the user has not run --bibtex once. This makes the release alpha quality. - --compress will compress temporary files. If the output is big and very compressible this will take up less disk space in $TMPDIR and possibly be faster due to less disk I/O. - --compress-program comtrols which program to use for compressing temporary files. - --bar show progress as a progress bar compatible with zenity. - --resume can now be used with --result: Jobs already run will be skipped. - --transfer and --basefile support paths relative to the --workdir by inserting /./ into the path. - Bug fixes and man page updates. 20131022 - --transfer files with /./ in the path will copy the files relative to the --workdir. - The maximal command length is now cached in a file halfing the startup time. - Bug fixes and man page updates. 20130922 - PDF-files of documentation is now included. - Bug fixes and man page updates. 20130822 - A tutorial showing much of GNU Parallel's functionality. Spend an hour walking through the tutorial. Your commandline will love you for it. man parallel_tutorial - --line-buffer will buffer output on line basis. --group keeps the output together for a whole job. --ungroup allows output to mixup with half a line coming from one job and half a line coming from another job. --line-buffer fits between these two; it prints a full line, but will allow for mixing lines of different jobs. - --record-env records all variables and functions set. Useful to record the normal environment before using --env _. - --env _ exports all functions and variables to the remote server except for the ones recorded. - New signing key. Due to recommendations from NIST http://www.keylength.com/en/4/ the signing key was changed from 1024D/ID:FFFFFFF1 to 9888R/ID:88888888. - Bug fixes and man page updates. 20130722 - --round-robin with --pipe will write all blocks to already running jobs. - --env can now transfer Bash function for remote execution. That is pretty cool! - Quite a few bug fixes and man page updates. 20130622 - --xapply now recycles arguments if an input source has more arguments than others. - The sleep time between jobs is now both increased and decreased exponentially. - 10 seconds installation check the signature using GnuPG if GnuPG is installed. - Bug fixes and man page updates. 20130522 - --ctrlc is now default if --pipe is not used; thus making it possible to kill remotely started jobs simply by pressing Ctrl-C. - --timeout 200% now means kill jobs that take more than twice the time of the median run time of a job. - Jobs are now distributed round robin when having mulitiple --sshlogin instead of filling up one --sshlogin at a time. - niceload: darwin version of --io --mem --swap. Thanks to Anders F Bjorklund. - Bug fixes and man page updates. 20130422 - 10 seconds installation: wget -O - pi.dk/3 | sh - HPUX CPU/core detection. Thanks to Javier Tarradas. - CPU detection for HURD, IRIX, SCO OpenServer and (old) HPUX. - --ctrlc will send SIGINT to tasks running on remote computers and thus killing them. - --load now uses 'ps' to see immediately see the number of running processes instead of 'uptime' thus making --load react much faster. - Testing on Centos 3.9 and FreeBSD 7 revealed a lot of compability bugs. Some of these required quite extensive changes making this release beta quality. - --retries works with --onall. - The new --load computation now works on FreeBSD 7. - --nice works under tcsh. - Bug fixes and man page updates. 20130222 - --resume works with --pipe. - --resume-failed will go through --joblog, redo the failed jobs and then continue like --resume. - Negative positional arguments count from the end: {-1} means the last argument, {-2} the second to last. - NetBSD CPU detection. - --blocksize increases exponentially if it smaller than a full record. - Processing n-line records (--pipe -L n) is now much faster. - --tollef is obsoleted after discussion on the mailing list. - Bug fixes and man page updates.
2016-07-09Bump PKGREVISION for perl-5.24.0 for everything mentioning perl.wiz7-14/+14
2016-06-19Remove INSTALLATION_PREFIX; no change to package.wiz1-2/+1
2016-06-08Bump PKGREVISION for perl-5.24.wiz1-2/+2
2016-06-08Switch to MASTER_SITES_PYPI.wiz1-2/+2
2016-06-08Remove the stability entity, it has no meaning outside of an official context.jperkin1-1/+0
2016-06-08Change the service_bundle name to "export" to reduce diffs between thejperkin1-1/+1
original manifest.xml file and the output from "svccfg export".
2016-06-01Include correct directory and rpath for finding bdb.joerg2-1/+15
2016-03-05Bump PKGREVISION for security/openssl ABI bump.jperkin3-5/+6
2016-02-28Let the compiler decide whether or not DT_RPATH vs DT_RUNPATH should bejoerg4-2/+147
used, don't force the crap in places that definitely don't know the right answer. Bump revision.
2016-02-16Update parallel/ganglia-monitor-core to 3.7.2. Add SMF support.fhajny17-72/+379
Upstream changelog follows. 3.7.2 * Fix for a memory leak if override_hostname or override_ip are used 3.7.1 * Hash table in gmetad has been reworked to support much higher metric counts and larger number of metrics * A number of GMond python modules have been rewritten and enhanced 3.7.0 * Add support for sending Ganglia metrics to Riemann * Performance improvements for gmetad * Export internal gmetad metrics 3.6.0 * Improvements - Add ability to gzip XML output in gmond - Forward Graphite metrics over UDP for increased scalability - Configurable graphite prefix - Memcached publish support - export metrics into memcached - Improve metric summarization performance under heavy load * Bug fixes - Reduce CPU utilization when running in deaf mode 3.5.0 * gmond uses a separate thread to service gmetad requests. This should improve performance * Updates to a number of core Python collection modules 3.4.0 * web - now distributed as a separate source tarball, web/ tree is not present in the ganglia-3.4.0 tarball * gmond: - setgid() and initgroups() are invoked when setuid() is invoked - gmond no longer runs in root group (there may be side-effects from this change) - new UDP RX channel parameter `buffer' to set the socket buffer size 3.3.7 * gmond - BUG100: fails to start in Solaris zones - fails to start when no address on the network interface (added retry_bind parameter) - BUG321: fails to start when Solaris CPU in state FAILED (seg fault) 3.3.6 * gmond - BUG327: memory leak when receive channel is not configured or not hearing any data 3.3.5 * General - Return to the use of autotools (make dist) for building the release - Release with a valid version number in configure.in * Gmetad - Fix for gmetad buffer overflow with metric value of DOUBLE_MAX * Gmond python modules - multi traffic module has been replaced with multi interface which correctly reports all interfaces * Gmond - Fix for issues/11 - Expose some useful functions in the public C API to assist module developers 3.3.1 * Web - Graph Zooming off when Ganglia Web server and web browser in different time zones - Metrics chooser in cluster view now provides metric drop down in addition to autocomplete - Ability to specify default size for graphs in Views Gmetad - You can now disable RRD writing - useful if you are using Ganglia only to feed Graphite - Export to graphite now handles spaces in metrics, groups - Minor code clean up and fixes * Gmond - Diskstat python module incorrect IO time reporting has been fixed. 3.3.0 * Web UI shipped with Ganglia has been upgraded to the Ganglia Web 2.0 interface. Changes to web UI from Gweb 2.2.0 are as follows - Additional Nagios integration scripts - Contributed by Etsy - stacked graph in the cluster view - Ability to hide/show events on individual graphs or whole pages - Numerous bug fixes and small enhancements * metric names are sanitized to avoid breaking gmetad/UI when characters such as <, > are supplied. Those are now replaced with underscores * gmetad supports sending metrics to graphite. Uncomment proper section of gmetad.conf to enable * Default RRD resolution changed. It affects only newly created RRDs. To change to old behavior please modify the RRA field in gmetad.conf * Number of Python modules from Ganglia gmond_python_modules repository have been included such as disk statistics, individual interface statistics, etc. Most of the modules are disabled. To enable rename the files in /etc/ganglia/conf.d. * sFlow has been enhanced to support Virtual machine pool/Java VM/Memcache/Apache/NGINX/Tomcat and node.js metrics. You can read more at sFlow blog 3.2.0 * sFlow support - more and even more * hostname/ip override - you can specify an arbitrary host name and IP to be shown in the UI. * FreeBSD patches * Python module improvements * Bugfixes and improvements over 3.1.7 3.1.7: * Fix build support for RHEL5/issue with commas in CFLAGS * multicpu module: show CPU utilization as a value between 0-100% for each core 3.1.3 - 3.1.6: * Merge commit 1966 from trunk to fix "contrib/removespikes.pl" * Bootstrapping with Debian 5.0 (lenny) versions of autotools for this and future releases. * Require user to explicitly specify sysconfdir when building from source, due to the fact that the old behavior was not consistent with the documented behavior. * Configuration files and scripts are now created during the install phase rather than during configure. This allows values such as @sysconfdir@ to be used in the template configuration files. * Abolish the use of release names - only release numbers will be used to distinguish versions in future * libmetrics: workaround system header conflict in DFBSD >= 2.4 (BUG245) * Use PCRE regex matching to configure metrics using the name_match directive * rrdcached support * gmetad now uses apr and the sleep intervals between polls are randomized in a way that supports shorter polling intervals * FreeBSD support: fixes for crashes and disk statistics (BUG153) * Further tweaks to Solaris build support (remove C99 hack) * Eliminate conflict with ncpus symbol name on older Solaris * AIX support: determine if the host is a virtual server (BUG226) * AIX support: setting linker flags (BUG227), add -lm * AIX support: tweaks for AIX >= v6.1 * AIX support: revised init scripts for gmond and gmetad * Check for Python.h explicitly * Include the necessary Python files in the distribution tarball, regardless of how BUILD_PYTHON is set (r2215). * Remove references to GNU toolchain in documentation * Fortify write_data_to_rrd against overflows * Web interface: minor formatting changes * mcast_if implementation tweaked so that the send channel will be bound to the IP of the outgoing interface * Documentation updates relating to the options for multihomed hosts, particularly bind, bind_hostname and mcast_if * gmond: Limit the use of APR_POLLSET_THREADSAFE to Linux >= 2.6 * gmond: improve/revert setuid behavior in configure script * gmond: Fix the allow_extra_data configuration directive (BUG199) * gmond: Ensure that a complete XML dump is delivered before closing the send socket. Submitted by: Jerry <adawzq 126 com> * gmond: add bind and bind_hostname parameters for udp_send_channel() * gmetad: BUG232: eliminate case-sensitive hostname bug, user can choose to maintain legacy behavior though. * gmond: BUG237: revise fix for segfault on Solaris where first CPU not in slot 0 * gmond: support for HUP signal on platforms with execve * gmond: status module: return gmond version info as string metrics * gmond: Check return status of apr_pollset_create. Use APR_POLLSET_THREADSAFE on Linux. * build: various configure options: Solaris 8 with Sun Studio 11 support, extra modules for static linking, default setuid, release number, build multicpu and status during static builds, support for SYSCONFDIR (BUG16) * RPM: include status module, allow packager to supply own gmond.conf * build: Look in lib64 rather than lib for apr, confuse and expat on x86_64
2015-12-29Fix missing/broken rcsids.dholland3-3/+11
2015-12-25-export-dynamic is a linker option, mark it accordingly. Fix non-catmanjoerg4-11/+54
build as root. Fix rpath path of spooledit.
2015-12-13Add a java option.markd5-13/+276
Fix PLIST for the sge option. Fix PLIST for linux.
2015-12-13Update to Grid Engine 2011.11p1asau6-131/+40
Give it chance to build on amd64.
2015-12-12Update to OpenMPI 1.10.1asau6-122/+216
Changes: 1.10.1 ------ - Workaround an optimization problem with gcc compilers >= 4.9.2 that causes problems with memory registration, and forced mpi_leave_pinned to default to 0 (i.e., off). Thanks to @oere for the fix. - Fix use of MPI_LB and MPI_UB in subarray and darray datatypes. Thanks to Gus Correa and Dimitar Pashov for pointing out the issue. - Minor updates to mpi_show_mpi_alloc_mem_leaks and ompi_debug_show_handle_leaks functionality. - Fix segv when invoking non-blocking reductions with a user-defined operation. Thanks to Rupert Nash and Georg Geiser for identifying the issue. - No longer probe for PCI topology on Solaris (unless running as root). - Fix for Intel Parallel Studio 2016 ifort partial support of the !GCC$ pragma. Thanks to Fabrice Roy for reporting the problem. - Bunches of Coverity / static analysis fixes. - Fixed ROMIO to look for lstat in <sys/stat.h>. Thanks to William Throwe for submitting the patch both upstream and to Open MPI. - Fixed minor memory leak when attempting to open plugins. - Fixed type in MPI_IBARRIER C prototype. Thanks to Harald Servat for reporting the issue. - Add missing man pages for MPI_WIN_CREATE_DYNAMIC, MPI_WIN_ATTACH, MPI_WIN_DETACH, MPI_WIN_ALLOCATE, MPI_WIN_ALLOCATE_SHARED. - When mpirun-launching new applications, only close file descriptors that are actually open (resulting in a faster launch in some environments). - Fix "test ==" issues in Open MPI's configure script. Thank to Kevin Buckley for pointing out the issue. - Fix performance issue in usnic BTL: ensure progress thread is throttled back to not aggressively steal CPU cycles. - Fix cache line size detection on POWER architectures. - Add missing #include in a few places. Thanks to Orion Poplawski for supplying the patch. - When OpenSHMEM building is disabled, no longer install its header files, help files, or man pages. Add man pages for oshrun, oshcc, and oshfort. - Fix mpi_f08 implementations of MPI_COMM_SET_INFO, and profiling versions of MPI_BUFFER_DETACH, MPI_WIN_ALLOCATE, MPI_WIN_ALLOCATE_SHARED, MPI_WTICK, and MPI_WTIME. - Add orte_rmaps_dist_device MCA param, allowing users to map near a specific device. - Various updates/fixes to the openib BTL. - Add missing defaults for the Mellanox ConnectX 3 card to the openib BTL. - Minor bug fixes in the OFI MTL. - Various updates to Mellanox's MXM, hcoll, and FCA components. - Add OpenSHMEM man pages. Thanks to Tony Curtis for sharing the man pages files from openshmem.org. - Add missing "const" attributes to MPI_COMPARE_AND_SWAP, MPI_FETCH_AND_OP, MPI_RACCUMULATE, and MPI_WIN_DETACH prototypes. Thanks to Michael Knobloch and Takahiro Kawashima for bringing this to our attention. - Fix linking issues on some platforms (e.g., SLES 12). - Fix hang on some corner cases when MPI applications abort. - Add missing options to mpirun man page. Thanks to Daniel Letai for bringing this to our attention. - Add new --with-platform-patches-dir configure option - Adjust relative selection priorities to ensure that MTL support is favored over BTL support when both are available - Use CUDA IPC for all sized messages for performance 1.10.0 ------ ** NOTE: The v1.10.0 release marks the transition to Open MPI's new ** version numbering scheme. The v1.10.x release series is based on ** the v1.8.x series, but with a few new features. v2.x will be the ** next series after the v1.10.x series, and complete the transition ** to the new version numbering scheme. See README for more details ** on the new versioning scheme. ** ** NOTE: In accordance with OMPI version numbering, the v1.10 is *not* ** API compatible with the v1.8 release series. - Added libfabric support (see README for more details): - usNIC BTL updated to use libfabric. - Added OFI MTL (usable with PSM in libfabric v1.1.0). - Added Intel Omni-Path support via new PSM2 MTL. - Added "yalla" PML for faster MXM support. - Removed support for MX - Added persistent distributed virtual machine (pDVM) support for fast workflow executions. - Fixed typo in GCC inline assembly introduced in Open MPI v1.8.8. Thanks to Paul Hargrove for pointing out the issue. - Add missing man pages for MPI_Win_get|set_info(3). - Ensure that session directories are cleaned up at the end of a run. - Fixed linking issues on some OSs where symbols of dependent libraries are not automatically publicly available. - Improve hcoll and fca configury library detection. Thanks to David Shrader for helping track down the issue. - Removed the LAMA mapper (for use in setting affinity). Its functionality has been largely superseded by other mpirun CLI options. - CUDA: Made the asynchronous copy mode be the default. - Fix a malloc(0) warning in MPI_IREDUCE_SCATTER_BLOCK. Thanks to Lisandro Dalcin for reporting the issue. - Fix typo in MPI_Scatter(3) man page. Thanks to Akshay Venkatesh for noticing the mistake. - Add rudimentary protection from TCP port scanners. - Fix typo in Open MPI error handling. Thanks to ¥ke Sandgren for pointing out the error. - Increased the performance of the CM PML (i.e., the Portals, PSM, PSM2, MXM, and OFI transports). - Restored visibility of blocking send requests in message queue debuggers (e.g., TotalView, DDT). - Fixed obscure IPv6-related bug in the TCP BTL. - Add support for the "no_locks" MPI_Info key for one-sided functionality. - Fixed ibv_fork support for verbs-based networks. - Fixed a variety of small bugs in OpenSHMEM. - Fixed MXM configure with additional CPPFLAGS and LDFLAGS. Thanks to David Shrader for the patch. - Fixed incorrect memalign threshhold in the openib BTL. Thanks to Xavier Besseron for pointing out the issue. 1.8.8 ----- - Fix a segfault in MPI_FINALIZE with the PSM MTL. - Fix mpi_f08 sentinels (e.g., MPI_STATUS_IGNORE) handling. - Set some additional MXM default values for OSHMEM. - Fix an invalid memory access in MPI_MRECV and MPI_IMRECV. - Include two fixes that were mistakenly left out of the official v1.8.7 tarball: - Fixed MPI_WIN_POST and MPI_WIN_START for zero-size messages - Protect the OOB TCP ports from segfaulting when accessed by port scanners 1.8.7 ---- ** NOTE: v1.8.7 technically breaks ABI with prior versions ** in the 1.8 series because it repairs two incorrect API ** signatures. However, users will only need to recompile ** if they were using those functions - which they couldn't ** have been, because the signatures were wrong :-) - Plugged a memory leak that impacted blocking sends - Fixed incorrect declaration for MPI_T_pvar_get_index and added missing return code MPI_T_INVALID_NAME. - Fixed an uninitialized variable in PMI2 support - Added new vendor part id for Mellanox ConnectX4-LX - Fixed NBC_Copy for legitimate zero-size messages - Fixed MPI_Win_post and MPI_Win_start for zero-size messages - Protect the OOB ports from segfaulting when accessed by port scanners - Fixed several Fortran typos - Fixed configure detection of XRC support - Fixed support for highly heterogeneous systems to avoid memory corruption when printing out the bindings 1.8.6 ----- - Fixed memory leak on Mac OS-X exposed by TCP keepalive - Fixed keepalive support to ensure that daemon/node failure results in complete job cleanup - Update Java binding support - Fixed MPI_THREAD_MULTIPLE bug in vader shared memory BTL - Fixed issue during shutdown when CUDA initialization wasn't complete - Fixed orted environment when no prefix given - Fixed trivial typo in MPI_Neighbor_allgather manpage - Fixed tree-spawn support for sh and ksh shells - Several data type fixes 1.8.5 ----- - Fixed configure problems in some cases when using an external hwloc installation. Thanks to Erick Schnetter for reporting the error and helping track down the source of the problem. - Fixed linker error on OS X when using the clang compiler. Thanks to Erick Schnetter for reporting the error and helping track down the source of the problem. - Fixed MPI_THREAD_MULTIPLE deadlock error in the vader BTL. Thanks to Thomas Klimpel for reporting the issue. - Fixed several Valgrind warnings. Thanks for Lisandro Dalcin for contributing a patch fixing some one-sided code paths. - Fixed version compatibility test in OOB that broke ABI within the 1.8 series. NOTE: this will not resolve the problem between pre-1.8.5 versions, but will fix it going forward. - Fix some issues related to running on Intel Xeon Phi coprocessors. - Opportunistically switch away from using GNU Libtool's libltdl library when possible (by default). - Fix some VampirTrace errors. Thanks to Paul Hargrove for reporting the issues. - Correct default binding patterns when --use-hwthread-cpus was specified and nprocs <= 2. - Fix warnings about -finline-functions when compiling with clang. - Updated the embedded hwloc with several bug fixes, including the "duplicate Lhwloc1 symbol" that multiple users reported on some platforms. - Do not error when mpirun is invoked with with default bindings (i.e., no binding was specified), and one or more nodes do not support bindings. Thanks to Annu Desari for pointing out the problem. - Let root invoke "mpirun --version" to check the version without printing the "Don't run as root!" warnings. Thanks to Robert McLay for the suggestion. - Fixed several bugs in OpenSHMEM support. - Extended vader shared memory support to 32-bit architectures. - Fix handling of very large datatypes. Thanks to Bogdan Sataric for the bug report. - Fixed a bug in handling subarray MPI datatypes, and a bug when using MPI_LB and MPI_UB. Thanks to Gus Correa for pointing out the issue. - Restore user-settable bandwidth and latency PML MCA variables. - Multiple bug fixes for cleanup during MPI_FINALIZE in unusual situations. - Added support for TCP keepalive signals to ensure timely termination when sockets between daemons cannot be created (e.g., due to a firewall). - Added MCA parameter to allow full use of a SLURM allocation when started from a tool (supports LLNL debugger). - Fixed several bugs in the configure logic for PMI and hwloc. - Fixed incorrect interface index in TCP communications setup. Thanks to Mark Kettenis for spotting the problem and providing a patch. - Fixed MPI_IREDUCE_SCATTER with single-process communicators when MPI_IN_PLACE was not used. - Added XRC support for OFED v3.12 and higher. - Various updates and bug fixes to the Mellanox hcoll collective support. - Fix problems with Fortran compilers that did not support REAL*16/COMPLEX*32 types. Thanks to Orion Poplawski for identifying the issue. - Fixed problem with rpath/runpath support in pkg-config files. Thanks to Christoph Junghans for notifying us of the issue. - Man page fixes: - Removed erroneous "color" discussion from MPI_COMM_SPLIT_TYPE. Thanks to Erick Schnetter for spotting the outdated text. - Fixed prototypes for MPI_IBARRIER. Thanks to Maximilian for finding the issue. - Updated docs about buffer usage in non-blocking communications. Thanks to Alexander Pozdneev for citing the outdated text. - Added documentation about the 'ompi_unique' MPI_Info key with MPI_PUBLISH_NAME. - Fixed typo in MPI_INTERCOMM_MERGE. Thanks to Harald Servat for noticing and sending a patch. - Updated configure paths in HACKING. Thanks to Maximilien Levesque for the fix. - Fixed Fortran typo in MPI_WIN_LOCK_ALL. Thanks to Thomas Jahns for pointing out the issue. - Fixed a number of MPI one-sided bugs. - Fixed MPI_COMM_SPAWN when invoked from a singleton job. - Fixed a number of minor issues with CUDA support, including registering of shared memory and supporting reduction support for GPU buffers. - Improved support for building OMPI on Cray platforms. - Fixed performance regression introduced by the inadvertent default enabling of MPI_THREAD_MULTIPLE support. 1.8.4 ----- - Fix MPI_SIZEOF; now available in mpif.h for modern Fortran compilers (see README for more details). Also fixed various compiler/linker errors. - Fixed inadvertant Fortran ABI break between v1.8.1 and v1.8.2 in the mpi interface module when compiled with gfortran >= v4.9. - Fix various MPI_THREAD_MULTIPLE issues in the TCP BTL. - mpirun no longer requires the --hetero-nodes switch; it will automatically detect when running in heterogeneous scenarios. - Update LSF support, to include revamped affinity functionality. - Update embedded hwloc to v1.9.1. - Fixed max registerable memory computation in the openib BTL. - Updated error message when debuggers are unable to find various symbols/types to be more clear. Thanks to Dave Love for raising the issue. - Added proper support for LSF and PBS/Torque libraries in static builds. - Rankfiles now support physical processor IDs. - Fixed potential hang in MPI_ABORT. - Fixed problems with the PSM MTL and "re-connect" scenarios, such as MPI_INTERCOMM_CREATE. - Fix MPI_IREDUCE_SCATTER with a single process. - Fix (rare) race condition in stdout/stderr funneling to mpirun where some trailing output could get lost when a process terminated. - Removed inadvertent change that set --enable-mpi-thread-multiple "on" by default, thus impacting performance for non-threaded apps. - Significantly reduced startup time by optimizing internal hash table implementation. - Fixed OS X linking with the Fortran mpi module when used with gfortran >= 4.9. Thanks to Github user yafshar for raising the issue. - Fixed memory leak on Cygwin platforms. Thanks for Marco Atzeri for reporting the issue. - Fixed seg fault in neighborhood collectives when the degree of the topology is higher than the communicator size. Thanks to Lisandro Dalcin for reporting the issue. - Fixed segfault in neighborhood collectives under certain use-cases. - Fixed various issues regarding Solaris support. Thanks to Siegmar Gross for patiently identifying all the issues. - Fixed PMI configure tests for certain Slurm installation patterns. - Fixed param registration issue in Java bindings. Thanks to Takahiro Kawashima and Siegmar Gross for identifying the issue. - Several man page fixes. - Silence several warnings and close some memory leaks (more remain, but it's better than it was). - Re-enabled the use of CMA and knem in the shared memory BTL. - Updated mpirun manpage to correctly explain new map/rank/binding options. - Fixed MPI_IALLGATHER problem with intercommunicators. Thanks for Takahiro Kawashima for the patch. - Numerous updates and performance improvements to OpenSHMEM. - Turned off message coalescing in the openib BTL until a proper fix for that capability can be provided (tentatively expected for 1.8.5) - Fix a bug in iof output that dates back to the dinosaurs which would output extra bytes if the system was very heavily loaded - Fix a bug where specifying mca_component_show_load_errors=0 could cause ompi_info to segfault - Updated valgrind suppression file 1.8.3 ----- - Fixed application abort bug to ensure that MPI_Abort exits appropriately and returns the provided exit status - Fixed some alignment (not all) issues identified by Clang - Allow CUDA-aware to work with nonblocking collectives. Forces packing to happen when using GPU buffers. - Fixed configure test issue with Intel 2015 Fortran compiler - Fixed some PGI-related errors - Provide better help message when encountering a firewall - Fixed MCA parameter quoting to protect multi-word params and params that contain special characters - Improved the bind-to help message to clarify the defaults - Add new MPI-3.1 tools interface - Several performance optimizations and memory leak cleanups - Turn off the coll/ml plugin unless specifically requested as it remains in an experimental state - Fix LSF support by adding required libraries for the latest LSF releases. Thanks to Joshua Randal for supplying the initial patches. 1.8.2 ----- - Fix auto-wireup of OOB, allowing ORTE to automatically test all available NICs - "Un-deprecate" pernode, npernode, and npersocket options by popular demand - Add missing Fortran bindings for MPI_WIN_LOCK_ALL, MPI_WIN_UNLOCK_ALL, and MPI_WIN_SYNC. - Fix cascading/over-quoting in some cases with the rsh/ssh-based launcher. Thanks to multiple users for raising the issue. - Properly add support for gfortran 4.9 ignore TKR pragma (it was erroneously only partially added in v1.7.5). Thanks to Marcus Daniels for raising the issue. - Update/improve help messages in the usnic BTL. - Resolve a race condition in MPI_Abort. - Fix obscure cases where static linking from wrapper compilers would fail. - Clarify the configure --help message about when OpenSHMEM is enabled/disabled by default. Thanks to Paul Hargrove for the suggestion. - Align pages properly where relevant. Thanks to Paul Hargrove for identifying the issue. - Various compiler warning and minor fixes for OpenBSD, FreeBSD, and Solaris/SPARC. Thanks to Paul Hargrove for the patches. - Properly pass function pointers from Fortran to C in the mpi_f08 module, thereby now supporting gfortran 4.9. Thanks to Tobias Burnus for assistance and testing with this issue. - Improve support for Cray CLE 5. - Fix mpirun regression: ensure exit status is non-zero if mpirun is terminated due to signal. - Improved CUDA efficiency of asynchronous copies. - Fix to parameter type in MPI_Type_indexed.3. Thanks to Bastian Beischer for reporting the mistake. - Fix NUMA distance calculations in the openib BTL. - Decrease time required to shut down mpirun at the end of a job. - More RMA fixes. - More hostfile fixes from Tetsuya Mishima. - Fix darray issue where UB was not computed correctly. - Fix mpi_f08 parameter name for MPI_GET_LIBRARY_VERSION. Thanks to Junchao Zhang for pointing out the issue. - Ensure mpirun aborts properly when unable to map processes in scheduled environments. - Ensure that MPI RMA error codes show up properly. Thanks to Lisandro Dalcin for reporting the issue. - Minor bug fixes and improvements to the bash and zsh mpirun autocompletion scripts. - Fix sequential mpirun process mapper. Thanks to Bill Chen for reporting the issue. - Correct SLURM stdout/stderr redirection. - Added missing portals 4 files. - Performance improvements for blocking sends and receives. - Lots of cleanup to the ml collective component - Added new Java methods to provide full MPI coverage - Many OSHMEM cleanups - Prevent comm_spawn from automatically launching a VM across all available nodes - Close many memory leaks to achieve valgrind-clean operation - Better handling of TCP connection discovery for mismatched networks where we don't have a direct 1:1 subnet match between nodes - Prevent segfault when OMPI info tools are used in pipes and user exits one step of that pipe before completing output 1.8.1 ----- - Fix for critical bug: mpirun removed files (but not directories) from / when run as root. Thanks to Jay Fenlason and Orion Poplawski for bringing the issue to our attention and helping identify the fix. 1.8 --- - Commit upstream ROMIO fix for mixed NFS+local filesystem environments. - Several fixes for MPI-3 one-sided support. For example, arbitrary-length datatypes are now supported. - Add config support for the Mellanox ConnectX 4 card. - Add missing MPI_COMM_GET|SET_INFO functions, and missing MPI_WEIGHTS_EMPTY and MPI_ERR_RMA_SHARED constants. Thanks to Lisandro Dalcin for pointing out the issue. - Update some help messages in OSHMEM, the usnic BTL, the TCP BTL, and ORTE, and update documentation about ompi_info's --level option. - Fix some compiler warnings. - Ensure that ORTE daemons are not bound to a single processor if TaskAffinity is set on by default in Slurm. Thanks to Artem Polyakov for identifying the problem and providing a patch 1.7.5 ----- ********************************************************************** * Open MPI is now fully MPI-3.0 compliant ********************************************************************** - Add Linux OpenSHMEM support built on top of Open MPI's MPI layer. Thanks to Mellanox for contributing this new feature. - Allow restricting ORTE daemons to specific cores using the orte_daemon_cores MCA param. - Ensure to properly set "locality" flags for processes launched via MPI dynamic functions such as MPI_COMM_SPAWN. - Fix MPI_GRAPH_CREATE when nnodes is smaller than the size of the old communicator. - usnic BTL now supports underlying UDP transport. - usnic BTL now checks for common connectivty errors at first send to a remote server. - Minor scalability improvements in the usnic BTL. - ompi_info now lists whether the Java MPI bindings are available or not. - MPI-3: mpi.h and the Fortran interfaces now report MPI_VERSION==3 and MPI_SUBVERSION==0. - MPI-3: Added support for new RMA functions and functionality. - Fix MPI_Info "const buglet. Thanks to Orion Poplawski for identifying the issue. - Multiple fixes to mapping/binding options. Thanks to Tetsuya Mishima for his assistance. - Multiple fixes for normal and abnormal process termination, including singleton MPI_Abort and ensuring to kill entire process groups when abnormally terminating a job. - Fix DESTDIR install for javadocs. Thanks to Orion Poplawski for pointing out the issue. - Various performance improvements for the MPI Java bindings. - OMPI now uses its own internal random number generator and will not perturb srand() and friends. - Some cleanups for Cygwin builds. Thanks to Marco Atzeri for the patches. - Add a new collective component (coll/ml) that provides substantially improved performance. It is still experimental, and requires setting coll_ml_priority > 0 to become active. - Add version check during startup to ensure you are using the same version of Open MPI on all nodes in a job. - Significantly improved the performance of MPI_DIMS_CREATE for large values. Thanks to Andreas Sch„fer for the contribution. - Removed ASYNCHRONOUS keyword from the "ignore TKR" mpi_f08 module. - Deprecated the following mpirun options: --bynode, --bycore, --byslot: replaced with --map-by node|core|slot. --npernode, --npersocket: replaced with --map-by ppr:N:node and --map-by ppr:N:socket, respectively - Pick NFS "infinitely stale" fix from ROMIO upstream. - Various PMI2 fixes and extension to support broader range of mappings. - Improve launch performance at large scale. - Add support for PBS/Torque environments that set environment variables to indicate the number of slots available on each nodes. Set the ras_tm_smp MCA parameter to "1" to enable this mode. - Add new, more scalable endpoint exchange (commonly called "modex") method that only exchanges endpoint data on a per-peer basis on first message. Not all transports have been updated to use this feature. Set the rte_orte_direct_modex parameter to "1" to enable this mode. 1.7.4 ----- ********************************************************************** * CRITICAL CHANGE * * As of release 1.7.4, OpenMPI's default mapping, ranking, and binding * settings have changed: * * Mapping: * if #procs <= 2, default to map-by core * if #procs > 2, default to map-by socket * Ranking: * if default mapping is used, then default to rank-by slot * if map-by <obj> is given, then default to rank-by <obj>, * where <obj> is whatever object we mapped against * Binding: * default to bind-to core * * Users can override any of these settings individually using the * corresponding MCA parameter. Note that multi-threaded applications * in particular may want to override at least the binding default * to allow threads to use multiple cores. ********************************************************************** - Restore version number output in "ompi_info --all". - Various bug fixes for the mpi_f08 Fortran bindings. - Fix ROMIO compile error with Lustre 2.4. Thanks to Adam Moody for reporting the issue. - Various fixes for 32 bit platforms. - Add ability to selectively disable building the mpi or mpi_f08 module. See the README file for details. - Fix MX MTL finalization issue. - Fix ROMIO issue when opening a file with MPI_MODE_EXCL. - Fix PowerPC and MIPS assembly issues. - Various fixes to the hcoll and FCA collective offload modules. - Prevent integer overflow when creating datatypes. Thanks to original patch from Gilles Gouaillardet. - Port some upstream hwloc fixes to Open MPI's embedded copy for working around buggy NUMA node cpusets and including mising header files. Thanks to Jeff Becker and Paul Hargrove for reporting the issues. - Fix recursive invocation issues in the MXM MTL. - Various bug fixes to the new MCA parameter back-end system. - Have the posix fbtl module link against -laio on NetBSD platforms. Thanks to Paul Hargrove for noticing the issue. - Various updates and fixes to network filesystem detection to support more operating systems. - Add gfortran v4.9 "ignore TKR" syntax to the mpi Fortran module. - Various compiler fixes for several BSD-based platforms. Thanks to Paul Hargrove for reporting the issues. - Fix when MPI_COMM_SPAWN[_MULTIPLE] is used on oversubscribed systems. - Change the output from --report bindings to simply state that a process is not bound, instead of reporting that it is bound to all processors. - Per MPI-3.0 guidance, remove support for all MPI subroutines with choice buffers from the TKR-based mpi Fortran module. Thanks to Jed Brown for raising the issue. - Only allow the usnic BTL to build on 64 bit platforms. - Various bug fixes to SLURM support, to include ensuring proper exiting on abnormal termination. - Ensure that MPI_COMM_SPAWN[_MULTIPLE] jobs get the same mapping directives that were used with mpirun. - Fixed the application of TCP_NODELAY. - Change the TCP BTL to not warn if a non-existent interface is ignored. - Restored the "--bycore" mpirun option for backwards compatibility. - Fixed debugger attach functionality. Thanks to Ashley Pittman for reporting the issue and suggesting the fix. - Fixed faulty MPI_IBCAST when invoked on a communicator with only one process. - Add new Mellanox device IDs to the openib BTL. - Progress towards cleaning up various internal memory leaks as reported by Valgrind. - Fixed some annoying flex-generated warnings that have been there for years. Thanks to Tom Fogal for the initial patch. - Support user-provided environment variables via the "env" info key to MPI_COMM_SPAWN[_MULTIPLE]. Thanks to Tom Fogal for the feature request. - Fix uninitialized variable in MPI_DIST_GRAPH_CREATE. - Fix a variety of memory errors on SPARC platforms. Thanks to Siegmar Gross for reporting and testing all the issues. - Remove Solaris threads support. When building on Solaris, pthreads will be used. - Correctly handle the convertor internal stack for persistent receives. Thanks to Guillaume Gouaillardet for identifying the problem. - Add support for using an external libevent via --with-libevent. See the README for more details. - Various OMPIO updates and fixes. - Add support for the MPIEXEC_TIMEOUT environment variable. If set, mpirun will terminate the job after this many seconds. - Update the internal copy of ROMIO to that which shipped in MPICH 3.0.4. - Various performance tweaks and improvements in the usnic BTL, including now reporting MPI_T performance variables for each usnic device. - Fix to not access send datatypes for non-root processes with MPI_ISCATTER[V] and MPI_IGATHER[V]. Thanks to Pierre Jolivet for supplying the initial patch. - Update VampirTrace to 5.14.4.9. - Fix ptmalloc2 hook disable when used with ummunotify. - Change the default connection manager for the openib BTL to be based on UD verbs data exchanges instead of ORTE OOB data exchanges. - Fix Fortran compile error when compiling with 8-byte INTEGERs and 4-byte ints. - Fix C++11 issue identified by Jeremiah Willcock. - Many changes, updates, and bug fixes to the ORTE run-time layer. - Correctly handle MPI_REDUCE_SCATTER with recvcounts of 0. - Update man pages for MPI-3, and add some missing man pages for MPI-2.x functions. - Updated mpi_f08 module in accordance with post-MPI-3.0 errata which basically removed BIND(C) from all interfaces. - Fixed MPI_IN_PLACE detection for MPI_SCATTER[V] in Fortran routines. Thanks to Charles Gerlach for identifying the issue. - Added support for routable RoCE to the openib BTL. - Update embedded hwloc to v1.7.2. - ErrMgr framework redesigned to better support fault tolerance development activities. See the following RFC for details: http://www.open-mpi.org/community/lists/devel/2010/03/7589.php - Added database framework to OPAL and changed all modex operations to flow thru it, also included additional system info in the available data - Added staged state machine to support sequential work flows - Added distributed file system support for accessing files across nodes that do not have networked file systems - Extended filem framework to support scalable pre-positioning of files for use by applications, adding new "raw" component that transmits files across the daemon network - Native Windows support has been removed. A cygwin package is available from that group for Windows-based use. - Added new MPI Java bindings. See the Javadocs for more details on the API. - Wrapper compilers now add rpath support by default to generated executables on systems that support it. This behavior can be disabled via --disable-wrapper-rpath. See note in README about ABI issues when using rpath in MPI applications. - Added a new parallel I/O component and multiple new frameworks to support parallel I/O operations. - Fixed MPI_STATUS_SIZE Fortran issue when used with 8-byte Fortran INTEGERs and 4-byte C ints. Since this issue affects ABI, it is only enabled if Open MPI is configured with --enable-abi-breaking-fortran-status-i8-fix. Thanks to Jim Parker for supplying the initial patch. - Add support for Intel Phi SCIF transport. - For CUDA-aware MPI configured with CUDA 6.0, use new pointer attribute to avoid extra synchronization in stream 0 when using CUDA IPC between GPUs on the same node. - For CUDA-aware MPI configured with CUDA 6.0, compile in support of GPU Direct RDMA in openib BTL to improve small message latency. - Updated ROMIO from MPICH v3.0.4. - MPI-3: Added support for remaining non-blocking collectives. - MPI-3: Added support for neighborhood collectives. - MPI-3: Updated C bindings with consistent use of []. - MPI-3: Added the const keyword to read-only buffers. - MPI-3: Added support for non-blocking communicator duplication. - MPI-3: Added support for non-collective communicator creation. 1.7.3 ----- - Make CUDA-aware support dynamically load libcuda.so so CUDA-aware MPI library can run on systems without CUDA software. - Fix various issues with dynamic processes and intercommunicator operations under Torque. Thanks to Suraj Prabhakaran for reporting the problem. - Enable support for the Mellanox MXM2 library by default. - Improve support for Portals 4. - Various Solaris fixes. Many thanks to Siegmar Gross for his incredible patience in reporting all the issues. - MPI-2.2: Add reduction support for MPI_C_*COMPLEX and MPI::*COMPLEX. - Fixed internal accounting when openpty() fails. Thanks to Michal Peclo for reporting the issue and providing a patch. - Fixed too-large memory consumption in XRC mode of the openib BTL. Thanks to Alexey Ryzhikh for the patch. - Add bozo check for negative np values to mpirun to prevent a deadlock. Thanks to Upinder Malhi for identifying the issue. - Fixed MPI_IS_THREAD_MAIN behavior. Thanks to Lisandro Dalcin for pointing out the problem. - Various rankfile fixes. - Fix functionality over iWARP devices. - Various memory and performance optimizations and tweaks. - Fix MPI_Cancel issue identified by Fujitsu. - Add missing support for MPI_Get_address in the "use mpi" TKR implementation. Thanks to Hugo Gagnon for identifying the issue. - MPI-3: Add support for MPI_Count. - MPI-2.2: Add missing MPI_IN_PLACE support for MPI_ALLTOALL. - Added new usnic BTL to support the Cisco usNIC device. - Minor VampirTrace update to 5.14.4.4. - Removed support for ancient OS X systems (i.e., prior to 10.5). - Fixed obscure packing/unpacking datatype bug. Thanks to Takahiro Kawashima for identifying the issue. - Add run-time support for PMI2 environments. - Update openib BTL default parameters to include support for Mellanox ConnectX3-Pro devices. - Update libevent to v2.0.21. - "ompi_info --param TYPE PLUGIN" now only shows a small number of MCA parameters by default. Add "--level 9" or "--all" to see *all* MCA parameters. See README for more details. - Add support for asynchronous CUDA-aware copies. - Add support for Mellanox MPI collective operation offload via the "hcoll" library. - MPI-3: Add support for the MPI_T interface. Open MPI's MCA parameters are now accessible via the MPI_T control variable interface. Support has been added for a small number of MPI_T performance variables. - Add Gentoo memory hooks override. Thanks to Justin Bronder for the patch. - Added new "mindist" process mapper, allowing placement of processes via PCI locality information reported by the BIOS. - MPI-2.2: Add support for MPI_Dist_graph functionality. - Enable generic, client-side support for PMI2 implementations. Can be leveraged by any resource manager that implements PMI2; e.g. SLURM, versions 2.6 and higher. 1.7.2 ----- - Major VampirTrace update to 5.14.4.2. (** also appeared: 1.6.5) - Fix to set flag==1 when MPI_IPROBE is called with MPI_PROC_NULL. (** also appeared: 1.6.5) - Set the Intel Phi device to be ignored by default by the openib BTL. (** also appeared: 1.6.5) - Decrease the internal memory storage used by intrinsic MPI datatypes for Fortran types. Thanks to Takahiro Kawashima for the initial patch. (** also appeared: 1.6.5) - Fix total registered memory calculation for Mellanox ConnectIB and OFED 2.0. (** also appeared: 1.6.5) - Fix possible data corruption in the MXM MTL component. (** also appeared: 1.6.5) - Remove extraneous -L from hwloc's embedding. Thanks to Stefan Friedel for reporting the issue. (** also appeared: 1.6.5) - Fix contiguous datatype memory check. Thanks to Eric Chamberland for reporting the issue. (** also appeared: 1.6.5) - Make the openib BTL more friendly to ignoring verbs devices that are not RC-capable. (** also appeared: 1.6.5) - Fix some MPI datatype engine issues. Thanks to Thomas Jahns for reporting the issue. (** also appeared: 1.6.5) - Add INI information for Chelsio T5 device. (** also appeared: 1.6.5) - Integrate MXM STREAM support for MPI_ISEND and MPI_IRECV, and other minor MXM fixes. (** also appeared: 1.6.5) - Fix to not show amorphous "MPI was already finalized" error when failing to MPI_File_close an open file. Thanks to Brian Smith for reporting the issue. (** also appeared: 1.6.5) - Add a distance-based mapping component to find the socket "closest" to the PCI bus. - Fix an error that caused epoll to automatically be disabled in libevent. - Upgrade hwloc to 1.5.2. - *Really* fixed XRC compile issue in Open Fabrics support. - Fix MXM connection establishment flow. - Fixed parallel debugger ability to attach to MPI jobs. - Fixed some minor memory leaks. - Fixed datatype corruption issue when combining datatypes of specific formats. - Added Location Aware Mapping Algorithm (LAMA) mapping component. - Fixes for MPI_STATUS handling in corner cases. - Add a distance-based mapping component to find the socket "closest" to the PCI bus. 1.7.1 ----- - Fixed compile error when --without-memory-manager was specified on Linux - Fixed XRC compile issue in Open Fabrics support. 1.7 --- - Added MPI-3 functionality: - MPI_GET_LIBRARY_VERSION - Matched probe - MPI_TYPE_CREATE_HINDEXED_BLOCK - Non-blocking collectives - MPI_INFO_ENV support - Fortran '08 bindings (see below) - Dropped support for checkpoint/restart due to loss of maintainer :-( - Enabled compile-time warning of deprecated MPI functions by default (in supported compilers). - Revamped Fortran MPI bindings (see the README for details): - "mpifort" is now the preferred wrapper compiler for Fortran - Added "use mpi_f08" bindings (for compilers that support it) - Added better "use mpi" support (for compilers that support it) - Removed incorrect MPI_SCATTERV interface from "mpi" module that was added in the 1.5.x series for ABI reasons. - Lots of VampirTrace upgrades and fixes; upgrade to v5.14.3. - Modified process affinity system to provide warning when bindings result in being "bound to all", which is equivalent to not being bound. - Removed maffinity, paffinity, and carto frameworks (and associated MCA params). - Upgraded to hwloc v1.5.1. - Added performance improvements to the OpenIB (OpenFabrics) BTL. - Made malloc hooks more friendly to IO interprosers. Thanks to the bug report and suggested fix from Darshan maintainer Phil Carns. - Added support for the DMTCP checkpoint/restart system. - Added support for the Cray uGNI interconnect. - Fixed header file problems on OpenBSD. - Fixed issue with MPI_TYPE_CREATE_F90_REAL. - Wrapper compilers now explicitly list/link all Open MPI libraries if they detect static linking CLI arguments. - Open MPI now requires a C99 compiler to build. Please upgrade your C compiler if you do not have a C99-compliant compiler. - Fix MPI_GET_PROCESSOR_NAME Fortran binding to set ierr properly. Thanks to LANL for spotting the error. - Many MXM and FCA updates. - Fixed erroneous free of putenv'ed string that showed up in Valgrind reports. - Fixed MPI_IN_PLACE case for MPI_ALLGATHER. - Fixed a bug that prevented MCA params from being forwarded to daemons upon launch. - Fixed issues with VT and CUDA --with-cuda[-libdir] configuration CLI parameters. - Entirely new implementation of many MPI collective routines focused on better performance. - Revamped autogen / build system. - Add new sensor framework to ORTE that includes modules for detecting stalled applications and processes that consume too much memory. - Added new state machine framework to ORTE that converts ORTE into an event-driven state machine using the event library. - Added a new MCA parameter (ess_base_stream_buffering) that allows the user to override the system default for buffering of stdout/stderr streams (via setvbuf). Parameter is not visible via ompi_info. - Revamped the launch system to allow consideration of node hardware in assigning process locations and bindings. - Added the -novm option to preserve the prior launch behavior. - Revamped the process mapping system to utilize node hardware by adding new map-by, rank-by, and bind-to cmd line options. - Added new MCA parameter to provide protection against IO forwarding backlog. - Dropped support for native Windows due to loss of maintainers. :-( - Added a new parallel I/O component and multiple new frameworks to support parallel I/O operations. - Fix typo in orte_setup_hadoop.m4. Thanks to Aleksej Saushev for reporting it - Fix a very old error in opal_path_access(). Thanks to Marco Atzeri for chasing it down. 1.6.5 ----- - Updated default SRQ parameters for the openib BTL. (** also to appear: 1.7.2) - Major VampirTrace update to 5.14.4.2. (** also to appear: 1.7.2) - Fix to set flag==1 when MPI_IPROBE is called with MPI_PROC_NULL. (** also to appear: 1.7.2) - Set the Intel Phi device to be ignored by default by the openib BTL. (** also to appear: 1.7.2) - Decrease the internal memory storage used by intrinsic MPI datatypes for Fortran types. Thanks to Takahiro Kawashima for the initial patch. (** also to appear: 1.7.2) - Fix total registered memory calculation for Mellanox ConnectIB and OFED 2.0. (** also to appear: 1.7.2) - Fix possible data corruption in the MXM MTL component. (** also to appear: 1.7.2) - Remove extraneous -L from hwloc's embedding. Thanks to Stefan Friedel for reporting the issue. (** also to appear: 1.7.2) - Fix contiguous datatype memory check. Thanks to Eric Chamberland for reporting the issue. (** also to appear: 1.7.2) - Make the openib BTL more friendly to ignoring verbs devices that are not RC-capable. (** also to appear: 1.7.2) - Fix some MPI datatype engine issues. Thanks to Thomas Jahns for reporting the issue. (** also to appear: 1.7.2) - Add INI information for Chelsio T5 device. (** also to appear: 1.7.2) - Integrate MXM STREAM support for MPI_ISEND and MPI_IRECV, and other minor MXM fixes. (** also to appear: 1.7.2) - Improved alignment for OpenFabrics buffers. - Fix to not show amorphous "MPI was already finalized" error when failing to MPI_File_close an open file. Thanks to Brian Smith for reporting the issue. (** also to appear: 1.7.2)
2015-11-25Remove mk/find-prefix.mk usage from the parallel category.jperkin3-25/+9
The find-prefix infrastructure was required in a pkgviews world where packages installed from pkgsrc could have different installation prefixes, and this was a way for a dependency prefix to be determined. Now that pkgviews has been removed there is no longer any need for the overhead of this infrastructure. Instead we use BUILDLINK_PREFIX.pkg for dependencies pulled in via buildlink, or LOCALBASE/PREFIX where the dependency is coming from pkgsrc. Provides a reasonable performance win due to the reduction of `pkg_info -qp` calls, some of which were redundant anyway as they were duplicating the same information provided by BUILDLINK_PREFIX.pkg.
2015-11-14Update hwloc to 1.11.1asau2-12/+8
pkgsrc changes: stop regenerating build system and drop build-time dependency on autotools. Version 1.11.1 -------------- * Detection fixes + Hardwire the topology of Fujitsu K-computer, FX10, FX100 servers to workaround buggy Linux kernels. Thanks to Takahiro Kawashima and Gilles Gouaillardet. + Fix L3 cache information on AMD Opteron 61xx Magny-Cours processors in the x86 backend. Thanks to Guillaume Beauchamp for the patch. + Detect block devices directly attached to PCI without a controller, for instance NVMe disks. Thanks to Barry M. Tannenbaum. + Add the PCISlot attribute to all PCI functions instead of only the first one. * Miscellaneous internal fixes + Ignore PCI bridges that could fail assertions by reporting buggy secondary-subordinate bus numbers Thanks to George Bosilca for reporting the issue. + Fix an overzealous assertion when inserting an intermediate Group object while Groups are totally ignored. + Fix a memory leak on Linux on AMD processors with dual-core compute units. Thanks to Bob Benner. + Fix a memory leak on failure to load a xml diff file. + Fix some segfaults when inputting an invalid synthetic description. + Fix a segfault when plugins fail to find core symbols. Thanks to Guy Streeter. * Many fixes and improvements in the Windows backend: + Fix the discovery of more than 32 processors and multiple processor groups. Thanks to Barry M. Tannenbaum for the help. + Add thread binding set support in case of multiple process groups. + Add thread binding get support. + Add get_last_cpu_location() support for the current thread. + Disable the unsupported process binding in case of multiple processor groups. + Fix/update the Visual Studio support under contrib/windows. Thanks to Eloi Gaudry for the help. * Tools fixes + Fix a segfault when displaying logical indexes in the graphical lstopo. Thanks to Guillaume Mercier for reporting the issue. + Fix lstopo linking with X11 libraries, for instance on Mac OS X. Thanks to Scott Atchley and Pierre Ramet for reporting the issue. + hwloc-annotate, hwloc-diff and hwloc-patch do not drop unavailable resources from the output anymore and those may be annotated as well. + Command-line tools may now import XML from the standard input with -i -.xml + Add missing documentation for the hwloc-info --no-icaches option.
2015-11-02Add SHA512 digests for distfiles for parallel categoryagc19-19/+38
Existing SHA1 digests verified, all found to be the same on the machine holding the existing distfiles (morden). Existing SHA1 digests retained for now as an audit trail.
2015-09-23libtool can be found at least 3 levels deep.joerg1-1/+3
2015-08-18Bump all packages that depend on curses.bui* or terminfo.bui* since theywiz3-5/+6
might incur ncurses dependencies on some platforms, and ncurses just bumped its shlib. Some packages were bumped twice now, sorry for that.
2015-08-15Update to hwloc 1.11.1asau3-31/+56
From Benedek Gergely on pkgsrc-users. Version 1.11.1 -------------- * hwloc-annotate does not drop unavailable resources from the output anymore and those may be annotated as well. * Fix an overzealous assertion when intermediate Group object while Groups are totally ignored. * Fix a memory leak on failure to load a xml diff file. * Fix some segfaults when inputting an invalid synthetic description. Version 1.11.0 -------------- * API + Socket objects are renamed into Package to align with the terminology used by processor vendors. The old HWLOC_OBJ_SOCKET type and "Socket" name are still supported for backward compatibility. + HWLOC_OBJ_NODE is replaced with HWLOC_OBJ_NUMANODE for clarification. HWLOC_OBJ_NODE is still supported for backward compatibility. "Node" and "NUMANode" strings are supported as in earlier releases. * Detection improvements + Add support for Intel Knights Landing Xeon Phi. Thanks to Grzegorz Andrejczuk and Lukasz Anaczkowski. + Add Vendor, Model, Revision, SerialNumber, Type and LinuxDeviceID info attributes to Block OS devices on Linux. Thanks to Vineet Pedaballe for the help. - Add --disable-libudev to avoid dependency on the libudev library. + Add "MemoryDevice" Misc objects with information about DIMMs, on Linux when privileged and when I/O is enabled. Thanks to Vineet Pedaballe for the help. + Add a PCISlot attribute to PCI devices on Linux when supported to identify the physical PCI slot where the board is plugged. + Add CPUStepping info attribute on x86 processors, thanks to Thomas Röhl for the suggestion. + Ignore the device-tree on non-Power architectures to avoid buggy detection on ARM. Thanks to Orion Poplawski for reporting the issue. + Work-around buggy Xeon E5v3 BIOS reporting invalid PCI-NUMA affinity for the PCI links on the second processor. + Add support for CUDA compute capability 5.x, thanks Benjamin Worpitz. + Many fixes to the x86 backend - Add L1i and fix L2/L3 type on old AMD processors without topoext support. - Fix Intel CPU family and model numbers when basic family isn't 6 or 15. - Fix package IDs on recent AMD processors. - Fix misc issues due to incomplete APIC IDs on x2APIC processors. - Avoid buggy discovery on old SGI Altix UVs with non-unique APIC IDs. + Gather total machine memory on NetBSD. * Tools + lstopo - Collapse identical PCI devices unless --no-collapse is given. This avoids gigantic outputs when a PCI device contains dozens of identical virtual functions. - The ASCII art output is now called "ascii", for instance in "lstopo -.ascii". The former "txt" extension is retained for backward compatibility. - Automatically scales graphical box width to the inner text in Cairo, ASCII and Windows outputs. - Add --rect to lstopo to force rectangular layout even for NUMA nodes. - Add --restrict-flags to configure the behavior of --restrict. - Objects may have a "Type" info attribute to specify a better type name and display it in lstopo. - Really export all verbose information to the given output file. + hwloc-annotate - May now operate on all types of objects, including I/O. - May now insert Misc objects in the topology. - Do not drop instruction caches and I/O devices from the output anymore. + Fix lstopo path in hwloc-gather-topology after install. * Misc + Fix hwloc/cudart.h for machines with multiple PCI domains, thanks to Imre Kerr for reporting the problem. + Fix PCI Bridge-specific depth attribute. + Fix hwloc_bitmap_intersect() for two infinite bitmaps. + Fix some corner cases in the building of levels on large NUMA machines with non-uniform NUMA groups and I/Os. + Improve the performance of object insertion by cpuset for large topologies. + Prefix verbose XML import errors with the source name. + Improve pkg-config checks and error messages. + Fix excluding after a component with an argument in the HWLOC_COMPONENTS environment variable. * Documentation + Fix the recommended way in documentation and examples to allocate memory on some node, it should use HWLOC_MEMBIND_BIND. Thanks to Nicolas Bouzat for reporting the issue. + Add a "Miscellaneous objects" section in the documentation. + Add a FAQ entry "What happens to my topology if I disable symmetric multithreading, hyper-threading, etc. ?" to the documentation. Version 1.10.1 -------------- * Actually remove disallowed NUMA nodes from nodesets when the whole-system flag isn't enabled. * Fix the gathering of PCI domains. Thanks to James Custer for reporting the issue and providing a patch. * Fix the merging of identical parent and child in presence of Misc objects. Thanks to Dave Love for reporting the issue. * Fix some misordering of children when merging with ignore_keep_structure() in partially allowed topologies. * Fix an overzealous assertion in the debug code when running on a single-PU host with I/O. Thanks to Thomas Van Doren for reporting the issue. * Don't forget to setup NUMA node object nodesets in x86 backend (for BSDs) and OSF/Tru64 backend. * Fix cpuid-x86 build error with gcc -O3 on x86-32. Thanks to Thomas Van Doren for reporting the issue. * Fix support for future very large caches in the x86 backend. * Fix vendor/device names for SR-IOV PCI devices on Linux. * Fix an unlikely crash in case of buggy hierarchical distance matrix. * Fix PU os_index on some AIX releases. Thanks to Hendryk Bockelmann and Erik Schnetter for helping debugging. * Fix hwloc_bitmap_isincluded() in case of infinite sets. * Change hwloc-ls.desktop into a lstopo.desktop and only install it if lstopo is built with Cairo/X11 support. It cannot work with a non-graphical lstopo or hwloc-ls. * Add support for the renaming of Socket into Package in future releases. * Add support for the replacement of HWLOC_OBJ_NODE with HWLOC_OBJ_NUMANODE in future releases. * Clarify the documentation of distance matrices in hwloc.h and in the manpage of the hwloc-distances. Thanks to Dave Love for the suggestion. * Improve some error messages by displaying more information about the hwloc library in use. * Document how to deal with the ABI break when upgrading to the upcoming 2.0 See "How do I handle ABI breaks and API upgrades ?" in the FAQ. Version 1.10.0 -------------- * API + Add hwloc_topology_export_synthetic() to export a topology to a synthetic string without using lstopo. See the Synthetic topologies section in the documentation. + Add hwloc_topology_set/get_userdata() to let the application save a private pointer in the topology whenever it needs a way to find its own object corresponding to a topology. + Add hwloc_get_numanode_obj_by_os_index() and document that this function as well as hwloc_get_pu_obj_by_os_index() are good at converting nodesets and cpusets into objects. + hwloc_distrib() does not ignore any objects anymore when there are too many of them. They get merged with others instead. Thanks to Tim Creech for reporting the issue. * Tools + hwloc-bind --get <command-line> now executes the command after displaying the binding instead of ignoring the command entirely. Thanks to John Donners for the suggestion. + Clarify that memory sizes shown in lstopo are local by default unless specified (total memory added in the root object). * Synthetic topologies + Synthetic topology descriptions may now specify attributes such as memory sizes and OS indexes. See the Synthetic topologies section in the documentation. + lstopo now exports in this fully-detailed format by default. The new option --export-synthetic-flags may be used to revert back the old format. * Documentation + Add the doc/examples/ subdirectory with several real-life examples, including the already existing hwloc-hello.C for basics. Thanks to Rob Aulwes for the suggestion. + Improve the documentation of CPU and memory binding in the API. + Add a FAQ entry about operating system errors, especially on AMD platforms with buggy cache information. + Add a FAQ entry about loading many topologies in a single program. * Misc + Work around buggy Linux kernels reporting 2 sockets instead 1 socket with 2 NUMA nodes for each Xeon E5 v3 (Haswell) processor. + pciutils/libpci support is now removed since libpciaccess works well and there's also a Linux-specific PCI backend. For the record, pciutils was GPL and therefore disabled by default since v1.6.2. + Add --disable-cpuid configure flag to work around buggy processor simulators reporting invalid CPUID information. Thanks for Andrew Friedley for reporting the issue. + Fix a racy use of libltdl when manipulating multiple topologies in different threads. Thanks to Andra Hugo for reporting the issue and testing patches. + Fix some build failures in private/misc.h. Thanks to Pavan Balaji and Ralph Castain for the reports. + Fix failures to detect X11/Xutil.h on some Solaris platforms. Thanks to Siegmar Gross for reporting the failure. + The plugin ABI has changed, this release will not load plugins built against previous hwloc releases. Version 1.9.1 ------------- * Fix a crash when the PCI locality is invalid. Attach to the root object instead. Thanks to Nicolas Denoyelle for reporting the issue. * Fix -f in lstopo manpage. Thanks to Jirka Hladky for reporting the issue. * Fix hwloc_obj_type_sscanf() and others when strncasecmp() is not properly available. Thanks to Nick Papior Andersen for reporting the problem. * Mark Linux file descriptors as close-on-exec to avoid leaks on exec. * Fix some minor memory leaks. Version 1.9.0 ------------- * API + Add hwloc_obj_type_sscanf() to extend hwloc_obj_type_of_string() with type-specific attributes such as Cache/Group depth and Cache type. hwloc_obj_type_of_string() is moved to hwloc/deprecated.h. + Add hwloc_linux_get_tid_last_cpu_location() for retrieving the last CPU where a Linux thread given by TID ran. + Add hwloc_distrib() to extend the old hwloc_distribute[v]() functions. hwloc_distribute[v]() is moved to hwloc/deprecated.h. + Don't mix total and local memory when displaying verbose object attributes with hwloc_obj_attr_snprintf() or in lstopo. * Backends + Add CPUVendor, CPUModelNumber and CPUFamilyNumber info attributes for x86, ia64 and Xeon Phi sockets on Linux, to extend the x86-specific support added in v1.8.1. Requested by Ralph Castain. + Add many CPU- and Platform-related info attributes on ARM and POWER platforms, in the Machine and Socket objects. + Add CUDA info attributes describing the number of multiprocessors and cores and the size of the global, shared and L2 cache memories in CUDA OS devices. + Add OpenCL info attributes describing the number of compute units and the global memory size in OpenCL OS devices. + The synthetic backend now accepts extended types such as L2Cache, L1i or Group3. lstopo also exports synthetic strings using these extended types. * Tools + lstopo - Do not overwrite output files by default anymore. Pass -f or --force to enforce it. - Display OpenCL, CUDA and Xeon Phi numbers of cores and memory sizes in the graphical output. - Fix export to stdout when specifying a Cairo-based output type with --of. + hwloc-ps - Add -e or --get-last-cpu-location to report where processes/threads run instead of where they are bound. - Report locations as likely-more-useful objects such as Cores or Sockets instead of Caches when possible. + hwloc-bind - Fix failure on Windows when not using --pid. - Add -e as a synonym to --get-last-cpu-location. + hwloc-distrib - Add --reverse to distribute using last objects first and singlify into last bits first. Thanks to Jirka Hladky for the suggestion. + hwloc-info - Report unified caches when looking for data or instruction cache ancestor objects. * Misc + Add experimental Visual Studio support under contrib/windows. Thanks to Eloi Gaudry for his help and for providing the first draft. + Fix some overzealous assertions and warnings about the ordering of objects on a level with respect to cpusets. The ordering is only guaranteed for complete cpusets (based on the first bit in sets). + Fix some memory leaks when importing xml diffs and when exporting a "too complex" entry. Version 1.8.1 ------------- * Fix the cpuid code on Windows 64bits so that the x86 backend gets enabled as expected and can populate CPU information. Thanks to Robin Scher for reporting the problem. * Add CPUVendor/CPUModelNumber/CPUFamilyNumber attributes when running on x86 architecture. Thanks to Ralph Castain for the suggestion. * Work around buggy BIOS reporting duplicate NUMA nodes on Linux. Thanks to Jeff Becker for reporting the problem and testing the patch. * Add a name to the lstopo graphical window. Thanks to Michael Prokop for reporting the issue. Version 1.8.0 ------------- * New components + Add the "linuxpci" component that always works on Linux even when libpciaccess and libpci aren't available (and even with a modified file-system root). By default the old "pci" component runs first because "linuxpci" lacks device names (obj->name is always NULL). * API + Add the topology difference API in hwloc/diff.h for manipulating many similar topologies. + Add hwloc_topology_dup() for duplicating an entire topology. + hwloc.h and hwloc/helper.h have been reorganized to clarify the documentation sections. The actual inline code has moved out of hwloc.h into the new hwloc/inlines.h. + Deprecated functions are now in hwloc/deprecated.h, and not in the official documentation anymore. * Tools + Add hwloc-diff and hwloc-patch tools together with the new diff API. + Add hwloc-compress-dir to (de)compress an entire directory of XML files using hwloc-diff and hwloc-patch. + Object colors in the graphical output of lstopo may be changed by adding a "lstopoStyle" info attribute. See CUSTOM COLORS in the lstopo(1) manpage for details. Thanks to Jirka Hladky for discussing the idea. + hwloc-gather-topology may now gather I/O-related files on Linux when --io is given. Only the linuxpci component supports discovering I/O objects from these extended tarballs. + hwloc-annotate now supports --ri to remove/replace info attributes with a given name. + hwloc-info supports "root" and "all" special locations for dumping information about the root object. + lstopo now supports --append-legend to append custom lines of text to the legend in the graphical output. Thanks to Jirka Hladky for discussing the idea. + hwloc-calc and friends have a more robust parsing of locations given on the command-line and they report useful error messages about it. + Add --whole-system to hwloc-bind, hwloc-calc, hwloc-distances and hwloc-distrib, and add --restrict to hwloc-bind for uniformity among tools. * Misc + Calling hwloc_topology_load() or hwloc_topology_set_*() on an already loaded topology now returns an error (deprecated since release 1.6.1). + Fix the initialisation of cpusets and nodesets in Group objects added when inserting PCI hostbridges. + Never merge Group objects that were added explicitly by the user with hwloc_custom_insert_group_object_by_parent(). + Add a sanity check during dynamic plugin loading to prevent some crashes when hwloc is dynamically loaded by another plugin mechanisms. + Add --with-hwloc-plugins-path to specify the install/load directories of plugins. + Add the MICSerialNumber info attribute to the root object when running hwloc inside a Xeon Phi to match the same attribute in the MIC OS device when running in the host. Version 1.7.2 ------------- * Do not create invalid block OS devices on very old Linux kernel such as RHEL4 2.6.9. * Fix PCI subvendor/device IDs. * Fix the management of Misc objects inserted by parent. Thanks to Jirka Hladky for reporting the problem. * Add a Port<n>State into attribute to OpenFabrics OS devices. * Add a MICSerialNumber info attribute to Xeon PHI/MIC OS devices. * Improve verbose error messages when failing to load from XML.
2015-06-29Deal with linux 4.xmarkd2-1/+17
2015-06-12Recursive PKGREVISION bump for all packages mentioning 'perl',wiz8-16/+16
having a PKGNAME of p5-*, or depending such a package, for perl-5.22.0.
2015-04-25Recursive revbump following MesaLib update, categories p through x.tnn1-2/+2
2015-03-19config.guess / config.sub are found in very deep subdirectories, adjust.joerg1-1/+3