From e56285e0e6d4dcc3961dea2405b5ef0a321b679e Mon Sep 17 00:00:00 2001 From: wiz Date: Fri, 19 Aug 2016 22:34:14 +0000 Subject: Updated py-borgbackup to 1.0.7. Important note about pre-1.0.4 potential repo corruption -------------------------------------------------------- Some external errors (like network or disk I/O errors) could lead to corruption of the backup repository due to issue #1138. A sign that this happened is if "E" status was reported for a file that can not be explained by problems with the source file. If you still have logs from "borg create -v --list", you can check for "E" status. Here is what could cause corruption and what you can do now: 1) I/O errors (e.g. repo disk errors) while writing data to repo. This could lead to corrupted segment files. Fix:: # check for corrupt chunks / segments: borg check -v --repository-only REPO # repair the repo: borg check -v --repository-only --repair REPO # make sure everything is fixed: borg check -v --repository-only REPO 2) Unreliable network / unreliable connection to the repo. This could lead to archive metadata corruption. Fix:: # check for corrupt archives: borg check -v --archives-only REPO # delete the corrupt archives: borg delete --force REPO::CORRUPT_ARCHIVE # make sure everything is fixed: borg check -v --archives-only REPO 3) In case you want to do more intensive checking. The best check that everything is ok is to run a dry-run extraction:: borg extract -v --dry-run REPO::ARCHIVE Version 1.0.7 (2016-08-19) -------------------------- Security fixes: - borg serve: fix security issue with remote repository access, #1428 If you used e.g. --restrict-to-path /path/client1/ (with or without trailing slash does not make a difference), it acted like a path prefix match using /path/client1 (note the missing trailing slash) - the code then also allowed working in e.g. /path/client13 or /path/client1000. As this could accidentally lead to major security/privacy issues depending on the pathes you use, the behaviour was changed to be a strict directory match. That means --restrict-to-path /path/client1 (with or without trailing slash does not make a difference) now uses /path/client1/ internally (note the trailing slash here!) for matching and allows precisely that path AND any path below it. So, /path/client1 is allowed, /path/client1/repo1 is allowed, but not /path/client13 or /path/client1000. If you willingly used the undocumented (dangerous) previous behaviour, you may need to rearrange your --restrict-to-path pathes now. We are sorry if that causes work for you, but we did not want a potentially dangerous behaviour in the software (not even using a for-backwards-compat option). Bug fixes: - fixed repeated LockTimeout exceptions when borg serve tried to write into a already write-locked repo (e.g. by a borg mount), #502 part b) This was solved by the fix for #1220 in 1.0.7rc1 already. - fix cosmetics + file leftover for "not a valid borg repository", #1490 - Cache: release lock if cache is invalid, #1501 - borg extract --strip-components: fix leak of preloaded chunk contents - Repository, when a InvalidRepository exception happens: - fix spurious, empty lock.roster - fix repo not closed cleanly New features: - implement borg debug-info, fixes #1122 (just calls already existing code via cli, same output as below tracebacks) Other changes: - skip the O_NOATIME test on GNU Hurd, fixes #1315 (this is a very minor issue and the GNU Hurd project knows the bug) - document using a clean repo to test / build the release Version 1.0.7rc2 (2016-08-13) ----------------------------- Bug fixes: - do not write objects to repository that are bigger than the allowed size, borg will reject reading them, #1451. Important: if you created archives with many millions of files or directories, please verify if you can open them successfully, e.g. try a "borg list REPO::ARCHIVE". - lz4 compression: dynamically enlarge the (de)compression buffer, the static buffer was not big enough for archives with extremely many items, #1453 - larger item metadata stream chunks, raise archive item limit by 8x, #1452 - fix untracked segments made by moved DELETEs, #1442 Impact: Previously (metadata) segments could become untracked when deleting data, these would never be cleaned up. - extended attributes (xattrs) related fixes: - fixed a race condition in xattrs querying that led to the entire file not being backed up (while logging the error, exit code = 1), #1469 - fixed a race condition in xattrs querying that led to a crash, #1462 - raise OSError including the error message derived from errno, deal with path being a integer FD Other changes: - print active env var override by default, #1467 - xattr module: refactor code, deduplicate, clean up - repository: split object size check into too small and too big - add a transaction_id assertion, so borg init on a broken (inconsistent) filesystem does not look like a coding error in borg, but points to the real problem. - explain confusing TypeError caused by compat support for old servers, #1456 - add forgotten usage help file from build_usage - refactor/unify buffer code into helpers.Buffer class, add tests - docs: - document archive limitation, #1452 - improve prune examples Version 1.0.7rc1 (2016-08-05) ----------------------------- Bug fixes: - fix repo lock deadlocks (related to lock upgrade), #1220 - catch unpacker exceptions, resync, #1351 - fix borg break-lock ignoring BORG_REPO env var, #1324 - files cache performance fixes (fixes unneccessary re-reading/chunking/ hashing of unmodified files for some use cases): - fix unintended file cache eviction, #1430 - implement BORG_FILES_CACHE_TTL, update FAQ, raise default TTL from 10 to 20, #1338 - FUSE: - cache partially read data chunks (performance), #965, #966 - always create a root dir, #1125 - use an OrderedDict for helptext, making the build reproducible, #1346 - RemoteRepository init: always call close on exceptions, #1370 (cosmetic) - ignore stdout/stderr broken pipe errors (cosmetic), #1116 New features: - better borg versions management support (useful esp. for borg servers wanting to offer multiple borg versions and for clients wanting to choose a specific server borg version), #1392: - add BORG_VERSION environment variable before executing "borg serve" via ssh - add new placeholder {borgversion} - substitute placeholders in --remote-path - borg init --append-only option (makes using the more secure append-only mode more convenient. when used remotely, this requires 1.0.7+ also on the borg server), #1291. Other changes: - Vagrantfile: - darwin64: upgrade to FUSE for macOS 3.4.1 (aka osxfuse), #1378 - xenial64: use user "ubuntu", not "vagrant" (as usual), #1331 - tests: - fix fuse tests on OS X, #1433 - docs: - FAQ: add backup using stable filesystem names recommendation - FAQ about glibc compatibility added, #491, glibc-check improved - FAQ: 'A' unchanged file; remove ambiguous entry age sentence. - OS X: install pkg-config to build with FUSE support, fixes #1400 - add notes about shell/sudo pitfalls with env. vars, #1380 - added platform feature matrix - implement borg debug-dump-repo-objs Version 1.0.6 (2016-07-12) -------------------------- Bug fixes: - Linux: handle multiple LD_PRELOAD entries correctly, #1314, #1111 - Fix crash with unclear message if the libc is not found, #1314, #1111 Other changes: - tests: - Fixed O_NOATIME tests for Solaris and GNU Hurd, #1315 - Fixed sparse file tests for (file) systems not supporting it, #1310 - docs: - Fixed syntax highlighting, #1313 - misc docs: added data processing overview picture Version 1.0.6rc1 (2016-07-10) ----------------------------- New features: - borg check --repair: heal damaged files if missing chunks re-appear (e.g. if the previously missing chunk was added again in a later backup archive), #148. (*) Also improved logging. Bug fixes: - sync_dir: silence fsync() failing with EINVAL, #1287 Some network filesystems (like smbfs) don't support this and we use this in repository code. - borg mount (FUSE): - fix directories being shadowed when contained paths were also specified, #1295 - raise I/O Error (EIO) on damaged files (unless -o allow_damaged_files is used), #1302. (*) - borg extract: warn if a damaged file is extracted, #1299. (*) - Added some missing return code checks (ChunkIndex._add, hashindex_resize). - borg check: fix/optimize initial hash table size, avoids resize of the table. Other changes: - tests: - add more FUSE tests, #1284 - deduplicate fuse (u)mount code - fix borg binary test issues, #862 - docs: - changelog: added release dates to older borg releases - fix some sphinx (docs generator) warnings, #881 Notes: (*) Some features depend on information (chunks_healthy list) added to item metadata when a file with missing chunks was "repaired" using all-zero replacement chunks. The chunks_healthy list is generated since borg 1.0.4, thus borg can't recognize such "repaired" (but content-damaged) files if the repair was done with an older borg version. Version 1.0.5 (2016-07-07) -------------------------- Bug fixes: - borg mount: fix FUSE crash in xattr code on Linux introduced in 1.0.4, #1282 Other changes: - backport some FAQ entries from master branch - add release helper scripts - Vagrantfile: - centos6: no FUSE, don't build binary - add xz for redhat-like dists Version 1.0.4 (2016-07-07) -------------------------- New features: - borg serve --append-only, #1168 This was included because it was a simple change (append-only functionality was already present via repository config file) and makes better security now practically usable. - BORG_REMOTE_PATH environment variable, #1258 This was included because it was a simple change (--remote-path cli option was already present) and makes borg much easier to use if you need it. - Repository: cleanup incomplete transaction on "no space left" condition. In many cases, this can avoid a 100% full repo filesystem (which is very problematic as borg always needs free space - even to delete archives). Bug fixes: - Fix wrong handling and reporting of OSErrors in borg create, #1138. This was a serious issue: in the context of "borg create", errors like repository I/O errors (e.g. disk I/O errors, ssh repo connection errors) were handled badly and did not lead to a crash (which would be good for this case, because the repo transaction would be incomplete and trigger a transaction rollback to clean up). Now, error handling for source files is cleanly separated from every other error handling, so only problematic input files are logged and skipped. - Implement fail-safe error handling for borg extract. Note that this isn't nearly as critical as the borg create error handling bug, since nothing is written to the repo. So this was "merely" misleading error reporting. - Add missing error handler in directory attr restore loop. - repo: make sure write data hits disk before the commit tag (#1236) and also sync the containing directory. - FUSE: getxattr fail must use errno.ENOATTR, #1126 (fixes Mac OS X Finder malfunction: "zero bytes" file length, access denied) - borg check --repair: do not lose information about the good/original chunks. If we do not lose the original chunk IDs list when "repairing" a file (replacing missing chunks with all-zero chunks), we have a chance to "heal" the file back into its original state later, in case the chunks re-appear (e.g. in a fresh backup). Healing is not implemented yet, see #148. - fixes for --read-special mode: - ignore known files cache, #1241 - fake regular file mode, #1214 - improve symlinks handling, #1215 - remove passphrase from subprocess environment, #1105 - Ignore empty index file (will trigger index rebuild), #1195 - add missing placeholder support for --prefix, #1027 - improve exception handling for placeholder replacement - catch and format exceptions in arg parsing - helpers: fix "undefined name 'e'" in exception handler - better error handling for missing repo manifest, #1043 - borg delete: - make it possible to delete a repo without manifest - borg delete --forced allows to delete corrupted archives, #1139 - borg check: - make borg check work for empty repo - fix resync and msgpacked item qualifier, #1135 - rebuild_manifest: fix crash if 'name' or 'time' key were missing. - better validation of item metadata dicts, #1130 - better validation of archive metadata dicts - close the repo on exit - even if rollback did not work, #1197. This is rather cosmetic, it avoids repo closing in the destructor. - tests: - fix sparse file test, #1170 - flake8: ignore new F405, #1185 - catch "invalid argument" on cygwin, #257 - fix sparseness assertion in test prep, #1264 Other changes: - make borg build/work on OpenSSL 1.0 and 1.1, #1187 - docs / help: - fix / clarify prune help, #1143 - fix "patterns" help formatting - add missing docs / help about placeholders - resources: rename atticmatic to borgmatic - document sshd settings, #545 - more details about checkpoints, add split trick, #1171 - support docs: add freenode web chat link, #1175 - add prune visualization / example, #723 - add note that Fnmatch is default, #1247 - make clear that lzma levels > 6 are a waste of cpu cycles - add a "do not edit" note to auto-generated files, #1250 - update cygwin installation docs - repository interoperability with borg master (1.1dev) branch: - borg check: read item metadata keys from manifest, #1147 - read v2 hints files, #1235 - fix hints file "unknown version" error handling bug - tests: add tests for format_line - llfuse: update version requirement for freebsd - Vagrantfile: - use openbsd 5.9, #716 - do not install llfuse on netbsd (broken) - update OSXfuse to version 3.3.3 - use Python 3.5.2 to build the binaries - glibc compatibility checker: scripts/glibc_check.py - add .eggs to .gitignore Version 1.0.3 (2016-05-20) -------------------------- Bug fixes: - prune: avoid that checkpoints are kept and completed archives are deleted in a prune run), #997 - prune: fix commandline argument validation - some valid command lines were considered invalid (annoying, but harmless), #942 - fix capabilities extraction on Linux (set xattrs last, after chown()), #1069 - repository: fix commit tags being seen in data - when probing key files, do binary reads. avoids crash when non-borg binary files are located in borg's key files directory. - handle SIGTERM and make a clean exit - avoids orphan lock files. - repository cache: don't cache large objects (avoid using lots of temp. disk space), #1063 Other changes: - Vagrantfile: OS X: update osxfuse / install lzma package, #933 - setup.py: add check for platform_darwin.c - setup.py: on freebsd, use a llfuse release that builds ok - docs / help: - update readthedocs URLs, #991 - add missing docs for "borg break-lock", #992 - borg create help: add some words to about the archive name - borg create help: document format tags, #894 --- sysutils/py-borgbackup/Makefile | 4 ++-- sysutils/py-borgbackup/PLIST | 5 +---- sysutils/py-borgbackup/distinfo | 10 +++++----- 3 files changed, 8 insertions(+), 11 deletions(-) (limited to 'sysutils') diff --git a/sysutils/py-borgbackup/Makefile b/sysutils/py-borgbackup/Makefile index 767445a55c8..d9607fedc15 100644 --- a/sysutils/py-borgbackup/Makefile +++ b/sysutils/py-borgbackup/Makefile @@ -1,6 +1,6 @@ -# $NetBSD: Makefile,v 1.6 2016/07/09 13:04:07 wiz Exp $ +# $NetBSD: Makefile,v 1.7 2016/08/19 22:34:14 wiz Exp $ -DISTNAME= borgbackup-1.0.2 +DISTNAME= borgbackup-1.0.7 PKGNAME= ${PYPKGPREFIX}-${DISTNAME} CATEGORIES= sysutils MASTER_SITES= ${MASTER_SITE_PYPI:=b/borgbackup/} diff --git a/sysutils/py-borgbackup/PLIST b/sysutils/py-borgbackup/PLIST index c6be903d0a5..15f492f80a2 100644 --- a/sysutils/py-borgbackup/PLIST +++ b/sysutils/py-borgbackup/PLIST @@ -1,4 +1,4 @@ -@comment $NetBSD: PLIST,v 1.2 2016/04/15 13:31:29 wiz Exp $ +@comment $NetBSD: PLIST,v 1.3 2016/08/19 22:34:14 wiz Exp $ bin/borg-${PYVERSSUFFIX} ${PYSITELIB}/${EGG_INFODIR}/PKG-INFO ${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt @@ -30,9 +30,6 @@ ${PYSITELIB}/borg/crypto.so ${PYSITELIB}/borg/fuse.py ${PYSITELIB}/borg/fuse.pyc ${PYSITELIB}/borg/fuse.pyo -${PYSITELIB}/borg/hash_sizes.py -${PYSITELIB}/borg/hash_sizes.pyc -${PYSITELIB}/borg/hash_sizes.pyo ${PYSITELIB}/borg/hashindex.so ${PYSITELIB}/borg/helpers.py ${PYSITELIB}/borg/helpers.pyc diff --git a/sysutils/py-borgbackup/distinfo b/sysutils/py-borgbackup/distinfo index ae792da7243..e13187c4a6b 100644 --- a/sysutils/py-borgbackup/distinfo +++ b/sysutils/py-borgbackup/distinfo @@ -1,6 +1,6 @@ -$NetBSD: distinfo,v 1.3 2016/04/30 06:25:05 wiz Exp $ +$NetBSD: distinfo,v 1.4 2016/08/19 22:34:14 wiz Exp $ -SHA1 (borgbackup-1.0.2.tar.gz) = 1cda61522798e13c134494fdb032ac67e0a239cd -RMD160 (borgbackup-1.0.2.tar.gz) = a96435825742d1f81e7efefca591e495e0035333 -SHA512 (borgbackup-1.0.2.tar.gz) = 721f840b55a385976b7d01e03690cf0e73029f1566809fee5770622b3455aba6a35ca8b246127b5a8e07bc26e0cc31d6fd35a4e188906e5714d8dac912844189 -Size (borgbackup-1.0.2.tar.gz) = 422181 bytes +SHA1 (borgbackup-1.0.7.tar.gz) = 55d8eb92b57c22d274ab4847ae44c8d880d8b5c1 +RMD160 (borgbackup-1.0.7.tar.gz) = 95e6c45232097159983ffe76ae29cf5138826c4c +SHA512 (borgbackup-1.0.7.tar.gz) = 750e85597043049775a2480fb32df5a10e4ba581a91911e93eeb29dbc517db43f537b54206f8f2ce8fa51530866ed9e490c31e6c35d181853481fb3e6269d8af +Size (borgbackup-1.0.7.tar.gz) = 462509 bytes -- cgit v1.2.3