rsync.git
4 years agoSearch for double-fuzzy files only when needed
benrubson [Tue, 26 May 2020 14:06:07 +0000 (16:06 +0200)]
Search for double-fuzzy files only when needed

4 years agoSilence a strncpy() warning.
Wayne Davison [Sat, 30 May 2020 00:35:56 +0000 (17:35 -0700)]
Silence a strncpy() warning.

4 years agoSilence some g++ warnings.
Wayne Davison [Fri, 29 May 2020 21:38:19 +0000 (14:38 -0700)]
Silence some g++ warnings.

4 years agoGive configure's snprintf() test a guaranteed short string at the start.
Wayne Davison [Fri, 29 May 2020 21:37:49 +0000 (14:37 -0700)]
Give configure's snprintf() test a guaranteed short string at the start.

4 years agoSilence gcc7.1 warnings about snprintf().
Wayne Davison [Fri, 29 May 2020 21:18:08 +0000 (14:18 -0700)]
Silence gcc7.1 warnings about snprintf().

4 years agoFix regex issue due to python 3.8 bug.
Wayne Davison [Fri, 29 May 2020 03:48:24 +0000 (20:48 -0700)]
Fix regex issue due to python 3.8 bug.

4 years agoA few minor xxhash changes.
Wayne Davison [Fri, 29 May 2020 03:21:54 +0000 (20:21 -0700)]
A few minor xxhash changes.

4 years agoMerge pull request #5 from benrubson/daemonstats
Wayne Davison [Thu, 28 May 2020 20:42:47 +0000 (13:42 -0700)]
Merge pull request #5 from benrubson/daemonstats

Have daemon log data sent/received even when exiting with an error.

4 years agoAdd stub for canonical_checksum().
Wayne Davison [Thu, 28 May 2020 19:46:46 +0000 (12:46 -0700)]
Add stub for canonical_checksum().

4 years agoThe xxh* checksums don't need to be reversed on output.
Wayne Davison [Thu, 28 May 2020 19:05:54 +0000 (12:05 -0700)]
The xxh* checksums don't need to be reversed on output.

4 years agoSwitch to using LZ4_compress_default().
Wayne Davison [Thu, 28 May 2020 18:40:52 +0000 (11:40 -0700)]
Switch to using LZ4_compress_default().

4 years agoUse MSG_FLUSH in a couple more spots.
Wayne Davison [Thu, 28 May 2020 07:41:39 +0000 (00:41 -0700)]
Use MSG_FLUSH in a couple more spots.

4 years agoMerge pull request #2 from benrubson/flush
Wayne Davison [Thu, 28 May 2020 06:35:25 +0000 (23:35 -0700)]
Merge pull request #2 from benrubson/flush

Help final error messages get to the sender. Is particularly helpful when talking with an older rsync client. Adds a new flush type of MSG_FLUSH.

4 years agoA couple minor tweaks.
Wayne Davison [Wed, 27 May 2020 05:28:22 +0000 (22:28 -0700)]
A couple minor tweaks.

4 years agoLog data sent/received even if error
benrubson [Tue, 26 May 2020 17:53:25 +0000 (19:53 +0200)]
Log data sent/received even if error

4 years agoSimplify the capabilities array.
Wayne Davison [Tue, 26 May 2020 14:38:45 +0000 (07:38 -0700)]
Simplify the capabilities array.

4 years agoCorrectly send last error to sender
benrubson [Tue, 26 May 2020 14:24:30 +0000 (16:24 +0200)]
Correctly send last error to sender

4 years agoConvert NEWS & OLDNEWS into .md files.
Wayne Davison [Tue, 26 May 2020 09:14:05 +0000 (02:14 -0700)]
Convert NEWS & OLDNEWS into .md files.

4 years agoFix the parsing of the --version capabilities.
Wayne Davison [Tue, 26 May 2020 06:42:33 +0000 (23:42 -0700)]
Fix the parsing of the --version capabilities.

4 years agoInstall yodl for the CI builds.
Wayne Davison [Tue, 26 May 2020 06:41:39 +0000 (23:41 -0700)]
Install yodl for the CI builds.

4 years agoGet indent right.
Wayne Davison [Tue, 26 May 2020 06:33:11 +0000 (23:33 -0700)]
Get indent right.

4 years agoImprove output of capabilities in --version list.
Wayne Davison [Tue, 26 May 2020 05:43:33 +0000 (22:43 -0700)]
Improve output of capabilities in --version list.

It now wraps better as the "no " prefixes change, and it makes it easier
to maintain patches that add items into the capabilities list.

4 years agoRemove xxhash from capabilities list.
Wayne Davison [Tue, 26 May 2020 04:27:21 +0000 (21:27 -0700)]
Remove xxhash from capabilities list.

It's now listed in the "Checksum list:" output.

4 years agoFix a couple deb names.
Wayne Davison [Tue, 26 May 2020 04:21:58 +0000 (21:21 -0700)]
Fix a couple deb names.

4 years agoImprove CI setup.
Wayne Davison [Tue, 26 May 2020 04:17:51 +0000 (21:17 -0700)]
Improve CI setup.

4 years agoImprove how negotiated info affects batch files.
Wayne Davison [Tue, 26 May 2020 01:03:02 +0000 (18:03 -0700)]
Improve how negotiated info affects batch files.

4 years agoTweak the --zz option to --zc (aka --compress-choice).
Wayne Davison [Mon, 25 May 2020 23:57:47 +0000 (16:57 -0700)]
Tweak the --zz option to --zc (aka --compress-choice).

4 years agoRename a couple files to .md
Wayne Davison [Mon, 25 May 2020 22:00:30 +0000 (15:00 -0700)]
Rename a couple files to .md

4 years agoAdd some extra indent.
Wayne Davison [Mon, 25 May 2020 21:59:51 +0000 (14:59 -0700)]
Add some extra indent.

4 years agoCreate ccpp.yml
Wayne Davison [Mon, 25 May 2020 21:43:25 +0000 (14:43 -0700)]
Create ccpp.yml

4 years agoTwo sparse fixes from Yuxuan Shui.
Wayne Davison [Mon, 25 May 2020 21:01:52 +0000 (14:01 -0700)]
Two sparse fixes from Yuxuan Shui.

- Make "len" parameter of do_punch_hole an OFF_T.
- Clear sparse_past_write in sparse_end(), otherwise when write_sparse()
  is called for the next file, do_punch_hole() will be called with a pos
  that's not actually the current position in file, causing it to fail.

4 years agoInclude lz4 compression support.
Wayne Davison [Mon, 25 May 2020 20:43:25 +0000 (13:43 -0700)]
Include lz4 compression support.

Based on a patch that was emailed to me without a valid return address.

4 years agoInclude zstd compression support.
Wayne Davison [Mon, 25 May 2020 20:31:30 +0000 (13:31 -0700)]
Include zstd compression support.

Based on a patch by Sebastian A. Siewior. Fixes bug #14338.

4 years agoFix handling of a compressor that has no off_level.
Wayne Davison [Mon, 25 May 2020 20:02:56 +0000 (13:02 -0700)]
Fix handling of a compressor that has no off_level.

4 years agoOutput the default checksum & compress lists in the --version output.
Wayne Davison [Mon, 25 May 2020 19:45:28 +0000 (12:45 -0700)]
Output the default checksum & compress lists in the --version output.

4 years agoA couple minor configure.ac tweaks.
Wayne Davison [Mon, 25 May 2020 18:50:44 +0000 (11:50 -0700)]
A couple minor configure.ac tweaks.

4 years agoMake compression-level handling generic.
Wayne Davison [Mon, 25 May 2020 17:47:57 +0000 (10:47 -0700)]
Make compression-level handling generic.

4 years agoSome minor tweaks & tidying up.
Wayne Davison [Mon, 25 May 2020 05:31:28 +0000 (22:31 -0700)]
Some minor tweaks & tidying up.

4 years agoChange odd-ball map_ptr() to use remainder like the others.
Wayne Davison [Mon, 25 May 2020 03:38:48 +0000 (20:38 -0700)]
Change odd-ball map_ptr() to use remainder like the others.

4 years agoImprove performance of file_checksum()
Jorrit Jongma [Sun, 24 May 2020 22:30:55 +0000 (00:30 +0200)]
Improve performance of file_checksum()

Previously files were hashed in blocks of CSUM_CHUNK (64) bytes. This
causes significant overhead. The CSUM_CHUNK define cannot be changed as
md5.c depends on it, but there is no obvious reason to use it in
file_checksum(). By using CHUNK_SIZE (32 kB) instead, in some test
cases throughput more than doubles.

4 years agoMore NEWS updates.
Wayne Davison [Mon, 25 May 2020 03:19:15 +0000 (20:19 -0700)]
More NEWS updates.

4 years agoPrefer zlibx compression consistently instead of having 2 possible default preference...
Wayne Davison [Mon, 25 May 2020 02:51:11 +0000 (19:51 -0700)]
Prefer zlibx compression consistently instead of having 2 possible default preference orders.

4 years agoDon't auto-foward debug options to the server side anymore.
Wayne Davison [Mon, 25 May 2020 02:35:22 +0000 (19:35 -0700)]
Don't auto-foward debug options to the server side anymore.

4 years agoFix -z choice with older rsyncs.
Wayne Davison [Mon, 25 May 2020 02:16:05 +0000 (19:16 -0700)]
Fix -z choice with older rsyncs.

4 years agoA few more compression tweaks.
Wayne Davison [Mon, 25 May 2020 01:08:09 +0000 (18:08 -0700)]
A few more compression tweaks.

4 years agoMore compress changes
Wayne Davison [Sun, 24 May 2020 22:45:59 +0000 (15:45 -0700)]
More compress changes

- Add the zlibx (external-code compatible) compression name.
- Re-enable zlib support with the external library so it can be
  tried as a fallback if zlibx isn't available.
- Add --compress-choice=STR (aka -zz=STR) option.
- Make --cc=STR an alias for --checksum-choice=STR.
- Hook up the new compression negotiation logic.

4 years agoFix negotiation of none & improve NSTR debug msgs.
Wayne Davison [Sun, 24 May 2020 20:34:47 +0000 (13:34 -0700)]
Fix negotiation of none & improve NSTR debug msgs.

4 years agoAdd support for negotiated checksum names.
Wayne Davison [Sun, 24 May 2020 19:56:00 +0000 (12:56 -0700)]
Add support for negotiated checksum names.

4 years agoMake xxh64 the "main_name" for the current xxhash.
Wayne Davison [Sun, 24 May 2020 09:07:43 +0000 (02:07 -0700)]
Make xxh64 the "main_name" for the current xxhash.

4 years agoFix some warnings.
Wayne Davison [Sun, 24 May 2020 09:04:14 +0000 (02:04 -0700)]
Fix some warnings.

4 years agoSome more checksum improvements
Wayne Davison [Sun, 24 May 2020 05:05:16 +0000 (22:05 -0700)]
Some more checksum improvements

- Add/improve --debug=CSUM2 messages.
- Add an "xxh64" alias for "xxhash" name because we should be
  getting a few more xxhash variations in the future.
- Tweak the matching code to handle entries that have multiple
  names.
- Tweak some of the vars/defines.

4 years agoImprove some early debug-message newlines.
Wayne Davison [Sun, 24 May 2020 04:46:15 +0000 (21:46 -0700)]
Improve some early debug-message newlines.

Avoid a newline issue during the output of --DEBUG=CSUM info from
both the server and the client -- we need to output the full message
with its newline as much as possible.

4 years agoAvoid re-evaluating the args of SIVAL* w/CAREFUL_ALIGNMENT.
Wayne Davison [Sun, 24 May 2020 04:43:53 +0000 (21:43 -0700)]
Avoid re-evaluating the args of SIVAL* w/CAREFUL_ALIGNMENT.

4 years agoUnify the checksum context memory, since we only use one at a time.
Wayne Davison [Sun, 24 May 2020 01:25:11 +0000 (18:25 -0700)]
Unify the checksum context memory, since we only use one at a time.

4 years agoFixed the use of openssl MD4 for transfer checksums.
Wayne Davison [Sat, 23 May 2020 23:22:36 +0000 (16:22 -0700)]
Fixed the use of openssl MD4 for transfer checksums.

4 years agoOptional openssl support for MD4 pre-transfer checksums (but, sadly, not transfer...
Wayne Davison [Sat, 23 May 2020 19:26:06 +0000 (12:26 -0700)]
Optional openssl support for MD4 pre-transfer checksums (but, sadly, not transfer checksums).

4 years agoAdd optional use of the openssl crypto lib for MD5.
Wayne Davison [Sat, 23 May 2020 16:23:01 +0000 (09:23 -0700)]
Add optional use of the openssl crypto lib for MD5.

4 years agoMention a few more news items.
Wayne Davison [Sat, 23 May 2020 06:26:25 +0000 (23:26 -0700)]
Mention a few more news items.

4 years agoAdd "input" handling for cmd_txt_*() pkglib.py.
Wayne Davison [Sat, 23 May 2020 05:40:54 +0000 (22:40 -0700)]
Add "input" handling for cmd_txt_*() pkglib.py.

4 years agoOptimized assembler version of md5_process() for x86-64
Jorrit Jongma [Fri, 22 May 2020 17:38:37 +0000 (19:38 +0200)]
Optimized assembler version of md5_process() for x86-64

Originally created by Marc Bevand and placed in the public domain.
Enable/disabled via the same --enable-simd configure switch as
the rolling checksum optimizations.

4 years agoA little more safety in negotiate_checksum().
Wayne Davison [Sat, 23 May 2020 02:26:30 +0000 (19:26 -0700)]
A little more safety in negotiate_checksum().

4 years agoPromoting xxhash support.
Wayne Davison [Sat, 23 May 2020 00:40:08 +0000 (17:40 -0700)]
Promoting xxhash support.

4 years agoSome checksum improvements
Wayne Davison [Fri, 22 May 2020 22:06:20 +0000 (15:06 -0700)]
Some checksum improvements

- Improve csum negotation logic.
- Define the csum names in a single structure.
- Add --debug=CSUM.

4 years agoAVX2 optimized version of get_checksum1() for x86-64
Jorrit Jongma [Fri, 22 May 2020 11:03:55 +0000 (13:03 +0200)]
AVX2 optimized version of get_checksum1() for x86-64

Additionally restructures build switches and defines from SSE2 to SIMD,
to allow potential reuse should patches become available with SIMD
instructions for other processor architectures.

(Some minor tweaks of Jorrit's patch to avoid requiring GNU make and to
avoid C++ comments in .c files.)

4 years agoChecksum negotiation & more bits for compat_flags
Wayne Davison [Fri, 22 May 2020 15:23:26 +0000 (08:23 -0700)]
Checksum negotiation & more bits for compat_flags

- Add checksum negotiation to the protocol so that we can easily add new
  checksum algorithms and each will be used when both sides support it.
- Increase the size of the compat_flags value in the protocol from a
  byte to an int.

4 years agoFix a bug in the writing of the batch.sh file
Wayne Davison [Fri, 22 May 2020 15:10:41 +0000 (08:10 -0700)]
Fix a bug in the writing of the batch.sh file

Fix the code that writes the options and the default destination path
into the batch.sh file to be able to handle options being specified
after source/dest args.

4 years agoSSE2/SSSE3 optimized version of get_checksum1() for x86-64
Jorrit Jongma [Tue, 19 May 2020 12:52:40 +0000 (14:52 +0200)]
SSE2/SSSE3 optimized version of get_checksum1() for x86-64

Requires compilation using GCC C++ front end, build scripts have been
modified accordingly. C++ is only used when the optimization is enabled
(g++ as compiler, x86-64 build target, --enable-sse2 is passed to
configure).

(Wayne made a few tweaks, including making it disabled by default.)

4 years agoTweak the accept/refuse strings a bit.
Wayne Davison [Mon, 18 May 2020 07:03:09 +0000 (00:03 -0700)]
Tweak the accept/refuse strings a bit.

4 years agoAdd back a lost "*" and document the refusing of log-file* opts.
Wayne Davison [Mon, 18 May 2020 06:26:13 +0000 (23:26 -0700)]
Add back a lost "*" and document the refusing of log-file* opts.

4 years agoMore manpage improvements for "refuse options".
Wayne Davison [Mon, 18 May 2020 05:16:58 +0000 (22:16 -0700)]
More manpage improvements for "refuse options".

4 years agoAdd the ability to negate matches for the daemon's "refuse options".
Wayne Davison [Mon, 18 May 2020 04:29:11 +0000 (21:29 -0700)]
Add the ability to negate matches for the daemon's "refuse options".

4 years agoNeed to flush early errors before we exit.
Wayne Davison [Mon, 18 May 2020 03:21:45 +0000 (20:21 -0700)]
Need to flush early errors before we exit.

4 years agoSome pkglib improvements.
Wayne Davison [Sun, 17 May 2020 21:39:18 +0000 (14:39 -0700)]
Some pkglib improvements.

4 years agoA couple more manpage fixes.
Wayne Davison [Wed, 13 May 2020 07:20:03 +0000 (00:20 -0700)]
A couple more manpage fixes.

4 years agoA bit more manpage tweaking.
Wayne Davison [Wed, 13 May 2020 07:11:57 +0000 (00:11 -0700)]
A bit more manpage tweaking.

4 years agoChange r'\1%s\2' to r'\g<1>%s\2'.
Wayne Davison [Thu, 7 May 2020 00:23:17 +0000 (17:23 -0700)]
Change r'\1%s\2' to r'\g<1>%s\2'.

4 years agoA couple extra function checks for future features.
Wayne Davison [Thu, 30 Apr 2020 05:14:49 +0000 (22:14 -0700)]
A couple extra function checks for future features.

4 years agoConvert another packaging script to python3.
Wayne Davison [Thu, 30 Apr 2020 04:24:26 +0000 (21:24 -0700)]
Convert another packaging script to python3.

4 years agoA few superficial tweaks.
Wayne Davison [Thu, 30 Apr 2020 02:28:23 +0000 (19:28 -0700)]
A few superficial tweaks.

4 years agoUse a varint to send the file-list flags
Wayne Davison [Thu, 30 Apr 2020 00:55:41 +0000 (17:55 -0700)]
Use a varint to send the file-list flags

If both sides support the "V" compatibility flag, we send the file-list
flags as a varint instead of a 1-or-2 byte value.  This upgrades the
number of reserved flag bits from 1 to 17 with very few extra bytes in
typical file-list data.

4 years agoA resumed partial-dir file is transferred in-place.
Wayne Davison [Wed, 29 Apr 2020 23:51:02 +0000 (16:51 -0700)]
A resumed partial-dir file is transferred in-place.
Fixed bug #13071.

4 years agoFix a couple issues with the atime file-list value.
Wayne Davison [Mon, 27 Apr 2020 01:39:15 +0000 (18:39 -0700)]
Fix a couple issues with the atime file-list value.

4 years agoAllow a missing parent dir when --delete-missing-args was specified.
Wayne Davison [Mon, 27 Apr 2020 01:02:17 +0000 (18:02 -0700)]
Allow a missing parent dir when --delete-missing-args was specified.

4 years agoAdd extensions to the default no-compress list.
Wayne Davison [Mon, 27 Apr 2020 00:05:02 +0000 (17:05 -0700)]
Add extensions to the default no-compress list.
Fixes bug #13749.

4 years agoAdd a few new opts to rrsync.
Wayne Davison [Sun, 26 Apr 2020 21:50:40 +0000 (14:50 -0700)]
Add a few new opts to rrsync.

4 years agoSome var cleanup; move test-util vars into t_stub.c.
Wayne Davison [Sun, 26 Apr 2020 21:05:27 +0000 (14:05 -0700)]
Some var cleanup; move test-util vars into t_stub.c.

4 years agoUpdate a few more copyright years.
Wayne Davison [Sun, 26 Apr 2020 06:34:16 +0000 (23:34 -0700)]
Update a few more copyright years.

4 years agoChange pending version to 3.2.0 (currently 3.2.0dev).
Wayne Davison [Sun, 26 Apr 2020 06:30:42 +0000 (23:30 -0700)]
Change pending version to 3.2.0 (currently 3.2.0dev).

4 years agoChange do_setattrlist_times() to use an stp arg.
Wayne Davison [Sun, 26 Apr 2020 04:39:11 +0000 (21:39 -0700)]
Change do_setattrlist_times() to use an stp arg.

4 years agoChange --set-notime to --open-noatime.
Wayne Davison [Thu, 23 Apr 2020 21:27:44 +0000 (14:27 -0700)]
Change --set-notime to --open-noatime.

4 years agoAdded --atimes and --set-noatime options.
Wayne Davison [Thu, 23 Apr 2020 20:17:41 +0000 (13:17 -0700)]
Added --atimes and --set-noatime options.

4 years agoMention more changes in the NEWS.
Wayne Davison [Thu, 23 Apr 2020 20:13:35 +0000 (13:13 -0700)]
Mention more changes in the NEWS.

4 years agoEliminate .in for rsync-ssl.
Wayne Davison [Wed, 22 Apr 2020 21:53:06 +0000 (14:53 -0700)]
Eliminate .in for rsync-ssl.

4 years agoTurn nightly-rsync into a python script.
Wayne Davison [Wed, 22 Apr 2020 19:00:08 +0000 (12:00 -0700)]
Turn nightly-rsync into a python script.

4 years agoSome more rsync-ssl improvements:
Wayne Davison [Wed, 22 Apr 2020 17:53:38 +0000 (10:53 -0700)]
Some more rsync-ssl improvements:

- Make the rsync-ssl default behavior more user friendly.
- Install rsync-ssl & rsync-ssl-rsh in the regular install rules.
- Add a manpage for rsync-ssl (which is also installed).
- Get rid of the rsync-ssl-client package in our spec file.

4 years agoChange the name of the rsh-ssl-rsync script.
Wayne Davison [Wed, 22 Apr 2020 17:52:01 +0000 (10:52 -0700)]
Change the name of the rsh-ssl-rsync script.

4 years agoChange the rsync-ssl helper script
Wayne Davison [Sun, 19 Apr 2020 19:08:01 +0000 (12:08 -0700)]
Change the rsync-ssl helper script

The new rsh-ssl-rsync helper script (replacing stunnel-rsync) supports
openssl in addition to stunnel.  The RSYNC_SSL_TYPE environment variable
can be set to specify which type of connection to use, and the first arg
to rsync-ssl can be --type=stunnel or --type=openssl to override the env
var or the default of "stunnel".  The helper script now looks for
stunnel4 or stunnel on the PATH at runtime instead of having configure
look for it at compile time.

4 years agoTwo more spelling fixes and some year updates.
Wayne Davison [Thu, 16 Apr 2020 16:29:22 +0000 (09:29 -0700)]
Two more spelling fixes and some year updates.

4 years agoSpelling fixes from a Fossies run done by Jens.
Wayne Davison [Thu, 16 Apr 2020 00:42:23 +0000 (17:42 -0700)]
Spelling fixes from a Fossies run done by Jens.

4 years agoA few file-data improvements.
Wayne Davison [Sun, 12 Apr 2020 22:45:22 +0000 (15:45 -0700)]
A few file-data improvements.