From ca8aad6c264a5bc1cd761b2dc780c7780576db05 Mon Sep 17 00:00:00 2001 From: Martin Pool Date: Tue, 26 Mar 2002 01:34:33 +0000 Subject: [PATCH] initial cvs checkin --- FAQ.html | 7 + GPL.html | 7 + README.html | 7 + about.html | 6 + backup.txt | 36 ++++ documentation.html | 28 +++ download.html | 49 +++++ examples.html | 49 +++++ faq.html | 0 faq.txt | 220 +++++++++++++++++++++ features.html | 23 +++ footer.html | 29 +++ header.html | 40 ++++ horus.txt | 28 +++ index.html | 21 ++ nt.html | 80 ++++++++ nt.txt | 74 +++++++ resources.html | 51 +++++ rsync_wrapper.pl | 471 +++++++++++++++++++++++++++++++++++++++++++++ susan.txt | 31 +++ vger.txt | 35 ++++ y2k.html | 19 ++ 22 files changed, 1311 insertions(+) create mode 100644 FAQ.html create mode 100644 GPL.html create mode 100644 README.html create mode 100644 about.html create mode 100644 backup.txt create mode 100644 documentation.html create mode 100644 download.html create mode 100644 examples.html create mode 100644 faq.html create mode 100644 faq.txt create mode 100644 features.html create mode 100644 footer.html create mode 100644 header.html create mode 100644 horus.txt create mode 100644 index.html create mode 100644 nt.html create mode 100644 nt.txt create mode 100644 resources.html create mode 100644 rsync_wrapper.pl create mode 100644 susan.txt create mode 100644 vger.txt create mode 100644 y2k.html diff --git a/FAQ.html b/FAQ.html new file mode 100644 index 0000000..454ef85 --- /dev/null +++ b/FAQ.html @@ -0,0 +1,7 @@ + + +

Frequently Asked Questions

+ + + + diff --git a/GPL.html b/GPL.html new file mode 100644 index 0000000..c22851f --- /dev/null +++ b/GPL.html @@ -0,0 +1,7 @@ + + +

+
+
+ + diff --git a/README.html b/README.html new file mode 100644 index 0000000..47905a0 --- /dev/null +++ b/README.html @@ -0,0 +1,7 @@ + + +

+
+
+ + diff --git a/about.html b/about.html new file mode 100644 index 0000000..4b5e877 --- /dev/null +++ b/about.html @@ -0,0 +1,6 @@ + + +

rsync resources

+ + + diff --git a/backup.txt b/backup.txt new file mode 100644 index 0000000..c09fb57 --- /dev/null +++ b/backup.txt @@ -0,0 +1,36 @@ +#!/bin/sh + +# This script does personal backups to a rsync backup server. You will end up +# with a 7 day rotating incremental backup. The incrementals will go +# into subdirectories named after the day of the week, and the current +# full backup goes into a directory called "current" +# tridge@linuxcare.com + +# directory to backup +BDIR=/home/$USER + +# excludes file - this contains a wildcard pattern per line of files to exclude +EXCLUDES=$HOME/cron/excludes + +# the name of the backup machine +BSERVER=owl + +# your password on the backup server +export RSYNC_PASSWORD=XXXXXX + + +######################################################################## + +BACKUPDIR=`date +%A` +OPTS="--force --ignore-errors --delete-excluded --exclude-from=$EXCLUDES + --delete --backup --backup-dir=/$BACKUPDIR -a" + +export PATH=$PATH:/bin:/usr/bin:/usr/local/bin + +# the following line clears the last weeks incremental directory +[ -d $HOME/emptydir ] || mkdir $HOME/emptydir +rsync --delete -a $HOME/emptydir/ $BSERVER::$USER/$BACKUPDIR/ +rmdir $HOME/emptydir + +# now the actual transfer +rsync $OPTS $BDIR $BSERVER::$USER/current diff --git a/documentation.html b/documentation.html new file mode 100644 index 0000000..f9379be --- /dev/null +++ b/documentation.html @@ -0,0 +1,28 @@ + + +

Documentation

+ + + + + + diff --git a/download.html b/download.html new file mode 100644 index 0000000..0802aad --- /dev/null +++ b/download.html @@ -0,0 +1,49 @@ + + + +

rsync download

+ +You can download rsync via ftp, http, cvs or rsync. + +

http

+ + + +

ftp

+ + + +

rsync

+ + + +

cvs

+ + + +

Contributing binaries

+ +If you can build binaries for a platform that we currently don't offer +binaries for then please let me know (via the bug tracking +system). I'll then give you a way of uploading binaries to +rsync.samba.org. Please only do this if you have the time to build and +package binaries in a timely fashion for every rsync release. + + diff --git a/examples.html b/examples.html new file mode 100644 index 0000000..d8fa5b7 --- /dev/null +++ b/examples.html @@ -0,0 +1,49 @@ + + +

rsync examples

+ +If you have an interesting example of how you use rsync then please +submit it to the +rsync-bugs@samba.org +for inclusion on this page. + +

backup to a central backup server with 7 day incremental

+ +

+
+
+ +

backup to a spare disk

+ +

+
+
+ +

mirroring vger CVS tree

+ +

+
+
+ +

automated backup at home

+ +

+
+
+ +

Fancy footwork with remote file lists

+ +

+One little known feature of rsync is the fact that when run over a
+remote shell (such as rsh or ssh) you can give any shell command as
+the remote file list. The shell command is expanded by your remote
+shell before rsync is called. For example, see if you can work out
+what this does:
+
+	rsync -avR remote:'`find /home -name "*.[ch]"`' /tmp/
+
+note that that is backquotes enclosed by quotes (some browsers don't
+show that correctly).
+
+ + diff --git a/faq.html b/faq.html new file mode 100644 index 0000000..e69de29 diff --git a/faq.txt b/faq.txt new file mode 100644 index 0000000..1f27bfd --- /dev/null +++ b/faq.txt @@ -0,0 +1,220 @@ +
+HP compile
+Read-only file system
+copies every file
+is your shell clean
+memory usage
+out of memory
+
    +rsync 2.4.3 with rsh
    +rsync and cron
    +rsync: Command not found
    +spaces in filenames
    +stderr & stdout
    +subscribe
    +
+

HP compile

+

+For HPUX apparently you need to add the option -Ae to the CFLAGS. Edit
+the Makefile and change CFLAGS to:
+
+ CFLAGS=-Ae -O
+
+

Read-only file system

+

+if you get "Read-only file system" as an error when sending to a rsync
+server then you probably forgot to set "read only = no" for that
+module.
+
+

copies every file

+

+Some people occasionally report that rsync copies every file when they 
+expect it to copy only a small subset. In most cases the explanation
+is that rsync is not in fact copying every file it is just trying
+to update file permissions or ownership and this is failing for some
+reason. rsync lists files with the -v option if it makes any change
+to the file, including minor changes such as group changes.
+
+If you think that rsync is erroneously copying every file then look
+at the stats produced with -v and see if rsync is really sending all
+the data. 
+
+
+
+

is your shell clean

+

+The "is your shell clean" message and the "protocol mismatch"
+message are usually caused by having some sort of program
+in your .cshrc, .profile, .bashrc or equivalent file that
+writes a message every time you connect. Data written
+in this way corrupts the rsync data stream. rsync detects this
+at startup and produces those error messages.
+
+A good way to test this is something like:
+
+	rsh remotemachine /bin/true > test.dat
+
+you should get a file called test.dat created of 0 length. If
+test.dat is not of zero length then your shell is not clean.
+Look at the contents of test.dat to see what was sent. Look
+at all the startup files on remotemachine to try and find the
+problem.
+
+
+
+

memory usage

+

+yes, rsync uses a lot of memory. The majority of the memory is used to
+hole the list of files being transferred. This takes about 100 bytes
+per file, so if you are transferring 800,000 files then rsync will consume
+about 80M of memory. It will be higher if you use -H or --delete.
+
+To fix this requires a major rewrite of rsync. I do plan on doing that, but
+I don't know when I'll get to it.
+
+

out of memory

+

+The usual reason for "out of memory" when running rsync is that you
+are transferring a _very_ large number of files.  The size of the
+files doesn't matter, only the total number of files.
+
+As a rule of thumb you should expect rsync to consume about 100 bytes per
+file in the file list. This happens because rsync builds a internal
+file list structure containing all the vital details of each file. 
+rsync needs to hold structure in memory because it is being constantly
+traversed.
+
+I do have a plan for how to rewrite rsync so that it consumes a fixed
+(small) amount of memory no matter how many files are transferred, but
+I haven't yet found a spare week of coding time to implement it!
+
+
+
+

rsync 2.4.3 with rsh

+

+rsync 2.4.3 has a problem with some versions of rsh. The versions of rsh (such as the
+one on Solaris) that don't handle non-blocking IO will cause all sorts of errors,
+including "unexpected tag" "multiplexing overflow" etc.
+
+The fix is to either use an earlier version of rsync or use ssh instead of rsh
+or wait for rsync 2.4.4
+
+
+

rsync and cron

+

+On some systems (notably SunOS4) cron supplies what looks like a
+socket to rsync, so rsync thinks that stdin is a socket. This means
+that if you start rsync with the --daemon switch from a cron job you
+end up rsync thiking it has been started from inetd. The fix is simple
+- just redirect stdin from /dev/null in your cron job.
+
+
+

rsync: Command not found

+

+> rsync: Command not found
+
+This error is produced when the remote shell is unable to locate the rsync
+binary in your path. There are 3 possible solutions:
+
+1) install rsync in a "standard" location that is in your remote path. 
+
+2) modify your .cshrc, .bashrc etc on the remote machine to include the path
+that rsync is in
+
+3) use the --rsync-path option to explicitly specify the path on the
+remote machine where rsync is installed
+
+You may echo find the command:
+
+	rsh samba 'echo $PATH'
+
+for determining what your remote path is.
+
+
+
+

spaces in filenames

+

+Jim wrote:
+> This seems to imply rsync can't copy files with names containing
+> spaces.  A couple quick greps through the man page suggests that
+> this limitation isn't mentioned.
+
+Short answer: rsync can handle filenames with spaces
+
+Long answer: 
+
+rsync handles spaces just like any other unix command line application.
+Within the code spaces are treated just like any other character so
+a filename with a space is no different from a filename with any
+other character in it.
+
+The problem of spaces is in the argv processing done to interpret the
+command line. As with any other unix application you have to escape
+spaces in some way on the command line or they will be used to
+separate arguments. 
+
+It is slightly trickier in rsync because rsync sends a command line
+to the remote system to launch the peer copy of rsync. The command
+line is interpreted by the remote shell and thus the spaces need
+to arrive on the remote system escaped so that the shell doesn't
+split such filenames into multiple arguments.
+
+For example:
+
+	rsync -av fjall:'a long filename' /tmp/
+
+won't work because the remote shell gets an unquoted filename. Instead
+you have to use:
+
+	rsync -av fjall:'"a long filename"' /tmp/
+
+or a similar construct (there are lots of varients that work).
+
+As long as you know that the remote filenames on the command line
+are interpreted by the remote shell then it all works fine. 
+
+I should probably provide the above examples in the docs :-)
+
+Cheers, Andrew
+
+
+
+

stderr & stdout

+

+> Why does rsync produce some things on stdout and some on stderr?
+
+All messages which originate from the remote computer are sent to stderr.
+All informational messages from the local computer are sent to stdout.
+All error messages from the local computer are sent to stderr.
+
+There is a reason to this system, and it would be quite difficult to change.
+The reason is that rsync uses a remote shell for execution. The remote
+shell provides stderr/stdout. The stdout stream is used for the rsync 
+protocol. Mixing error messages into this stdout stream would involve
+lots of extra overhead and complexity in the protocol because each message
+would need to be escaped, which means non-messages would need to be encoded
+in some way. Instead rsync always sends remote messages to stderr. This means
+they appear on stderr at the local computer. rsync can't intercept them.
+
+If you have a problem with scripts or cron jobs that produce stderr then I
+suggest you use your shell to redirect stderr to stdout. For example you
+could do a cron line like this:
+
+0 0 * * * /usr/local/bin/rsync -avz /foobar /foo 2>&1 > logfile
+
+this would send both stderr and stdout to "logfile". The magic
+bit is the "2>&1" which says to redirect stderr to stdout.
+
+
+
+
+

subscribe

+

+> How to subscribe to rsync mailing list?
+
+Send a email to listproc@samba.org with no subject and a
+body of "subscribe rsync Your Name"
+
+See http://lists.samba.org/ for more info.
+
+
diff --git a/features.html b/features.html new file mode 100644 index 0000000..6704bcd --- /dev/null +++ b/features.html @@ -0,0 +1,23 @@ + + +

rsync features

+ +rsync is a file transfer program for Unix systems. rsync uses the +"rsync algorithm" which provides a very fast method for bringing +remote files into sync. It does this by sending just the differences +in the files across the link, without requiring that both sets of +files are present at one of the ends of the link beforehand.

+ +Some features of rsync include + +

+ + diff --git a/footer.html b/footer.html new file mode 100644 index 0000000..66cd8eb --- /dev/null +++ b/footer.html @@ -0,0 +1,29 @@ + + + + + +


=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= + + +

+

+ + + + + + +
Search WWW +Search rsync web pages and list archives
+
+ + + + + + + diff --git a/header.html b/header.html new file mode 100644 index 0000000..d9fa56e --- /dev/null +++ b/header.html @@ -0,0 +1,40 @@ + + + +rsync + + + + + + + + + + + + + +
+ + + rsync + + +

+ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= +


+
diff --git a/horus.txt b/horus.txt new file mode 100644 index 0000000..d861dc8 --- /dev/null +++ b/horus.txt @@ -0,0 +1,28 @@ +I do local backups on several of my machines using rsync. I have an +extra disk installed that can hold all the contents of the main +disk. I then have a nightly cron job that backs up the main disk to +the backup. This is the script I use on one of those machines. + + #!/bin/sh + + export PATH=/usr/local/bin:/usr/bin:/bin + + LIST="rootfs usr data data2" + + for d in $LIST; do + mount /backup/$d + rsync -ax --exclude fstab --delete /$d/ /backup/$d/ + umount /backup/$d + done + + DAY=`date "+%A"` + + rsync -a --delete /usr/local/apache /data2/backups/$DAY + rsync -a --delete /data/solid /data2/backups/$DAY + + + +The first part does the backup on the spare disk. The second part +backs up the critical parts to daily directories. I also backup the +critical parts using a rsync over ssh to a remote machine. + diff --git a/index.html b/index.html new file mode 100644 index 0000000..ad076f8 --- /dev/null +++ b/index.html @@ -0,0 +1,21 @@ + + + +

Welcome to the rsync web pages

+ +rsync is an open source +utility that provides fast incremental file transfer. rsync is freely +available under the GNU General Public +License + +

rsync 2.5.4

+ +The latest version of rsync is version 2.5.4. + +

This version includes the following changes: + +

+
+
+ + diff --git a/nt.html b/nt.html new file mode 100644 index 0000000..8ebc09e --- /dev/null +++ b/nt.html @@ -0,0 +1,80 @@ + + +

rsync on NT

+ +
+From:   "Mike McHenry" 
+Subject: Rsync 2.3.1 WinNT binaries and instructions available
+Date:   Fri, 15 Oct 1999 02:53:30 +1000
+
+Hello all,
+
+I have created Windows NT binaries for rsync 2.3.1 and have decided to make
+them available for others to download. These binaries have been tested on
+Windows NT Server 4.0 SP5 and WILL run in daemon mode if you follow my
+instructions below. I make no guarantees about these binaries, they have
+however been working for me for weeks on several NT machines.
+
+Binaries at ftp://ftp.minn.net/usr/mmchen/
+
+Instructions for running in daemon mode:
+
+1. You will need two files, rsync.exe and cygwin1.dll. Place rsync.exe
+anywhere you like (I chose c:\program files\rsync\rsync.exe) and put
+cygwin1.dll in c:\winnt\system32
+
+2. You will need a program from the NT Server resource kit called
+srvany.exe. This program allows you to run any executable as a service. If
+you simply install the entire service pack it will be located in c:\ntreskit
+
+3. Create a service for rsync by typing the following:
+  instsrv Rsync "C:\ntreskit\srvany.exe"
+
+4. You should now have a new service called Rsync and you can verify by
+looking in Start->Control Panel->Services DON'T START IT YET!
+
+5. If you want to run rsync in daemon mode you will need a configuration
+file. Here is the one I use, call it rsyncd.conf and place it in the same
+directory as rsync (C:\Program files\rsync\rsyncd.conf)
+  use chroot = false
+  strict modes = false
+  hosts allow = *
+
+  [backup]
+     path = /
+     read only = yes
+     list = no
+
+This example configuration will make one big anonymous anonymous rsync area
+available, I use this to backup my NT machines from a central Unix machine.
+This configuration might not be ideal for you, change to suit your tastes.
+The first two lines are important for rsync to work on Windows NT however.
+
+6. You are going to need to hack some keys in the registry to make it work.
+Don't do this unless you are comfortable with the changes! Run regedit32 and
+add the following keys and values (quotation marks ARE IMPORTANT):
+  HKEY_LOCAL_MACHINE->SYSTEM->CurrentControlSet->Services->Rsync
+    Edit->Add Key->  Key Name: Paramaters
+    Edit->Add Value->  Value Name: AppDirectory  Value: "C:\programfiles\rsync"
+    Edit->Add Value->  Value Name: Application  Value: "C:\programfiles\rsync\rsync.exe"
+    Edit->Add Value->  Value Name: AppParamters  Value: --config="C:\programfiles\rsync\rsyncd.conf" --daemon
+
+7. That's it, you should be able to start and stop the rsync service at will
+using the Services Control Panel. When running with the above configuration
+you should be able to test by attempting to telnet to port 873 from a remote
+machine.
+  telnet rsync.server.com 873 (replacing rsync.server.com with your own
+server's address)
+You should get a connection to the rsync daemon running on your NT box.
+
+8. If you have problems you are on your own, sorry, I have enough to do :) I
+would suggest triple-checking your spelling on EVERYTHING (filenames,
+configs, reg keys). If you have any comments or suggestions I would be happy
+to hear them at mmchen@minn.net.
+
+Mike McHenry
+ Systems Administrator
+ MinnNet Communications, Inc.
+
+ + diff --git a/nt.txt b/nt.txt new file mode 100644 index 0000000..a2a4311 --- /dev/null +++ b/nt.txt @@ -0,0 +1,74 @@ +From: "Mike McHenry" +Subject: Rsync 2.3.1 WinNT binaries and instructions available +Date: Fri, 15 Oct 1999 02:53:30 +1000 + + +Hello all, + +I have created Windows NT binaries for rsync 2.3.1 and have decided to make +them available for others to download. These binaries have been tested on +Windows NT Server 4.0 SP5 and WILL run in daemon mode if you follow my +instructions below. I make no guarantees about these binaries, they have +however been working for me for weeks on several NT machines. + +Binaries at ftp://ftp.minn.net/usr/mmchen/ + +Instructions for running in daemon mode: + +1. You will need two files, rsync.exe and cygwin1.dll. Place rsync.exe +anywhere you like (I chose c:\program files\rsync\rsync.exe) and put +cygwin1.dll in c:\winnt\system32 + +2. You will need a program from the NT Server resource kit called +srvany.exe. This program allows you to run any executable as a service. If +you simply install the entire service pack it will be located in c:\ntreskit + +3. Create a service for rsync by typing the following: + instsrv Rsync "C:\ntreskit\srvany.exe" + +4. You should now have a new service called Rsync and you can verify by +looking in Start->Control Panel->Services DON'T START IT YET! + +5. If you want to run rsync in daemon mode you will need a configuration +file. Here is the one I use, call it rsyncd.conf and place it in the same +directory as rsync (C:\Program files\rsync\rsyncd.conf) + use chroot = false + strict modes = false + hosts allow = * + + [backup] + path = / + read only = yes + list = no + +This example configuration will make one big anonymous anonymous rsync area +available, I use this to backup my NT machines from a central Unix machine. +This configuration might not be ideal for you, change to suit your tastes. +The first two lines are important for rsync to work on Windows NT however. + +6. You are going to need to hack some keys in the registry to make it work. +Don't do this unless you are comfortable with the changes! Run regedit32 and +add the following keys and values (quotation marks ARE IMPORTANT): + HKEY_LOCAL_MACHINE->SYSTEM->CurrentControlSet->Services->Rsync + Edit->Add Key-> Key Name: Paramaters + Edit->Add Value-> Value Name: AppDirectory Value: "C:\programfiles\rsync" + Edit->Add Value-> Value Name: Application Value: "C:\programfiles\rsync\rsync.exe" + Edit->Add Value-> Value Name: AppParamters Value: --config="C:\programfiles\rsync\rsyncd.conf" --daemon + +7. That's it, you should be able to start and stop the rsync service at will +using the Services Control Panel. When running with the above configuration +you should be able to test by attempting to telnet to port 873 from a remote +machine. + telnet rsync.server.com 873 (replacing rsync.server.com with your own +server's address) +You should get a connection to the rsync daemon running on your NT box. + +8. If you have problems you are on your own, sorry, I have enough to do :) I +would suggest triple-checking your spelling on EVERYTHING (filenames, +configs, reg keys). If you have any comments or suggestions I would be happy +to hear them at mmchen@minn.net. + +Mike McHenry + Systems Administrator + MinnNet Communications, Inc. + diff --git a/resources.html b/resources.html new file mode 100644 index 0000000..079e2e5 --- /dev/null +++ b/resources.html @@ -0,0 +1,51 @@ + + +

rsync resources

+ +Please let me know if you have any rsync related documents to add to +this list + + + + + + diff --git a/rsync_wrapper.pl b/rsync_wrapper.pl new file mode 100644 index 0000000..b760a97 --- /dev/null +++ b/rsync_wrapper.pl @@ -0,0 +1,471 @@ +# __ +# /\ \ From the mind of +# / \ \ +# / /\ \ \_____ Lee Eakin +# / \ \ \______\ or +# / /\ \ \/____ / +# \ \ \ \____\/ / Wrapper module for the rsync program +# \ \ \/____ / rsync can be found at http://rsync.samba.org/rsync/ +# \ \____\/ / +# \/______/ + +package Rsync; +require 5.004; + +use FileHandle; +use IPC::Open3 qw(open3); +use Carp; + +use strict; + +=head1 NAME + +Rsync - perl module interface to B http://rsync.samba.org/rsync/ + +=head1 SYNOPSIS + +use Rsync; + +$obj = Rsync->new(qw(-az C<-e> /usr/local/bin/ssh + --rsync-path /usr/local/bin/rsync)); + +$obj->exec(qw(localdir rhost:remdir)) + or warn "rsync failed\n"; + +=head1 DESCRIPTION + +Perl Convenience wrapper for B program. Written for B 2.3.1 but +should perform properly with most versions. + +=cut + +# options from the rsync man pae +###### Boolean flags ###### +# -h, --help show this help screen +# -v, --verbose increase verbosity +# -q, --quiet decrease verbosity +# -c, --checksum always checksum +# -a, --archive archive mode +# -r, --recursive recurse into directories +# -R, --relative use relative path names +# -b, --backup make backups (default ~ suffix) +# -u, --update update only (don't overwrite newer files) +# -l, --links preserve soft links +# -L, --copy-links treat soft links like regular files +# --copy-unsafe-links copy links outside the source tree +# --safe-links ignore links outside the destination tree +# -H, --hard-links preserve hard links +# -p, --perms preserve permissions +# -o, --owner preserve owner (root only) +# -g, --group preserve group +# -D, --devices preserve devices (root only) +# -t, --times preserve times +# -S, --sparse handle sparse files efficiently +# -n, --dry-run show what would have been transferred +# -W, --whole-file copy whole files, no incremental checks +# -x, --one-file-system don't cross filesystem boundaries +# -C, --cvs-exclude auto ignore files in the same way CVS does +# RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS +# .make.state .nse_depinfo *~ #* .#* ,* *.old *.bak +# *.BAK *.orig *.rej .del-* *.a *.o *.obj *.so *.Z +# *.elc *.ln core +# --delete delete files that don't exist on the sending side +# --delete-excluded also delete excluded files on the receiving side +# --partial keep partially transferred files +# --force force deletion of directories even if not empty +# --numeric-ids don't map uid/gid values by user/group name +# -I, --ignore-times don't exclude files that match length and time +# --size-only only use file size when determining if a file +# should be transferred +# -z, --compress compress file data +# --version print version number +# --daemon run as a rsync daemon +# --stats give some file transfer stats +# --progress show progress during transfer +###### scalar values ###### +# --csum-length=LENGTH <=16 bit md4 checksum size +# -B, --block-size=SIZE checksum blocking size (default 700) +# --timeout=TIME set IO timeout in seconds +# --port=PORT specify alternate rsyncd port number +# -e, --rsh=COMMAND specify rsh replacement +# -T, --temp-dir=DIR create temporary files in directory DIR +# --compare-dest=DIR also compare destination files relative to DIR +# --exclude-from=FILE exclude patterns listed in FILE +# --include-from=FILE don't exclude patterns listed in FILE +# --config=FILE specify alternate rsyncd.conf file +# --password-file=FILE get password from FILE +# --log-format=FORMAT log file transfers using specified format +# --suffix=SUFFIX override backup suffix +# --rsync-path=PATH specify path to rsync on the remote machine +###### array values ###### +# --exclude=PATTERN exclude files matching PATTERN +# --include=PATTERN don't exclude files matching PATTERN +# + +=over 4 + +=item Rsync::new + +$obj = new Rsync; + + or + +$obj = Rsync->new; + + or + +$obj = Rsync->new(@options); + +=back + +Create an Rsync object. Any options passed at creation are stored in +the object as defaults for all future exec call on that object. Options +are the same as those in L with the addition of +--path-to-rsync which can be used to override the hardcoded default of +/usr/local/bin/rsync, and --debug which causes the module functions to +print some debugging information to STDERR. + +=cut + +sub new { + my $class=shift; + + # seed the options hash, booleans, scalars, excludes, data, + # status, stderr/stdout storage for last exec + my $self={ + # the full path name to the rsync binary + 'path-to-rsync' => '/usr/local/bin/rsync', + # these are the boolean flags to rsync, all default off, including them + # in the args list turns them on + 'flag' => {qw( + --archive 0 --backup 0 --checksum 0 + --compress 0 --copy-links 0 --copy-unsafe-links 0 + --cvs-exclude 0 --daemon 0 --delete 0 + --delete-excluded 0 --devices 0 --dry-run 0 + --force 0 --group 0 --hard-links 0 + --help 0 --ignore-times 0 --links 0 + --numeric-ids 0 --one-file-system 0 --owner 0 + --partial 0 --perms 0 --progress 0 + --quiet 0 --recursive 0 --relative 0 + --safe-links 0 --size-only 0 --sparse 0 + --stats 0 --times 0 --update 0 + --verbose 0 --version 0 --whole-file 0 + )}, + # these have simple scalar args we cannot easily check + 'scalar' => {qw( + --block-size 0 --compare-dest 0 --config 0 + --csum-length 0 --exclude-from 0 --include-from 0 + --log-format 0 --password-file 0 --port 0 + --rsh 0 --rsync-path 0 --suffix 0 + --temp-dir 0 --timeout 0 + )}, + # these can be specified multiple times and are additive, the doc also + # specifies that it is an ordered list so we must preserve that order + 'exclude' => [], + # source/destination path names and hostnames + 'data' => [], + # return status from last exec + 'status' => 0, + 'realstatus' => 0, + # whether or not to print debug statements + 'debug' => 0, + # stderr from last exec in array format (messages from remote rsync proc) + 'err' => [], + # stdout from last exec in array format (messages from local rsync proc) + 'out' => [], + }; + if (@_) { + &defopts($self,@_) or return undef; + } + bless $self, $class; +} + +=over 4 + +=item Rsync::defopts + +defopts $obj @options; + + or + +$obj->defopts(@options); + +=back + +Set default options for future exec calls for the object. See L +for a complete list of valid options. This is really the internal +function that B calls but you can use it too. Presently there is no way +to turn off the boolean options short of creating another object, but if it is +needed and the B guys don't use it, I may add hooks to let + and ++ or a +leading no- toggle it back off similar to B (the GNU way). + +=cut + +sub defopts { + my $self=shift; + my @opts=@_; + + # need a conversion table in case someone uses the short options + my %short=(qw( + -B --block-size -C --cvs-exclude -D --devices -H --hard-links + -I --ignore-times -L --copy-links -R --relative -T --temp-dir + -W --whole-file -a --archive -b --backup -c --checksum + -e --rsh -g --group -h --help -l --links + -n --dry-run -o --owner -p --perms -q --quiet + -r --recursive -s --sparse -t --times -u --update + -v --verbose -x --one-file-system -z --compress + )); + while (my $opt=shift @opts) { + my $arg; + print(STDERR "setting debug flag\n"),$self->{debug}=1,next + if $opt eq '--debug'; + print STDERR "processing option: $opt\n" if $self->{debug}; + if ($opt=~/^-/) { + # handle short opts first + if ($opt=~/^-(\w+)$/) { + foreach (split '',$1) { + print STDERR "short option: -$_\n" if $self->{debug}; + if (exists $short{'-'.$_}) { + $opt=$short{'-'.$_}; + # convert it to the long form + $opt=$short{$opt} if exists $short{$opt}; + # handle the 3 short opts that require args + $self->{scalar}{$opt}=shift(@opts),next if (/^[BeT]$/); + # handle the rest + $self->{flag}{$opt}=1,next if exists $self->{flag}{$opt}; + } + carp "$opt - unknown option\n"; + return undef; + } + } + # handle long opts with = args + if ($opt=~/^(--\w+[\w-]*)=(.*)$/) { + print STDERR "splitting longopt: $opt ($1 $2)\n" if $self->{debug}; + ($opt,$arg)=($1,$2); + } + # handle boolean flags + $self->{flag}{$opt}=1,next if exists $self->{flag}{$opt}; + # handle simple scalars + $self->{scalar}{$opt}=($arg || shift @opts),next + if exists $self->{scalar}{$opt}; + # handle excludes + if ($opt eq '--exclude') { + $arg||=shift @opts; + # if they sent a reset, we will too + $self->{exclude}=[],next if $arg eq '!'; + # otherwise add it to the list + push @{$self->{exclude}},$arg; + next; + } + # to preserve order we store both includes and excludes in the same + # array. We use the leading '+ ' (plus space) trick from the man + # page to accomplish this. + if ($opt eq '--include') { + $arg||=shift @opts; + # first check to see if this is really an exclude + push(@{$self->{exclude}},$arg),next if $arg=~s/^- //; + # next see if they sent a reset, if they did, we will too + $self->{exclude}=[],next if $arg eq '!'; + # if it really is an include, fix it first, since we use exclude + $arg='+ '.$arg unless $arg=~/^\+ /; + push @{$self->{exclude}},$arg; + next; + } + # handle our special case to override hard-coded path to rsync + $self->{'path-to-rsync'}=($arg || shift @opts),next + if $opt eq '--path-to-rsync'; + # if we get this far nothing matched so it must be an error + carp "$opt - unknown option\n"; + return undef; + } else { # must be data (source/destination info) + print STDERR "adding to data array: $opt\n" if $self->{debug}; + push(@{$self->{data}},$opt); + } + } + 1; +} + +=over 4 + +=item Rsunc::exec + +exec $obj @options or warn "rsync failed\n"; + + or + +$obj->exec(@options) or warn "rsync failed\n"; + +=back + +This is the function that does the real work. Any options passed to this +routine are appended to any pre-set options and are not saved. They effect +the current execution of B only. It returns 1 if the return status was +zero (or true), if the B return status was non-zero it returns undef and +stores the return status. You can examine the return status from rsync and +any output to stdout and stderr with the functions listed below. + +=cut + +sub exec { + my $self=shift; + my @cmd=($self->{'path-to-rsync'}); + + foreach (sort keys %{$self->{flag}}) { + push @cmd,$_ if $self->{flag}{$_}; + } + foreach (sort keys %{$self->{scalar}}) { + push @cmd,$_.'='.$self->{scalar}{$_} if $self->{scalar}{$_}; + } + foreach (@{$self->{exclude}}) { + push @cmd,'--exclude='.$_; + } + foreach (@{$self->{data}}) { + push @cmd,$_; + } + push @cmd,@_ if @_; + print STDERR "exec: @cmd\n" if $self->{debug}; + my $in=FileHandle->new; my $out=FileHandle->new; my $err=FileHandle->new; + my $pid=eval{ open3 $in,$out,$err,@cmd }; + if ($@) { + $self->{realstatus}=0; + $self->{status}=255; + $self->{err}=[$@,"Execution of rsync failed.\n"]; + return undef; + } + $in->close; # we don't use it and neither should rsync (at least not yet) + $self->{err}=[ $err->getlines ]; + $self->{out}=[ $out->getlines ]; + $err->close; + $out->close; + waitpid $pid,0; + $self->{realstatus}=$?; + $self->{status}=$?>>8; + return undef if $self->{status}; + return 1; +} + +=over 4 + +=item Rsync::status + +$rval = status $obj; + + or + +$rval = $obj->status; + +=back + +Returns the status from last B call right shifted 8 bits. + +=cut + +sub status { + my $self=shift; + return $self->{status}; +} + +=over 4 + +=item Rsync::realstatus + +$rval = realstatus $obj; + + or + +$rval = $obj->realstatus; + +=back + +Returns the real status from last B call (not right shifted). + +=cut + +sub realstatus { + my $self=shift; + return $self->{realstatus}; +} + +=over 4 + +=item Rsync::err + +$aref = err $obj; + + or + +$aref = $obj->err; + +=back + +Returns an array or a reference to the array containing all output to stderr +from the last B call. B sends all messages from the remote +B process to stderr. This functions purpose is to make it easier for +you to parse that output for appropriate information. + +=cut + +sub err { + my $self=shift; + return(wantarray ? @{$self->{err}} : $self->{err}); +} + +=over 4 + +=item Rsync::out + +$aref = out $obj; + + or + +$aref = $obj->out; + +=back + +Similar to the B function, this returns an array or a reference to the +array containing all output to stdout from the last B call. B +sends all messages from the local B process to stdout. + +=cut + +sub out { + my $self=shift; + return(wantarray ? @{$self->{out}} : $self->{out}); +} + +=head1 Author + +Lee Eakin Eleakin@nostrum.comE + +=head1 Credits + +Gerard Hickey C + +Russ Allbery C + +Graham Barr C + +Andrew Tridgell and Paul Mackerras C + +John Steele Esteele@nostrum.comE + +Philip Kizer Epckizer@nostrum.comE + +Larry Wall C + +I borrowed many clues on wrapping an external program from the PGP modules, +and I would not have had such a useful tool to wrap except for the great work +of the B authors. Thanks also to Graham Barr, the author of the libnet +modules and many others, for looking over this code. Of course I must mention +the other half of my brain, John Steele, and his good friend Philip Kizer for +finding B and bringing it to my attention. And I would not have been +able to enjoy writing useful tools if not for the creator of the B +language. + +=head1 Copyrights + + Copyleft (l) 1999, by Lee Eakin + +=cut + +1; diff --git a/susan.txt b/susan.txt new file mode 100644 index 0000000..2d8a67e --- /dev/null +++ b/susan.txt @@ -0,0 +1,31 @@ +I use rsync to backup my wifes home directory across a modem link each +night. The cron job looks like this + + #!/bin/sh + cd ~susan + { + echo + date + dest=~/backup/`date +%A` + mkdir $dest.new + find . -xdev -type f \( -mtime 0 -or -mtime 1 \) -exec cp -aPv "{}" + $dest.new \; + cnt=`find $dest.new -type f | wc -l` + if [ $cnt -gt 0 ]; then + rm -rf $dest + mv $dest.new $dest + fi + rm -rf $dest.new + rsync -Cavze ssh . samba:backup + } >> ~/backup/backup.log 2>&1 + + +note that most of this script isn't anything to do with rsync, it just +creates a daily backup of Susans work in a ~susan/backup/ directory so +she can retrieve any version from the last week. The last line does +the rsync of her directory across the modem link to the host +samba. Note that I am using the -C option which allows me to add +entries to .cvsignore for stuff that doesn't need to be backed up. + + + diff --git a/vger.txt b/vger.txt new file mode 100644 index 0000000..da629c9 --- /dev/null +++ b/vger.txt @@ -0,0 +1,35 @@ +The vger.rutgers.edu cvs tree is mirrored onto cvs.samba.org via +anonymous rsync using the following script. + + #!/bin/bash + + cd /var/www/cvs/vger/ + PATH=/usr/local/bin:/usr/freeware/bin:/usr/bin:/bin + + RUN=`lps x | grep rsync | grep -v grep | wc -l` + if [ "$RUN" -gt 0 ]; then + echo already running + exit 1 + fi + + rsync -az vger.rutgers.edu::cvs/CVSROOT/ChangeLog $HOME/ChangeLog + + sum1=`sum $HOME/ChangeLog` + sum2=`sum /var/www/cvs/vger/CVSROOT/ChangeLog` + + if [ "$sum1" = "$sum2" ]; then + echo nothing to do + exit 0 + fi + + rsync -az --delete --force vger.rutgers.edu::cvs/ /var/www/cvs/vger/ + exit 0 + +Note in particular the initial rsync of the ChangeLog to determine if +anything has changed. This could be omitted but it would mean that the +rsyncd on vger would have to build a complete listing of the cvs area +at each run. As most of the time nothing will have changed I wanted to +save the time on vger by only doing a full rsync if the ChangeLog has +changed. This helped quite a lot because vger is low on memory and +generally quite heavily loaded, so doing a listing on such a large +tree every hour would have been excessive. diff --git a/y2k.html b/y2k.html new file mode 100644 index 0000000..b018124 --- /dev/null +++ b/y2k.html @@ -0,0 +1,19 @@ + + +

Year 2000 Issues

+ +We are starting to get a lot of Y2K compliance questions. The answer +is that rsync has no Y2K problems. rsync uses 32 bit "seconds since +1970" date formats and thus has no problems with the turn of the +century.

+ +You must remember though that rsync is part of a larger system. It +relies on a transport mechanism (rsh, ssh etc) and many operating +systems services (such as inetd, syslog etc). If any of these other +parts fail then rsync will of course be affected.

+ +Also remember that rsync comes with absolutely no warranty. It is up +to you to test whether rsync is suitable in your computing environment, +both before and after the turn of the century. + + -- 2.34.1