Uwe Hermann - Simple backups using rsync - Comments

Two years later...

Michaël — Sun, 29 Jun 2008 00:36:56 +0200

You have probably perfected the art of doing back-ups0 by now. Time for that blog post with more details?

Yeah, backup is nice in some

Uwe Hermann — Fri, 26 May 2006 17:38:24 +0200

Yeah, --backup is nice in some situations, but not for me in this situation.

I want to have an (almost) exact copy of my whole file tree on the backup medium, renaming files to .bak etc. would make that a complete mess...

rsync --backup?

Inetpro — Fri, 26 May 2006 16:00:54 +0200

Hi Uwe,

Enjoy reading your posts. Keep it up.

Have you ever considered the --backup options in rsync. Very powerful in my view.

From the man page:
-b, --backup With this option, preexisting destination files are renamed as each file is transferred or deleted. You can control where the backup file goes and what (if any) suffix gets appended using the --backup-dir and --suffix options.


       --backup-dir=DIR

              In combination with the --backup option,  this  tells  rsync  to

              store  all backups in the specified directory. This is very use‐

              ful for incremental backups.  You  can  additionally  specify  a

              backup  suffix  using  the  --suffix option (otherwise the files

              backed up in the specified directory will  keep  their  original

              filenames).   If  DIR  is a relative path, it is relative to the

              destination directory (which changes in a recursive transfer).

--suffix=SUFFIX This option allows you to override the default backup suffix used with the --backup (-b) option. The default suffix is a ~ if no --backup-dir was specified, otherwise it is an empty string.

Regards.

I liked also this one very

Martin Weis — Mon, 22 May 2006 22:39:36 +0200

I liked also this one very much:
http://www.pollux.franken.de/hjb/rsback/

which is a similar concept like rsnapshot. It has a different configuration file, you might like that ;-)

Untrusted media

Uwe Hermann — Wed, 17 May 2006 14:10:12 +0200

Ah, ok. I hope you properly wiped the data afterwards, using shred, wipe , dd if=/dev/urandom or some similar method...

~/.a is for me to know, and

Erik — Wed, 17 May 2006 10:29:13 +0200

~/.a is for me to know, and for you to find out... or something...

No.. At this point I didn't trust the target media, which is why I didn't backup .gnupg. I have a few versions of this script, the one I took here was taken when I was at a customers site, and the laptop was starting to behave badly, I needed to make a backup, but didn't want real sensitive information to be backed up. So I "borrowed" a machine and backed up to it :) They never knew, but still.....

Backup solutions

Uwe Hermann — Tue, 16 May 2006 23:52:55 +0200

Hm, maybe I'll have a look at duplicity, rdiff-backup, and rsnapshot, too... I was under the impression that Amanda was the backup solution when you have enough time to learn how to use it (Bacula being similar, but with some features which Amanda doesn't seem to have)...

I certainly don't want to have a frontend to rsync, I'm looking for a more sophisticated server/client solution which works nice for >= 10 machines running various Unix-like OSes, Windows, and whatnot, and has the ability for full/incremental backups and restoring data from any point in time...

Expect a blog post with more details eventually...

rsync script

Uwe Hermann — Tue, 16 May 2006 23:35:38 +0200

Hm, you're leaking information. What's in ~/.a? ;)

Why don't you backup ~/.gnupg? Is the target media not trustworthy or under your control? Why then, do you backup the rest of your stuff there?

Anyways, thanks for your script and the --link-dest trick, I'll try that soon...

Woot

Uwe Hermann — Tue, 16 May 2006 23:30:23 +0200

Yeah, yeah. Bow before my Über-hacker skillz ;)

More useful script

Erik — Tue, 16 May 2006 19:37:40 +0200

I have a more useful script here.

rsync supports hardlinking only things that has changed between old versions, and new. So with some magic you can have a setup where you do a rsync each day, and on the server you'll find
2006-01-01/
2006-01-02/
2006-01-03/

And everything that hasn't changed between these dates are hardlinked together, and will not take any extra space

Here is a script that does this for you (the magic is --link-dest :) )

#!/bin/bash
# You can call this script with -n to just run in dryrun mode

remotebackdir=/path/to/backup/
remotehost=backuphost
user=$(whoami)

date=$(date --iso-8601)
last=$(ssh $remotehost ls -tr $remotebackdir| grep 200 | tail -1)
if [ "$last" != "" ]; then
        link="--link-dest=$remotebackdir/$last"
fi
rsync -avb $1 -W -e ssh \
        $link \
        --exclude /.gnupg/ \
        --exclude /.a/ \
        --exclude /.ssh/ \
        --exclude cache/ \
        --exclude Cache/ \
        --exclude Media/ \
        --exclude old/ \
        /home/$user $remotehost:$remotebackdir/$date

rsync -avb $1 -W -e ssh \
        $link \
        --exclude /shadow \
        /etc $remotehost:$remotebackdir/$date

Regards,
Erik

Or for that matter, rsnapshot

Craig Box — Tue, 16 May 2006 03:35:38 +0200

rsnapshot is also great here; lets you take snapshots at certain points in time and automatically rotates them.

http://www.rsnapshot.org/

Why not duplicity or rdiff-backup?

Tim Stoop — Mon, 15 May 2006 21:41:01 +0200

Why not duplicity or rdiff-backup? I don't know your needs of course, but we've been using both of them (depending on the requirements) for a while now and they work really nice. What are your requirements for deciding you need Amanda or Bacula? (Not that they're evil or something, I just think duplicity or rdiff-backup usually do the job as well.)

Great!!!

Piotr Esden-Tempski — Mon, 15 May 2006 20:44:52 +0200

Wooot! Now I know! You are my hero. *waving_hands* *evil_grin*

Simple backups using rsync

Uwe Hermann — Mon, 15 May 2006 18:41:15 +0200

I'm currently looking into "serious" backup solutions (such as Amanda or Bacula) for work and personal backups. While I may switch to one of those soonishly, I'm currently using a pretty simple script to backup my laptop contents to an external, dm-crypt/LUKS encrypted USB hard drive via rsync.

#!/bin/sh
#------------------------------------------------------------------------------
# File: rsync_backup
# Author: Uwe Hermann 
# URL: http://www.hermann-uwe.de/files/rsync_backup
# License: GNU GPL (http://www.gnu.org/licenses/gpl.html)
# $Id: rsync_backup 511 2006-05-15 16:44:08Z uh1763 $
#------------------------------------------------------------------------------

SOURCE='/svn /etc /var/www /var/log /root /home'
DEST=/mnt/samsung300gb/backup

time rsync -a --progress --stats --delete-during $SOURCE $DEST

Just to let you know ;)