Rsync for back up, external HD

Hello all!

I am quite unsure with all options of rsync. Here my backup configuration: I am on Mac X (10.8) and want an exact copy of my HD to an external HD. I formatted the new USB drive with Mac OS extended (Journaled, Encrypted) and made in my shell the following command and got the following errors:

sudo rsync -a / /Volumes/backup_2013/
Password:
rsync: opendir "/dev/fd/3" failed: Bad file descriptor (9)
file has vanished: "/Users/mstep/Library/Saved Application State/com.apple.DiskUtility.savedState/data.data"
file has vanished: "/Users/mstep/Library/Saved Application State/com.apple.DiskUtility.savedState/window_1.data"
file has vanished: "/Users/mstep/Library/Saved Application State/com.apple.DiskUtility.savedState/window_2.data"
file has vanished: "/Users/mstep/Library/Saved Application State/com.apple.DiskUtility.savedState/windows.plist"
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]

I think, this is due to the fact, that I was working while this very long backup was running. And probably there were some temporary files of the format just before I run the backup ...

My questions:

1. Is this the good option (rsync -a) for the first backup?
2. Is for the following backups `rsync -a --delete HD/ backup` right?
3. Is it possible to make the backup bootable (it is password protected)?

Thank you for your hints!

marek - greetings from very hot Munich

Well Marek I suppose the lack of response comes from the usage you make of rsync... designed for remote copy...
And so most of us to do what you do would use cpio (or even tar...) or dd for exact copies
As to your Q3, with rsync like that you can forget about it... but I would give a try with dd ( your external disk must be same size or bigger, only if bigger, the difference will be lost...) I cant promise anything since I only did this on scsi devices almost identical or same standards... but it be worth trying ( but will it damage your external disk??). For if it works you would have an exact copyand then you could try to boot from it, if this works, you can return to your rsync for following backups I suppose

My 2 cents...

Hi.

For backup of files, I use rsnapshot, possibly in your repositories, more information at rsnapshot . It uses rsync as part of a larger organization process. I have used it for more than a year, it allows recovery of individual files, and uses hard-links to create a very compact backup, yet still "addressable" by date-time of collection. You can set up cron jobs to do hourly, daily, weekly, monthly backups from the remote gathering computer. I backup at 4-hour intervals.

For a very quick backup of everything, I often used a tar.gz file -- it quickly restores my files to updated, and newly installed systems with no trouble (although you may not want every old file).

For cloning to create a bootable image, I suggest you look at Clonezilla - About -- it will use dd as a last resort, but will first check to see if it can copy by omitting unused sectors. (I use software RAID, which causes a dd copy.)

Always experiment before you commit important data.

Best wishes ... cheers, drl

Thank you vbe and del for your hints!

Sorry for answering so late. I have had to read tons of manual pages of rsync, cp, dd, cpio, tar, rsnapshot.

My question was concerning an exact backup of my HD on a USB-HD and how to do it incrementally.

@drl

Off course rsync is also a tool for external HDs not only over network.

If I am right the command I will use in future is the following to make a incremental backup - you may still prevent a catastrophe with some helpful hints:

sudo rsync -tavu --delete / /Volumes/backup_2013/ 

The option -t preserves modification times. -a archive mode. -v verbose. -u update (only newer files are transferred - this switch I was looking for). --delete means all files which are deleted on the source HD are deleted on the destination too. Because I asked for an exact copy of my actual HD, this is the option I was looking for.

To make the external HD bootable (MacOS Lion) I have to make some researches again ... It is for a later post. (I am not sure, whether it will possible with an encrypted HD).

marek

---------- Post updated at 03:52 PM ---------- Previous update was at 02:23 PM ----------

To make the external HD bootable (only to make this thread complete for some MacOS X users):

First try: I opened the HD Utility in the Utilities folder. I was hoping to find something like "Make Disk Bootable", but no help. I will not claim, that there is no such "button". I simply did not find it.

I have no Install MacOS X Disk Image in my Applications Folder. My idea was, to run the installer over the external HD to make it bootable. But to download this large installer file over the App Store, just to make an external HD bootable, I thought that this is overkill. So I decided to try the following and it worked:

The old version of CCC (Carbon Copy Cloner), which is now payable, is working! I just have had to run a copy of the HD over the external HD ...

I expected from this forum, to help me to find the right switches for rsync, and perhaps tell me, how to make an external HD bootable over the Shell. Disappointed!

Hope this helps somebody out there

marek

Wanted to thank you for your contribution!
As I already said rsync wasnt what it was supposed to be for that is why you have so little feed back, as for rsync making something bootable, that certainly cant be done the only standard unix tool to my knowlege able to do so is dd for you will have to create a bootstrap and that no ordinary copy command can do..that is why backup utilities that are able to create bootable tapes use dd to write at the beginning of tape then pass to tar... I might give a try when I have time and a bit $$ to spend on an extra external usb disk and see what happens with dd from a MacOS at home (my laptop in my bedroom...)

It can just as easily do local copy. More easily, since daemons and authentication aren't involved.

rsync dir1 dir2

You could use Apple's proprietary tool asr which can clone your operating system using block copy. The version of rsync Apple supplies is horribly hack. I'm not going to get into the details. If you want to use rsync to create a clone then roll your own by downloading the latest version of rsync (3.0.9) and applying the patches fileflags, crtimes, and hfs-compression. You'll find a list of files that should be excluded from the clone at the following link- Some files and folders are automatically excluded from a backup task / Backup and archiving settings / Knowledge Base - Bombich Software Support

Finally, you need to "bless" the clone (see man bless) so it can boot.

Thank you xbin!

I did not know the asr command! Interesting! I already did install the latest version of rsync, but did not succeeded to apply the patches. Will dip in again one more time. And thank you for the link!

marek

Hello all!

rsync version: 3.0.9
MacOSX: 10.8.4
Subject: make an exact copy of HD to an external HD with rsync

After long tests I put up my rsync script like follows:

First a long exclude file, which is read in by rsync with the switch

--exclude-from=path/to/exclude/file.txt

in this file are some legacy files from old MacOSes. So this exclude file should also work for older Macs. But probably not with an older rsync version ...

The following exclude file is from: Some files and folders are automatically excluded from a backup task / Backup and archiving settings / Knowledge Base - Bombich Software Support

.HFS+ Private Directory Data*
/.journal
/.journal_info_block
.AppleDouble
/lost+found
.metadata_never_index
/.com.apple.timemachine.donotpresent
.VolumeIcon.icns
/TheVolumeSettingsFolder
Saved Application State
.DocumentRevisions-V100
.Spotlight-V100
/.fseventsd
/.hotfiles.btree
/private/var/db/dyld/dyld_*
/System/Library/Caches/com.apple.bootstamps/*
/System/Library/Caches/com.apple.corestorage/*
/System/Library/Caches/com.apple.kext.caches/*
/.com.apple.NetBootX
/Volumes/*
/dev/*
/automount
/Network
/.vol/*
/net
/private/var/folders/*
/private/var/vm/*
/private/tmp/*
/cores
.Trash
.Trashes
/Backups.backupdb
/.MobileBackups
/private/tmp/kacta.txt
/private/tmp/kactd.txt
/PGPWDE01
/PGPWDE02
/.bzvol
/Library/Application Support/Comodo/AntiVirus/Quarantine
.DS_Store

Contrary to Slicehost Articles: rsync - exclude files and folders I found, that spaces and wild cards "" don't need to escaped with "[ ]" (for spaces) or surrounded by quotes like "file" for wild cards. And "/" means root folder - as expected -, not the folder from were the script was started.

Here the rules for this exclude files: every excluded file or folder has to be separated by a line.

Suppose, we have following folders, with some contents:

dev
#dev
dev#

dev # backups the folders "#dev" and "dev#", but not "dev"
dev/ # backups the folder "dev#" and nothing else!
dev/ # backups the folders "#dev" and "dev#"
dev/ # backups the folders "#dev" and "dev#" with its contents and the folder "dev" without
*dev/* # backups an empty "#dev" and "dev" and "dev#" with contents.
dev*/ # backups "#dev" with contents.
*dev*/ # nothing ...
*dev*/* # empty folders: "dev", "#dev", "dev#"

And now the backup script, which you start with, being in the folder where you saved this script:

sudo ./rsync.sh

#!/bin/sh

EXCLUDES=$HOME/Documents/programming/backup_script/rsync_exclude.txt

OPTS="-tavu --exclude-from=$EXCLUDES --delete"

EXTERN="backup_2013"
USBDisk="/Volumes/$EXTERN/"

if ls /Volumes/ | grep -q $EXTERN; then
	echo "$EXTERN is mounted! Starting backup ...\n"
else
	echo "Please mount the external HD \"$EXTERN\" before running this script!"
	exit
fi


rsync $OPTS / $USBDisk

Hope this helps some people out there ...

marek

Thanks for your contribution