rsync incremental question

UNIX for Advanced & Expert Users
1

Hi, I have problem with incremental copy of files from source to destination.
I would like to copy different sources to different destinations. I made a backup script which look like that:

curd=$(date +%w)
day=$(date +%A)

mkdir -p /home/backup/DATA/$day

case $curd in
1|2|3|4|5)
            rsync -rP --ignore-errors --force /home/backup/Data-Finance /home/backup/DATA/$day;
            rsync -rP --ignore-errors --force /home/backup/data /home/backup/DATA/$day;
            rsync -rP --ignore-errors --force /home/backup/data-tb /home/backup/DATA/$day;;
6)
            rsync -rWP --delete --ignore-errors --force /home/backup/Data-Finance /home/backup/DATA/$day;
            rsync -rWP --delete --ignore-errors --force /home/backup/data /home/backup/DATA/$day;
            rsync -rWP --delete --ignore-errors --force /home/backup/data-tb /home/backup/DATA/$day;;
*) exit

The problem is that I have source almost 400 GB.I want to back up only those files which was modify only on the current day without those are older.

When rsync start for first time it starts coping all files from src to dest.
Is there any option which to resolve this?

Thanks in advance!

2

Hi zhivko.neychev (BG ? :slight_smile: )
I don't see such option in rsync man, however, you may sort the files with "find" and "-ctime" option, and then use "--exclude=PATTERN" of rsync to move only them ? Just a wild suggestion.

3

Thank you sisgate, I'll try it and I'll post the result. It's look logical.
Yes, I am from Bulgaria. Do you speak bulgarian :)?

4

It is a great Idea but I have some problems:

  1. I use -mtime 0 to became day-modified files.
  2. I use rsync --include`find * mtime 0` /dest
    the problem is that find back files such as: "My files" but bash understand only "My\ Files" - this is the problem. I can't use PATTERN.
5

Yeah, i see... Can't you do something like (roughly) :
dig through the source files, determine which ones have space, and substitute the space with underscore " _ " for example ? I mean, prepare the source files for better pattern matching by avoiding the space. I'm not sure how feasible is that, it's up to you.
Offtopic : I speak bulgarian, it's my native language :smiley: Have fun in unix.com :wink:

6

You can use a pattern with find - find /path -name 'pattern'.log
where pattern contains metacharacters like ? and *

-mtime -1 finds files less than 24 hours old. If you have bash, are you on a GNU-based system (linux)? If so, try unison instead. It has more features.

7

I resolved my problem yesterday. Thanks everybody who tried to help me.
This is my final script and it works fine for me:

#!/bin/bash

mount /dev/sda1 /home/backup/DATA/
cd /home/backup/src/

case $(date +%w) in

1|2|3|4|5)find * -mtime 0 > ../upd_list;rsync --progress --files-from=../upd_list /home/backup/src /home/backup/DATA/$(date +%A);;
        6)rsync -rWlP --delete --ignore-errors --force --exclude-from=../exclude /home/backup/src/Data-Finance /home/backup/src/data /home/backup/src/data-tb /home/backup/DATA/$(date +%A);;
        *)exit
esac
8

Interesting problem, interesting solution... i've never had a situation like this with rsync but I could easily see how this could be useful.