Compare & Copy Directories : Bash Script Help

Beginner/Intermediate shell; comfortable in the command line.

I have been looking for a solution to a backup problem. I need to compare Directory 1 to Directory 2 and copy all modified or new files/directories from Directory 1 to Directory 3. I need the directory and file structure to be mirrored on Directory 3. Another way of thinking about the logic is: Dir1 - Dir2 = Dir3. I want Dir3 to be portable between users/machines so I don't think an incremental backup with hardlinks will work. Seems like it should be easy right?

I found a scripts that seems like it could do the trick. However, I'm having some trouble getting it to work properly. It compares Dir1 to Dir2 and copies all new or modified files into Dir3 but it does NOT recreate the directory structure on the target. Instead all files end up in a flat folder.

From Statckoverflow . com

#!/bin/bash
#
# setup folders for our different stages
#
Dir1=/Users/username/source1
Dir2=/Users/username/source2
Dir3=/Users/username/target
#
cd $Dir1
for f in *f
do
# Diff the files - ignore the output...
    diff $f $Dir2 > /dev/null 2>&1
# ...but get the status
    status=$?
    if [ $status -eq 0 ] ; then
# Files are identical - don't copy the file
        echo $f unchanged
    elif [ $status -eq 1 ] ; then
# Files differ - copy new file
        echo $f changed
        cp $f $diff
    elif [ $status -eq 2 ] ; then
# Old file doesn't exist - copy new file
        echo old $f does not exist
        cp $f $diff
    fi
done

I started my reading with rsync. I've used it for back up - locally and over ssh between NAS servers. It should be able to do the job, right? I just couldn't find examples to guide me and I don't know enough and / or have enough time to do it unaided. Researching it I got side tracked by diff which I didn't know much about but it looked promising.

My preference is to do this with a bash script. Anyone have any ideas?

Thanks!

What OS & do you have the stat command?

I'm working in OpenBSD (OSX) which does support 'stat'.

Rod, here is a crude script that I wrote in bash. It uses cksum to check if the file in DIR_1 is modified. I have commented most of the steps. This one is just to give you an idea of using a different approach other than diff , I hope it helps.

#!/bin/bash

for file_1 in $( find DIR_1/* -type f | sed 's/\.\.\///g' )             # For each file in DIR_1
do
        file_2=$( echo $file_1 | sed 's/DIR_1/DIR_2/g' )                # Getting file path in DIR_2
        dir_2=$( dirname $file_2 )

        if [ ! -d $dir_2 ]                                              # Checking if sub-dir exists in DIR_2
        then
                echo -e "Dir: $dir_2 does not exist. Creating...\c"
                mkdir -p $dir_2                                         # Creating if sub-dir missing
                echo "Done"
        fi

        if [ -f $file_2 ]                                               # Checking if file exists in DIR_2
        then
                cksum_file_1=$( cksum $file_1 | cut -f 1 -d " " )       # Get cksum of file in DIR_1
                cksum_file_2=$( cksum $file_2 | cut -f 1 -d " " )       # Get cksum of file in DIR_2

                if [ $cksum_file_1 -ne $cksum_file_2 ]                  # Check if cksum matches
                then
                        echo -e "File: $file_1 is modified. Copying..\c"
                        cp $file_1 $file_2                              # Copy if cksum mismatch
                        echo "Done"
                fi
        else
                echo -e "File: $file_2 does not exist. Copying...\c"
                cp $file_1 $file_2                                      # Copy if file does not exist.
                echo "Done"
        fi
done

BTW I am using only 2 directory structures in this script: DIR_1 & DIR_2 .

Hi Rod

You can make a script in order to get all the differences between Dir1 and Dir2 with the rsync parameter --dry-run, after that, you only need to do a copy of this files/folders from Dir1 to Dir3, something like that in csh:

#!/bin/csh
setenv Dir1 "path1"
setenv Dir2 "path2"
setenv Dir3 "path3"
rsync -Havx --dry-run "$Dir1" "$Dir2" > /tmp/differences
foreach "element" ( 'cat /tmp/differences' )
            cp -rpf "$element" "$Dir3"
end
exit

This script, or something like that, will copy any file or folder from the Dir1 to Dir3 with the same structure.

BR

Hombreopaco