Rsync files check

Shell Programming and Scripting
1

Hello everybody,

I sent some files a remote server using simple ssh command line:

rsync -e 'ssh -p SSH-PORT' -vr --log-file=/var/log/rsync.log /home/USER/www/* USER@IP:/home/USER/www

Then I would like to check if files in server1 are the same, file size check or any other way to make sure everything is fine.

How can I do this kind of comparison?

Thanks in advance

2

rsync does that, called a hash code. If the hash code fails rsync barfs all over the place. However to answer your question:

There are several programs for this, but cksum is more than adequate.
Example
In the oldfile directory

cd /path/to/oldfile
old=$(cksum oldfile)
cd /path/to/new/file (or use ssh for a remote file): new=$(ssh USER@IP cd /path/to/newfile && cksum newfile)
new=$(cksum oldfile)
# you get output that looks like this :
# echo "$old" 
# 2201537306 122235 oldfile
# first number == checksum, second number == line count, name of file
# test example code 
if [ "$old" = "$new"  ] ; then 
   echo 'OK'
else
   echo 'NOT OK'
fi

Beware that ssh can add extra text all by itself - nothing to do with cksum

1 Like
3

I think I should loop through the specified directory and get all files, this is my last step:

dirName='/root/Scripts/Misc/'

cd $dirName


listFiles=$(find $dirName -iname '*' -exec ls -l {} \; | awk '{print $9}')

md5=$(cksum $listFiles)

But it does not look to work properly

4

If you want the md5 operation output, use md5sum instead of ckcum

If you have lots of files and you should be ensuring that they are identical (i.e. no files missing or extra on the target) you can use sum, cksum or md5sum on a wildcard and catch the output to a file, then compare the files. As an alternate with md5sum you can run it on the source system into a file, transfer the file and run it again on the target system using the -c flag, a bit like this:-

md5sum -c /tmp/md5sums.file 2>/dev/null |grep ": FAILED"

Any extra files on the target side will be ignored though. Is that acceptable?

Robin

1 Like
5

Well, this is the first part of the script, hope is good:

#!/bin/bash

dirName='/root/Scripts/Misc'

# Loop through directories
for elems in $(find $dirName -mindepth 1)
do

# if is file
if [[ -f $elems ]]; then

# Get file md5sum
md5=$(md5sum $elems | awk '{print $1}')

# if is directory
else

# get how many files inside this directory
md5=$(find $elems -type f | wc -l)

fi

# Get files size
fileSize=$(stat -c %s $elems)

# Print everything in one line
echo -e "$elems : $md5 : $fileSize" >> log

done