I need to transport a number of files from one server to other. I like to ensure the integrity using file checksum values.
The action plan is,
create the list of checksum values for all the files using cksum command in source server.
Transfer all the files including the file genered in step 1 to destination server.
Take the checksum of each file and compared with the file generated in step 1 using fgrep command.
I have a bash shell script to verify this.
The issue for me, in step 3 when i compared, the fgrep didn't find out even the checksum values changes.
My script is as below:
clear
cksum *.* > tmp.txt
while read N
do
fgrep -c $'(echo $N)' tmp.txt
done<x.txt
Remarks: tmp.txt - the file which generated in the destination server after copied. x.txt - the file which generated in source server (step 1) and transfered along with the files.
Please help me in this regard. Also let me know, any other ideas.
FTP will not copy the ownership or permissions of the files, so why not tar all the files into a single archive first, ftp the archive, compare checksums, then extract the files from the archive.
You are right. But the problem here is the number of files are so high and volume of the each files in GB's. It will take a huge amount of time to zip and unzipping process. Also It is not possible to send the entire files at a time. The file transfered on the basis of network traffice. Thats why I need this kind of checking.
I have two questions about your script.
Why do you use *.* instead of * ?
Only files with an extension will be processed.
The output of your fgrep command is presumably 0 or 1 so how will you know which file has failed?
If the txt files are in the same sequence why not use 'diff' to find differences.
So:
sort <tmp.txt >tmp.srt
sort <x.txt >x.srt
diff x.srt tmp.srt
I welcome the points which you raised. Let me explain.
I copied a list of files and the files are listed in x.txt as my script. I dont know all the files are copied successfully. It is possible to missed out one or more files in the destination. Also it is natural that the destination directory may have some other files also. That's why I use the *.* which create checksum for all the files in the destination directory. Then I will loop thru my x.txt file line by line and check the files is copied and the checksum value comparison. Also, the files are not with a same extension.
The file 1 or 0 indicates the checksum comparison result. In case the value is 0 then the file identified that it is not copied correctly or failed.
In this case, I cant use diff command to compare the two files becuase the no. of files in x.txt may differ with tmp.txt. The reason is, the destination directory may contains many other files including the copied files. If the directory have only the copied files then it is possible.
I hope you understand. In case further doubts, I am ready to explain in brief again.
My problem is, the fgrep always return 0 even though the checksum values are same.