Shell Script to Compare Files and Email the differences

UNIX for Beginners Q & A
1

Hi,

I have 2 files abc.txt and bdc.txt.

I am using

$diff -y abc.txt bcd.txt  -- compared the files side by side

I would like to write a Shell Script to cmpare the files side by side and print the results( which are not matched) in a side by side format and save the results in another file.

Please help me to write a script.

Thanks,
Vasu

2

Please post your work (what you have tried so far using CODE tags).

Thanks.

3

By following the forum have copied the below code.
below code displaying result. But I would like to mail the differences as attachment which can be easy to understand for any user. ( I didnt add mailx command in the below given code)

#! /bin/sh
# rt.sh

if [[ $# -ne 2 ]] ; then
echo "Need just 2 files to compare"
exit 1
fi ;

# Take the diff by ignoring the blank and whitespaces.
diff -b -w ${1} ${2} > ${1}.diff
        if [[ $? -eq 0 ]] ; then
        echo "No files were added/removed"
        else
        echo "Files were added/removed"
        fi ;

# Process the diff file.
# A line might look like 
# < text-which-went-out
# > text-which-came-in

while read line
do
        if [[ ${line:0:2} == "< " ]] ; then
        echo ${line:2} is removed. 
        fi ;

        if [[ ${line:0:2} == "> " ]] ; then
        echo ${line:2} is added.  
        fi ;

done < ${1}.diff

rm ${1}.diff
4

If you are looking for differences between two sorted list files, would you be better with grep -vf file1 file2 and then the reverse, grep -vf file2 file1 ?

I am guessing at your input because of the comments. It might save you a headache, or it might be completely wrong.

Do you have some sample input and expected output?

Robin

5

Did you consider the comm utility?

6

Here , where can we find the output? I gave the

grep -vf file1.txt  file2.txt

. then in 3 minutes(approx) its showing the $ prompt again. Please advise where can i find the comparison results.

Thanks

7

No output mean that there are no difference. Any differences would be displayed to the screen, or you could choose to direct them to a file as usual.

As for the return code, a zero value of $? indicates that there are differences and a value of one indicates that the files match.

I presume that these are quite large files if they take 3 minutes to work through. I know that there can be performance issues with grep and the -f flag with files over a certain size. is this time critical?

I hope that this helps,
Robin

8

To avoid having a filename in one file like file1 match longer names in the other file like file10 , file101 , etc. and to speed things up a little bit, you might want to try:

grep -vFxf file1.txt file2.txt

instead of:

grep -vf file1.txt file2.txt

The -F option tells grep to match fixed strings instead of basic regular expressions and the -x option tells grep to only look for full-line matches.

On some older systems, you might need to use fgrep instead of grep -F :

fgrep -vxf file1.txt file2.txt
2 Likes
9

There is difference in the files as the size of the files as are below.

FILE1.txt   156665

FILES2.txt 154359

But its not displaying the differences. Please advise how to get the differences and mail it as attachment.

Thanks

---------- Post updated at 06:22 PM ---------- Previous update was at 06:15 PM ----------

Don,

Now am able to see the differences in a file. Can you advise How can I export these differences (which will be Side by side) into local desktop to understand and compare easily.

As I can see the differences by using
diff -y file1.txt files2,txt

Thanks

10

You may want to add - given your diff version provides it - the --suppress-common-lines option.

11

There are at least two issues here.

First, knowing that the files FILE1.txt and FILES2.txt have a different size tells us those two files are different, but it doesn't tell us whether or not any line in either file is unique. The command diff -y f1 f2 will point out line-by-line any similarities and any differences between those two files. The command grep -vFxf f1 f2 will tell you if there are any lines in f2 that do not also appear as a line in f1 . For example, if f1 contains the single line:

a

and f2 contains the three lines:

a
a
a

that diff will tell you:

a								a
							      >	a
							      >	a

while that grep command will find no differences (because every line in f2 is also a line somewhere in f1 ). Similarly, if f1 contains:

a
b
c

and f2 contains:

c
b
a

the diff command will show you that at most one line is the same between the two files while the grep will not find any differences. If your two files contains lists of filenames (especially if those files are not sorted), the grep commands will give you useful information. The diff -y command will give you a lot of information, but it might not be useful.

Second, knowing that files FILE1.txt and FILES2.txt are different says absolutely nothing about whether or not file1.txt and files2,txt are different. UNIX, Linux, and BSD filesystems are case sensitive. And, even on filesystems that are not case-sensitive, a comma and a period are different.

Until you tell us what operating system you're using AND show us what you have tried with mailx there is little that we can do to help you with adding attachments to mail. The mailx utility in the standards doesn't have an option to add attachments. The mailx utility on many systems does have an option to add an attachment to a mail message, but that option can vary from system to system or not be provided at all.

We may also need to know what mail reader will be used by the person receiving your email. Some mail readers recognize a wide variety of attachment types; others do not.