File Comparision

Hi All,

I want to write a script which will compare two files and tell me if the files are different.

Actually my files will be same but order of lines will be different,so diff is not working.

I have written a script to do this:-

while read line; do
cnt=`grep -i $line /etl/symdev/hp/load/$1 | wc -l`
[ ${cnt} -le 1 ] && { echo "$line" >> /home/pkuma93/Error_log.log}
done < /etl/symdev/td/hp/load/$1

But its giving an error message Like:-

+ grep -i -12007122118835640491 P 2 4373 4373 26 2 79795 2 2 4359 4359 5 2 4359 4359 9 1069 78492 5 4373 3396 528 1 850 403 242 3063 -1 3063 1001378554 38 1 1000515551236727527 1177397160601 47.32 1 134.28 134.28 86.96 0.00 47.32 0.00 0.00 0.00 0.00 1 0 134.28 0.00 47.32 1 0.00 0 47.320000000010594994 0.000000075471949 0.00Y 2 0 14 542 2 1 0.00NM 0001N 6270 47.32 134.28 1 1 78492 210 133 210 210 210 1467405274 1001 11350733EE /etl/symdev/hp/load/HP74.LOAD.unet_phy_mkt_seg_refresh.load
grep: illegal option -- 1
grep: illegal option -- 2
grep: illegal option -- 0
grep: illegal option -- 0
grep: illegal option -- 7
grep: illegal option -- 1
grep: illegal option -- 2
grep: illegal option -- 2
grep: illegal option -- 1
grep: illegal option -- 1
grep: illegal option -- 8
grep: illegal option -- 8
grep: illegal option -- 3
grep: illegal option -- 5
grep: illegal option -- 6
grep: illegal option -- 4
grep: illegal option -- 0
grep: illegal option -- 4
grep: illegal option -- 9
grep: illegal option -- 1
usage: grep [-r] [-R] [-H] [-L] [-E|-F] [-c|-l|-q] [-insvxbhwy] [-p[parasep]] -e pattern_list...
        [-f pattern_file...] [file...]
usage: grep [-E|-F] [-c|-l|-q] [-insvxbhwy] [-p[parasep]] [-e pattern_list...]
        -f pattern_file... [file...]
usage: grep [-E|-F] [-c|-l|-q] [-insvxbhwy] [-p[parasep]] pattern_list [file...]
cnt=       0
+ echo 0
0

Can anyone help me to resolve this issue?

Thanks

try to sort the files first, then use diff

Sorry i m new to unix,there are mutiple columns in files then how can i sort....and i want generic script for all the files which will be of different columns....

an example:

# cat file1
line1 a b c
line3 c d e
line2 q l r
# cat file2
line1 a b c
line2 q l r
line3 c d e
# cat file3
line3 c d e
line2  l r
line1 a b c

without sort

# diff file1 file2
2d1
< line3 c d e
3a3
> line3 c d e

echo $? # output return code
1

with sort:

#sort file1 > file1.sort
#sort file2 >file2.sort
#diff file1.sort file2.sort
#echo $?
0

to put this in a script:

#!/bin/ksh
sort ${1} > /tmp/${1}.sort
sort ${2} > /tmp/${2}.sort

diff /tmp/${1}.sort /tmp/${2}.sort >/dev/null 2>&1
if [ $? -eq 0 ] 
then echo "files are equal"  ; RC=0
else echo "files are not equal" ; RC=1
fi

rm /tmp/${1}.sort
rm /tmp/${2}.sort 

exit $RC

usage:

# ./compare.ksh file1 file3
files are not equal
# ./compare.ksh file2 file1
files are equal

they way you sort the files doesn't matter, unless you sort both files the same way

Hi,

I have written file comparision script using diff and sort command,and its working fine for small files but i m facing issue with the large files having size around 300MB.

Is there any other way to compare two large files?

---------- Post updated at 01:45 PM ---------- Previous update was at 12:00 PM ----------

While sorting large file i m getting error message
sort: 0653-657 A write error occurred while sorting.
sort: 0653-657 A write error occurred while sorting.

hm sort/diff may take longer, but there should be no write error

perhaps diff or sort temporary store information in the filesystem

check the amount of free disk space in /tmp I guess

or you have too less memory/paging space?

Hi,

I have checked the space

Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 0.50 0.10 81% 5526 20% /
/dev/hd2 4.94 1.04 79% 72352 22% /usr
/dev/hd9var 0.50 0.09 82% 5159 19% /var
/dev/hd3 0.50 0.49 2% 740 1% /tmp
/dev/hd1 0.12 0.02 88% 2190 27% /home
/proc - - - - - /proc

Space is less....so is there any other alternative to do this..??