awk code to find difference in second file which is not present in first file .

UNIX for Beginners Q & A
1

Hi All,

I want to find difference between two files and output only lines which are not present in second file .I am using awk and I am getting only the first difference but I want to get all the lines which are not present in file2 .Below is the code I am using . Please help to get the desired output .

awk ' NR==FNR { a[NR]=$0; next } $0 != a[FNR] { print a[FNR];a[FNR]++; exit }' file1.txt file2.txt

Thanks ,
Srinivas

2

This prints any lines in file1.txt that don't appear in file2.txt:

awk ' NR==FNR { a[$0]; next } !($0 in a)' file2.txt file1.txt
1 Like
3

Hello Chubler_XL ,

Thanks for your prompt response . But I am getting few common lines as well with the below code i.e lines which are common to both the files but not all few of them .Your help is highly appreciated .

awk ' NR==FNR { a[$0]; next } !($0 in a)' file2.txt file1.txt

Thanks ,
Srinivas

4

There will be a difference between the two lines. Possible different case or white space characters (tabs vrs spaces, etc.).

1 Like
5

Thanks again , do we have any way out to suppress/ignore them and find the difference .

6

Pls show detailed, abbreviated but representative, samples of respective files, supported by e.g. a hexdump of each. so we stop shooting in the dark and start a data driven analysis.

7

Hi Rudic ,

I am attaching two files namely file1 and file2.I have removed /proc and /opt file system in file2 . Now when I use below command it is giving me two more file system along with /proc and /opt .

awk ' NR==FNR { a[$0]; next } !($0 in a)' file2.txt file1.txt

Expected output :
/proc - - - - /proc
/dev/hd15opt 2.50 1.37 0.13 92% /opt

Actual Output :
/proc - - - - /proc
/dev/hd15opt 2.50 1.37 0.13 92% /opt
abcdef:/ghi/USER_DATA/Tech-ABC/nmon 349479.97 211239.03 138240.94 61% /nmon_data
abcdef:/ghi/ABC/NIK 349479.97 211239.03 138240.94 61% /mpc

8

Your files are NOT *nix text files - they use DOS <CR> (= ^M = 0x0D = \r) line terminators, which usually cause problems with *nix text tools:

file /tmp/file*
/tmp/file1.txt:  ASCII text, with CRLF line terminators
/tmp/file2.txt:  ASCII text, with CRLF line terminators

On top, file2.txt doesn't have a terminating line feed mandatory for *nix text files.

Trying to reproduce your output, I get

awk ' NR==FNR { a[$0]; next } !($0 in a)' /tmp/file2.txt /tmp/file1.txt
/proc                 -         -         -    - /proc
/dev/hd05opt       2.50      1.37      0.13   92% /opt
abcdef:/ghi/ABC/NIK 349479.97 211239.03 138240.94   61% /mpc

, which, after adding the terminating line feed in file2, reduces to

/proc                 -         -         -    - /proc
/dev/hd05opt       2.50      1.37      0.13   92% /opt

BTW, diff might be the tool of choice for this case:

diff /tmp/file1.txt /tmp/file2.txt
7,8d6
< /proc                 -         -         -    - /proc
< /dev/hd05opt       2.50      1.37      0.13   92% /opt
16c14
< abcdef:/ghi/ABC/NIK 349479.97 211239.03 138240.94   61% /mpc
---
> abcdef:/ghi/ABC/NIK 349479.97 211239.03 138240.94   61% /mpc
\ No newline at end of file