AWK match $1 $2 pattern in file 1 to $1 $2 pattern in file2

Shell Programming and Scripting
1

Hi, I have 2 files that I have modified to basically match each other, however I want to determine what (if any) line in file 1 does not exist in file 2. I need to match column $1 and $2 as a single string in file1 to $1 and $2 in file2 as these two columns create a match.

I'm stuck in an AWK command and can't get any further. Below is an example of my 2 files. Note the difference in the 2 files: file1 has TEST in $1 on a line and has an additional line at the bottom where $1 match in both files but $2 does not.

file1

 
AO|IO.SYS.123|ABCXYZ
AR|IO.SYS.XYZ|ABC123
AT|IO.SYS.ABC|ABC321
TEST|IO.SYS.ABC|444555
AW|IO.SYS.XXX|123456
AZ|IO.SYS.ABC|222555
BB|IO.SYS.ABC|ABCABC
BC|IO.SYS.ABC|111222
BI|IO.SYS.ABC|8889BC
BR|IO.SYS.ABC|ABCVER
BR|IO.SYS.XYZ|ABCVER

file2

 
AO|IO.SYS.123|ABCXYZ
AR|IO.SYS.XYZ|ABC123
AT|IO.SYS.ABC|ABC321
AW|IO.SYS.XXX|123456
AZ|IO.SYS.ABC|222555
BB|IO.SYS.ABC|ABCABC
BC|IO.SYS.ABC|111222
BI|IO.SYS.ABC|8889BC
BR|IO.SYS.ABC|ABCVER

I can't figure out how to match both $1 and $2 as 1 string in both files. $1 in file1 might exist in file2 but $1 and $2 must match each other. If not determine which line does not exist.

2

Is the second column supposed to be unique for each line?

3 
awk -F\| 'NR == FNR {
  _[$1, $2]; next
  }
!(($1, $2) in _ )' file2 file1
4

I'm curious how that works.

5

It builds an associative array indexed by the concatenation of the first field, a special character (the current value of SUBSEP, default \034) and the second field, then it checks if such key is present (while reading the next file and constructing the key dynamically ...). Of course, I could be missing something ...
To the OP: does this code produce the expected output?

6

You can use grep as well

grep -F -v -f file2 file1

Regards
Peasant.

7

yes i tested it and it seems to work perfect. thanks!

8

Hm, no, if only the key columns need to be compared:

% head file[12]
==> file1 <==
AO|IO.SYS.123|ABCXYZ

==> file2 <==
AO|IO.SYS.123|NOT THIS ONE
% awk -F\| 'NR == FNR {
  _[$1, $2]; next
  }
!(($1, $2) in _ )' file2 file1  
% grep -F -v -f file2 file1
AO|IO.SYS.123|ABCXYZ
9

Part of the OP post says :
If not determine which line does not exist.

So if you require that, grep would be fine i guess or ? :stuck_out_tongue:

10

Maybe I'm just reading it wrongly ..., could the OP clarify ? :slight_smile: