Print only lines where fields concatenated match strings

Shell Programming and Scripting
1

Hello everyone,

Maybe somebody could help me with an awk script.

I have this input (field separator is comma ","):

547894982,M|N|J,U|Q|P,98,101,0,1,1
234900027,M|N|J,U|Q|P,98,101,0,1,1
234900023,M|N|J,U|Q|P,98,54,3,1,1
234900028,M|H|J,S|Q|P,98,101,0,1,1
234900030,M|N|J,U|F|P,98,101,0,1,1
547894932,M|H|J,U|Q|P,98,101,0,1,1
547894900,M|N|J,U|Q|P,98,101,0,1,1
547894995,M|T|J,U|Q|P,98,101,0,1,1
547894977,M|N|J,U|Q|P,0,101,0,1,1
300888261,R|N|J,U|Q|P,98,101,0,1,1

If the resulting string of concatenate fields from field2 to last field is different to the following strings:
M|N|JU|Q|P98101011
M|H|JU|Q|P98101011
M|H|JS|Q|P98101011

Then print those lines, so the output should be:

234900023,M|N|J,U|Q|P,98,54,3,1,1,Different
234900030,M|N|J,U|F|P,98,101,0,1,1,Different
547894995,M|T|J,U|Q|P,98,101,0,1,1,Different
547894977,M|N|J,U|Q|P,0,101,0,1,1,Different
300888261,R|N|J,U|Q|P,98,101,0,1,1,Different

I hope the explanation is good enough.

Thanks in advance

2

This seems to do what you want:

awk '{  x = substr($0, index($0, ","))
        gsub(/,/, "", x)
        if(x !~ /M\|(N\|JU|H\|JU|H\|JS)\|Q\|P98101011/) print $0, "Different"
}' OFS="," input

If you're using a Solaris/Sun OS system, use /usr/xpg4/bin/awk or nawk, instead of awk.

3

Hello Don,

Many thanks for your answer, I've tested with 53 lines and worked fine.

My real file has more than a million lines, I hope work relative fast and with correct output.

Many thanks again!