Cant get awk 1liner to remove duplicate lines from Delimited file, get "event not found" error..help

Shell Programming and Scripting
1

Hi,

I am on a Solaris8 machine

If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error"

...or

Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great.
******************

Here is the comma delimited InputFile i have (File1.dat):

It has (7 records each with 3 fields)

I want to define a duplicate record as any record where Field1 & Field2 values repeat across 2 or more records so that in by input file below...I have 3 duplicate records. Then I want to only keep the 1st occurrence of this duplicate record.

kk,12,a
aa,11,n --> duplicate
ee,13,b
aa,11,f --> duplicate
bb,17,k
pp,12,t
aa,11,w --> duplicate

What I want my awk script or perlscript to do is produce the following output (that is reduce the file from the 7records to 5records) , in essence keeping the 1st duplicate line, record only (again duplicate is defined as Field1 & Field2 having the same value across the records)

Desired Output File is here (5records, it kept only the 1st duplicate):

kk,12,a
aa,11,n
ee,13,b
bb,17,k
pp,12,t

I tried the following awk 1-liner at the csh prompt

awk '!x[$1,$2]++' FS="," File1.dat

but got the error message "Event not found"

***********

If someone knows how i can correct the above awk 1liner or how to make an alternative multi-line awk or perl script to do this ..that okay with me also. Just let me know where in your code it references

a) the fact that the files are comma delmited
b) Field1 & Field 2 as the fields by which to classify a record as a duplicate

Thanks in advance (this has stumped me for hours ..)

andy b

2

In Solaris / SunOS try using /usr/xpg4/bin/awk or nawk

3

"event not found" is an error in csh/tcsh when it meets an un-escaped ! . Other shells handle it better.
Try to avoid the !

awk '0==x[$1 FS $2]++' FS="," File1.dat

Better, avoid csh/tcsh!

exec bash
echo 'hello !world!'
4

Perl

perl -nle '@flds=split(/,/);
if (not exists ($seen{$flds[0].",".$flds[1]})) {
$seen{$flds[0].",".$flds[1]}=$_;
}
END{
print $seen{$_} foreach (keys %seen);}' filename