Extract UNIque records from File

Shell Programming and Scripting
1

Hi,

I have a file with 20GB Pipe Delimited file where i have too many duplicate records.
I need an awk script to extract the unique records from the file and put it into another file.

Kindly help.

Thanks,
Arun

2

Unique records based on any particular field?

Did you look at uniq / sort -u command as well?