Finding the duplicate in a file....

Shell Programming and Scripting
1

Hi Unix Guru's

I had generated the uniqe code for every day date ranging from 20000101 to 21990101(200 years alomost 73000 uniqe codes ) and redirected it to text file.

Now My problem is i want to check whether there are any duplicates in unique code not PRESENT in the textfile ?

unique.txt Text file is in the following format.

YYYYMMDD|YYYYMM|UNIQUE CODE(ignore this row)

21120101|211201|F5CA1DD7746029E9C1CEF3137345D987
21120102|211201|F98804977D03F72DBC0AA0163B26F89E
21120103|211201|F01F29EC62E943978C934BCA79CD0140
21120104|211201|C943B6AB6BE9275D4D52B06BB484C59E
21120105|211201|A42873466FD7EF8FD211C82C52B2E1B2
21120106|211201|0179BB5B69E1433758E17DCA7D5A7D10
21120107|211201|30801625DDF75D0CC74E0255E994629E
21120108|211201|B758F26C1DCBC48F5BA62F38CED8B880

And also i want to print all the duplicates found. in the same formate of the record.

Thanks in Advance.

2

Julian date for '20000101' = 2451545
Julian date for '22000101' = 2524594

That is more than 73000 days.
to check to see if all dates are there try:

wc -l unique.txt

To see if the date key field is duplicated anywhere

awk -F'|'  '{arr[$1]++} END {for (i in arr) { if (arr[i>1]) {print i  }  }}'  unique.txt
3

Thanks Jim.