Search based on 1,2,4,5 columns and remove duplicates in the same file.

Shell Programming and Scripting
1

Hi,

I am unable to search the duplicates in a file based on the 1st,2nd,4th,5th columns in a file and also remove the duplicates in the same file.

Source filename: Filename.csv

"1","ccc","information","5000","temp","concept","new"
"1","ddd","information","6000","temp","concept","new"
"1","aaa","information","4000","temp","concept","new"
"1","aaa","information","4000","temp","concept","new"
"1","bbb","information","4000","temp","concept","new"
"1","bbb","information","4000","temp","concept","new"
"1","abc","information","7000","temp","concept","new"

Output filename: Filename.csv

Output:

"1","ccc","information","5000","temp","concept","new"
"1","ddd","information","6000","temp","concept","new"
"1","aaa","information","4000","temp","concept","new"
"1","bbb","information","4000","temp","concept","new"
"1","abc","information","7000","temp","concept","new"

Any help greatly appreciated.

thanks
Suri

2 
awk -F, '!_[$1, $2, $4, $5]++' infile
3 
for i in `sed = file1 | sed -e N -e 's/\(.*\)\n\(.*\)/\1  \2/' 
| sed -e 's/\(.*\),\(.*\),.*,\(.*\),\(.*\),.*,.*/\1\2\3\4/' 
| sed 'N;s/\([0-9]*\)  \(.*\)\n[0-9]*  \2/\1 \2/' 
| sed 's/\([0-9]*\).*/\1/'`
   do 
    sed -n "$i p" file1 
   done
"1","ccc","information","5000","temp","concept","new"
"1","ddd","information","6000","temp","concept","new"
"1","aaa","information","4000","temp","concept","new"
"1","bbb","information","4000","temp","concept","new"
"1","abc","information","7000","temp","concept","new"