remove duplicated columns

kamel.seg · February 21, 2008, 5:08am

hi all,

i have a file contain multicolumns, this file is sorted by col2 and col3.

i want to remove the duplicated columns if the col2 and col3 are the same in another line.

example
fileA

AA BB CC DD
CC XX CC DD
BB CC ZZ FF
DD FF HH HH

the output is
AA BB CC DD
BB CC ZZ FF
DD FF HH HH

thanks

manikantants · February 21, 2008, 5:11am

uniq -f1 -f2 -c file1

kamel.seg · February 21, 2008, 5:20am

it works but i have also in the file i get the occurence number of the line
can i remove it ??

manikantants · February 21, 2008, 5:23am

just remove '-c' option

kamel.seg · February 21, 2008, 5:26am

yes it work but the file a have have many columns and i want just to compare the colx and coly and ignor the rest what can i do, because f1 and f2 ignore the two first columns??

manikantants · February 21, 2008, 5:56am

yes give the columns to be avoided with -f option.

kamel.seg · February 21, 2008, 6:36am

i try sort -u -kx,y

with x and y are the columns numbers it works well