Remove duplicate entries from the same line

Hello,

I have a file which have several duplicate entries on the same line:

File

ID    source    
1    GM    GF    GM
2    GM    GF    GM    GF    GM    GF    GM    GF    GM    GF        
3    GM    GF    GM    SF    GM    GF    GM    SF
4    FF    FF    FF    FF
5    FF    GM    FF    GM
6    FF    FF    FF    FF 

and the output needs to be

ID    source
1    GM    GF    
2    GM    GF        
3    GM    GF    SF    
4    FF    
5    FF    GM    
6    FF

I tried using sort {file-name} | uniq -u but that removes duplicates from the whole file!

Thank you

 awk '{for(i=1;i<=NF;i++){if(!A[$i,NR]++){S=S?S OFS $i:$i}}print S;S=""}' OFS="\t" file

ID      source
1       GM      GF
2       GM      GF
3       GM      GF      SF
4       FF
5       FF      GM
6       FF
1 Like

This one saves some memory by deleting the Array:

awk '{printf "%s", $1; delete A; for (i=2;i<=NF;i++) if (!A[$i]++) printf "\t%s",$i; print ""}' file
1 Like