remove duplicates and sort

Shell Programming and Scripting
1

Hi,

I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another.

Thanks

2 
sort -k 1,1 -u file > file.tmp; mv file.tmp file
1 Like
3

I had a similar problem with a list of IP addresses. Best solution I was able to come up with was:

TMPFILE=$(mktemp /tmp/rbip.XXXXXX)

sort -n -t '.' -k 1,1 -k 2,2 -k 3,3 -k 4,4 <ip_list|uniq >>$TMPFILE

cp $TMPFILE ip_list

rm $TMPFILE
1 Like
4

If you have the data comma separated and want to sort the second column:

 $ sort -t, file1 -k 2,2 | uniq -u >file2
5

cp and mv are inode destructive for the destination file.
so some situation may prefer

cat  $TMPFILE >$FILE && rm $TMPFILE
6

This method also preserves ownership and permissions.

7

Thanks ctsgnb, I have never tried your method but I will now.