Select distinct rows in a file by last column

Shell Programming and Scripting
1

Hi, I have the following file:

LOG:015608:[00:23:52]:ERR:2310:map_spsrec:Invalid parameter
LOG:015608:[00:23:54]:ERR:2471:map_dgdrec:Invalid parameter
LOG:015608:[00:23:55]:ERR:2487:map_nnmrec:Invalid number
LOG:015608:[00:23:56]:ERR:2310:map_nmrec:Invalid number
LOG:015608:[00:23:57]:ERR:2438:map_nmrec:Invalid number

As a delimiter I use ":". I want to have distinct rows based only on the last column, but also to keep the rest of the fields in front. The sorting is doesn't metter, e.g. just need one line with "Invalid parameter" and one line with "Invalid number". So desired output is:

LOG:015608:[00:23:54]:ERR:2471:map_dgdrec:Invalid parameter
LOG:015608:[00:23:55]:ERR:2487:map_nnmrec:Invalid number

Thanks in advance!

2 
awk -F: '!arr[$(NF)]++'  oldfile > newfile
1 Like
3

Thanks a lot!!