Awk/sed : help on:Filtering multiple lines to one:

Experts Good day,

I want to filter multiple lines of same error of same day , to only 1 error of each day, the first line from the log.

Here is the file:

May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:31 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:34 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 10:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 10:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 10:01:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 10:02:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 10:03:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)

The output should be:

May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 11:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 10:00:02 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)

may be this is quite simple, but not able to figure out:
Thanks ,

awk '{E=$0;gsub(/.*..:..:.. /,X,E)}!A[$1,$2,E]++' file

Yoda Thanks,
Correction. I saw the lines were different so it is working as per the logic I had given.

---------- Post updated at 12:46 PM ---------- Previous update was at 12:39 PM ----------

In an another set of data : I found a different out put so I thought initially that it is not working but realized that the lines are not identical in the data , so reporting multiple times:

May 26 11:29:33 cmihpx02 vmunix: NFS getattr failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:33 cmihpx02 vmunix: NFS getattr failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 26 11:29:35 cmihpx02 vmunix: NFS getattr failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:34:30 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:34:30 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:52 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:52 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:34:31 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)

awk '{E=$0;gsub(/.*..:..:.. /,X,E)}!A[$1,$2,E]++' file2
May 26 11:29:33 cmihpx02 vmunix: NFS getattr failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 29 14:33:35 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 30 15:45:40 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.
May 31 09:33:43 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS write error on host cmiauxe1: error 238.

• I think I should be more specific about the string I am looking here. I am trying to filter the lines based on the string :

vmunix: NFS

Thanks a lot..

---------- Post updated at 12:50 PM ---------- Previous update was at 12:46 PM ----------

Hi Yoda, I made a slight change in the code: and it is working .

$ awk '{E=$5 " " $6;gsub(/.*..:..:.. /,X,E)}!A[$1,$2,E]++' file2
May 26 11:29:33 cmihpx02 vmunix: NFS getattr failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)

It is nice..
Thanks a lot,

$ sort -u -k2n -t' ' file
May 26 11:29:33 cmihpx02 vmunix: NFS getattr failed for server cmiauxe1: error 5 (RPC: Timed out)
May 29 14:33:35 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 30 15:45:40 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)
May 31 09:33:43 cmihpx02 vmunix: NFS commit failed for server cmiauxe1: error 5 (RPC: Timed out)

anbu23 , Thanks this too works, However noticed if -t' ' is really needed since the data already contains space separators.

sort -u -k2n -t' ' 

Thanks,