PERL "filtering the log file removing the duplicates

Shell Programming and Scripting
1

Hi folks,

I have a log file in the below format and trying to get the output of the unique ones based on mnemonic IN PERL.

Could any one please let me know with the code and the logic ?

Severity   Mnemonic	              Log Message
7	CLI_SCHEDULER	         Logfile for scheduled CLI type1
5      UPTIME                       Line protocol on Interface TYPE3
7	CLI_SCHEDULER	         Logfile for scheduled CLI type2
7	CLI_SCHEDULER	         Logfile for scheduled CLI type4
5      UPTIME                       Line protocol on Interface TYPE1
4      INACTIVE_LOCAL	        Local interface Port-Channel2 is link down.type2
5      UPTIME                       Line protocol on Interface TYPE2
7	CLI_SCHEDULER	         Logfile for scheduled CLI type3
4      INACTIVE_LOCAL	        Local interface Port-Channel2 is link down type1 
4      INACTIVE_LOCAL	        Local interface Port-Channel2 is link down.type3
6	INTERFACE_STATE	 Interface Ethernet21 state1
6	INTERFACE_STATE	 Interface Ethernet21 state2
6       INTERFACE_DEL           Interface Down

Final output should be

Severity   Mnemonic	  occurrence             Log Message

7      CLI_SCHEDULER	    4         Logfile for scheduled CLI <type>
5      UPTIME                  2         Line protocol on Interface <type>
4      INACTIVE_LOCAL	    3         Local interface Channel2 is down.<type>
5      UPTIME                  1         Line protocol on Interface <type>
5      NEIGHBOR_NEW	    1         LLDP neighbor <type>
5      UPDOWN	            1         Line protocol on Interface.<type>
6      INTERFACE_STATE   2         Interface Ethernet21 state
6      INTERFACE_DEL       1         Interface Down

Thanks in advance .......

2

This is a start, the first column is the occurrence:

perl -F"\t" -lane '$_ =~ s/(type|state|TYPE)[0-9]/<$1>/; $h{$_}++; END{while (($k, $v) = each %h){print "$h{$k}\t$k"}}' file
1 Like
3

my tons o thanks Subbeh ... This is exactly what I require...

and also Could you please make the above to Perl program instead of perl one liner..

Thanks once again ..

---------- Post updated at 09:40 PM ---------- Previous update was at 11:03 AM ----------

Could anyone please help on this....

Thanks in advance....

4

scriptscript, did you try to do it yourself? With a little bit of research on how to open files and how to use variables in perl you could do it yourself:

#!/usr/bin/perl

my (%h, $k, $v);

open(my $fh, '<', '/path/to/file') or die "Unable to open file, $!";
while (<$fh>) {
        $_ =~ s/(type|state|TYPE)[0-9]/<$1>/;
        $h{$_}++;
}
close($fh);

while (($k, $v) = each %h) {
        print "$h{$k}\t$k"
}