Getting last section of data from logfile

Shell Programming and Scripting
1

Hi, I have a log file from Munin like this:

2012/12/04 13:45:31 [INFO]: Munin-update finished (29.68 sec)
2012/12/04 13:50:01 Opened log file
2012/12/04 13:50:01 [INFO]: Starting munin-update
2012/12/04 13:50:01 [ERROR] Error occured in under [] in the configuration.
2012/12/04 13:50:01 [Warning] Could not parse datafile /var/lib/munin/datafile: [ERROR] Error occured in under [] in the configuration.  Please refer to the log if details are missing here.
2012/12/04 13:50:11 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:50:21 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:50:31 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:50:33 [INFO] Reaping Munin::Master::UpdateWorker<localdomain;localhost.localdomain>.  Exit value/signal: 0/0
2012/12/04 13:50:33 [INFO]: Munin-update finished (31.71 sec)
2012/12/04 13:55:02 Opened log file
2012/12/04 13:55:02 [INFO]: Starting munin-update
2012/12/04 13:55:02 [ERROR] Error occured in under [] in the configuration.
2012/12/04 13:55:02 [Warning] Could not parse datafile /var/lib/munin/datafile: [ERROR] Error occured in under [] in the configuration.  Please refer to the log if details are missing here.
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_testing
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_testing
2012/12/04 13:55:12 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:22 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:32 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:37 [INFO] Reaping Munin::Master::UpdateWorker<localdomain;localhost.localdomain>.  Exit value/signal: 0/0
2012/12/04 13:55:37 [INFO]: Munin-update finished (35.48 sec)

How do I get only the last section from the log starting with last Opened log file so I get output like this:

2012/12/04 13:55:02 Opened log file
2012/12/04 13:55:02 [INFO]: Starting munin-update
2012/12/04 13:55:02 [ERROR] Error occured in under [] in the configuration.
2012/12/04 13:55:02 [Warning] Could not parse datafile /var/lib/munin/datafile: [ERROR] Error occured in under [] in the configuration.  Please refer to the log if details are missing here.
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_testing
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_testing
2012/12/04 13:55:12 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:22 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:32 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:37 [INFO] Reaping Munin::Master::UpdateWorker<localdomain;localhost.localdomain>.  Exit value/signal: 0/0
2012/12/04 13:55:37 [INFO]: Munin-update finished (35.48 sec)
2

Try:

perl -lp0e 's/.*\n([^\n]*Opened log file)/$1/s' file.log
3

You can try this.

sed -n '/Opened log file/,$p' logfilename

It prints from the line matches the given pattern to end of file.

4

I would like to have all data from last found Opened log file , not from first found :slight_smile:
Prefer sed/awk soultion. My knowledge about perl is missing.

5

Did you try my code?

6

Yes, your solution works :slight_smile:
But prefer not to use perl

awk '/Opened log file/ {i=NR} END {for (x=i;x>i;i++) {print $NR}}' munin-update.log

This does not work, but it should be some like it.
Find last hits, by setting i to NR
Then print all record form i and out

7

Then you can open the log file and type in the end

set nu

and then you just need to see the line number of the last Opened log file suppose its line number is 4 and the last line number is 50 then you just need to type on command line.

sed -n '4,50'p logfilename

It will give result as per your need.:slight_smile:

8 
$ count=$(grep -c "Opened log file" munin.txt)
$ awk -v c="$count" '/Opened log file/{i++}i==c{print}' munin.txt
9

Another AWK approach:

awk '{a[NR]=$0}/Opened log file/{x=NR}END{for (i=x;i<=NR;i++) print a}' file.log
10

re-use the array

$ awk '/Opened log file/{i=0}{a[i++]=$0}END{for(j=0;j<i;j++)print a[j]}' munin.txt