get last section from large logfile

Shell Programming and Scripting
1

Hi all,

I need to pull the (last section) from the header to the end of file. This logfile will have many sections appended, but I only need to capture the last one and email it to someone. Any ideas? Thanks Kathy

Example Logfile:

=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr

= Wed May 19 13:10:25 EST 2004

blah
blah
blah

=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr

= Wed May 19 13:21:25 EST 2004

blah
blah
blah

2

This will do the trick - using awk

#!/bin/sh

awk 'BEGIN { FS="\n"; 
RS="" }
{
     record=$0
}
END { print record }' your_log_file

exit 0

Cheers
ZB
http://www.zazzybob.com

3

ZB,

That was close. I don't even understand the code.
But it returned the last 3 lines of the file. I needed probably about 60 lines back.

I think it picked up from the last blank line to the end. Let me describe my format again. These lines are varied with blank line in between various multiple lines.

Example Logfile:

=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr

= Wed May 19 13:10:25 EST 2004

blah
blah
blah
blank line
blah
blah
blank line
blah
blank line

=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr

= Wed May 19 13:21:25 EST 2004

blah
blah
blah
blank line
blah
blah
blank line
blah
blank line

4

Change the RS value from RS="" to RS="===\n" The RS value is the Record Separator value. FS is field separator. Changing RS to "===\n" specifies that 1 record is delimited by multiple === values with a new line at the end of the record. Each field within one of these records is delimited by FS, in this case a new line.

this could be shortened to
cat yourfile | awk 'BEGIN{RS="===*\n";FS="\n";} END{print $NR}'

5

I tried the cat version and my screen went blank and nothing came back. What's up with that.

6

print $NR wouldn't have the expected result, it would print the field at the position referenced by the Number of Records (NR) variable, which means that if you've more records than fields, you'll get nothing. For example if there were 6 records, then NR would be set to 6, thus "print $NR" would evaluate to "print $6" hence the sixth field would be printed.

The shortest version of this is (from the command line):

awk 'BEGIN {RS="===*\n";FS="\n"}{result=$0} END {print result}' my_log_file

........ I think.......

Cheers
ZB

7

Close again,

The problem seems to be defining a RS. Currently, a single = creates a RS.
this is the bottom of the file:
>>>>>>>>>>>>>>>>>>>>>>>>>
R3V45(66)->launch: debug: /apps/autosys/autosys/bin/sendevent -S TST -E SET_GLOBAL -G 'TSAPNOHO_C_072R_CFI_CONCFI=succeeded'
R3V45(67)->adapter_fork: debug: pids: adapter=6189098, child_task=5316686

R3V45(68)->adapter: exiting with status 0 at Wed May 19 14:33:04 2004

R3V45(69)->adapter_log_#2: debug: Flushing buffers and exiting with _exit(0)
<<<<<<<<<<<<<<<<<<<<<<<<<<
this is the value of result:
>>>>>>>>>>>>>>>>>>>>>>>
5316686

R3V45(68)->adapter: exiting with status 0 at Wed May 19 14:33:04 2004

R3V45(69)->adapter_log_#2: debug: Flushing buffers and exiting with _exit(0)
<<<<<<<<<<<<<<<<<<<<<<<<

Any more ideas?
Thanks,
Kathy

8

I think this problem could be due to the version of awk you're using.

I tried it with the (crap) version of awk that comes with Microsoft's Interix toolkit, and it doesn't work (we get the same thing, where "===*\n" matches even just one "="). Whereas trying it with the latest version of GNU Awk running under SuSE Linux 8.2, it works fine, and the RS is "two or more equals followed by a newline", which is what we want.

See if there are any other versions of awk on your system (gawk or nawk would probably work).

EDIT: Just tried it under GNU Awk 3.1.3 and that works fine too!

Cheers
ZB
http://www.zazzybob.com

9

tried nawk and got the same thing.... tried gawk, but it wasn't found...

looks like this might work...
1st line.... VAR=`grep -n "====" log|tail -1|awk -F: '{print $1}'`
NEXT line... tail -$VAR log > shortlog

only problem is that tail does not have same line count as the var says... the shortlog has 4 additional line at the top!!!

Thanks for your help!
Kathy

10

This awk code (untested) should print everything from the last line that begins with "=============".

/^=============/ {delete arr; c=0 }
{ arr[c++]=$0 }
END { for (x=0; x<c; x++) print arr[x] }