String search between patterns using sed

Shell Programming and Scripting
1

Hi,
I am trying to find a way to get sed/awk/grep to help me find a string in a log file that exists between two datestamps and then print the preceding datestamp up to the next datestamp.

Here is an example of my logfile:

+++ 2013/03/28 17:01:37.085 SIGNALING HIGH ACTIVE

Failure Response Measurements:
400: 1

+++ 2013/03/28 17:01:37.085 SIGNALING HIGH ACTIVE

Failure Response Measurements:
408: 2
487: 4

+++ 2013/03/28 17:02:37.085 SIGNALING HIGH ACTIVE

Failure Response Measurements:
428: 2

+++ 2013/03/28 17:09:49.885 SIGNALING HIGH ACTIVE 

Failure Response Measurements:
487: 3

+++ 2013/03/28 18:44:48.926 SIGNALING HIGH ACTIVE

Failure Response Log:
Function Trace:
1188ce09 107f9f0f 11845800 11818639 11816b3c 118069cf
11807613 1075d735 11849f10 11848b58 11afb295 f7f4384e
f7d970aa

Response Code: 487
Request: 
<several lines of data here>

+++ 2013/03/28 19:41:28.369 SIGNALING HIGH ACTIVE

Failure Response Log:
Function Trace:
1188ce09 107f9f0f 10799a15 10797f42 107f6607 1073b770
11afb295 f7f7f84e f7dd30aa

Response Code: 408
Request:
<several lines of data here>

So for instance I have been trying to use:

sed -n '/\+\+\+/,/\+\+\+/ p' <logfile.log> |grep 487

This of course doesn't work. My desired result would be to search for "487" and get that line and all the lines that exist between the preceding datestamp and the following datestamp so that my results would look something like this:

Example desired output:

+++ 2013/03/28 17:01:37.085 SIGNALING HIGH ACTIVE

Failure Response Measurements:
408: 2
487: 4

+++ 2013/03/28 17:09:49.885 SIGNALING HIGH ACTIVE 

Failure Response Measurements:
487: 3

+++ 2013/03/28 18:44:48.926 SIGNALING HIGH ACTIVE

Failure Response Log:
Function Trace:
1188ce09 107f9f0f 11845800 11818639 11816b3c 118069cf
11807613 1075d735 11849f10 11848b58 11afb295 f7f4384e
f7d970aa

Response Code: 487
Request: 
<several lines of data here>

Any insight would be appreciated.

2 
awk '
{
        R = ( R == "" ? $0 : R RS $0 )
        if ( $0 ~ /487/ )
                ++v
        if ( $0 ~ /^\+/ )
        {
                if ( v == 1 )
                        print R
                v = 0
                R = RS $0
        }

} ' logfile
3

It is relatively easy: read the log file in parts, where "part" is from one timestamp to the next. Accumulate these parts in the hold space. When you find a time stamp you finish the last part read in: exchange hold space and pattern space, check if the last part contains the string you searched for and print it if this is the case.

The following script which does that assumes a POSIX-compatible "sed", if you use a GNU-sed you might have to escape the "+":

sed -n '/^+++/ {
                  x
                  /<searchpattern here>/p
                  d
              }
        H' /path/to/input

I hope this helps.

bakunin

4

Try with GNU awk (or mawk):

gawk '/487/{print "+++" $0}' RS='\+\+\+' ORS= file

With regular awk you could try

awk '/487/{print "+++" $0}' RS=+ ORS= file

But that would break if there is so much as a single + -sign in the record.
You could try replacing the +++ with a single character first if you know for sure that it will not occur in the text.

sed 's/^+++/|/' file | awk '/487/{print "+++" $0}' RS=\| ORS=

Anyway, probably best to use something like this:

awk '/487/{f=1} /^\+\+\+/{ if(f)print s ; f=0; s=$0; next } {s=s RS $0} END{ if(f)print s }' file

--
@Bakunin: very nice, but it would not work for the last record in the file.
@Yoda: also would not work for the last record and it prints the next "+++ header"..

1 Like
5

Are you stuck with the log file format?

I think it would be a lot simpler to process, and the resulting scripts easier to maintain, if you had a marker (### or whatever you might choose) for the end of each time stamp record.

+++ 2013/03/28 17:01:37.085 SIGNALING HIGH ACTIVE

Failure Response Measurements:
400: 1

###

+++ 2013/03/28 17:01:37.085 SIGNALING HIGH ACTIVE
6

Thanks for the help and nice catch on the others not grabbing that last line. This seems to be working for me, so I can build around it.
Appreciate the help!

Ray