a better way to grep until end of error message, although most seem to be 1 or 2 line

Shell Programming and Scripting
1

My file creates an output log after which includes a few sql queries.
I segregate them into warnings and errors and then get a total count.
The errors' and warnings' lines always start with SQL[0-9]{4}[A-Z] followed by the details of the error.

This is what im doing as o now...

errors=`grep -A 1 -E "^SQL3196N|^SQL3147N|^SQL3148W" $output.log`

This gets me only the first line after a match is obtained.

Some errors have 1 line and others stretch out to 10 lines...
How would do you capture this info for errors and warnings and later on count the number of distinct errors and warnings encountered???

2

Can you please provide a sample of the log file and your requirement as per the sample?

3

heres' a snippet... it's basically a lotta errors like this. The part in bold is what i have to pull out...
This may vary from 1 line to 10 lines.

This is a log of whatever is seen on the screen
.........
......
...
DROP VIEW V_VALUES

DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name. SQLSTATE=42704
....
........
............
........

4

Try this:

sed -n '/^SQL[0-9]\{4\}/p'  $output.log >  errors.txt

If you have an old version of sed:

sed -n '/^SQL[0-9][0-9][0-9][0-9]/p' $output.log >  errors.txt
5

Thanks for that man...
but Iguess that was not a good example. Basically I want to capture the description of the error along with the error and count how many times it occurred.
I want to capture the part in bold

DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:

SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name.
SQLSTATE=42704
blah blah blah....
[blank line] <-all the information upto this blank line and then how many times it appears in the file

6

Here's one way to do it:

$
$ cat input.txt
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
 
SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name. 
SQLSTATE=42704
blah blah blah....
 
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
 
SQL0204N "ITUAM.VW_TT_VALUES_2" is an undefined name. 
SQLSTATE=42705
blah blah blah....
 
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
 
SQL0204N "ITUAM.VW_TT_VALUES_3" is an undefined name. 
SQLSTATE=42706
blah blah blah....
 
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned: success
$
$ perl -ne '{$n++ if /SQL\d{4}[A-Z].*/s}END{print "The error occurred $n times.\n"}' input.txt
The error occurred 3 times.
$
$

HTH,
tyler_durden

7

The output i was looking for :

SQL0204N occoured 3 times
[description] SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name.
SQLSTATE=42704
blah blah blah....

SQL3196N occoured 7 times
[description] SQL3196N The input file was not found
blah blah...
......
.... [The no. of lines varies for this error description]
..
blah blah blah....

how do i get the entire description part?
The only cue is that there is a blank line after the description is over in the output.log file which i am reading.

8

well.... any ideas on how this is done?

9

show more samples of your error file.

10

Here's one way to do it:

$ 
$ cat input.txt
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:   

SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name. 
SQLSTATE=42704                                      
blah blah blah....                                  

DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:   

SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name. 
SQLSTATE=42704                                      
blah blah blah....                                  

DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:

SQL3196N The input file was not found
blah blah...
......
.... [The no. of lines varies for this error description]
..
blah blah blah....

DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned: success
$
$
$ perl -ne 'BEGIN {$/=""} while (/((SQL\d{4}[A-Z]).*)/sg) {$count{$2.":".$1}++}
> END {foreach $key (keys %count){
>        $x = (split(/:/, $key))[0];
>        print "$x occurred $count{$key} times\n[description] $key";
>      }}' input.txt
SQL3196N occurred 1 times
[description] SQL3196N:SQL3196N The input file was not found
blah blah...
......
.... [The no. of lines varies for this error description]
..
blah blah blah....

SQL0204N occurred 2 times
[description] SQL0204N:SQL0204N "ITUAM.VW_TT_VALUES" is an undefined name.
SQLSTATE=42704
blah blah blah....

$
$

HTH,
tyler_durden

____________________________________________________________
"This is your life and it's ending one minute at a time."

11 
use strict;
use Data::Dumper;

open (FILE, 'input.txt') or die "input.txt : $!\n";
my @wanted = ();
my $in_section = 0;          # this is a flag that indicates start of 'data-store'
                                        # '0' indicates 'do not-store-data'
                                        # '1' indicates 'store-data'
my $line = '';
while (<FILE>) {
    chomp;
    if (/^\s*SQL\d{4}[A-Z].*/i) {           # start 'data-store'
        $line .= "$_\n";
        $in_section = 1;                      # set our flag to one
        next;
    }
    if (/^\s*$/ and $in_section) {         # end the 'data-store'
        $in_section = 0;                     # unset our flag
        push @wanted, $line;
        $line = '';
    }
    $line .= "$_\n" if ($in_section);
}
close FILE;
print Dumper(\@wanted);

You may change the regexp patterns to have different start 'data-store' & end 'data-store' criterion
___________
My Perl blog

12

Try this:

awk '$0 ~ /^SQL/{
p=$1;
while ($0 !~ /^ *$/)
{
s=(n=0)? $0:s"\n"$0
getline
};c[p]++;a[p]=s;n=0;s="";next
}END{for(p in a)printf("%s%s%d%s%s%s\n\n",p," occured ",c[p]," times\n","[description] ",a[p])}' input.txt

cheers,
Devaraj Takhellambam