Extract data from file

Dear All ,
I am posting first time in this forum . Please ignore my mistakes .

I am learning Unix and i need help to extract specific data from file .

  1. I want to grep number of fails from log . The file contains "fails" word in line if test cases are failed .

  2. The log contains results for different version . i want to count total number of fails on different version .

  3. The log contains more than 1000 lines and searching fails is very difficult for me .

Example-
version 10--- passed
vesrion 10 --- fails
version 10 -- passed
version9 -- passed
version9 - fails

please help me to write shell scripting

regards
dips

#!/usr/bin/ksh

IFS="
"

for name in $(grep fails tmp1 | cut -d"-" -f1 | sort -u)
do
echo $name failed $(egrep -c "$name.*fails" tmp1) times
done

it is giving me error . Please help me and write full shell script with example
I am sorry i am in learning stage of unix
waiting for reply
regards
dips

That is complete script, you just need to creat a file with tmp1 contains the data... or have whatever file name you want.

#!/usr/bin/ksh

IFS="
"

for name in $(grep fails <filename> | cut -d"-" -f1 | sort -u)
do
echo $name failed $(egrep -c "$name.*fails" <filename> ) times
done

Replace <filename> with the actual file name.

Also please post the error message when you get any error, otherwise we would not be able to help without knowing the error.

getting error as syntax error at line no 5 : $ unexpected ..

pls help me . ..

Onemorething i have several files . i dont want to hardcord in shell script

please give me soultion

:slight_smile: sirs,

just would like to clarify if the version number and the word version is space delimited or not :confused:

if NOT delimited use this code, save it to a file... use file name as arguments

example:

script.sh filename


echo "file ${1} contains `grep -c fail ${1}` fails"

for x in `grep fail ${1} | sort -u | cut -f1 -d" "`
do
	echo "version ${x} contains `grep -c fail ${1} | grep -c ${x}` count"
done

if delimited just change cut -f1 -d" " to cut -f2 -d" "

thanks

Sorry it is not working . i am writing my question again in details .

Example
File name :2006-01-17-test-daily-batch-limo.log

The logs conatins 1000 records row by row . if test cases are failed then it prints Fail(s) at end of row .

I want to grep number of fails .

Please help me to grep this ...

waiting for reply thanks in advance

can you post the first 20 lines from the log file with few failure messages in it... then i can give you exact script

2006-01-17 09:32:39 #### Testsuite started (plugin: sicat)
2006-01-17 09:32:40 (devel: 1/526) Running daily/DB/CmdLn/CmdLn1.exp ...
2006-01-17 09:34:40 (devel: 2/526) Running daily/DB/CmdLn/CmdLn2.exp ...
2006-01-17 09:34:41 (devel: 3/526) Running daily/DB/CmdLn/CmdLn3.exp ...
2006-01-17 09:34:42 (devel: 4/526) Running daily/DB/CmdLn/cliCS.exp ...
2006-01-17 09:34:42 (devel: 5/526) Running daily/DB/CmdLn/comma.exp ...
2006-01-17 09:34:43 (devel: 6/526) Error :daily/DB/CmdLn/no_licnse.exp :2 fails
2006-01-17 09:34:43 (devel: 7/526) Running daily/DB/CmdLn/parfile.exp ...
2006-01-17 09:34:44 (devel: 8/526) Running daily/DB/CmdLn/range.exp ...
2006-01-17 09:34:45 (devel: 9/526) Error : daily/DB/Filter/Filter1.exp :1 fails
2006-01-17 09:34:46 (devel: 10/526) Error : daily/DB/SFM/SFMDoublePointHole.exp : 5 fails

2006-01-17 09:34:47 (devel: 11/526) Running daily/DB/SFM/SFMDoublePointHole2.exp ...

2006-01-17 09:34:48 (devel: 12/526) Running daily/DB/SFM/SFMDoublePointHull.exp ...
2006-01-17 09:34:49 (devel: 13/526) Running daily/DB/SFM/SFMDoublePointHull2.exp ...
2006-01-17 09:34:50 (devel: 14/526) Running daily/DB/SFM/SFMSelfIntersection.exp ...
2006-01-17 09:34:51 (devel: 15/526) Running daily/DB/SFM/SFMSpeedAllAngleFlat.exp ...

i posted the samle log file . i have to count number of test modules failing for devel version . ( count as 1 record as 1 fail ).
I dont want to sum number of test cases failuers . i want to count total number of fail record .

The log contains data for differnt version .

I need output like this

Version : Devel
Total Fail : 15

Version : 9
Total Fail :10

Please help me

mahendraji pls reply me waiting for ur reply

Please remember the rules when posting to forums. For instance:

(4) Do not 'bump up' questions if they are not answered promptly. No duplicate or cross-posting and do not report a post where your goal is to get an answer more quickly.

Cheers,

Keith

sorry keith i wont repeat this mistake

deleted....

thanks a lot for ur reply and help ..
but i am getting one problem .. i am getting records but not count of all records ..

the fails records are filterd but not sum ..

one more thing i dont want to sum number of fails ...

i want to add number of failuers one record is one fail doesnt matter how many fails ..

pls do needful

$ more logfile
2006-01-17 09:32:39 #### Testsuite started (plugin: sicat)
2006-01-17 09:32:40 (devel: 1/526) Running daily/DB/CmdLn/CmdLn1.exp ...
2006-01-17 09:34:40 (devel: 2/526) Running daily/DB/CmdLn/CmdLn2.exp ...
2006-01-17 09:34:41 (devel: 3/526) Running daily/DB/CmdLn/CmdLn3.exp ...
2006-01-17 09:34:42 (devel: 4/526) Running daily/DB/CmdLn/cliCS.exp ...
2006-01-17 09:34:42 (devel: 5/526) Running daily/DB/CmdLn/comma.exp ...
2006-01-17 09:34:43 (devel: 6/526) Error :daily/DB/CmdLn/no_licnse.exp :2 fails
2006-01-17 09:34:43 (devel: 7/526) Running daily/DB/CmdLn/parfile.exp ...
2006-01-17 09:34:44 (devel: 8/526) Running daily/DB/CmdLn/range.exp ...
2006-01-17 09:34:45 (devel: 9/526) Error : daily/DB/Filter/Filter1.exp :1 fails
2006-01-17 09:34:46 (devel: 10/526) Error : daily/DB/SFM/SFMDoublePointHole.exp : 5 fails

2006-01-17 09:34:47 (devel: 11/526) Running daily/DB/SFM/SFMDoublePointHole2.exp ...

2006-01-17 09:34:48 (devel: 12/526) Running daily/DB/SFM/SFMDoublePointHull.exp ...
2006-01-17 09:34:49 (devel: 13/526) Running daily/DB/SFM/SFMDoublePointHull2.exp ...
2006-01-17 09:34:50 (devel: 14/526) Running daily/DB/SFM/SFMSelfIntersection.exp ...
2006-01-17 09:34:51 (devel: 15/526) Running daily/DB/SFM/SFMSpeedAllAngleFlat.exp ...
2006-01-17 09:34:45 (prod: 9/526) Error : daily/DB/Filter/Filter1.exp :1 fails
2006-01-17 09:34:46 (prod: 10/526) Error : daily/DB/SFM/SFMDoublePointHole.exp : 5 fails

$ grep fails logfile | cut -d":" -f3 | sed 's/.*(\(.*\)/\1/g' | uniq -c
3 devel
2 prod
$

or this,

awk -F":" ' /fails/ {print $3}' logfile | sed -e 's/^.*(//' | uniq -c

hello

sorry i am getting errors ,, Pls mahendraji reply me .. i need ur help
the script which u wrote for me it is giving records of fails row by row
i need only total count ..

regards

please post the error when you are getting error, also post me the output what you are getting, i could not guess what is going on at your end..

i guess the file format is different from what you have posted above..

i have provided the command based on the file format you have given, if there is any change to that, you need to modify the command accordingly...

initially you gaved one example, i have provided the command based on that..
later you have provided different log file format...

please note that this command would not work for any format, we need to make changes according to the format chat.

the command i have provided above works perfectly for the file format you have provided.

i would suggest you to take out 100 lines and excute the command i have provided above.

if it is not giving the output you expected, post me the 100 lines here and also the output from my command above and also your expected output...

replace urfile by the logfile name u want

Add the following lines in a script, give it execute permissions and execute it.

## grep the lines with word fails, take the last field delimited by :, replace leading
## spaces, sort the output and store it in a tmp file
grep fails urfile|awk -F":" '{printf("%s\n",$NF)}'|sed 's/^ *//g'|sort > /tmp/tmp

## incr a counter till the first field on each line is unchanged, once changed print out
## old valued of the field and counter

awk 'BEGIN {
count=1
fld=$1;
}
{
if ( $1 == fld ) {
count++;
}
else {
printf ("version : %d\nTotal Fail : %d\n\n", fld, count);
fld=$1;
count=1;
}
}
END {
printf ("version : %d\nTotal Fail : %d\n\n", fld, count);
}' /tmp/tmp;