Problem assigning a counter for particular pattern

Hi,
I have a script that compares two files(which are updated dynamically by a daemon) and evaluate results from the comparision.
For the first line of comparision from the file1, i will grep some part of the line in file with file1 and set a counter for that particular comparison. So for each and every line there should be a counter variable assigned and set.

So now the first run of script ends... and the next run starts.
For the next run, the script may find some more patterns from file and will grep for that pattern in file1 and set a counter for the new patterns...., above process continues.

Here, I have a problem storing a counter variable for a particular pattern. If script matches the particular pattern on its next run, it should increment the counter assigned for that pattern - else it should leave the counter for that pattern as it is.

This is what i have tried:

#!/bin/bash
file="/home/reddybs/test/failures.log"
file1="/home/reddybs/test/failures.log1"
file3="/home/reddybs/test/pc.chk"
ff=0
while read LINE
do
 
         i=`echo $LINE | awk '{print $4,$5}'`
#echo $i
 
         grep -w "$i" $file1 >>/dev/null
 
         if [ `echo $?` -eq 0 ]
         then
                echo "matched"
#Please guide me to assign a counter for this particular pattern               
         ff=`expr $ff + 1`
                echo "$ff ,Matched count for $i" >>$file3
         else
                echo "Yet to implement"
         fi
 
done < $file

Struk in assiging/set a counter variable for the particluar pattern/match/comparision/loop.

Thanks in advance for all your time.....

Cheers,
Sai

what problem you are facing in you code?

Hi anchal: Thanks for the reply...

Problem in assiging a counter for particular match.

I would be happy if you can read my post again. The script posted in the post doesn't have any problems but it is not that intelligent to assign/store a counter for the particular macth.

Hope you are getting me!!!

Basically, to "transfer" a value from one script (run) to another, you may want to store the value in question to a plain text file, e.g. by updating the "intro" of your script approximately as follows:

if [ -f value.data ]
then
  VALUE=$( more value.data ) && rm value.data
else
  VALUE=0 # for better understanding only ;-)
fi

Hi dr.house....
Thanks a ton for your response...

Yes - your logic is fair enough to implement... I've tried some thing related to that.....! But this works only if script finds the same pattern in the next run...

Let me elaborate things...

Once the comparision is done, i store that in a file(file4 in script) and for the next time the script runs, If the pattern matches, i'll do a grep of the pattern and do a wc -l of the pattern on file4 and print that output to the file that i want.

So know the first part of IF works fine i.e assigning a counter. I will have to reset the counter - so what ever the entry that goes into file4 for the nth run, should be removed from the file4 if it n+1th run doesn't find it/them...

How can this be achieved????

Any different logic is also accepted with great pleasure and i'd give it a try definetly.

Script:

while read LINE
do
        i=`echo $LINE | awk '{print $4,$5}'`
        grep -w "$i" $file1 >>/dev/null
        if [ `echo $?` -eq 0 ]
        then
#               echo "matched"
                echo "$i" >>$file4  -----> stroe in some file
                ff=`grep "$i" "$file4" | wc -l` --------> get the count
#               echo "$ff ,Failure count for $i" >>$file3
                echo "FF $ff, $LINE" >>$file3
                echo "$LINE" >>$file1
        else
                echo "$LINE" >>$file3
                echo "$LINE" >>$file1
        fi
done < $tmpfile1
 

Cheers,
Sai

#create the following file before yor script runs

touch count_file

Type the following after the line "#Please guide me to assign a counter for this particular pattern "

var=`grep -w "$i" count_file`
if [ `echo $?` -eq 0 ]
then
grep -v "$i" count_file >> temp
mv temp count_file
new_count=`grep -w "$i" $file1 | wc -l | tr -s " " " " | cut -d " " -f3`
echo $i" "$new_count >>count_file
fi

Maybe you should post a sample data for each file and required output.
echo , awk and grep don't look nice, I think that awk can do all in one shot.

@girish: Thanks for the reply. I am trying to understand your logic and get back to you once done.

@danmero: I really dont know how to put forward this in posts and as i felt this is bit tedious to explain through posts, i am pasting what actually i am trying to do.
This is what i am trying to do...
I have a file(mainfile) with the below format:

Date:Warning: client clientname1, grp grpouname1, sometext
Date:Warning: client clientname2, grp grpouname2, sometext
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname4, db dbname4,(sometext)
.
.
.
.

I am writing a script which will run for every 5 min read the content of main file and update a file(pcfile) to contain the client name (entire name as in mainfile ) along with the count of how many times the line has been repeated.
i have divided the script into following stages

1. Check the mainfile every time the script runs and store only the updated lines in that. Say this file as tmpfile.
   2.I will compare each and every line in the tmpfile with the file called failedfile. Comparison will be done with the below format:
   clientname1, grp grpouname1
   clientname2, db dbname2

about failedfile:
----------------
failed file contains the filtered entries of mainfile. For the first run of script it will be empty.
Comparision starts:
3.
a. If the first comparision(line1) is positive, i will assign a counter,increment it and send the line with counter to pcfile. Keep in mind that each and every line is a different comparision and requires a seperate counter. Now append the line to pcfile and failedfile. For the next time the script runs, if the script doesn't find this line in tmpfile, the counter should be reset.
b. else, append the line to pcfile and failedfile.
4. Now remove the tmpfile.
----Script ends-----

Right now i am struggling with 3rd point.

Cheers,
Sai

Time to replace the beforementioned, single value by an array of values, then ...

Hi dr.house....!

Me just in!!!! Thanks for the reply.

Can you guide me how to start with!!! Mean while i will be going through 'arrays' in shell scripting - which is a bit new to me.

Many thanks,
Sai

I am also having troubles to get what you want but could something like this be a start?

Here comes the example file:

Date:Warning: client clientname1, grp groupname1, sometext
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname4, db dbname4,(sometext)
Date:Warning: client clientname1, grp groupname1, sometext
Date:Warning: client clientname4, db dbname4,(sometext)
Date:Warning: client clientname4, db dbname4,(sometext)
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname1, grp groupname1, sometext
Date:Warning: client clientname1, grp groupname1, sometext
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname1, grp groupname1, sometext
Date:Warning: client clientname4, db dbname4,(sometext)
Date:Warning: client clientname1, grp groupname1, sometext
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname4, db dbname4,(sometext)
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname3, db dbname3,(sometext)
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname2, grp groupname2, sometext
Date:Warning: client clientname3, db dbname3,(sometext)

Counting the occurence per client:

awk -F" |," '{_[$3]+=1} END{for(a in _){print a,_[a]}}' infile
clientname2 7
clientname3 8
clientname4 5
clientname1 6

Now you would like to compare this with a file you've written 5 mins ago?

Hello Zaxxon,

Many thanks for the reply.

Yes, i will now need to compare this with the updated contents of the file and check if the pattern has occurred again or not and accordingly i should increment/decrement the counter.

I will try again and explain things in a better way so that it can be understood properly - in the next post ASAP.

However, many thanks for your time.

Cheers,
Sai

The command that Zaxxon suggested worked out. I have modified some parts of my script and embedded this to get the problem resolved.

@Zaxxon: Many thanks for your reply.

@Others: My heart full thanks for all those whose supported me regarding this. I don't remember all the names, but i hope my thanks should reach you with this FINAL POST.

and Finally, to UNIX.COM for this excellent forum. Really - I love this.

Regards,
Sai