Last occurrence of code between two tags

Shell Programming and Scripting
1

Hi, I am new to linux and have a challenge while I am debugging my application logs n linux boxes.

our log file xxx.log will have different responses coming in its way while an user logs in. Each response might be of 2000 lines or more. Many users do login at a time and our log file goes big and big.

What command or script should I write to take last occurrence of particular response of one user. In the following example what should I do to take last occurrence of response2 of user1 ?

I am not interested in running less command and copy from the screen. could you guys suggest me with a command or script ?

Responses in logs are like below:

<response1>
1000 lines or more data 
<userId>user1<userId1>
1000 lines or more data
</response1>

<response2>
1000 lines or more data 
<userId>user1<userId1>
1000 lines or more data
</response2>

<response1>
1000 lines or more data 
<userId>user2<userId1>
1000 lines or more data
</response1>

<response3>
 1000 lines or more data 
 <userId>user1<userId1>
 1000 lines or more data
 </response3>

<response2>
 1000 lines or more data 
 <userId>user2<userId1>
 1000 lines or more data
 </response2>

<response1>
  1000 lines or more data 
  <userId>user3<userId1>
  1000 lines or more data
  </response1>

so on ...

Thanks in advance,
Narayana.V

2

If you have GNU awk, you could try this:

tac xxx.log  | awk '/user1/{print; exit}' RS="</*response[0-9]*>" | tac
3

Hi Mirni, Thanks for replying !

How do I know if I have GNU awk or not ?

I am getting the following exception when I run the command that you have given me bash: tac: command not found

is there any awk or sed command that I can execute to get the work done ?

Thanks
Narayana.V

4

You find your version of awk by typing

awk --version

What system are you on?
Try:

awk '{a[i++]=$0}END{for(j=i-1;j>=0;j--) if(a[j]~/user1/) {print a[j];exit} }' RS="</*response[0-9]*>" xxx.log
5

Thanks again Mirni !

I am working on Solaris.
awk --version is not working to get the version.

your awk code ran and given me some random one line output from the log, it has not given me the output that I required. This line is not part of the response also.

do you have anything to suggest ?

6

On Solaris it is best to use /usr/xpg4/bin/awk, but that is not gawk (GNU awk)..

See if this works:

/usr/xpg4/bin/awk '/<userId>user1<userId1>/&&/<response2>/{p=$0}END{print p}' RS= infile

-or-

/usr/xpg4/bin/awk '/user1/&&/response2/{p=$0}END{print p}' RS= infile
7

Thank You Scrutinizer !
First command is having syntax error.

Second command worked partially as it has not printed the complete response2 of user1 where it missed to have part of response below and also it printed some part of response1 of some other user on top ..

It has given me the output like below

some 100 lines from response1
<response2>
got till
<userId>user1</userId>
-- some lines from response2
-- missed most of the lines here ..

can you pls suggest any changes ?

Thanks
Narayana.V

8

Scrutinizer's solutions are elegant and correct for the input you posted.
What syntax error are you getting from the first command? Awk should tell you where the syntax error is:

$ awk '/user1/& &/response2/{p=$0}END{print p}' RS= xxx.log
awk: /user1/& &/response2/{p=$0}END{print p}
awk:         ^ syntax error

Notice the caret ('^') in the last line. It points to the position of syntax error.
Are your responses really separated by an empty line?
Can you please post a sample from your real input? (Use code tags).

9

Hi, I am searching for a response xml. As you might know xml tags ends with </.
My user tag starts with <user> and ends with </user> but not with <user>.
I believe, We need to use escape patterns for that. Anyway I am trying with

<userId>user1

as follows which is working without syntax error but couldnt give me the
the result that I wanted.

/usr/xpg4/bin/awk '/<userId>user1/&&/<response2>/{p=$0}END{print p}' RS= infile

For security reasons, I can not paste the sample log from my actual log, but I am pasting the replica of my log with some dummy tags and values in it.

Please check and help me in getting response2 of user1.

I am using correct tags while I am running on actual log, so pls ignore if you find tag used for user is not correct in the following snippet.

    Log line response 1 user user1
    Logs of some other user ... task done by user X 
    Logs of other user tasks done by user Y
    log line user1 doing some task 
    <response1>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response1>
    <reponse2>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </reponse2>
    Log line response 1 user user2
    log line user2 doing some task 
    <response1>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user2</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response1>
    Log line response3 user user2
    log line user1 doing some task 
    <reponse3>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </reponse3>
    
    Log line response 2 user user2
    log line user2 doing some task  
    <reponse2>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user2</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </reponse2>

Thanks for your continuous effort.

---------- Post updated at 11:05 AM ---------- Previous update was at 10:54 AM ----------

Some points to be noted

. There may be some n number of response1 responses and n number of response2 responses of each user.

So I need the last response2 of user1.

I ran the command

/usr/xpg4/bin/awk '/<user>user1/&&/response2>/{p=$0}END{print p}' RS= logForum.txt

on the code snippet (saved in logForum.txt) given in previous post.

It gives me nothing in the output.

10

That's because you didn't copy it correctly from Scrutinizer's post:

/usr/xpg4/bin/awk '/<userId>user1/&&/<response2>/{p=$0}END{print p}' RS=

since your sample input contained <userId> tag but not <user> tag.

The reason why it's not giving you what you want on your real input is because from what i see now, the responses are not separated by empy lines, like you posted before. Try to pre-process the file by inserting an empty line before opening <response[0-9]*> tag and after closing tag:

sed -e 's/.*<response[0-9]*>.*/\n&/; s/<\/response[0-9]*>/&\n/' inputFile | 
/usr/xpg4/bin/awk '/<user>user1<\/user>/&&/response2/{p=$0}END{print p}' RS=
11

I have used correct tags while I ran the commands. Yes I agree, I had given one time as userId and now I am using user. I had given a note on this in my previous post.

I tried to run with your sed command on my log, it still didnt give me any response.

My log now has <user> tag, but not <userId> tag..

Sorry for confusion.

12

In your posted input, there is no 'response2', there is only 'reponse2'. Is this a typo, or is there really "reponse" in the input?

1 Like
13

Ohh.. How did I do that ? Yes Mirni it is typo error. My sincere apologies for inconvenience.
I modified my template log file with the correct tag .. But it is not replacing the empty lines where it is just displaying n before and after response tags .. Pasted the output below.. I think we should better use escape pattern for that as well ..

 Log line response 1 user user1
    Logs of some other user ... task done by user X
    Logs of other user tasks done by user Y
    log line user1 doing some task
n    <response1>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response1>n
n    <response2>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response2>n
    Log line response 1 user user2
    log line user2 doing some task
n    <response1>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user2</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response1>n
    Log line response3 user user2
    log line user1 doing some task
n    <response3>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response3>n
   
    Log line response 2 user user2
    log line user2 doing some task 
n    <response2>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user2</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response2>n

Here is the modified log file

    Log line response 1 user user1
    Logs of some other user ... task done by user X
    Logs of other user tasks done by user Y
    log line user1 doing some task
    <response1>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response1>
    <response2>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response2>
    Log line response 1 user user2
    log line user2 doing some task
    <response1>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user2</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response1>
    Log line response3 user user2
    log line user1 doing some task
    <response3>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user1</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response3>
   
    Log line response 2 user user2
    log line user2 doing some task 
    <response2>
        <abc>
            <ddd>asdfasd</ddd>
            <ggg>4545</ggg>
        </abc>
        <abc>
            <aaaa>4545</aaaa>
            <ggg>0455</ggg>
        </abc>
        <abc>
            <ffff>455445</ffff>
            <ggg>0454545</ggg>
        </abc>
        <gggttrr>
            <aaaa>
                <rtyrry>
                    <weere>
                        <werfwe>serwe</werfwe>
                        <d>1</d>
                    </weere>
                    <sadaf>dddfdf</sadaf>
                    <user>user2</user>
                </rtyrry>
            </aaaa>
            <werwe>werwe</werwe>
            <werwew>fasdsadfa</werwew>
        </gggttrr>
        <dddd>
            <wertty>sdfadfss</wertty>
        </dddd>
        <dddd>
            <wertty>Paasdf</wertty>
        </dddd>
        <dddd>
            <wertty>asdfasd</wertty>
        </dddd>
    </response2>
14

If there are no empty lines, you need something more complicated like this, or you can preprocess first like mirni suggested...

/usr/xpg4/bin/awk '$0~"<"resp{sect=1} 
     sect{p=p (p?RS:x) $0} 
     $0~"<user>"user && sect{found=1} 
     $0~"</"resp{if(found)n=p;found=0;p=x;sect=0}
     END{print n}' resp=response2 user=user1 infile

-or-

/usr/xpg4/bin/sed '/<\/response/G' infile | 
/usr/xpg4/bin/awk '/<user>user1/&&/response2>/{p=$0}END{print p}' RS=
1 Like
15

Perfect ! Both commands are working perfect for the example that I posted. \

But it is not working if I change the script to my actual.

Some useful points about my real logs..

  1. Responses can have some names like ListABCRes, ABCRes, DDARes, etc..
    where in example log I gave you response1,response2 etc ..

To get ListABCRes, I changed your command as below

ddddddd$ /usr/xpg4/bin/awk '$0~"<"ListABCRes{sect=1}
/admin/home/abc>      sect{p=p (p?RS:x) $0}
/admin/home/abc>      $0~"<user>"user && sect{found=1}
/admin/home/abc>      $0~"</"Res{if(found)n=p;found=0;p=x;sect=0}
/admin/home/abc>      END{print n}' resp=ListABCRes user=narayana12345 infile

I am getting the error like below

/usr/xpg4/bin/awk: line 3 (NR=8499138): Record too long (LIMIT: 19999 bytes)

So, I thought I should need to filter something and so I changed like below

ddddddd$ /usr/xpg4/bin/awk '$0~"<"ListABCRes{sect=1}
/admin/home/abc>      sect{p=p (p?RS:x) $0}
/admin/home/abc>      $0~"<user>narayana"user && sect{found=1}
/admin/home/abc>      $0~"</"Res{if(found)n=p;found=0;p=x;sect=0}
/admin/home/abc>      END{print n}' resp=ListABCRes user=12345 infile

it is still having the same error.

Responses are big and There are too many responses for one user and too many same responses for different users say for example many users can have ListABCRes response and one user can have ListABCRes in the logs many times.

So do you suggest anything on this ?

16

Mirni or Scrutinizer,

Thanks for your continuous effort in helping me.

Do you have anything for me to do to rewrite the command to work on my real logs ?

17

You did not modify Scrutinizer's code well. His code was as follows:

/usr/xpg4/bin/awk '$0~"<"resp{sect=1} 
   sect{p=p (p?RS:x) $0} 
   $0~"<user>"user && sect{found=1} 
   $0~"</"resp{if(found)n=p;found=0;p=x;sect=0}
   END{print n}' resp=response2 user=user1 infile

This is setting a variable 'resp' to have the value 'response2', and variable 'user' to a string 'user1'. Do not change the code inside the single quotes, just change the "response2" to what is your response actually named. If you are hunting a section

<ListABCResp>
...
</ListABCResp>

, with user "user1", then you'd do:

/usr/xpg4/bin/awk '$0~"<"resp{sect=1} 
   sect{p=p (p?RS:x) $0} 
   $0~"<user>"user && sect{found=1} 
   $0~"</"resp{if(found)n=p;found=0;p=x;sect=0}
   END{print n}' resp=ListABCResp user=user1 infile

However, I don't see why you would be getting "record too long" error... the code is appending to an internal variable. Have you tried to preprocess the file with the sed command and then pipe it to awk as suggested before?

18

Mirni,

It is still getting me record too long exception with the following code as you suggested.

/usr/xpg4/bin/awk '$0~"<"resp{sect=1} sect{p=p (p?RS:x) $0} $0~"<user>"user && sect{found=1} $0~"</"resp{if(found)n=p;found=0;p=x;sect=0}END{print n}' resp=ListABCResp user=user1 infile

I tried to preprocess with sed , then also it is giving me the same exception

Record too long (LIMIT: 19999 bytes)

sed -e 's/.*<ListABCResp[0-9]*>.*/\n&/; s/<\/ListABCResp[0-9]*>/&\n/' infile | /usr/xpg4/bin/awk '$0~"<"resp{sect=1} sect{p=p (p?RS:x) $0} $0~"<user>"user && sect{found=1} $0~"</"resp{if(found)n=p;found=0;p=x;sect=0}END{print n}' resp=ListABCResp user=user1 infile

getting same error with the following code also

sed -e 's/.*<ListABCResp>.*/\n&/; s/<\/ListABCResp>/&\n/' infile | /usr/xpg4/bin/awk '$0~"<"resp{sect=1} sect{p=p (p?RS:x) $0} $0~"<user>"user && sect{found=1} $0~"</"resp{if(found)n=p;found=0;p=x;sect=0}END{print n}' resp=ListABCResp user=user1 infile

I am again noting that one user can have two or more ListABCResp in the logs, Logs can be big, Response can be big and Many users can have ListABCResp responses. Similarly other responses also.

I have no clue how to proceed. Hope you will help me again.

19

I recommend getting GNU awk, it may prove useful in the future also. You can download it from here:
ftp://ftp.gnu.org/gnu/gawk/gawk-3.1.8.tar.gz
and then standard build:

tar xvzf gawk-3.1.8.tar.gz
cd gawk-3.1.8
./configure
make
make install

If you don't have the privilege to install it system wide, just install it in a non-standard directory:

./configure --prefix=/path/to/MyDir
make && make install

---------- Post updated at 06:54 PM ---------- Previous update was at 06:34 PM ----------

Do you have perl?

r=response1 #substitute with the real name
u=user1  #substitute with the real name
sed -e '/.*<'$r'*>.*/ {x;p;x}' -e '/<\/'$r'*>/ {G}' logFile | 
perl -ne '$/="\n\n"; if(/<'$r'/&&/<user>'$u'/){print;}'
20

Hi Mirni,

Downloaded gawk-3.1.8.tar.gz
Extracted in windows machine as tar was not working for me.
uploaded the extracted gawk-3.1.8 folder to my directory in unix machine.
ran the command ./configure make make install

I am getting the following error while installing.

 ./configure make && make install
configure: WARNING: you should use --build, --host, --target
checking for a BSD-compatible install... ./install-sh -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... nawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking for make-gcc... no
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/xpg4/bin/grep
checking for egrep... /usr/xpg4/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for egrep... (cached) /usr/xpg4/bin/grep -E
checking for bison... no
checking for byacc... no
checking whether ln -s works... yes
checking for make-gcc... gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking how to run the C preprocessor... gcc -E
checking whether make sets $(MAKE)... (cached) yes
checking for special development options... no
checking for z/OS USS compilation... no
checking for strerror in -lcposix... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking for AIX compilation hacks... no
checking for Linux/Alpha compilation hacks... no
checking for function prototypes... yes
checking for string.h... (cached) yes
checking whether NLS is requested... yes
checking for msgfmt... no
checking for gmsgfmt... :
checking for xgettext... no
checking for msgmerge... no
checking build system type... Invalid configuration `make': machine `make' not recognized
configure: error: /bin/bash ./config.sub make failed

I have perl installed. When I ran your suggested perl code by modifying like below, I am getting the following error.

r=ListABCResp #substitute with the real name
u=user1 #substitute with the real name
sed -e '/.*<'$r'*>.*/ {x;p;x}' -e '/<\/'$r'*>/ {G}' abc.log  | 
perl -ne '$/="\n\n"; if(/<'$r'/&&/<user>'$u'/){print;}'


$ sed -e '/.*<'$r'*>.*/ {x;p;x}' -e '/<\/'$r'*>/ {G}' abc.log | 
perl -ne '$/="\n\n"; if(/<'$r'/&&/<user>'$u'/){print;}'
sed: command garbled: /.*<ListABCResp*>.*/ {x;p;x}

do you have anything to suggest ?