HELP !! script not working..

Shell Programming and Scripting
1

Hi all,
I have a piece of code which cork perfect. It copy the no of lines from some files (*.log) into finalLog file.
It read the lines after the line have text "ntot".
Here it is :

#!/bin/bash                                                                                                                                                            

awk '/ntot/{p=lines;next}                                                                                                                                             
p>0{                                                                                                                                                                   
       split($0,arr,"=")                                                                                                                                               
       if(!h[arr[1]"HDR"])h[arr[1]"HDR"]=arr[1]                                                                                                                        
       a[arr[1]]=a[arr[1]]" "$NF                                                                                                                                       
       if(j<lines) b[++j]=arr[1]                                                                                                                                       
       sum[arr[1]]+=$NF                                                                                                                                                
       p--                                                                                                                                                             
}                                                                                                                                                                      
END{                                                                                                                                                                   
       for(i=1;i<=j;i++)                                                                                                                                               
               print h[b"HDR"]"="a[b]" "sum                                                                                                                
[b]}' lines=1 *.log > finallOG

I modified the script to make it more flexible and so could provide the information of
1.) lines
2.) TEXT
3.) finalLOg file

through terminal. but it is not working. Can anybody suggest what wrong in the new script. Find the new script here:

#!/bin/bash                                                                                                                                                            
TEXT=$1
LINES=$2
FINAL=$3

awk '/$TEXT/{p=$LINES;next}                                                                                                                                            
p>0{                                                                                                                                                                   
echo 'working'                                                                                                                                                         
     split($0,arr,"=")                                                                                                                                                 
       if(!h[arr[1]"HDR"])h[arr[1]"HDR"]=arr[1]                                                                                                                        
       a[arr[1]]=a[arr[1]]" "$NF                                                                                                                                       
       if(j<lines) b[++j]=arr[1]                                                                                                                                       
       sum[arr[1]]+=$NF                                                                                                                                                
       p--                                                                                                                                                             
}                                                                                                                                                                      
END{                                                                                                                                                                   
       for(i=1;i<=j;i++)                                                                                                                                               
               print h[b"HDR"]"="a[b]" "sum                                                                                                                
[b]}' *.log > $FINAL

thanks in advance
pooja..

2

Your outer shells vars (TEXT, LINES & FINAL) won't be substituted inside the hard quotes around the awk argument string.

3

hi,
thanks for the reply.
Actually, I am not good with the shell scripting.
could you please elaborate your point?

Thanks
pooja..

4

You should learn how to pass variables to awk

#!/bin/bash
TEXT=$1
LINES=$2
FINAL=$3

awk '{if($0~TEXT){p=LINES;next}}
p>0{
  split($0,arr,"=")
  if(!h[arr[1]"HDR"])h[arr[1]"HDR"]=arr[1]
  a[arr[1]]=a[arr[1]]" "$NF
  if(j<lines) b[++j]=arr[1]
  sum[arr[1]]+=$NF
  p--
}
END{
  for(i=1;i<=j;i++)
  print h[b"HDR"]"="a[b]" "sum
[b]}' LINES=$LINES TEXT=$TEXT *.log > $FINAL

You can also use -v option to pass the variables.

awk -v arg1="hello" -v arg2="how are you" 'BEGIN{print arg1,arg2}'

HTH
--ahamed

5

hi ahamed,
THANKS again...:slight_smile: :slight_smile:

pooja..

---------- Post updated at 09:10 AM ---------- Previous update was at 09:06 AM ----------

Hi ahamed,
I am afraid it is not working..:frowning:

I provided :

./MakeFile.sh ntot 3 finalLOG

but finalLOG is blank file.

Pooja..

6

Please past the sample input file...

--ahamed

7

Hi,
Here is the section of one of the input file:

Error in <TTree::SetBranchStatus>: unknown branch -> genndau
Error in <TTree::SetBranchStatus>: unknown branch -> gennmoth
ntot = 208190
ev_passjson = 208190
ev_goodvtx = 208189
wp85ev = 6021
phoidev= 15
invev = 14
hltev = 14
=======using photon counter ======
tot pho is                 = 12680
nomatch pho is             = 595
phoeta pho is              = 539
nospike_counter is         = 539
et cut passed pho          = 0
ecal cut passed pho        = 0
hcal cut passed pho        = 282
trk cut passed pho         = 106
sieie cut passed pho       = 155

thnaks
pooja.

---------- Post updated at 09:29 AM ---------- Previous update was at 09:21 AM ----------

Hi,
I just faced a problem, I found that finalLOG is having some column showing 0 entries. Which means that log file has the 0 entries.
As there are 60 input files, will it be possible that input file information be written in the top column of finalLOG file.
So that I can trace which log file is having null entries.

If possible? please do the necessary changes.
** using working script.

thanks
pooja.

8 
root@bt:/tmp# cat *.log
Error in <TTree::SetBranchStatus>: unknown branch -> genndau
Error in <TTree::SetBranchStatus>: unknown branch -> gennmoth
ntot = 208190
ev_passjson = 208190
ev_goodvtx = 208189
wp85ev = 6021
phoidev= 15
invev = 14
hltev = 14
=======using photon counter ======
tot pho is                 = 12680
nomatch pho is             = 595
phoeta pho is              = 539
nospike_counter is         = 539
et cut passed pho          = 0
ecal cut passed pho        = 0
hcal cut passed pho        = 282
trk cut passed pho         = 106
sieie cut passed pho       = 155

#!/bin/bash
TEXT=$1
LINES=$2
FINAL=$3

awk '{if($0~text){p=lines;next}}
p>0{
        split($0,arr,"=")
        if(!h[arr[1]"HDR"])h[arr[1]"HDR"]=arr[1]
        a[arr[1]]=a[arr[1]]" "$NF
        if(j<lines) b[++j]=arr[1]
        sum[arr[1]]+=$NF
        p--
}
END{
        for(i=1;i<=j;i++)
                print h[b"HDR"]"="a[b]" "sum
[b]}' lines=$LINES text="$TEXT" *.log > $FINAL

root@bt:/tmp# ./run ntot 3 finalLOG
root@bt:/tmp# cat finalLOG
ev_passjson = 208190 208190
ev_goodvtx = 208189 208189
wp85ev = 6021 6021

--ahamed

9

Hi,
Yeah, I made the mistake. I modified at the different folder and was checking at different place.
If input file information can be added in the first column of the finalLog, please do the changes.

Thanks :slight_smile:
pooja..

10

Try this...

#!/bin/bash
TEXT=$1
LINES=$2
FINAL=$3

awk '{if($0~text){p=lines;next}}
{if(filename!=FILENAME){f[++g]=FILENAME;filename=FILENAME}}
p>0{
        split($0,arr,"=")
        if(!h[arr[1]"HDR"])h[arr[1]"HDR"]=arr[1]
        a[arr[1]]=a[arr[1]]"\t"$NF
        if(j<lines) b[++j]=arr[1]
        sum[arr[1]]+=$NF
        p--
}
END{
        printf "\t\t\t\t"
        for(i=1;i<=g;i++) printf f"\t"
        printf "Sum\n"
        for(i=1;i<=j;i++)
                print h[b"HDR"]"="a[b]"\t"sum
[b]}' lines=$LINES text="$TEXT" *.log > $FINAL

Try to learn from all these and implement...

--ahamed

1 Like
11

Yes, I will..

Thanks again
pooja.