Help nawk change external variable

Shell Programming and Scripting
1

Hello,

I have external variable rownumber, I am processing files within a loop and I would like to keep incrementing rownumber. What is happening is inside nawk section it passes rownumber but it never gets updated. I want to update rownumber inside the nawk, I would appreciate your help

rownumber=0
for FILE in $FILELIST
do
    filename=`basename $FILE`
    curvename=`echo $filename | awk '{FS="."}{print $1}'`
    rundate=`echo $filename | awk '{FS="."}{print $2}'`
    echo $curvename
    nawk '
    FNR==1 {
        numheadrecords = $1;
        rowstoprocess  = numheadrecords + 2;
        next;
    }
    FNR<rowstoprocess {
        RCount = RCount + 1;
        julianmdate = $1;
        rate        = $3;
        mdate      = $4
        printf "%s,%s,%s,%s,%s,%s\n", LowFile, RDate, mdate, julianmdate, rate, RCount;
    }
    ' LowFile=$curvename RDate=$rundate RCount=$rownumber $FILE >>$OUTPUTFILE
done
2

awk doesn't have access to shell variables, you'll have to store it somewhere.

 awk '

...

END {        print RCount > "/dev/stderr" }' filename >> outname 2> varfile
VAR=`cat varfile`

You could also just do everything inside awk, and wouldn't need to worry about getting variables into and out of it.

3

Hi,

Thanks for your reply. In my case If you see I do not have an END, so when would I know that nwak is done processing the rows and I should store RCount in a file.

I am assuming initially I will have to store RCount as 0 in varfile. Inside nwak I have to read it from file keep incrementing and thne once done I write it out to the file?

Please advise how should I accomplish it, I am new to Unix scripting so I would really appreciate your help.

4

That's okay, just add the END { } block after the other blocks. It should start on its own line, just like all the other blocks you have in that awk statement. END is a special block that gets run when awk runs out of input.

Nope, you don't. Just do what I showed you.

Nope, you don't, just do what I showed you.

Sequence of events is:

  1. Shell variable copied into awk
  2. awk finishes processing data, calls END {} block, prints new value into stderr, which is saved to varfile.
  3. awk quits.
  4. shell reads varfile back into the variable.

Awk and the shell are separate so awk can't affect each others' variables. So awk just prints the variable into a file when its done for the shell to read and update itself with.

5

Hi,

I still dont exactly follow. Will it be something like follows?

nawk '
    FNR==1 {
        numheadrecords = $1;
        rowstoprocess  = numheadrecords + 2;
        next;
    }
    FNR<rowstoprocess {
        RCount = RCount + 1;
        julianmdate = $1;
        rate        = $3;
        mdate      = $4
        printf "%s,%s,%s,%s,%s,%s\n", LowFile, RDate, mdate, julianmdate, rate, RCount;
    }
    END { print RCount > "/dev/stderr" }' filename >> outname 2> varfile
VAR=`cat varfile`}
    ' LowFile=$curvename RDate=$rundate RCount=$rownumber $FILE >>$OUTPUTFILE
6

You're taking my example way too literally. "filename >> outname 2> varfile" was outside the awk statement I showed you and certainly shouldn't be inside yours. (The > "/dev/null" belongs inside, though.)

The VAR=`cat varfile` is not part of the awk statement, it happens after. It reads varfile into VAR inside the shell, getting back the value awk saved when it quit. I don't know how you ended up with the } after it. Again it's not meant to be taken literally -- you wanted rowcount, not VAR.

Come to think of it a faster way to do it is read rownumber <varfile

So:

rownumber=0
for FILE in $FILELIST
do
    filename=`basename $FILE`
    curvename=`echo $filename | awk '{FS="."}{print $1}'`
    rundate=`echo $filename | awk '{FS="."}{print $2}'`
    echo $curvename
    nawk '
    FNR==1 {
        numheadrecords = $1;
        rowstoprocess  = numheadrecords + 2;
        next;
    }
    FNR<rowstoprocess {
        RCount = RCount + 1;
        julianmdate = $1;
        rate        = $3;
        mdate      = $4
        printf "%s,%s,%s,%s,%s,%s\n", LowFile, RDate, mdate, julianmdate, rate, RCount;
    }
    END { print RCount > "/dev/stderr" }
    ' LowFile=$curvename RDate=$rundate RCount=$rownumber $FILE >>$OUTPUTFILE 2> varfile
    read rowcount <varfile
done

# delete temporary file when you're done
rm -f varfile
7

Thank you very much, worked like a charm. Appreciate all your patience and help.