Awk Performance Issues

UNIX for Beginners Q & A UNIX for Dummies Questions & Answers
1

Hi All,

I'm facing an issue in my awk script.
The script is processing a large text file having the details of a number of persons, each person's details being written from 100 to 250 tags as given below:
100 START|
101klklk|
...
245 opr|
246 55|
250 END|
100 START|
...
245 pp|
246 65|
250 END|
In the code it writes each individual person's info into a temporary file and then process it for customization

We increased the tags from 250 to 450 leaving almost 200 tags blank.
like from 245 tag to 250 where moved to 445 to 450.(tags from 245-444 will not be coming in the file for now)
But after doing this rearrangment awk script has performance issues of almost double the time. So did changing these tags/identifiers degraded performance?

2

If I'm reading this right, you roughly doubled the length of the file you're processing, and now it takes roughly twice as long...

Sounds about right to me...

If you could post your script, or at least the core sections I'm sure someone could assist in some performance tuning...

3

Hi ,

Thanks for the update.
We found out the issue, its not due to the tag rearrangment.
But due to the removal of the temporary files in the code like given below:
{if(system("test -r fact_consbill")==0){system("rm fact_cc")}}
{if(system("test -s fact_consbill")==0){system("rm fact_bon")}}

But if we interchange this code then the code is taking the same as the old one, so is there any impact on the test variable regarding the names?

Thanks

4

instead of spawning a next shell with "test -r", use this function and check for the return value:

# Function to test for the existance of a file
#
function fileExists(file,   dummy,ret)
{
  ret=0;
  if ((getline dummy < file) > 0) {
  # file exists and can be read
      ret = 1;
      close(file);
  }
   return ret;
}
5

Hi,
Changing the removal using getline certainly helped in performance.
Thanks a lottt :slight_smile:

Thanks
P