Processing Multiple Files

Shell Programming and Scripting
1

Hello Everyone,

I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this.

I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this
177.994 847.9
178.996 858.0
179.996 847.8
180.992 851.2
.
.
.
I want to do the following with these files.

  1. Separate the second column, so I only have the Throughput column.
  2. Calculate the Max/Min/Average and Standard Deviation of that Column.
  3. Save the Min, Max, Average, SD in a separate file with the FileName from which they were calculated

The file names are like this

p1-x20-y0-w
p1-x20-y0-wo
p1-x20-y10-w
p1-x20-y10-wo
p1-x20-y20-w
p1-x20-y20-wo
.
.
.
values with 'y' change from 0 to 100 in steps of 10
values with 'x' change from 20 to 40 in steps of 10

I have to submit a project report and tons to data to process before that. Can anybody help me with that?

Thanks,
Shahjehan

2

Which part are you having trouble with? Show us your code so far.

Processing a large number of files is no problem, you can just use a shell for loop around it like:

for f in p1*
do
    awk 'your awk script' $f > $f.awked
done