Computing average and standard deviation from multiple text files

Hello there,

I found an elegant solution to computing average values from multiple text files

awk '{for (i=1;i<=NF;i++){if ($i!~"n/a"){a[FNR""i]+=$i}else{b[FNR""i]++}}}END{for (i=1;i<=FNR;i++){for (j=1;j<=NF;j++){printf (a[i""j]/(3-b[i""j]))((b[i""j]>0)?"~"b[i""j]" ":" ")};printf "\n"}}' file1 file2 file3

I tried to modify the code to get the standard deviations along with the averages, but without success.

I have 3 files with two columns and 10 rows each. I'd like to calculate averages and standard deviations and write them into the new file with 4 columns and 10 rows, where first two columns are the averages calculated from 3 files and the last two columns are the corresponding standard deviations.

Here is an example of the input/output files:

input1.txt

8.5607309334335007 9
8.8632036168606803 112
.
.
.

input2.txt

8.3999948195361487 15
8.6903391660951304 166
.
.
.

input3.txt

8.649238358457044 46
9.0008044868119814 365
.
.
.

---

output.txt

8.5  23    0.1  20      
8.8  198  0.1  145  
.
.
.

Could you please help me modifying the code? I am new to awk and to this forum as well. Thanks a lot in advance!

Put this in "script.awk":

{ if (FNR==1) file++
  for (i=1;i<=NF;i++) {
    sum[FNR" "i]+=$i
    count[FNR" "i]++
    data[file" "FNR" "i]=$i
  }
}END{
  for (i=1;i<=FNR;i++) {
    for (j=1;j<=NF;j++) {
      avg[i" "j]=sum[i" "j]/count[i" "j]
    }
  }
  for (f=1;f<=file;f++) {
    for (i=1;i<=FNR;i++) {
      for (j=1;j<=NF;j++) {
        tmp[i" "j]+=(data[f" "i" "j]-avg[i" "j])*(data[f" "i" "j]-avg[i" "j]);
      }
    }
  }
  for (i=1;i<=FNR;i++) {
    for (j=1;j<=NF;j++) {
      printf avg[i" "j]" "
    }
    for (j=1;j<=NF;j++) {
      printf sqrt(tmp[i" "j]/count[i" "j])" "
    }
    printf "\n"
  }
}

Then run:

awk -f script.awk input1.txt input2.txt input3.txt

Check if the resulting numbers are correct using Excel or something like that. I didn't have time to do that.

Works like magic! The numbers are correct and the script itself is very self-explanatory! Thank you very much for your help!!!