Calculating cumulative frequency using awk

UNIX for Beginners Q & A UNIX for Dummies Questions & Answers
1

Hi, I wanted to calculate cumulative frequency distribution of my data that involves several arithmetic calls. I did things in excel but its taking me forever. this is what I want to do:

var1.txt contains n observations which I have to compute for frequency which is given by 1/n and subsequently the cumulative frequency (cumfreq).
obs       freq (1/n)    cumfreq
2            1/7              1/7 
3            1/7              1/7+1/7
4            1/7              1/7+1/7+1/7
5            1/7              1/7+1/7+1/7+1/7
6            1/7              1/7+1/7+1/7+1/7+1/7 
7            1/7              1/7+1/7+1/7+1/7+1/7+1/7 
8            1/7              1/7+1/7+1/7+1/7+1/7+1/7+1/7

Any help on this is very much appreciated. Bunch of thanks in advance.

2

I'm assuming you want the evaluated cumulative value and simply not the string "1/7+1/7+1/7...."

perl -lane 'if($. == 1){print; next}else{$x+=eval $F[1]; print "$F[0]\t$F[1]\t$x"}' file
3

Hi balajesuri, thanks for your quick reply. yes, I need the evaluated cumulative frequency. There's a small glitch though, it doesn't print the first row and as such the last cumulative frequency value is a little off to 1.

And also quick question what does $. means? thanks again

4

When you say first row, you mean "obs freq cumFreq", right? If so, the perl code does print this.

In perl, scalar variables begin with a "$" symbol. That's the language grammar. Likewise, perl recognizes an array identifier by "@" as in @myArrVar a hash by "%" as in %myHash .

5

Hi again, Thanks much for the explanation on perl syntax. Anyway, This is the sample output am getting when I run the perl script. the columns correspond to the obs, frequency and cumulative frequency, respectively. the first value in 3rd column should be in the first row so that the last row would be equal to 1. Thanks again for looking into this.

 14.6826892861   0.0333333
 13.1351571538   0.0333333       0.0333333
 13.7802422086   0.0333333       0.0666666
 13.5688246806   0.0333333       0.0999999
 15.0765921598   0.0333333       0.1333332
 13.5727983428   0.0333333       0.1666665
 13.6297192026   0.0333333       0.1999998
 15.8496069483   0.0333333       0.2333331
 12.0589998927   0.0333333       0.2666664
 16.3124475586   0.0333333       0.2999997
 15.1092044636   0.0333333       0.333333
 16.3960654816   0.0333333       0.3666663
 14.3450835037   0.0333333       0.3999996
 14.9916008481   0.0333333       0.4333329
 15.6877187161   0.0333333       0.4666662
 13.2263422247   0.0333333       0.4999995
 16.241350749     0.0333333       0.5333328
 15.2386372531   0.0333333       0.5666661
 15.494540553     0.0333333       0.5999994
 16.0025388812   0.0333333       0.6333327
 13.8140452599   0.0333333       0.666666
 14.7094971241   0.0333333       0.6999993
 14.3596550983   0.0333333       0.7333326
 14.5479239223   0.0333333       0.7666659
 15.1549626225   0.0333333       0.7999992
 14.7984092946   0.0333333       0.8333325
 15.1006025684   0.0333333       0.8666658
 16.2515280804   0.0333333       0.8999991
 13.6803490414   0.0333333       0.9333324
 14.9813933073   0.0333333       0.9666657
6

Ha, ha! Another classic example of changing requirements. Ok, here's some light:

if($. == 1) {print; next} ==> This part of the code checks if line in consideration is line #1 and if so prints the line as-is and proceeds to next line. This is because, from your initial sample in post #1, first line is the header line ( obs freq (1/n) cumfreq ).

Here's one to suit your requirement from post #5:

perl -lane '$x+=eval $F[1]; print "$F[0]\t$F[1]\t$x"' file
7

Thanks again and sorry bout the confusion on the header on the previous post. So, in awk its like NR=1{print}, something to that effect? Thanks again.

8

= is assignment, == is comparison. This is true in both perl and awk.

awk 'NR==1 { print }' would print the first line and only the first line.