Calculate Correlation between two fields !

Shell Programming and Scripting
1

Hello,

I request your help with a shell script (awk) that ask for two inputs in order to calculate the correlation of the last rows between two fields ( 3 and 4).

Data:

EC-GLD,1/25/2011,41.270000,129.070000
EC-GLD,1/26/2011,41.550000,129.280000
EC-GLD,1/27/2011,42.260000,127.800000
EC-GLD,1/28/2011,41.940000,127.950000
EC-GLD,1/31/2011,42.380000,129.250000
EC-GLD,2/1/2011,42.580000,129.330000
EC-GLD,2/2/2011,42.650000,129.450000
EC-GLD,2/3/2011,41.990000,129.280000
EC-GLD,2/4/2011,41.170000,131.230000
EC-GLD,2/7/2011,41.300000,131.230000
EC-GLD,2/8/2011,41.650000,132.800000
EC-GLD,2/9/2011,41.900000,132.490000
EC-GLD,2/10/2011,41.400000,132.000000
EC-GLD,2/11/2011,41.240000,132.090000
EC-GLD,2/14/2011,41.200000,132.700000
EC-GLD,2/15/2011,41.010000,133.630000
EC-GLD,2/16/2011,41.040000,133.450000
EC-GLD,2/17/2011,40.540000,134.470000
EC-GLD,2/18/2011,39.690000,134.880000
EC-GLD,2/22/2011,39.660000,136.190000

For example:
----> Correlation 1 : 15 (Last rows)
----> Correlation 2 : 10 (Last rows)

Output

EC-GLD,15,-0.8979
EC-GLD,10,-0.9037

Thanks a lot for your help

2

cor.awk

awk -F, -v c="$1" '
{ d=$1; for(i=c;i;i--) {
    x=x[i-1]
    y=y[i-1] }
    x[0]=$3
    y[0]=$4
}
END { for(i=0;i<c;i++) {
        sx+=x
        sy+=y
        sxy+=x*y
        sx2+=x*x
        sy2+=y*y }
    print d, c, ( c * sxy - sx * sy ) / ( sqrt(c*sx2 - sx*sx) * sqrt(c*sy2 - sy * sy))
} ' OFS=, cor infile

$ ./cor.awk 10
EC-GLD,10,-0.903706
 
$ ./cor.awk 15
EC-GLD,15,-0.897988

Script could now be something like:

echo -e "Correlation 1: \c"
read c1
echo -e "Correlation 2: \c"
read c2
./cor.awk $c1
./cor.awk $c2

---------- Post updated 25-02-11 at 12:44 PM ---------- Previous update was 24-02-11 at 02:32 PM ----------

This is the 2nd time Iv'e done (what now looks like) stats homework for you Carlos, without any thanks or recognition, and to top things off you have been asked several times to put [code] tags around data files and scripts that you post which you just seem to ignore.

Well no more solutions from me sorry!

1 Like