awk solution for taking bins

Hi all, I'm looking for an awk solution for taking bins of data set.
For example, if I have two columns of data that I wish to use for a scatter plot, and it contains 5 million lines, how can I take averages of every 100 points, 1000, 10000 etc...
The idea is to take bins of the 5,000,000 points and reduce the density.

$ cat largefile.txt
x        y
1       45
2       46
3       87
4       34
5       36
6       36
7       23
...     ...
5mil    228       

how to take bins every "n" points of y.

Thanks in advance.

I could interpret your request several different ways. Please show us an example of the output you're trying to produce.

Thanks for the response, here is an example:

input

$ cat file.txt
1      3
2      2
3      1
4      10
5      10
6      10
7      25
8      30
9      60

output example 1 (using bins of 3 - average every third point)

2      2
5      10
8      38.3

output example 2 (using bins of 2 - average every second point)

1.5    2.5
3.5    5.5
5.5    10
7.5    27.5

Not sure what the best tool is!

Many thanks,
Torch

I am not sure I understand your question here.
What is a "bin"?
Can you please post your example with a clear explanation.

Hi, I'll try to be more clear with the example.

Thanks for the response, here is an example, i'll focus on just the second column

input

$ cat file.2.txt
3
2
1
10
10
10
25
30
60

The purpose is to reduce the data for an x,y scatterplot, because the file is millions of lines long. Instead of plotting every point, I want to take an average of every "n" number of points, and plot that one number. Bin might not be the correct word, perhaps a "rolling-average"? For example a bin of 3 would break the data down like so:

$ cat file.2.txt
#bin A
3
2         #average all three = 2
1

#bin B
10
10     #average all three = 10
10

#bin C
25
30     # average all 3 = 38.3
60

Output would then be:

2
10
38.3

For the case where bin is 2

$ cat file.2.txt
#bin A
3    # average = 2.5
2

#bin B
1    # average = 5.5
10

#binC
10    # average = 10
10

#bin D
25    # average = 27.5
30

#binE - ignored because only one value
60

Finally, doing this for both the x and y axis (the original file), for bin of 3:

Input:

$ cat file.txt
1      3
2      2
3      1
4      10
5      10
6      10
7      25
8      30
9      60

Output:
2      2
5      10
8      38.8

Many thanks, I hope this is more clear
Torch

awk -v bin=3 ' BEGIN {
                c = 1
} c <= bin {
                ++c
                f += $1
                s += $2
} c > bin {
                printf "%d %.1f\n", f / bin, s / bin
                c = 1
                f = 0
                s = 0
} ' file.txt
awk     '               {Xsum+=$1; Ysum+=$2}
         !(NR%bin)      {print Xsum/bin, Ysum/bin; Xsum=Ysum=0}
        ' bin=3 file.txt
2 2
5 10
8 38.3333

There's nothing foreseen for residual lines at the end, i.e. two orphan lines when bin=3. Pls specify.

Thanks so much that works perfectly!