How to open large datafie in awk?

Shell Programming and Scripting
1

I just tried

awk '{print}' all.plo 
awk: cannot open all.plo (Value too large for defined data type)

awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}' all.plo 
awk: cannot open all.plo (Value too large for defined data type)

datafile size is 4.8GB

any other provision ? only

cat

works

FS is space as well as tab(mixed)
2

this is usual case with large files and can be worked around with split, which in turn makes it possible to use the full data

#split -b 900m filename prefix
and then
#awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}'  prefix*

should help.

3

What is your OS and version? And what is your version of awk? Are you using a 32-bit version?

4

Certainly a 32-bit awk program; should be upgraded/patched for "largefile" support.
Perhaps the shell has "largefile" support?
Then it helps to open the file in the shell:

awk '{print $8"-"$7"-"$6,$9,$4,$5,$12,$15}' < all.plo
1 Like
5

Its Ubuntu 12.04 LTS 32-bit and awk is 1.2 default

I tried #4 solution its working Thank you all