problem in using cat and ended up with no space error in aix

Shell Programming and Scripting
1

While doing cat on a large file (3 GB file) , I am getting the no space error in the shell script hugefile.sh.

Eg: for i in `cat hugefile.txt`
do
echo "$i"
done

error: hugefile.sh[3]: no space

Please let me know your thoughts in handling this no space issue.

2

The use of cat is redundant, try to read the file like this:

while read i
do
  echo "$i"
done < hugefile.txt
3

The default shell of AIX is /bin/ksh (ksh88) and it uses an inefficient stack system. When you code the form "for i in <splat>" then ksh attempts to put <splat> on the stack. You're running out of memory. Splat.

However, ksh93 uses a different stack library that is more efficient and should work fine in this case. Other shells (like "bash") are more adept at handling such a large data set.

Wondering why ksh88 is not updated? It's the AIX default shell. All the system utilities, SMIT functions, etc. all use ksh. Changing the behavior even a little can have dramatic consequences.

You might also argue that IBM is simply protective of shell functions, or even just slow, since AIX's ksh93 is fairly far behind the opensource version. Or maybe the community is just that powerful...