I am running a program written in perl script and it is stopped with "Out of memory!" error. This is very strange because at the time then the program is aborted, it only used 4GB RAM and there are still 30GB free physical memory left in the system. I check the perl script and found the program stopped at the place when it tried to sort some string set. Look at this:
"foreach my $string (sort keys %{$stringSet->{'strings'}}) {"
where the stringSet is composed of 4731718 string and the length of each string is around 30 bytes.
Do you think the Out of memory issue is due to the sort function?
if yes, then how does the sort in perl is implemented and how can I rewrite it?
Thanks,
---------- Post updated at 05:37 PM ---------- Previous update was at 05:20 PM ----------
Some more details FYI:
The program calls sort function because it got the data and tried to sort them and then store them to an xml file.
Here is the prstat and vmstat result when the program stopped:
prstat -a |grep <this program>
15901 root 3949M 3517M cpu13 50 0 1:12:26 6.2% <this program>./1
it shows the program used 3517M RAM when it is dead;
note this machine has 16 CPUs and this program used 6.2% of them;
vmstat 5 5
Vmstat:
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr s0 s1 s2 s3 in sy cs us sy id
0 0 0 24500544 28493564 717 5122 0 0 0 0 0 0 0 0 0 27693 22769 32615 8 5 88
it shows the machine in which the program run still had 28.49GB free memory left when the program is dead;
this is the result of ulimit -a:
# ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 10
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 16357
virtual memory (kbytes, -v) unlimited