shmat() Failure While Using a Large Amount of Shared Memory

Linux and Unix-Like Linux
1

Hi,

I'm developing a data processing pipeline with multiple stages, with data being moved between the stages using shared memory segments. The size of the data is typically of the order of hundreds of megabytes, and there are typically a few tens of main shared memory segments each of size around 10MB, and a few other shared memory segments of total size less than 1MB.

The problem I'm facing is that after around 600MB has been created in various shared memory segments, further shmat() fails, throwing the error 'Cannot allocate memory'. I tried various combinations of shared memory sizes, and the results are tabulated below.

----------------------------------------------------------
Size of individual segment | Total allocated shared memory
----------------------------------------------------------
      10485760 (10MB)      |      639635428 (~ 610MB)
      15728640 (15MB)      |      644877668 (~ 615MB)
      31457280 (30MB)      |      660605700 (~ 630MB)
----------------------------------------------------------

The system limits regarding shared memory as defined in the directory /proc/sys/kernel are as follows:

shmall = 2097152
shmmax = 33554432
shmmni = 4096

The contents of /proc/swaps is as follows:

Filename      Type            Size    Used    Priority
/dev/sda7     partition       3903752 34792   -1

This program used to work on a different PC which I was using earlier, but not on the current one. I'm running Kubuntu 7.10 on an Intel Core 2 Duo machine with 4GB of RAM.

Can anyone help me figure out where the problem is? Appreciate any help!

Thanks,

Jayanth

2

1, shmat() is deprecated SYSV legacy use shm_open()

  1. shmat() is done by the kernel itself fill a bugreport for the linux kernel mailing list if you thing it is not your program that causes the problem
3

Thanks for the suggestions, grumpf!