posix ipc message queue

Hello,

My question is related to "pipcs -qa" command under HP-UX 11i PA-RISC 64 bits.

We have a little C program that creates posix ipc message queues using the mq_open() system function.

The program fail with 'No space left on device' error when we create big queues. What is the system limit of the size of all queues ?

If we calculates the sum of all the MNUM*QBYTES columns in the output of 'pipcs -qa' the error arrive when we are close to 860 or 900 Mbytes.

Is there a way to go beyond this amount of memory ?

How the system actually allocate the memory when we create a posix queue of 1024 bytes for a message and for example 100000 as the max number of messages ? Are the 1024 * 100000 bytes allocated when the queue is created or at usage time that is when we really put some messages on the queue ?

Is this related to the fact that HP-UX divide his memory space into 4 quadrants of 1GB each ? Is there a workaround ?

I already compiled the program using the 64 bit options of the K&R
style "bundled" compiler, I also tried 'setmemwindow' without success...

I'm not sure if posix IPC message queues are implemented using the mmap() function, if it is, then what is the max. shared memory size of mmap() ?

I also failed to convince users not to use so big queues...

If we cannot go beyond 900 Mbytes for the size of all queues we are planning to use something else like Java JMS or MQseries or a database ...

Thanks for your advice (and for reading this long post),

Cengiz ADANIR

Check this out. I do not know how valid the information is, but it seems to be quite good. From the info, it looks like you can have a default of 16384 bytes in a message queue at one time, and a default of 50 message queues on the system at one time.
The max bytes present on a message queue can be 65535 and the increase in max number of message queues is memory dependent.

Oh and one more thing, please remove the email address from your post. Against forum rules.

Thank you Blowtorch... It seems that all that things are "implementation specific" and related to PA-RISC "strangeley" designed memory management...

man mmap() says that HP-UX may or may not call shmat() to access shared memory... of course this doesn't help me much... and says that the max shareable mem size is 1.75 GBytes. But I still don't know why I cannot go beyond 900 MB... for my posix queues.

The link you give me is related to SYS V style queues not posix queues and is probably not related to our limitations because we have already created more than 200 queues of 256 bytes length and 10 000 or 100 000 messages per queue...

Thank you again...

The language I see is "There may be implementation-dependent limits on the number of memory regions that can be mapped (per process or per system). If such a limit is imposed, whether the number of memory regions that can be mapped by a process is decreased by the use of shmat() is implementation-dependent." That is different than claiming mmap calls shmat. Anyway, shared memory has nothing at all to do with message queues.

According to the mq_open man page: "If attr is NULL, the message queue is created with default attributes - MQ_MAXMSG and MQ_MSGSIZE (defined in sys/mqueue.h) If attr is non-NULL and the message queue mq_maxmsg and mq_msgsize attributes are set to the values of the corresponding members in the mq_attr structure referred to by attr."

Since you're not happy with the default size, you are specifying a different size, right? What values are you using for mq_maxmsg and mq_msgsize?

pipcs -qa gives me somthing like this :

POSIX IPC status as of Thu Apr 27 15:59:08 2006
T MODE OWNER GROUP REFCNT QNUM MNUM QBYTES NAME
POSIX Message Queues:
q rw-rw---- abmd AZZX 1 0 100000 22 /abLVxy
q rw-rw---- abcd ABZY 13 0 50000 256 /abgxy
q rw-rw---- abcd ABZY 3 0 1000 256 /abILxy
q rw-rw---- abcd ABZY 1 0 50000 256 /ab0xy
q rw-rw---- abcd ABZY 1 0 50000 256 /abMD1xy
... etc
q rw-rw---- acc4d AZZX 0 0 1024 256 /abSOxy
q rw-rw---- acc4d AZZX 0 901 1024 256 /ab0Axy
q rw-rw---- acc4d AZZX 0 4 1024 256 /ab10xy
q rw-rw---- abcd ABZY 0 0 100000 14 /abB1xy
q rw-rw---- u91134 ABZY 0 0 100 10 /aby
q rw-rw---- acc4d AZZX 0 5 1024 256 /abITxy
q rw-rw---- abcd ABZY 2 0 100000 14 /abW1xy

And if I try to create a new one, with msg size 32 and queue length 128 :

mqopen 128 32 -c /test_32x128

There is no pb.

...but if I use :

mqopen 1024 10000 -c /test (this also is ok)
mqopen 1024 10000 -c /test2 (this fails ...)
mq_open(): No space left on device

It's ok, if I reduce the number of messages on the queue :
mq_open -b 1024 -m 5000 -c /test2 (this works)

The pb is that we have 12 Go of physical RAM available...

This link http://docs.hp.com/en/32650-90879/ch03s03.html says :
"
MESSAGE [ENOSPC] No space left on device. (POSIX.1 ERROR 28)

CAUSE During a write() function on a regular file, or when extending a directory, there was no free space left on the device.

ACTION Deallocate unneeded disk space.
"

Here the error occurs when we try to create a new queue not when we try to write a message on a queue.

Under Linux kernel 2.6 and probably under FreeBSD 7, posix queues use a "virtual file system" to make permanent the content of the queue between reboot and use shared memory. This is something like a filesystem in memory... in turn shared memory is implemented using ipc sys v shared memory system calls : shmat(), etc ...We already allow max usage os shared mem with kernel params. I try to know what is limiting us now ? Max adressable space for shared mem is 1.75 GB under HP-UX, but we cannot create any queue after that all existing queues use approx. 900 MB...

Of course there is a lot of apps running and sharing the same space of 1.75 GB. vmstat says we still have 407000 free page (if a page is 4096 Bytes), this gives 450 MB available. Of course I don't know if this RAM is contiguous or fragmented... this perhaps explain why we fail with a queue of 10 MB and not 5MB... I don't know if there is a way to unfragment memory without rebooting...

It's very strange that on a system with 12GB RAM, HP-UX fixed the shared memory adress range from 0x8000 0000 to 0xefff ffff... Is this the way they understand virtual memory ? Or perhaps I misunderstand their man pages...

Thank you again for your interest on this problem,
Cengiz ADANIR.

I'm beginning to see why you keep mentioning shared memory and mmap(). It is looking to me like mq_open is creating a file system object (in the current directory?). Could the message be literally accurate? Are you out of disk space?

Yes, it may be a filesystem limit... I will survey more closely the disk space available, probably something like /tmp or /var/tmp... or $TMPDIR... or perhaps the swap space.. must be increased...

I will look to this tomorow...

The OS (HP-UX) shoud write the content of the queues actually in memory to disk. There is no other way to find again this content after reboot...

But where he writes the queues ? ...these things should be documented. I don't want to search in 'The single UNIX Specification version x' and finally learn that it's 'implementation specific'...

Thank you perderabo...