I have a lot of processes all of which need to write quite
a lot of data to the filesystem ( to a single file).
This is managed today in the following way : all the processes
write the data to a shared memory block, which is manged by a process that empties it to a file, thus allowing more space for
writing by the other processes.
It is now argued that this is slow and time- consuming,
specifically in times of high load on the shmem resource
(since each read/write is performed via a lock management
facility).
My question is : Will it be faster to dump the whole idea
of the shared memory and use the buffers that the O.S
provudes ? What type of locking will then be necessary ?
You don't give us many details. But if this this like a log file, I would simply use the write(2) system call. If many processes open a file in append mode and then issue write's to it (via the system call directly), the writes will be atomic. Each record will be appended to the file in the order that the writes occur. No additional locking is needed. And the file will be buffered in the buffer cache. The syncer and/or the unix write-behind policy will actually write to disk. This is a very simple solution with low overhead and should be very portable.
I don't understand exactly how you're using shared memory. This is also a viable solution. Most database packages use shared memory and the database vendor compete heavily on performance. But they typically have access to secret os facilities designed just for them.
I have never worked much with threads. But what I have read suggests that maybe they would be an option. The theory goes that when processes lean too much on ipc facilities, they should simply become threads and talk via global data.
Ultimately you should probably try several approaches and benchmark them to find the fastest. And sometimes, if there is a lot of processing to do, there is a lot of processing to do. You can't always wave a magic wand and render intense processing trivial.
I have some doubts about the solution you suggested:
I am using Ada83, and writing to the file using Text_Io
which is the standard way to perform basic IO in Ada.
I have no direct access to the system call used to actually
implement the write (although I can of course import the
system call WRITE and call it directly, which is quite ugly
in Ada terms.
The file is actually not a log file but a binary data file,
but I have a log file (Text file) in the system which is
written to directly with no Shared memory buffering,
and I can see instances where a write operation by a process
was interrupted by a call from another process (the log line
fromm the first is cut in half and in the middle lies the line from
the second process), so there is no automatic locking via this call.
The shared memory is needed, as I can understand, to minimize
access to the file for all the processes involved ( there are many),
and instead supply them with a memory area where they dump the data. A single process them accesses the data and pours it in the file. So no harm is caused to the working processes if the IO
is for some reason damaged or stuck.
I am looking for ideas how to reduce mutual wait problems due
to locking of the shared memory resource, maybe by eliminating it
or fragmenting it ?
About a decade ago I might have been able to help you.
Heck, I haven't even worked on a DoD project in nearly four years...
But just thinking about the problem, can you use a message queue to pass the data to a daemon to write the data at a (on the scale of the microprocessor) later time?
How important is the latency of the data being written?
Sending off a message would free the individual processes fairly quickly and if the writing daemon got behind, it wouldn't affect the individual processes. And because a single process daemon would be doing the writing, you wouldn't have to worry about locking contention.
Just some random thoughts on a lazy Tuesday afternoon.
Auswipe , in working with IPCs 'shared memory' is always the fastest cause it dosen't much interact with the kernel I/O during its operations countary to others which heavily depend on kernel I/O.
In fact data manipulation is the fastest in respect to 'shared memory' than any other IPC you might talk about.
Do we really know if the slowness observed is caused by resource contention (many process contenting one SHMEM) or is caused by resource utilisation (the path length of the code doing the SHMEN access)?
If the actual cause is contention, and the process es appeared slow because they spend most of their time waiting, one of the solutions is to have more resouces (multiple SHEM? and pack them up by tge background process).
I would look to the existing problem in a different way. The problem as stated intially is "formulating a faster technique for transfer of data from Shared Memory to a file" .
I would suggest to divide the shared memory into small segments and allow a pair of thread to read/write from each domain, in sync. Hence multiple threads operates on the data together but at mapped memory address. Each thread pointers should be intialized once to the domain bounday (addresses space). Once done a write thread of each process can write to a specific region incrementing a global resource each for itself. Meanwhile a read thread for each data division conditionally waits for the global variable to reach its max or upper domain limit. Once the event is initiated the write thread should conditionally wait for the variable to be initialized to the lower boud limit meanwhile the read thread should write the data to the file. Hence block I/O would be possible , which I think could be faster than the existing.
But however I do take an assumption that the order in which data has to writen to the file is immaterial.