fwrite takes extremely long time

After my previous thread, I think I found out what causes the long delays.

I run this program on several Linux computers, and the sometimes (after the file with the arrays becomes big) the fwrite takes between 100 ms to 900 ms.
This is very bad for me, as I want a timer to halt each 30 ms....

The simplified code (without the timer) is:

#include <sys/time.h>
#include <signal.h>
#include <sched.h>
#include <sys/mman.h>
#include <iostream.h>

int main()
{
FILE* stream = fopen ("log.txt","w");
double data[100][100];
struct timeval tv1,tv2;
struct timezone tz1;
int diff;
for (int i=0;i<10000;i++)
{
gettimeofday(&tv1, &tz1);
fwrite(data, sizeof(double), 100 * 100, stream);
fflush(stream);
gettimeofday(&tv2, &tz1);

  if \(tv2.tv_usec >= tv1.tv_usec\)
	diff =  \(tv2.tv_usec - tv1.tv_usec\)/1000;
  else
	diff = \(tv2.tv_usec \+ \(1000000-tv1.tv_usec\)\)/1000;

  if \(diff>1\)
  cout << diff << endl;

\}

fclose(stream);
return 1;
}

Waiting for you proposals how to fix this.

You may want to consider POSIX aio - asynchronous I/O. Try man aio. e.g., aio_write, aio_read

aio allows you to request an i/o operation, then do soemthing else in your code. When you are done with the 'something else' you go back and query the completion status. You are trying to do real time processing.

You really should consider a design change. Create two processes - one to get/read the data, then send it to shared memory, and a second process which just does file writing using the shared memory data.

Actually I use the file for exchanging data between 2 processes - 1 is only writing and the other is only reading. This is done each 30 ms.

Besides that, I need the data in the file anyway for post-processing.

Would you suggest a better way?

Thanks,
Inna

Without considering the post processing,
sounds like you would need a pipe, message queue or a shared memory segment to allow your processes to communicate.

You would have to consider the merits of each against your own requirement, but personally I would go for a message queue.

As to how this would fit into with your post processing, I cannot comment, since you have not elaborated.

MBB.

Real time I/O processing usually uses the aio calls. These let you make a write call while you are doing something else. Normally these processes run at special priority, so they can get the CPU whenever they need it.

As for IPC - interprocess communication - shared memory is very fast. The idea of message queues is also very good because you can send messages without regard to how fast they are being read. As long as they do get read....

However there is a limit to I/O throughput.

It actually sounds like your system is I/O bound, if you know that term. If you have 900ms waits on I/O completion, you are going to have a problem no matter what design you implement. That is about 13 x 30 x 30ms "sleep" cycles. Which will create another condition requiring a disk write by the kernel as soon as it comes back from a write. Every one of your writes (assuming an 8 byte double and all 10000 array elements are written) is 100 X 100 X 8 bytes = 80 thousand bytes every 30ms.

13 requests ~ 1MB of data every 900ms. This is on top of all the other system I/O.
You may want to have a sysadmin look at monitoring I/O for you. You could write to several different low I/O volume filesystems simultaneously, for example.

The kernel normally manages I/O requests -it lets them pile up for a while, then it writes all of the pending requests to disk. There are ways to reconfigure the kernel for real time I/O considerations, but it alters the system for everybody else. You can't do this to a multiuser production box without causing problems.

You might want to get a PC with a really high throughput disk I/O subsystem, maybe one that supports realtime streaming, put Linux on it, and go from there. You could also consider doing something to reduce I/O like compressing the data first. If compression gets you a lot.

Thanks for the detailed responses.
I will try to implement some of the ideas.

I.