Memmove fail on 4 Gb+ raster under RHEL 64 bit

I'm rotating huge image rasters (1+ GB) using a 3-shear algorithm. I rotate 'in-place' after reallocating the raster. A necessary first step is reorganizing the image in the new raster dimensions that will exist after the rotation. RGBA pixels byte packed into uint32.

The following code runs fine on rasters < 4 Gb. No issues.
When I run it on rasters > 4 Gb, memmove() crashes.

I was under the impression that 64 bit Linux could take on all size data structures.

Any ideas out there. I am at my wits end on this.

Any replies greatly appreciated.

Imagtek

****************************************************************

   lbuf1  = (int32u *)realloc(*outrast, (longu)(Rcols*Rrows*4));
   if( !lbuf1 )
      return( -2 );                // insufficient memory

    if( Rcols > icols ){       // fit icols,irows to new Rcols,Rrows
      fprintf(stderr, "fitting data...");
      il = lbuf1 + (longu)((irows-1) * icols);
      ol = lbuf1 + (longu)((irows-1) * Rcols);
      for(iy=1; iy<irows; ++iy){
        memmove(ol, il, (longu)(icols * 4));
        ol -= Rcols;
        il  -= icols;
      }
      fprintf(stderr, "done...");
    } 

***************************************************************

4294967296 is 4GB; 4294967295 is usually the limit to the size of what an uint32_t pointer can reference. Change your datatype.

However, consider:
Your object must exceed that value or you have exhausted heap memory or exceeded the process memory limit if one is set. What exact error did you get? Fix your code to display what the error is, i.e.

   // you need to include <errno.h>
   if( !lbuf1 )
   {
      fprintf(stderr, "Fatal error %s\n", strerror(errno));
     // or use perror
     // perror("Fatal error");
      return( -2 );                // insufficient memory
   }

Did you check:
ulimit -a for the process. What does free show? While the code is running?

There are other factors that can also limit allocating giant chunks of memory - heap fragmentation. I do not know how to report that on RH or CentOS from the command line. I do know you can get heap information by writing C code to look at anonymous memory in /proc system-wide.

Heap framentation can occur when you make a bunch of calls to malloc or realloc in succession where the size continually doubles. And there are other process that tie up substantial heap memory or shared memory. You get to a point where the contiguous size chunk you want is not available but the total free memory (made of lots of smaller chunks) exceeds what you need. java is notorious for that - so much so that one of the startup parms allows you to force java to get max possible heap when it first starts.

Start by finding out the exact error, you are assuming ENOMEM.

What are the values of Rcols and Rrows when realloc() is failing? What type are Rcols and Rrows ?

Making a wild guess that longu is an unsigned long and Rcols and Rrows are ints (signed or unsigned), you may be overflowing type int in the calculation (Rcols*Rrows*4) before casting the result to a long. Have you tried:

   lbuf1  = (int32u *)realloc(*outrast, Rcols*(longu)Rrows*4);

instead of:

   lbuf1  = (int32u *)realloc(*outrast, (longu)(Rcols*Rrows*4));

so the calculation will be performed using longs instead of ints?

As an example:

$ cat t.c
#include <stdio.h>
int
main() {
	printf("%lx\n", (unsigned long)(32768*16384*4));
	printf("%lx\n", 32768*(unsigned long)16384*4);
	printf("%ld\n", 32768*(unsigned long)16384*4);
}
$ make t
make: `t' is up to date.
$ ./t
ffffffff80000000
80000000
2147483648

In the future, please show us the declarations for all of the variables we're looking at and any non-standard typedefs used in those declarations. Evaluating problems like this is much easier if we can see full details about the variables being used instead of having to hypothesize about possible problems and guessing at types that would lead to those problems.

PS: Jim: (uint32_t *)realloc(...) on a 64-bit compile casts the void * pointer returned by realloc() to a (64-bit) pointer to a 32-bit integer, not a 32-bit pointer. Note that if the code had used sizeof(uint32_t) instead of 4 , this problem would disappear if my guess about integer overflow was correct.

@Don - good point - but my assumptions are different. Here is why:

CentOS ( -Therefore probably RH. Solaris 10 sparc as well, my platform at work) defaults to gcc -m32 unless you tell it otherwise. My assumption is RH defaults to that and the OP blew past 4GB. There are two gcc libraries - one for 32 and one for 64.

Dunno if that is a good assumption.

The OP seems to be silent. But if he posts let's ask about the -m64 option to gcc. Or posting output from the command file compiled_image_file

Hi Jim,
Agreed. We have two plausible explanations for the reported problems and no information to determine if either (or both) of us guessed correctly.

Hi imagetek,
The ball is in your court...

Many sincere thanks. Very useful info. I admit that I have been rather cavalier slinging gigabytes of image memory around on a 64 bit machine with 8 Gb RAM, simply because I've been getting away with it, not due to deep understanding.

I've been assuming that (longu)(a * b * c) would cast every int32 in the expression to 64 bit long BEFORE the calculation. If that is the problem, how embarrassing...I thought I knew how to code.

I will pursue these leads and get back with the resolution. I think one of you probably nailed it. I want to be able to rotate an image of ANY size, machine capable of course.
Thanks again guys.
imagtek

---------- Post updated at 10:01 AM ---------- Previous update was at 08:52 AM ----------

OK. The problem is fixed. I was overflowing the expression used to determine the reallocated raster size.

The array dimensions Rcols, Rrows were int32. Plenty big for
any realistic raster dimension, but computationally vulnerable.

I assumed that casting a C/C++ expression such as
(long)(a * b * c)
had the effect of
( (long)a * (long)b * (long)c )
it does NOT.

I misunderstood the language spec. In 32 bit environment this hid in the grass. In 64 bit, it is a very big deal. I read the language spec over and over but somehow missed this subtlety.

The variables inside the expression are promoted to the largest type INSIDE the expression before computation. Any cast outside the expression effects only the expression result.

Images are getting big enough now that it finally bit me. Simple fix in about a dozen routines vulnerable to this.

Now my 4 Gb image rotates nicely to a 5 Gb image. Just takes awhile... Need to revisit some vulnerable code. Now I don't understand why some things did NOT blow up, but not an issue for here.

You're never to old to learn something you should have known a long time ago.

Thanks Guys! Bugs based upon misconceptions are the worst.

imagtek

The real question is, why weren't those all long's or size_t's in the first place? In 32-bit they are 32-bit, in 64-bit they are 64-bit.