I want to run a C program on my linux machine. I want to allocate specific heap size for that process (C program) to run.
How can I do that?
I know in Java same can be done using -Xmx option.
There may be some option which I can specify in the C program like Java or may be in linux process.
Your allocator must have the same interface as the standard one, malloc, realloc, calloc, free, ... in your own shared library.
You can then preload your shared library as in:
CMD="LD_PRELOAD=/home/kbw/bin/myalloc.so prog arg1 arg2"
'$CMD'
or somthing like that.
Thanks for your Reply Kbw but I did not understand that how would I allocate memory using this.
I want to allocate some part of RAM to this process using C program
I assumed you had an allocator that you wanted to use. In fact, I assumed you had written an allocator that you wanted to use.
At any rate you have to have one in a shared library somewhere and my instructions show how to load it.
"I want to allocate some part of RAM to this process using C program" If that's what you want to do, what's wrong with the standard malloc?
C does not have a "virtual machine" or garbage collection like Java does, operates with virtual memory, and has its own heap segment which never shrinks, which all make preallocation generally pointless except for some extremely specialized tasks. C programmers are usually more concerned with limiting memory than preallocating it.
For these specialized tasks you can use mmap(), mlock(), and madvise() to control how/when the kernel pages memory.
In short, what exactly are you trying to optimize here?
"I want to allocate specific heap size for that process (C program) to run."
I see, I must have been answering the wrong question. Ok, lets start again.
C does have a "virtual machine". In fact, the C virtual machine pretty much maps onto a real machine, that's what makes C so fast, because there's no translation. The C virtual machine has a call stack and a heap formed at opposite ends of the same memory block. The stack may have a fixed size, but the heap is everything else. So if you put more RAM in the machine, it shows up as more heap.
If you need memory you use it and hand it back when done. There's no portable way to put an upper limit on the heap. You may be able to specify it some custom heap, but not in a general way,
If you need memory just allocate it. The Java VM should coexist with the underlying system allocator.
So, in short, you can't limit it in a general way, and it shouldn't bother you that you can't.
It has a real machine. 
It's not really a block of memory. It's all translated from virtual addresses to real addresses. Both the stack and the heap are automatically "sized" through the magic of virtual memory. The kernel itself, behind the scenes, cares very little about which is which -- it just assigns memory in fixed-sized blocks from the pile. See The Paging Game for a general explanation on how paging works.
Agreed. There's no java VM to worry about, only the kernel itself. You use it, and the kernel either lets you or not.
It doesn't get "handed back", though. Your program keeps it for later, just in case you need to allocate again -- this makes it faster, and probably why he wanted to preallocate, but the kernel handles this for you.
Check out setrlimit.
getrlimit/setrlimit, That's pretty cool. You learn something every day.
You can also use brk() if you are careful hhow you use it. In modern systems, cgroups (control groups) are commonly also used for this purpose.