About efficiency of parallel memory allocation

Hello, there.
I'm a new beginner to Linux kernel and curious about its memory management.
When multiple applications apply for memory space at the same time, how Linux kernel solve the resource contending problem for high performance?
I have known that there is a buddy system for allocating and recalling memory for big page and there is a slab allocator for little buffers.
Is the performance guaranteed by multi-layer memory management architecture?
Wish for some experienced help. :slight_smile:

1 Like

Do you understand virtual memory versus physical memory? When a process asks for memory, especially huge chunks of it, one common strategy is to allocate a small part of it and then mark virtual memory (which is on disk usually) as allocated for the rest of the request.

See

free
swapon -s

Thanks for replying.
I have learned about different mapping methods of virtual memory to physical memory.

But whole physical memory is divided into two parts: one for Kernel, the other for User and usually 1:3.
For user part, user threads or processes share the physical memory by virtual memory mapping, so it means that each thread gets it memory without contention in the allocation even though they actually share the same physical memory? (I know the conflict can happen when they load their virtual memory segment to the physical memory area)
For kernel part, there is part of memory(For 4GB memory, it's 896MB) that can be used directly without mapping from virtual memory to physical memory. If parallel allocation happens in this area(such as parallel system call), will contention exist? If yes, how does it solve the performance problem?
Thanks :slight_smile:

First off, virtual memory mapping does not necessarily mean "contention free". Kernel memory is virtual too. There's really no difference between kernel and user memory except who gets to use it.

The two ways a process asks the kernel for address space -- brk() and mmap() -- are unavoidably high-overhead. Every kernel call is a task-switch, after all, even mundane things like read()! brk() and mmap() aren't generally bottlenecks, though, because they're called so seldom -- a handful of times to set up libraries, then a call to brk() to assign a large block of undifferentiated memory that the process can use as it pleases without bothering the kernel. You may know this block as the heap segment.

The library function malloc() carves out chunks of the heap segment on demand. It keeps keeps a list of available chunks at the base of the heap. brk() expands the heap in large jumps if it runs out, otherwise the kernel isn't involved at all. Whether there's any contention depends on how malloc() is written.

Most Linux systems use GNU libc, which avoids malloc() contention by using several independent "arenas". I think this amounts to independent heaps.

Now, inside the kernel, there's two basic options for allocating memory depending on your needs - kmalloc() and vmalloc().

kmalloc() is the kernel's cousin of userspace malloc() - more or less a heap kept inside kernel space. It's fast, doesn't change the page table, and lets you avoid contention if absolutely necessary. But you usually can't allocate more than 4 megs at a time(architecture dependent), and the total limit on size might also be as small as 128 megs.

The other is vmalloc(), which alters the page table and forces a synchronization but doesn't have a lot of limits otherwise, since it's not restricted to a chunk of preallocated kernel memory.

2 Likes

Very thanks for the detailed explanation!
Now I know the main idea of system's memory management. Costly operations are completed mostly in creating new processes or allocating more memory, but general cheap operations are completed in each process. It ensures that kernel will not be frequently bothered.
Interesting, I have to read more about the kernel.
Thanks again!

1 Like