I am running a Dell PE R815 with 4 x AMD 12 core CPUs with 128GB of RAM and a RAID 5 array of 6 SAS disks. This is an HPC application and is definitely CPU bound, however once I run 16 of these processes (thus pinning 16 cores) the work performed slows down dramatically, to maybe 5 or 10% of what it normally is capable of. Top shows that all HPC processes are using 100% CPU but they rather dramatically shift to system usage (rather than user). Here is a vmstat output:
To me it doesn't look like a RAM or Disk issue, but perhaps an IO or buffering issue. Please help!
-mowmentous