100% memory usage in prstat

UNIX Solaris
1

Hi Guys,

I have observed the Oracle (DB USER) is utilizing 100% of the memory in the prstat -a output. I am bit confused is it normal and if not how to bring it down? ABout the machine it is a SunOS 5.10 Generic_125100-10 sun4v sparc SUNW,Sun-Fire-T200.

Please see below output of prstat -a
I have made bold the utilization...

bash-3.00# prstat -a
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP       
 26879 sogadm    517M  416M sleep   59    0  74:17:12 5.7% java/50
 29064 sogadm    278M  176M sleep   59    0  22:58:58 1.2% java/117
 11385 sogadm     53M   40M sleep   59    0   3:19:10 0.3% FDSServer/101
  1471 root     8752K 8752K cpu1    39    0   0:00:03 0.2% prstat/1
   228 root     7192K 4088K sleep   59    0   1:53:48 0.1% nscd/28
  5956 emadba     44M   25M sleep   59  -20   1:21:04 0.1% ocssd.bin/15
 21113 sogadm    220M  121M cpu0    59    0   0:37:36 0.1% PluginContainer/214
 11387 sogadm     52M   38M sleep   59    0   0:45:20 0.1% FDSServer/77
  2383 root     1464K  768K sleep   50    0   0:43:41 0.0% init.cssd/1
 11688 emadba   1160M  907M sleep   59    0   0:38:00 0.0% oracle/1
  5691 root       70M   55M sleep   59    0   1:01:34 0.0% crsd.bin/45
 22319 sogadm    568M  544M sleep   59    0   0:23:20 0.0% PluginContainer/181
  5794 emadba     36M   18M sleep   59    0   0:37:22 0.0% oclsomon.bin/1
  8924 emadba   1160M 1107M sleep   59    0   0:29:25 0.0% oracle/1
 11682 emadba   1153M  903M sleep   59    0   0:28:41 0.0% oracle/1
     1 root     2504K  696K sleep   59    0   0:25:51 0.0% init/1
 11808 emadba   1152M  902M sleep   59    0   0:30:46 0.0% oracle/1
 11737 emadba   1162M  902M sleep  100    -   0:22:48 0.0% oracle/1
 11729 emadba   1162M  902M sleep  100    -   0:22:55 0.0% oracle/1
 11725 emadba   1162M  902M sleep  100    -   0:22:51 0.0% oracle/1
 11733 emadba   1162M  902M sleep  100    -   0:22:47 0.0% oracle/1
  8961 emadba   1162M 1102M sleep  100    -   0:23:28 0.0% oracle/1
  8974 emadba   1162M 1102M sleep  100    -   0:23:25 0.0% oracle/1
  8970 emadba   1162M 1102M sleep  100    -   0:23:23 0.0% oracle/1
  8995 emadba   1162M 1102M sleep  100    -   0:23:21 0.0% oracle/1
 29517 sogadm     55M   37M sleep   59    0   0:20:08 0.0% PluginContainer/99
 28556 sogadm    281M  126M sleep   59    0   0:21:17 0.0% java/58
 24721 sogadm     46M   24M sleep   59    0   0:20:14 0.0% PluginContainer/93
  2228 root       35M 2104K sleep  100    -   0:41:18 0.0% rpc.pmfd/79
 13099 sogadm     83M   57M sleep   59    0   0:16:22 0.0% PluginContainer/333
 NPROC USERNAME  SIZE   RSS MEMORY      TIME  CPU                             
    46 sogadm   3427M 2107M   0.2% 107:28:27 7.7%
  1208 emadba   1326G 1062G   100%  12:03:42 0.6%
    80 root      778M  220M   0.0%   5:49:12 0.5%
     1 batchadm   15M  976K   0.0%   0:04:20 0.0%
     1 noaccess  163M   66M   0.0%   0:05:58 0.0%
     3 spmadmin   61M    0K   0.0%   0:00:00 0.0%
     1 smmsp    7392K 1496K   0.0%   0:00:01 0.0%
     4 daemon     12M 2664K   0.0%   0:00:04 0.0%

Thx.

2

Time to buy another terabyte of RAM :wink:

3

Please check for any idling..
Maybe your DBA will know better which are the processes that can be killed. There could be some processes not terminated by themselves

4

Thx incredible for the suggestion, but i didn't got your point what I have check?

What you mean by idling?

5

it just means the processor is doing nothing..as you run other apps you will see it dropping down from 100%....I would get worried if it were running at 0% as then you would have problems. It just means it is 100% idle.

---------- Post updated at 10:27 PM ---------- Previous update was at 10:25 PM ----------

And also would like to check the amount of physical memory in your system plus the assigned swap space.. Is your swap equivalent or more than your physical RAM?

6

Swap

bash-3.00# swap -s
total: 9051592k bytes allocated + 2947960k reserved = 11999552k used, 2640008k available

Physical

bash-3.00# prtdiag -v | grep Memory
Memory size: 8184 Megabytes
7

Don't focus on this bogus statistic. You simply can't have 1+ TB (1024 GB) used when 8 GB is installed. Have a look at the prstat manual page (last sentence) for a clue.

On the other hand, you have allocated 9 GB of memory so there is a RAM shortage in your box.

8

Thank you jlliagre for reply.
I am bit confused here.
What I guess, swap is used from the disk, so no mattter how big we have it. It's the virtual memory which is simulated from the disk.

PLease correct me if I am wrong, moreover do you think we have to reduce the swap here. Thank you.

9

Indeed, too large a swap is not a problem however your applications are using more memory than what can fit in the RAM. That means they would likely benefit from an increase in RAM size.

No, either reduce the RAM usage by your applications if doable or increase the RAM size.

10

Try

vmstat 5 5

and have a closer look to "sr" column, if there are for longer time period higher values / for example about 200 / you are facing to memory shortage.
Also have a look at column "id" and "r" where you can discover real CPU utilization.

11

The sr coulmn seems to hold normal values .

bash-3.00$ vmstat 5 5 
 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr m0 m1 m2 m3   in   sy   cs us sy id
 0 0 0 3168312 351264 453 2467 9 11 12 0 5  4  4  4  0 4143 26324 8783 5  8 87
 0 0 0 2480544 304608 451 2507 0 0 0  0  0  0  0  0  0 4050 32797 9758 7  9 84
 0 0 0 2446640 306536 342 2233 0 0 0  0  0 19 18 18  0 3923 24769 7633 5  7 88
 0 0 0 2388648 311328 359 2746 0 2 0  0  0  0  0  0  0 4309 24969 8615 5  9 86
 0 0 0 2488416 315232 325 2091 0 2 2  0  0  0  0  0  0 3882 27707 8424 4  8 88

After 5 minutes I again executed vmstat 5 5

bash-3.00$ vmstat 5 5 
 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr m0 m1 m2 m3   in   sy   cs us sy id
 0 0 0 3168192 351256 453 2467 9 11 12 0 5  4  4  4  0 4143 26326 8783 5  8 87
 0 0 0 2511488 337496 522 2940 38 0 0 0  0  1  1  1  0 4050 37562 8630 6  9 86
 2 0 0 2391296 337928 333 2497 0 2 0  0  0  0  0  0  0 4215 24412 8440 4  9 87
 0 0 0 2511944 337712 535 2544 0 2 2  0  0  0  0  0  0 4060 32147 9325 6  9 85
 0 0 0 2514160 338936 326 2293 0 0 0  0  0 19 18 18  0 4068 28599 8713 6  8 85
12

In the first line, the sr columns is not null so that means you had some RAM shortage in the past. There is no way to tell how important it was from this value as it is an average since last reboot.

There is currently no memory pressure. That means some of the allocated and used memory is not accessed and is just dormant in the swap area.

13

Thx jlliagre for all the assistance and suggestions.