We have 2 Rs6000 S-85
Each initially had/have 6 processors and 8 GIG of RAM each
Node 1 we added 14 processors and 32 GIG of RAM On May 19. (11 days ago)
My memory utilization reporting for node 1 showed a spike in available memory 25-30GB for May 19 to 25 . I kind of expected this because I did not think the app would try to take any more memory. Then suddenly since the 26th, available Real memory took a sudden drop down to 2 GB!
Virtual memory up until the 26th was hovering around 80% since then and now it is at 99%.
Application on the node is is Health care System called Cerner. It also Runs MQ Series Middleware.
It is almost like the system realized it had the memory to use Saturday morning and has since then decided to use it!
I have not yet added paging space for this memory, if it is needed. I need to look into that. I believe there is an economy of scale or breaking point for paging. Currently paging space is at 8192MB currently.
The memory and CPU we added were a total surprise. Our vender may not have known what they sent us. We thought we were getting 4 more CPU and anothe 8 GIG of RAM. Instead we got 14 CPU and 32GIG of RAM! Christmas in May!
We are going to give node 2 the same treatment for memory and CPU (which strictly runs a 300 GB Oracle database)
I am just not sure what to make of node 1 behaviour.
Any comments are appreciated. SkyyBugg
Can you or have you tried a re-boot. Also set NMON running (If you have not already) to analyse what app is taking most of the memory. Also with 32Gb of memory your paging is now far to small it should be 48 Gb.
It looks like when I list top proecesses on nmon, no single process ever takes more than 1% of the memory. Even then, only one or two processes use use 1% at once. The utilization report that uses nmon says that there is 2.6 GB of 40GB Available. It sounds like that nmon AIX "sees" the memory but the application os is actually not using all of it.
If you are intrested attached is some output from nmon.
The m option seemns to be reporting differently than the t option.
I don't plan to change paging because no paging is taking place and it will take alot of my disk space to build paging at 80GB.
You need to run NMON over a 24 hour period and use the NMON analyser to produce a set of Excel graphs which will show a comprehensive set of statistics for the environment. Using NMON the way you have will give no useful information. I take it the machine has been re-booted?
John thanks for the perspective and questions.
May 19: I shut down the box. IBM Installed the memory, then I booted up the box. Nmon reports show available memory hovered around 33-35GB.
I have CPU/memory monitoring data from nmon each hour 24/7/365. This is how I was able to see that on May 26 through today available memory went from 35GB all the way down to 2.5 GB.
My attachement is currently what is typical as far as how much memory the application is using. You can see that there is no paging. The box is not breaking a sweat.
A caveat. We are still running AIX 5.1 ML04. In a week we will upgrade to AIX 5.3 TL04. This series of downtimes were set to upgrade harware, Oracle, AIX, MQ, CPU/memory.
Also, I have put in an Excell Sheet the past 30 days of nmon CPU/Memory.
It is intresing to see what the CPU/ Memory, application adjustments did to this box. It is starke improvement to say the least!
