Regarding Upgrading RAM in AIX 6.1

Hi Folks,

Currently we are having the below configuration in one of our AIX 6.1 server.

RAM:- 8 GB
Paging space :- 19 GB
CPU processor:- 1
CPU type: 64 bit

But we would like to upgrade the configuration to below to improve the system performance and resolve some memory issues.

RAM:- 32 GB
paging space: 40 GB
CPU processor :-1

Could you please confirm and let us know the below points.?

1.) Is This configuraion is with 1 CPU processor will be possible.?
2.) Can we see any perfomance with 1 CPU core or any impact?
3.) There will be any limitation in CPU core if we upgrade the RAM to 32 GB?

~Have a Good Day

Thank You,
Suman B

Do you have any hard coded facts and figures to show the memory problem!

One cannot just provide feed back with so little info, can you clarify the below

  • Are you seeing the error logs in error report?
  • Is the system hanging?
  • What is running on the system? what kind of server is it? like DB or Application.
  • What kind of DB (oracle, db2) or what is the application?
  • Since when are you seeing this problem?

Why so much paging space?

As a workaround this will work for some time, but the issue why you have memory problems might still show up, can just take longer until the RAM is used up. But that depends on what you are running and how the problems shows itself.
Though additional CPU units will not help with memory issues.

So as the others suggested, you should investigate what is the reason for your memory problem.

And as blackrageous asked, there is no need for so much paging space. I would go for a fix Paging Space size of 5GB or 10GB. When your system starts to use it, the performance will usually degrade enormous and so the goal is to avoid that before it happens.

  • Are you seeing the error logs in error report?
    Out of Memory Errors in the logs. Jobs are getting failed due to this.

  • Is the system hanging?
    The system is slow when the memory levels reaching to 99%

  • What is running on the system? what kind of server is it? like DB or Application.
    Application

  • What kind of DB (oracle, db2) or what is the application?
    DB2

  • Since when are you seeing this problem?
    When the application goes on live

If it the out of memory occurs directly after the application or DB2 is started, you might want to check, to which value the parameter database_memory is set, or maybe one of the other memory parameter for DB2.

Trivia:
IBM DB2 9.5 Information Center for Linux, UNIX, and Windows

How to do check:
IBM DB2 9.5 Information Center for Linux, UNIX, and Windows

When it occurs, just to make sure, use svmon -P | grep -p Pid to determin which is using Paging Space. You will simply see it, when there is not a null in the column "Pgsp" to find the culprit.

So it could be, that you overcommit memory to the database, that you don't even have (in the old setup) as physical RAM available.

Another thing comes to my mind - has your OS been "tuned"? Maybe post the output of vmo -F -L lru_file_repage -L minperm% -L maxperm% -L maxclient% -L maxpin% .

1 Like

I would not worry about real memory reporting 99% with vmstat, nmon or such like, as AIX keeps real memory even when a process ends just in case the same data is needed later. You need to keep an eye on paging to see if you are really running out of real memory.

lsps -a

You also need to consider if your DB has a real-memory requirement. If you exceed the real memory, you system will start to page. Check vmstat for the pi & po columns to see read and writes to/from disks for paging.

If your DB requires (or is set for) large real memory, then that can be a problem too. This is an allocation that is not eligible for paging. if you run multiple instances, remember that they will each want their own allocation and that you may be exceeding the machine limits there. The settings in the DB startup parameters is not defining how big a machine you have, it's how much you are allocating to that instance - a mistake we have seen here.

I hope that these help,
Robin

1 Like

May be your team did not performed a stress test thoroughly, you will put the system to its max limits during that test (before you go live).

I guess, the no of users after go-live crossed the actual stress test, you guys under sized the system.

If you say NO and your answer is you did the stress test pushing to max, then my immediate guess is there might be change in code in application, ask them. Why there was no problem before and then sudden spike.

To the best of my experience either one of those should be it.

1 Like
$ svmon -P | grep -p Pid
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
15925256 java            466928     8595   173586   528345      Y     Y     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
13697236 BIService       332451    13225   474539   566291      N     Y     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
19595408 java            300374     8626   557791   647293      Y     Y     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
18088138 java             42401     8459    67900    99438      Y     Y     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
21233778 java             34438     8478   141277   170749      Y     Y     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
 8061030 das              31752    13226    35314    85014      N     Y     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
11665488 topas            21195    13204     5346    46727      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
14024892 ksh              20920    13204     4992    46101      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
 7077994 ksh              20776    13204     5125    46100      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
14549230 sh               20772    13204     5107    46083      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
15073398 ksh              20772    13204     5126    46101      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
14352432 ksh              20772    13204     5128    46103      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
20840500 sh               20772    13204     5172    46148      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
13631528 grep             20769    13204     4992    46008      N     N     N
-------------------------------------------------------------------------------
     Pid Command          Inuse      Pin     Pgsp  Virtual 64-bit Mthrd  16MB
14221472 svmon            19808     8416     5489    32695      Y     N     N
$ topas
Exiting
$ vmstat
System Configuration: lcpu=4 mem=8192MB
kthr    memory              page              faults        cpu
----- ----------- ------------------------ ------------ -----------
 r  b   avm   fre  re  pi  po  fr   sr  cy  in   sy  cs us sy id wa
 2  2 3164764  3003   0  29  28  93  282   0 353 6306 873 14  2 82  1
$ vmstat
System Configuration: lcpu=4 mem=8192MB
kthr    memory              page              faults        cpu
----- ----------- ------------------------ ------------ -----------
 r  b   avm   fre  re  pi  po  fr   sr  cy  in   sy  cs us sy id wa
 2  2 3128583  9792   0  29  28  93  283   0 353 6306 874 14  2 82  1
$ topas

Ok, your system is definetly actively paging in and out to Paging Space. Also some processes like your java and BIService (Business Intelligence?) are paged out already.
That together with a DB2 on this box, is a bit much maybe for 8 GB.

I assume your box has the default settings - the output of vmo -F -L lru_file_repage -L minperm% -L maxperm% -L maxclient% -L maxpin% is missing.

So with this input it seems, you are simply far from having enough memory. If you are increasing to 40 GB, there will be most probably no paging anymore anyway.

$ vmo -F -L lru_file_repage -L minperm% -L maxperm% -L maxclient% -L maxpin%
ksh: vmo: cannot execute

Make sure you are root when calling this command.

Start using code tags - you don't want to get banned by accumulating infractions for missing code tags.