consumption memory

Hi,

I have a problem with memory on AIX 5.3. On this server, we have JDE Edwards (ERP) and Oracle Database (9.2.0.7.0).
We have 4 Gb for physical memory and 3 Gb for paging space.
When I stop all services (JDE, Oracle and all other services), the physical memory is not free (4 Gb)
svmon -G
size inuse free pin virtual
memory 1048576 1033596 14980 153224 404179
pg space 786432 184887

Memory is never freed even if process is finished

When I reboot the server and start all services without users and so without transactions, the memory is 1 Gb.

svmon -G
size inuse free pin virtual
memory 1048576 269172 779404 97321 216993
pg space 786432 1041

Do you have a similary case ?

Several possiblities: It could be a memory leak. In this case the only option you have is to stop/start the application. To be honest i doubt that, because JDEdwards as well as Oracle is standard software. If they'd have memory leaks it would be a well-known fact throughout the industry.

The other possibility is a lock on the shared memory segments which isn't removed. Use "ipcrm" to delete such memory segments and look how far that takes you, then report back, please.

I hope this helps.

bakunin

AIX always tries to use ALL of your real memory. Every piece of RAM that is not used by your application is used for file caching. So "all" of the memory appears to be "in use" all the time. When you exit your app the memory is not cleared but the former content is kept for possible reuse later. This is normal behaviour and expected. No reason to worry as long as no paging takes place. Having no "free" memory does not equal shortage of memory. AIX is not Windoze.

If you want to sort of flush the server's cache for a short period of time you could copy a huge file somewhere within your server (from /blah to /tmp or vice versa) and delete it immediately after copying. Then you would see "free" memory :wink:

Normally running Oracle on AIX 5.3, you need to do some memory tuning so that you don't "double up" in buffering files. General rule is to reduce AIX amount of mem used to buffer IO and put this into Oracle's file buffers (in the SGA). This is because Oracle knows better what to cache, as it is its own data.

I don't recommend changin the following parameter by itself but it is (IMHO) the most important new parameter for an ERP/OLTP server (Introduced in AIX 5.3 ML1 and AIX 5.2 ML4).
vmo �p �o lru_file_repage=0

This means that program memory requests (e.g. Oracle file buffers) get priority over AIX's own file buffers. This is the behaviour you'd want.

Of course there are a whole heap of other parameters you'd want to check so please DO NOT just run this command without understanding what the other settings are.

And the previous post was right, just looking at "free memory" in AIX doesn't tell you much at all as AIX will try to use all memory all the time.

I hope you are using DIO or CIO access from Oracle if using JFS2 to hold your data (and DIO if using JFS). Also Asynch I/O.

Thanks a lot for your quick advices.

With nmon, I have the result hereafter :
Code Resource Stats Now Warn Danger
OK -> CPU %busy 2.6% >80% >90%
OK -> Paging size %free 24.0% <20% <10%
Warning -> Paging Space RAM:pg 31.7% <50% <10%
DANGER -> Page Faults faults 1045.3 >16/s >160/s

Obviously, I guess this server have a "small" problem of tuning.

post vmo -a and the amout of mem and cpus and vmstat -I 5 5 and vmstat -v if you need tuning assistance :slight_smile:

Rgds
zx

Hi zxmaus and thanks in advance for your help.

oslevel -r : 5300-03

  • Memory
    Physical memory : 16 Mb
    Paging space : only 5 Mb

  • Processor :
    Processor Type: PowerPC_POWER5
    Number Of Processors: 4
    Processor Clock Speed: 1896 MHz

Command : vmo -a
cpu_scale_memp = 8 ; data_stagger_interval = 161 ; defps = 1
force_relalias_lite = 0 ;framesets = 2; htabscale = n/a
kernel_heap_psize = 4096 ; large_page_heap_size = 0 ; lgpg_regions = 0
lgpg_size = 0 ; low_ps_handling = 1 ; lru_file_repage = 1
lru_poll_interval = 10 ; lrubucket = 131072 ; maxclient% = 80
maxfree = 1088 ; maxperm = 3096895 ; maxperm% = 80
maxpin = 3304704 ; maxpin% = 80 ; mbuf_heap_psize = 4096
memory_affinity = 1 ; memory_frames = 4079616 ; memplace_data = 2
memplace_mapped_file = 2 ; memplace_shm_anonymous = 2
memplace_shm_named = 2 ; memplace_stack = 2 ; memplace_text = 2
memplace_unmapped_file = 2 ; mempools = 0 ; minfree = 960
minperm = 774223 ; minperm% = 20 ; nokilluid = 0
npskill = 10112 ; npsrpgmax = 80896 ;npsrpgmin = 60672
npsscrubmax = 80896 ; npsscrubmin = 60672 ; npswarn = 40448
num_spec_dataseg = 0 ; numpsblks = 1294336 ; page_steal_method = 0
pagecoloring = n/a ; pinnable_frames = 3319403 ;
pta_balance_threshold = n/a ; relalias_percentage = 0 ;rpgclean = 0
rpgcontrol = 2 ; scrub = 0 ;scrubclean = 0 ; soft_min_lgpgs_vmpool = 0
spec_dataseg_int = 512 ; strict_maxclient = 1 ; strict_maxperm = 0
v_pinshm = 0 ; vm_modlist_threshold = -1 ; vmm_fork_policy = 1

Command : vmstat -v
4079616 memory pages
3871119 lruable pages
21738 free pages
0 memory pools
760743 pinned pages
80.0 maxpin percentage
20.0 minperm percentage
80.0 maxperm percentage
72.5 numperm percentage
2809328 file pages
0.0 compressed percentage
0 compressed pages
72.7 numclient percentage
80.0 maxclient percentage
2817268 client pages
0 remote pageouts scheduled
4018 pending disk I/Os blocked with no pbuf
48890 paging space I/Os blocked with no psbuf
2740 filesystem I/Os blocked with no fsbuf
110 client filesystem I/Os blocked with no fsbuf
89345 external pager filesystem I/Os blocked with no fsbuf

Command : vmstat -I 5 5
kthr memory page faults cpu
-------- ----------- ------------------------ ------------ -----------
r b p avm fre fi fo pi po fr sr in sy cs us sy id wa
1 1 0 1837530 20768 41 196 8 0 0 0 1110 29968 10573 17 10 70 3
1 1 0 1833453 24285 75 213 11 0 0 0 1043 32556 13921 16 14 62 7
1 1 0 1827655 28258 22 74 85 0 0 0 740 57171 6965 21 5 68 6
2 1 0 1832959 21916 60 151 99 0 0 0 434 46722 7054 31 11 47 11
1 1 0 1832622 20253 96 230 88 0 0 0 1351 32595 11736 19 10 63 8

Hi,
yeah seems pretty similar to what we are always getting from engineering ... so after having tuned a couple of 100 lpars - most of them DB boxes - I would go for following values I think ...

vmo -p -o minperm%=5
vmo -p -o maxperm%=90
vmo -p -o maxclient%=90
vmo -p -o maxperm%=80
vmo -p -o minfree=960
vmo -p -o maxfree=1088
vmo -p -o lru_file_repage=0
vmo -p -o lru_poll_interval=10

ioo -p -o j2_nBufferPerPagerDevice=1024 (if not sufficient you can go upwards 256-wise - but carefully)
ioo -p -o hd_pbuf_cnt=1024 - you can go upwards 256-wise
ioo -p -o numfsbufs=1536 (can be raised up to 2048 if needed)
ioo -p -o pv_min_pbuf=1024 (you can play here up to 1536 but carefully)

ioo -p -o j2_maxRandomWrite=32 - needs monitoring but you can go up to 128 in particular cases
ioo -p -o maxrandwrt=32

ioo -p -o j2_maxPageReadAhead=128
ioo -p -o maxpgahead=16

no -p -o tcp_sendspace=262144
no -p -o tcp_recvspace=262144
no -p -o udp_sendspace=65536
no -p -o udp_recvspace=655360

if you are running cio you can even amend
ioo -p -o lvm_bufcnt=16

But please be aware - all these values are 'best practices' but every system is different, every workload, every query. What can be really great on one system can cause a huge performance impact on another one ... so keep on monitoring.

What you have to avoid are the real problems :

fi fo pi po fr sr in sy cs us sy id wa
41 196 8 0 0 0 1110 29968 10573 17 10 70 3
75 213 11 0 0 0 1043 32556 13921 16 14 62 7
22 74 85 0 0 0 740 57171 6965 21 5 68 6
60 151 99 0 0 0 434 46722 7054 31 11 47 11
96 230 88 0 0 0 1351 32595 11736 19 10 63 8

that means your box is paging out computational pages - so you should fix this at the earliest
vmo -p -o lru_file_repage=0

72.5 numperm percentage
vmo -p -o minperm%=5
vmo -p -o maxperm%=90
vmo -p -o maxclient%=90
vmo -p -o maxperm%=80

4018 pending disk I/Os blocked with no pbuf
48890 paging space I/Os blocked with no psbuf
2740 filesystem I/Os blocked with no fsbuf
110 client filesystem I/Os blocked with no fsbuf
89345 external pager filesystem I/Os blocked with no fsbuf

with
ioo -p -o j2_nBufferPerPagerDevice=1024
ioo -p -o hd_pbuf_cnt=1024
ioo -p -o numfsbufs=1536
ioo -p -o pv_min_pbuf=1024

and afterwards you might even see a few memory pools instead of
0 memory pools

and the rest of the suggested values should at least be beneficial for performance ... and please dont make the mistake to think if a box is running cio there is no filecaching needed. There is ... depending on the amount of processes connecting on the database from any given application this can be immense. And do yourself a favour - put redo-logs into a separate filesystem with (smaller) blocksize of 512 ...

Rgds
zx

Thanks a lot zxmaus for your help and your advices.
I send these parameters to our consultant.

Hi

As we all know that AIX utilizes entire RAM and shows 100% Utilization of Memory.
Can anybody tell me the parameter in AIX which forces memory flushing???
And what are the pros and cons of using that??
I will be very thankful to you.

Regards:)