Help with ZFS arc cache

UNIX Solaris
1

Greetings Forumers!

I have a Solaris 10u9 M5000 with 32GB RAM and have noticed the ZFS arc cache is consuming large amount of memory. Here's what I see on my system:

# echo ::memstat|mdb -k
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     534548              4176   13%
ZFS File Data             2898766             22646   70%
Anon                       253850              1983    6%
Exec and libs               23498               183    1%
Page cache                  22613               176    1%
Free (cachelist)            33639               262    1%
Free (freelist)            350437              2737    9%

Total                     4117351             32166
Physical                  4095740             31997

How can I limit this excessive use of memory without impacting the zones that run on this system?

Thanks is Advance!

2

Correct me where I missed the point - the system allocates 'unused' memory which it gives up when needed. You still have lots of free pages. This an m4000:

# echo ::memstat |mdb -k
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     398633              3114    5%
Anon                      1074821              8397   13%
Exec and libs               58856               459    1%
Page cache                6013479             46980   73%
Free (cachelist)           229681              1794    3%
Free (freelist)            459750              3591    6%

Total                     8235220             64337
Physical                  8213603             64168

Is page cache is out of control? No. The system is doing to this m4000 exactly what you see on your box. Using idle memory to good advantage.

Are you experiecing problems? Really sluggish performance, lots of swapping?

---------- Post updated at 08:48 ---------- Previous update was at 08:42 ----------

Here found it - the Evil Tuning Guide

ZFS Evil Tuning Guide - Siwiki

Read the arc cache section.

1 Like
3

As far as I know limiting the zfs cache could make sense because of the overhead associated with the dynamic resizing.

Oracle/Sun consultants recommended the following settings in /etc/system on our database servers with 64G of physical memory:

* ZFS tuning
set zfs:zfs_immediate_write_sz=8000
set zfs:metaslab_df_free_pct=4
set zfs:zfs_nocacheflush=1
set zfs:zfs_vdev_max_pending=32
* limit zfs 16G
set zfs:zfs_arc_max=17179869184
1 Like
4

Actually no. However, I have created other zones on ZFS pools. These zones run Oracle software that looks for available RAM before starting. If there is no RAM available, the software pukes out an error msg and dies.

I also found this link about limiting the amount of ZFS ARC in memory.

I'm just looking for ways to get these zones up and running while managing memory effectively. I'm also new at using ZFS and zones so I'm going through a bit of a learning curve too. :o

Any help is greatly appreciated.

5

There are additional notes on MOS about this.

6

I was think about setting this limit (16G) on my M5000.

7

You could also check ZFS Best Practices Guide.

8

What is MOS ? :confused:

9

My Oracle Support (previously known as MetaLink).
You need a Customer Support Identifier (a license) in order to use it.

Flash interface at: support.oracle.com
HTML interface at: supporthtml.oracle.com

10

I just added another zone running on a ZFS pool. The memory stats are looking better:

# echo ::memstat|mdb -k
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     571744              4466   14%
ZFS File Data             1615127             12618   39%
Anon                       733311              5728   18%
Exec and libs               26113               204    1%
Page cache                  22748               177    1%
Free (cachelist)            12238                95    0%
Free (freelist)           1136070              8875   28%

Total                     4117351             32166
Physical                  4095740             31997

I was also looking at "top" before and after instaling the zone and noticed a 6G difference in available memory. I knew that the ZFS ARC memory allocation was dynamic but did not think it would release soo much RAM at once.