Hi Guys,
I have fresh new installed VIO 2.2.3.70 on a p710, 3 physical SAS disks, rootvg on hdisk0
and 3 VIO clients through vscsi, AIX7.1tl4 AIX6.1tl9 RHEL6.5ppc, each lpar has its rootvg installed on a LV on datavg (hdisk2) mapped to vhost0,1,2
There is no vg on hdisk1, I use it for my investigation.
hdisk1 is mapped to all 3 lpar AIX7.1tl4 AIX6.1tl9 RHEL6.5ppc through the same vhost0,1,2
from VIO server, "dd if=/dev/hdisk1 of=/dev/null bs=1M" running in parallel "iostat -DlT 1", gives me a throughput of 93.2MB/s (AS EXPECTED)
from lpar1 AIX7.1tl4 , "dd if=/dev/hdisk1 of=/dev/null bs=1M" running in parallel "iostat -DlT 1" on the VIOserver, gives me a throughput of 10MB/s (TOO LOW !)
from lpar2 AIX6.1tl9 , "dd if=/dev/hdisk1 of=/dev/null bs=1M" running in parallel "iostat -DlT 1" on the VIOserver, gives me a throughput of 10MB/s (TOO LOW !)
from lpar3 RHEL6.5ppc, "dd if=/dev/sdb of=/dev/null bs=1M" running in parallel "iostat -DlT 1" on the VIOserver, gives me a throughput of 113MB/s (AS EXPECTED)
ofcourse the dd commands are run separetely each time. there is clearly bad performance on the AIX lpars, I tried to modify few parameters such queue_depth or max_transfer with the same result.
See below the iostat outputs on the VIO for each test.
If I compare the bps and tps value :
on the VIO where the disk is local, 22761 I/O requests give 93MB/s (about 4k per I/O as per my calculation)
on the VIO clients which run AIX, 2500 I/O requests give 10MB/s (about 4k per I/O as per my calculation)
on the VIO client which runs RHEL, 1730 I/O requests give 113MB/s (about 66k per I/O as per my calculation)
it seems the io size is limited to 4k on AIX, and that could be the thing to modify ...
Do you have any idea which parameter I could modify to get acceptable performance on aix lpar ?
Please note that there is no filesystem nor LVM in the picture, I access directly the hdisk through vscsi, that's it. so the Filesystem tuning nor LVM tuning is not relevant in my case.
Disks: xfers read write queue time
--------------- -------------------------------- ------------------------------------ ------------------------------------ -------------------------------------- ---------
%tm bps tps bread bwrtn rps avg min max time fail wps avg min max time fail avg min max avg avg serv
act serv serv serv outs serv serv serv outs time time time wqsz sqsz qfull
dd on VIO server:
hdisk1 97.0 93.2M 22761.0 93.2M 0.0 22761.0 0.1 0.1 5.8 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 1.0 0.0 19:19:29
dd on AIX7.1tl4 :
hdisk2 100.0 10.3M 2506.0 10.3M 0.0 2506.0 0.1 0.1 32.4 0 0 0.0 0.0 1.7 30.5 0 0 0.0 0.0 0.0 0.0 1.0 0.0 19:30:21
dd on AIX6.1tl9 :
hdisk1 80.0 10.5M 2559.0 10.5M 0.0 2559.0 0.1 0.1 5.9 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 1.0 0.0 19:35:43
dd on RHEL6.5ppc, :
hdisk1 99.0 113.4M 1730.0 113.4M 0.0 1730.0 0.9 0.3 8.6 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.1 0.0 3.0 0.0 19:33:25
---------- Post updated at 10:43 PM ---------- Previous update was at 09:34 PM ----------
I think I got my answer, I was simply "dd if=/dev/hdiskX ..." and I should have done "dd if=/dev/rhdiskX ..."
using the raw device instead of the block device increased the iostat result significantly !
Although on linux I was using the block device, I didn't get such difference with the raw device.
To dd the raw device on linux I have done :
[root@lpar3 ~]# raw /dev/raw/raw1 /dev/sdb
/dev/raw/raw1: bound to major 8, minor 16
[root@lpar3 ~]# dd if=/dev/raw/raw1 of=/dev/null bs=1M
I have now similar results in all 4 scenarios :
Disks: xfers read write queue time
--------------- -------------------------------- ------------------------------------ ------------------------------------ -------------------------------------- ---------
%tm bps tps bread bwrtn rps avg min max time fail wps avg min max time fail avg min max avg avg serv
act serv serv serv outs serv serv serv outs time time time wqsz sqsz qfull
dd on VIO server:
hdisk1 98.0 172.0M 164.0 172.0M 0.0 164.0 6.0 4.3 13.0 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 0.0 0.0 21:25:12
dd on AIX7.1tl4 :
hdisk1 100.0 163.8M 1250.0 163.8M 0.0 1250.0 3.0 0.1 17.7 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 6.0 0.0 21:28:54
dd on AIX6.1tl9 :
hdisk1 100.0 164.4M 627.0 164.4M 0.0 627.0 3.3 0.2 16.7 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 2.0 0.0 21:29:57
dd on RHEL6.5ppc, :
hdisk1 100.0 118.0M 1801.0 118.0M 0.0 1801.0 0.9 0.3 6.0 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.1 0.0 3.0 0.0 21:32:07
dd on RHEL6.5ppc, using RAW
hdisk1 95.0 171.9M 2623.0 171.9M 0.0 2623.0 3.1 0.3 14.3 0 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.2 0.0 7.0 164.0 21:34:02
Thanks for your help guys.