iostat as a tool for generating disk IO

UNIX Solaris
1

Hi All

Currently i had a server directly connect to the EMC box.
The EMC storage is a FC Direct Attached Storage to the server without going through SAN switches

And i had download the Daily data file in /var/adm/sa/sardd

But i not sure which one in /var/adm/sa/sardd is the correct information i need to get for the I/O report

kindly advise !

Thanks =)

---------- Post updated 06-29-10 at 02:36 PM ---------- Previous update was 06-28-10 at 03:12 PM ----------

0:00:00	device	%busy	avque	r+w/s	blks/s	avwait	avserv

0:05:00	md0	2	0.1	4	31	3.7	30.7
	md1	0	0	0	0	0	0
	md3	1	0	1	17	3	18.3
	md10	2	0.1	4	31	0	28.4
	md11	0	0	0	0	0	0
	md13	1	0	1	17	0	13.9
	md20	1	0.1	3	6	0	26.4
	md21	0	0	0	0	0	0
	md23	1	0	1	17	0	12.7
	nfs1	0	0	0	0	0	0
	nfs2	0	0	0	0	0	0
	nfs3	0	0	0	0	0	0
	nfs4	0	0	0	0	0	0
	nfs5	0	0	0	0	0	0
	sd0	4	0.2	7	53	0	24.1
	sd0,a	2	0.1	4	31	0	28.3
	sd0,b	0	0	0	0	0	0
	sd0,c	0	0	0	0	0	0
	sd0,d	1	0	1	17	0	13.9
	sd0,h	2	0	2	2	0	22
	sd2	2	0.1	4	23	0	22.3
	sd2,a	1	0.1	3	6	0	26.4
	sd2,b	0	0	0	0	0	0
	sd2,c	0	0	0	0	0	0
	sd2,d	1	0	1	17	0	12.6
	sd2,h	0	0	0	0	0	0
	sd4	0	0	0	0	0	0
	ssd0	7	0.1	59	4138	0	1.9
	ssd0,c	7	0.1	59	4138	0	1.9
	ssd0,h	0	0	0	0	0	0
	ssd1	0	0	32	2	0	0
	ssd4	6	0.1	51	4229	0	1.9
	ssd4,c	6	0.1	51	4229	0	1.9
	ssd4,h	0	0	0	0	0	0
	ssd5	0	0	32	2	0	0
	st2	0	0	0	0	0	0

is it the above is the I/O performance data.
but how do i know those sd0, sd1 .... is refer to which device?
please help ! =)

2

I think you should check /etc/path_to_inst file.

3

Here is one way:

pr -tm <(iostat -x | tail +3 | nawk '{print $1}') <(iostat -xn | tail +3 | nawk '{print $11}')
4

thanks for help bartus11

but i currently still not able to understand and no idea to this file.
please advise =)

root@mercury # cat path_to_inst
#
#       Caution! This file contains critical kernel state
#
"/pseudo" 0 "pseudo"
"/scsi_vhci" 0 "scsi_vhci"
"/options" 0 "options"
"/pci@8,4000" 0 "pcicmu"
"/pci@8,4000/ebus@1" 0 "ebus"
"/pci@8,4000/ebus@1/serial@14,400000" 0 "su"
"/pci@8,4000/ebus@1/scfc@14,200000" 0 "scfd"
"/pci@8,4000/ebus@1/panel@14,280030" 0 "oplpanel"
"/pci@0,600000" 0 "px"
"/pci@0,600000/pci@0" 0 "pxb_plx"
"/pci@0,600000/pci@0/pci@8" 1 "pxb_plx"
"/pci@0,600000/pci@0/pci@8/pci@0" 0 "px_pci"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1" 0 "mpt"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@0,0" 0 "sd"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@1,0" 2 "sd"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@3,0" 4 "sd"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/st@2,0" 2 "st"
"/pci@0,600000/pci@0/pci@8/pci@0/network@2" 0 "bge"
"/pci@0,600000/pci@0/pci@8/pci@0/network@2,1" 1 "bge"
"/pci@0,600000/pci@0/pci@8/pci@0,1" 1 "px_pci"
"/pci@0,600000/pci@0/pci@8/pci@0,1/SUNW,XVR-100@1" 0 "pfb"
"/pci@0,600000/pci@0/pci@9" 2 "pxb_plx"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0" 0 "qlc"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0" 4 "fp"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/ssd@w5006016141e0a4cd,0" 0 "ssd"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/ssd@w5006016841e0a4cd,0" 2 "ssd"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/ssd@w5006016141e0a4cd,1" 4 "ssd"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0,1" 1 "qlc"
"/pci@0,600000/pci@0/pci@9/SUNW,qlc@0,1/fp@0,0" 0 "fp"
"/pseudo-console" 0 "oplmsu"
"/pci@2,600000" 2 "px"
"/pci@2,600000/SUNW,qlc@0" 2 "qlc"
"/pci@2,600000/SUNW,qlc@0/fp@0,0" 1 "fp"
"/pci@2,600000/SUNW,qlc@0/fp@0,0/ssd@w5006016841e0a4cd,0" 1 "ssd"
"/pci@2,600000/SUNW,qlc@0/fp@0,0/ssd@w5006016141e0a4cd,0" 3 "ssd"
"/pci@2,600000/SUNW,qlc@0/fp@0,0/ssd@w5006016841e0a4cd,1" 5 "ssd"
"/pci@2,600000/SUNW,qlc@0,1" 3 "qlc"
"/pci@2,600000/SUNW,qlc@0,1/fp@0,0" 2 "fp"
"/pseudo-mc@200,200" 0 "mc-opl"
"/pci@1,700000" 1 "px"
"/pci@1,700000/network@0" 0 "nxge"
"/pci@1,700000/network@0,1" 1 "nxge"
"/pci@1,700000/network@0,2" 2 "nxge"
"/pci@1,700000/network@0,3" 3 "nxge"
"/pci@3,700000" 3 "px"
"/iscsi" 0 "iscsi"
root@mercury #

---------- Post updated at 03:56 PM ---------- Previous update was at 03:51 PM ----------

erm.....
when i run the command

it come out the error, because me still try to learn the scripts, now still not very good in this area. :frowning: please help

root@mercury # pr -tm <(iostat -x | tail +3 | nawk '{print $1}') <(iostat -xn | tail +3 | nawk '{print $11}')
syntax error: `(' unexpected
5

Well, in theory you should be able to identify devices by looking at their device path, and comparing device driver name from "device" column in iostat output with last column of that file.

6

i try to run the below command, it show me this, but i still no idea sd0, sd1 .... is refer to which device :frowning:

root@mercury # iostat -xtc
                 extended device statistics                    tty         cpu
device    r/s    w/s   kr/s   kw/s wait actv  svc_t  %w  %b  tin tout  us sy wt id
md0       2.0    3.0   23.2    2.9  0.0  0.1   23.9   1   2    0   46  22  7  0 70
md1       0.2    0.0    1.7    2.0  0.0  0.0    5.3   0   0
md3       0.1    1.4    0.5    9.5  0.0  0.0   16.5   0   1
md10      2.0    3.0   23.2    2.9  0.0  0.1   19.1   0   2
md11      0.2    0.0    1.7    2.0  0.0  0.0    5.0   0   0
md13      0.1    1.4    0.5    9.5  0.0  0.0   12.4   0   1
md20      0.0    3.0    0.0    2.9  0.0  0.1   24.9   0   1
md21      0.0    0.0    0.0    2.0  0.0  0.0   22.2   0   0
md23      0.0    1.4    0.0    9.5  0.0  0.0    9.4   0   1
sd0       2.3    5.9   25.4   15.1  0.0  0.1   18.4   0   4
sd2       0.0    4.4    0.0   14.4  0.0  0.1   19.9   0   2
sd4       0.0    0.0    0.0    0.0  0.0  0.0    0.0   0   0
ssd0    120.2   51.4 4666.4 2836.7  0.0  0.4    2.5   0  19
ssd1      0.1    0.0    0.0    0.0  0.0  0.0    0.0   0   0
ssd4     84.6  105.4 2688.7 1529.4  0.0  0.3    1.3   0  14
ssd5      0.1    0.0    0.0    0.0  0.0  0.0    0.0   0   0
st2       0.0    0.0    0.0    0.0  0.0  0.0    0.0   0   0
nfs1      0.0    0.0    0.0    0.0  0.0  0.0    0.0   0   0
nfs2      0.0    3.9    0.0  125.8  0.1  0.1   50.1   1   1
nfs3      0.0    0.0    0.0    0.0  0.0  0.0    6.5   0   0
nfs4      0.0    0.0    0.0    0.0  0.0  0.0    3.6   0   0
nfs5      0.1   12.4    1.9  396.1  0.4  0.4   59.9   4   6
root@mercury #
7

To assign disk driver names to their logical names (c0t0d0s0 etc) run "format". It will display list of disks with their device path and logical name.

8

erm... i run the format command, how do know the EMC which device? sd0? ssd0 ?

root@mercury # format </dev/null
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@0,0
       1. c0t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@1,0
       2. c1t0d0 <DGC-RAID10-0326 cyl 61438 alt 2 hd 256 sec 40>
          /pci@0,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/ssd@w5006016141e0a4cd,0
       3. c1t0d1 <DGC-RAID10-0326 cyl 61438 alt 2 hd 256 sec 40>
          /pci@0,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/ssd@w5006016141e0a4cd,1
       4. c3t0d0 <DGC-RAID10-0326 cyl 61438 alt 2 hd 256 sec 40>
          /pci@2,600000/SUNW,qlc@0/fp@0,0/ssd@w5006016841e0a4cd,0
       5. c3t0d1 <DGC-RAID10-0326 cyl 61438 alt 2 hd 256 sec 40>
          /pci@2,600000/SUNW,qlc@0/fp@0,0/ssd@w5006016841e0a4cd,1
       6. emcpower0a <DGC-RAID10-0326 cyl 61438 alt 2 hd 256 sec 40>
          /pseudo/emcp@0
       7. emcpower1a <DGC-RAID10-0326 cyl 61438 alt 2 hd 256 sec 40>
          /pseudo/emcp@1
Specify disk (enter its number):
root@mercury #
9

Use ksh or bash.

10

erm..

i create iostat.sh file, and try to use all the bash,ksh,sh,
all come out the same error, any idea?

root@mercury # vi iostat.sh
"iostat.sh" 4 lines, 114 characters
#! /usr/bin/ksh

pr -tm <(iostat -x | tail +3 | nawk '{print $1}') <(iostat -xn | tail +3 | nawk
'{print $11}')

~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
"iostat.sh" 4 lines, 113 characters

root@mercury # sh /iostat.sh
/iostat.sh: syntax error at line 3: `(' unexpected
11

You are not using ksh or bash but still the legacy sh.

Run

ksh iostat.sh

or better

chmod +x iostat.sh
./iostat.sh
12

yes, work nice ! but how i know which one is refer to EMC storage ?
** u01 and u02 is the storage from EMC

root@mercury # ./iostat.sh
Usage: nawk [-f programfile | 'program'] [-Ffieldsep] [-v var=value] [files]
./iostat.sh[4]: {print $11}:  not found
md0
md1
md3
md10
md11
md13
md20
md21
md23
sd0
sd2
sd4
ssd0
ssd1
ssd4
ssd5
st2
nfs1
nfs2
nfs3
nfs4
nfs5
root@mercury #

root@mercury # df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          20G   6.4G    13G    33%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   9.9G   1.6M   9.9G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
fd                       0K     0K     0K     0%    /dev/fd
/dev/md/dsk/d3          24G   2.9G    21G    13%    /var
swap                   9.9G   1.0M   9.9G     1%    /tmp
swap                   9.9G    88K   9.9G     1%    /var/run
swap                   9.9G     0K   9.9G     0%    /dev/vx/dmp
swap                   9.9G     0K   9.9G     0%    /dev/vx/rdmp
/dev/vx/dsk/smsdg/vol02
                       300G   140G   150G    49%    /u02
/dev/vx/dsk/smsdg/vol01
                       300G   216G    79G    74%    /u01
centaurus:/u01/oracle/sms5
                       404G   380G    20G    96%    /u07
centaurus:/u02/oracle/dss5
                       404G   398G   1.4G   100%    /u08
gemini:/u09            492G    39G   449G     8%    /u09
gemini:/u10            492G   132G   356G    27%    /u10
root@mercury #
13

The command I posted is on a single line. It appears you split it.

That was quite an understatement :wink:

1 Like
14

haha.. you are right

# ./haha.sh
md0                                 md/d0
md1                                 md/d1
md3                                 md/d3
md10                                md/d10
md11                                md/d11
md13                                md/d13
md20                                md/d20
md21                                md/d21
md23                                md/d23
sd0                                 c0t0d0
sd2                                 c0t1d0
sd4                                 c0t3d0
ssd0                                c1t0d0
ssd1                                c3t0d0
ssd4                                c1t0d1
ssd5                                c3t0d1
st2                                 rmt/0
nfs1                                mercury:vold(pid1597)
nfs2                                centaurus:/u01/oracle/sms5
nfs3                                centaurus:/u02/oracle/dss5
nfs4                                gemini:/u09
nfs5                                gemini:/u10

but it din show any device from the EMC box,
is it the iostat cannot catch the I/O info for the SAN ???

15

Just use the -n switch of iostat to get the logical device names instead of the instance names.

$ iostat -xn
                    extended device statistics
    r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
    2.5    0.7   19.8    2.4  0.0  0.0    0.0    2.5   0   0 c0t0d0
    0.7    0.4   19.0    1.1  0.0  0.0    0.0    3.7   0   0 md/globfs/d10
    0.7    0.4   19.0    1.1  0.0  0.0    0.0    3.8   0   0 md/globfs/d20
    1.3    0.4   38.0    1.1  0.0  0.0    0.4    2.9   0   0 md/globfs/d0
    0.0    0.0    0.0    0.0  0.0  0.0    0.0   10.4   0   0 md/globfs/d51
    0.0    0.0    0.0    0.0  0.0  0.0    0.0    6.2   0   0 md/globfs/d50
    1.3    0.4   38.0    1.1  0.0  0.0    0.0    3.3   0   0 md/globfs/d52
    0.0    0.0    0.0    0.0  0.0  0.0    0.0    0.0   0   0 c1t0d0
    0.0    1.8    3.5  170.1  0.0  0.3    0.0  172.9   0   1 c6t60060160E1F02100686FCAB3D446DF11d0
    0.0    1.9    3.1  171.6  0.0  0.3    0.0  166.0   0   1 c6t60060160E1F021006A6FCAB3D446DF11d0
    0.0    2.0    2.5  186.8  0.0  0.4    0.0  177.1   0   1 c6t60060160E1F021006E6FCAB3D446DF11d0
    0.0    2.0    2.3  186.9  0.0  0.4    0.0  178.8   0   1 c6t60060160E1F021006F6FCAB3D446DF11d0
    0.0    2.0    2.6  186.8  0.0  0.4    0.0  173.9   0   1 c6t60060160E1F021006D6FCAB3D446DF11d0
...

---------- Post updated at 07:57 ---------- Previous update was at 07:41 ----------

You can see your EMC LUNs on Controller c1 and c3 without multipathing.

It seems, that EMC powerpath does not provide iostat information, because the pseudo devices are missing from the iostat output. I never used powerpath on our systems, always MPxIO. It works very well with Clariions and gives you disk IO measurements as shown above.

1 Like