Need Help with SDD / SDDPCM / MPIO

filosophizer · May 7, 2016, 4:35pm

This is getting very confusing for me, and appreciate if someone can help.

Platform: Power VM ( Virtual I/O Server)
ioslevel 2.1.3.10-FP23
# oslevel -s
6100-05-00-0000
Storage: IBM DS4300
Two HBAs - Dual Port Fibre Adapter Channels
Each card has two ports , so a total of 4 ports going through two San Switches into one SAN box

Support Matrix

# pcmpath
ksh: pcmpath:  not found.

# lsvpcfg
ksh: lsvpcfg:  not found.


# manage_disk_drivers -l
Device           Present Driver     Driver Options
2810XIV          AIX_AAPCM          AIX_AAPCM,AIX_non_MPIO
DS4100           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4200           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4300           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4500           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4700           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS4800           AIX_APPCM          AIX_APPCM,AIX_fcparray
DS3950           AIX_APPCM          AIX_APPCM
DS5020           AIX_APPCM          AIX_APPCM
DS5100           AIX_APPCM          AIX_APPCM
DS3500           AIX_APPCM          AIX_APPCM


# lslpp -l | grep mpio
  devices.common.IBM.mpio.rte
  devices.common.IBM.mpio.rte


# lslpp -Lc | grep -E "sdd|fcp.disk.ibm"
#


# lspath -H
status  name   parent
Enabled dac0   fscsi0
Enabled dac1   fscsi0
Enabled dac0   fscsi1
Enabled dac1   fscsi1
Enabled dac0   fscsi2
Enabled dac1   fscsi2
Enabled dac0   fscsi3
Enabled dac1   fscsi3
Enabled hdisk8 fscsi0
Enabled hdisk9 fscsi0
Enabled dac1   fscsi2
Enabled hdisk8 fscsi1
Enabled hdisk9 fscsi1
Enabled hdisk8 fscsi2
Enabled hdisk9 fscsi2
Enabled hdisk8 fscsi3
Enabled hdisk9 fscsi3
#

# lspv
hdisk8          00c7780e8979b683                    None
hdisk9          00c7780e8979b683                    None
#


# lsdev -Cc disk
hdisk8 Available 0A-08-02     MPIO Other DS4K Array Disk
hdisk9 Available 0A-08-02     MPIO Other DS4K Array Disk

# lsattr -El fscsi0
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10200      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True
#

# lsattr -El fscsi1
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10200      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True

# lsattr -El fscsi2
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10300      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True


# lsattr -El fscsi3
attach       switch       How this adapter is CONNECTED         False
dyntrk       no           Dynamic Tracking of FC Devices        True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id      0x10300      Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True
#

The confusing part for me, i read somewhere

However, the SDDPCM fileset might be required on a VIOS partition for attached DS8000 and/or SVC devices and you might want to attach also DS4000/DS5000 devices to the same VIOS. A supported configuration with SDDPCM on VIOS2.1 for DS8000 and/or SVC devices and native AIX MPIO for DS4k/DS5k devices might be achived by using SDDPCM 2.5.1.0 on VIOS 2.1.2.10-FP-22.1 (or higher) and manually deselecting the SDDPCM driver option for the DS4k/DS5k devics with the manage_disk_drivers command. Such a configuration would be based on:
VIOS 2.1.2.10-FP-22.1 (or higher) required by SDDPCM 2.5.1.0 and including the new manage_disk_drivers command
SDDPCM 2.5.1.0 (or higher) fileset with appropriate SDDPCM prerequisites (e.g. devices.fcp.disk.ibm.mpio.rte, V1.0.0.20)
selecting AIX_APPCM as present driver for attached DS5k/DS4k subsystems with manage_disk_drivers command (i.e. deselect AIX_SDDAPPCM)
AIX MPIO using SDDPCM for DS8000 / SVC devices (i.e. only the DS8000/SVC devices are managed by SDDPCM)
AIX MPIO using native AIX PCM for all DS4k/DS5k devices (i.e. all DS4k/DS5k devices are managed by the default AIX MPIO driver with the native AIX PCM)

How can I solve this issue of Same Disk appearing Twice (see hdisk8 and hdisk9 have the same PVIDs)

How should I proceed ?

---------- Post updated 05-07-16 at 12:35 PM ---------- Previous update was 05-06-16 at 01:06 PM ----------

+-----------------+
|  host           | 
+-----------------+
   |       |
   | 3     | 4
   |       |
+-----------------+
|  FC-Switch      |
+-----------------+
   |       |
   | 1     | 2
   |       |
+-----------------+
|  Storage        |
+-----------------+

After a little more study, I have come to known that it is important to have the right zoning, on this forum, I have read

I have 2 x Dual Port HBA

HBA 1
FCS0
FCS1

HBA 2
FCS2
FCS3

What is the best zoning practice ?

fcs0 (HBA 1) -----> switch A <------ controller A

fcs3 (HBA 2) -----> switch B <------ controller B

or could i make it like this

fcs0 (HBA 1) -----> switch A <------ controller A
fcs1 (HBA 1) -----> switch B <------ controller B

fcs3 (HBA 2) -----> switch B <------ controller B
fcs4 (HBA 2) -----> switch A <------ controller A

Explanation from Bakunin

It has nothing to do with how many ports one adapter has. A "path" is just a way of looking at one disk. The short answer is, a "path" is a certain connection from the system to the disk, using a specific way.

Here is the long version: consider the following setup. You have one FC-Adapter (two ports, FCa, FCb), which is connected to two FC-switches (SWa, SWb), which both connect to a storage system. The switches are themselves connected via a crossover-link. You create a single disk on this storage system. This disk will be seen over different paths: FCa-SWa-Disk, FCa-SWa-SWb-Disk, FCb-SWb-Disk, FCb-SWb-SWa-Disk. AIX will therefore create 4 different hdisk-devices when you run cfgmgr.

Now, this is not very comfortable, especially when you create several disks. Each disk will be seen via 4 paths (if you have a fabric more complex than the one i sketched above you might have even more paths) and soon you will have trouble to remember that hdisk137 is in fact hdisk49 and also hdisk87 and hdisk3.

This is where MPIO comes into play: it takes all the hdisk-devices which represent in fact one and only one disk (its like taking a picture of a building from several angles - regardless of how many pictures you take it is always the same one building) and builds a "super-device" above them. You can use this super-device to build VGs, LVs and all other sorts of things you use normally disks for. The MPIO (Multipath-I/O) driver will take care about the relationship of the different hdisk devices and do the bookkeeping for you (and the OS as well).

And there is an additional advantage to it: suppose Switch A in the above picture would fail. Two of the four hdisks (the two paths involving SWa) would be inaccessible, because these pathes would not be usable, but the super-device still works, because it uses ALL paths and if two fail there are still two more to use. In fact this is the main reason one builds such complex fabrics with parallel FC-switches, crossover-links and the like.

and old post

My only concern is not to see the HdiskX as double, with same PVID? any idea folks ?

filosophizer · May 8, 2016, 4:37pm

Further developments

Present Config

# lspcmcfg
hdisk8 (Avail pv ) 600A0B800019B057000010A3572E3AC5 = path0 (Enabled) path1 (Enabled) path2 (Ena
# pcmpath query device

Total Active/Passive Devices : 1


DEV#:   8  DEVICE NAME: hdisk8  TYPE: 1742-900  ALGORITHM:  Load Balance
SERIAL: 600A0B800019B057000010A3572E3AC5
==========================================================================
Path#      Adapter/Path Name          State     Mode     Select     Errors
    0           fscsi0/path0          CLOSE   NORMAL          0          0
    1           fscsi1/path1          CLOSE   NORMAL          0          0
    2           fscsi2/path2          CLOSE   NORMAL          0          0
    3           fscsi3/path3          CLOSE   NORMAL          0          0
#
#
#
#       lspath
Enabled hdisk0 scsi2
Enabled hdisk1 scsi2
Enabled hdisk2 scsi2
Enabled hdisk3 scsi2
Enabled hdisk4 scsi3
Enabled hdisk5 scsi3
Enabled hdisk6 scsi3
Enabled hdisk7 scsi3
Enabled hdisk8 fscsi0
Enabled dac0   fscsi0
Enabled hdisk9 fscsi0
Enabled dac1   fscsi0
Enabled hdisk8 fscsi1
Enabled dac0   fscsi1
Enabled hdisk9 fscsi1
Enabled dac1   fscsi1
Enabled hdisk8 fscsi2
Enabled dac0   fscsi2
Enabled hdisk9 fscsi2
Enabled dac1   fscsi2
Enabled hdisk8 fscsi3
Enabled dac0   fscsi3
Enabled hdisk9 fscsi3
Enabled dac1   fscsi3
#
#

# manage_disk_drivers -l
Device           Present Driver     Driver Options
2810XIV          AIX_AAPCM          AIX_AAPCM,AIX_non_MPIO
DS4100           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4200           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4300           AIX_APPCM          AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4500           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4700           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS4800           AIX_SDDAPPCM       AIX_APPCM,AIX_fcparray,AIX_SDDAPPCM
DS3950           AIX_SDDAPPCM       AIX_APPCM,AIX_SDDAPPCM
DS5020           AIX_SDDAPPCM       AIX_APPCM,AIX_SDDAPPCM
DS5100           AIX_SDDAPPCM       AIX_APPCM,AIX_SDDAPPCM
DS3500           AIX_APPCM          AIX_APPCM
#
#
#
# lspv
hdisk0          00c7780e79838606                    rootvg          active
hdisk1          00c7780e2e21ec86                    diskpool_4      active
hdisk2          00c7780ea5bd16bb                    diskpool_4      active
hdisk3          00c7780ee224f286                    disk_pool_5     active
hdisk4          00c7780e1b75933b                    diskpool_3      active
hdisk5          00c7780ece91bde2                    diskpool_2      active
hdisk6          00c7780ec2b65f4d                    diskpool_1      active
hdisk7          00c7780e5293914b                    None
hdisk8          00c7780e9119f6d7                    None
hdisk9          none                                None
# lsdev -Cc disk
hdisk0 Available 09-08-00-3,0 16 Bit LVD SCSI Disk Drive
hdisk1 Available 09-08-00-4,0 16 Bit LVD SCSI Disk Drive
hdisk2 Available 09-08-00-5,0 16 Bit LVD SCSI Disk Drive
hdisk3 Available 09-08-00-8,0 16 Bit LVD SCSI Disk Drive
hdisk4 Available 09-08-01-3,0 16 Bit LVD SCSI Disk Drive
hdisk5 Available 09-08-01-4,0 16 Bit LVD SCSI Disk Drive
hdisk6 Available 09-08-01-5,0 16 Bit LVD SCSI Disk Drive
hdisk7 Available 09-08-01-8,0 16 Bit LVD SCSI Disk Drive
hdisk8 Available 0A-08-02     IBM MPIO DS4300 Array Disk
hdisk9 Available 0B-08-02     IBM MPIO DS4500 Array Disk

# lspcmcfg
hdisk8 (Avail pv ) 600A0B800019B057000010A3572E3AC5 = path0 (Enabled)
hdisk9 (Avail ) 600A0B800019B057000010A3572E3AC5 = path0 (Enabled)

Both hdisk8 and hdisk9 are same disks but one says IBM MPIO DS4300 and the other IBM MPIO DS4500 apart from showing it double.

So to make it simple:

Direct Connection (removed the san switch)
HBA 1 -- Port 1 -- To Controller A (SAN Storage )
HBA 2 -- Port 1 -- To Controller B (SAN Storage )

# lsdev -Cc disk

hdisk8 Available 0A-08-02     IBM MPIO DS4300 Array Disk
hdisk9 Available 0B-08-02     IBM MPIO DS4500 Array Disk

again seeing two disks

any solution / suggestion ?

sh_1580 · May 14, 2016, 8:30am

What is firmware level on Ds4300 ?
Please check compatible OS & multipath with that Firmware on IBM SSIC website.

Sent from my Le X507 using Tapatalk

filosophizer · May 21, 2016, 3:44pm

Firmware on DS4300 is 6.0.3.x
ALready checked the compatibility matrix

I am trying to upgrade vios to the latest version and will check what happens and post here so everyone can benefit

upgraded ios

# oslevel -s
6100-09-06-1543

$ ioslevel
2.2.4.10

manage_disk_drivers -d DS4500 -o AIX_APPCM
manage_disk_drivers -d DS4300 -o AIX_APPCM

did not work

---------- Post updated at 11:44 AM ---------- Previous update was at 12:36 AM ----------

Problem resolved

# fget_config -Av

---dar0---

User array name = ''
dac0 ACTIVE dac2 ACTIVE

Disk     DAC   LUN Logical Drive
utm             31
hdisk8   dac0    0 1
hdisk9   dac2    1 2
# lsdev -p fcs0
fcnet0 Defined   0A-08-01 Fibre Channel Network Protocol Device
fscsi0 Available 0A-08-02 FC SCSI I/O Controller Protocol Device
# lsdev -p fscsi0
dac0     Available 0A-08-02    1722-600 (600) Disk Array Controller
sfwcomm0 Available 0A-08-02-FF Fibre Channel Storage Framework Comm
#

# manage_disk_drivers -l
Device              Present Driver        Driver Options
2810XIV             AIX_AAPCM             AIX_AAPCM,AIX_non_MPIO
DS4100              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4200              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4300              AIX_fcparray          AIX_APPCM,AIX_fcparray
DS4500              AIX_fcparray          AIX_APPCM,AIX_fcparray
DS4700              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS4800              AIX_APPCM             AIX_APPCM,AIX_fcparray
DS3950              AIX_APPCM             AIX_APPCM
DS5020              AIX_APPCM             AIX_APPCM
DCS3700             AIX_APPCM             AIX_APPCM
DS5100/DS5300       AIX_APPCM             AIX_APPCM
DS3500              AIX_APPCM             AIX_APPCM
XIVCTRL             MPIO_XIVCTRL          MPIO_XIVCTRL,nonMPIO_XIVCTRL
2107DS8K            NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
IBMFlash            NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
IBMSVC              NO_OVERRIDE           NO_OVERRIDE,AIX_AAPCM,AIX_non_MPIO
#

# lsdev -Cc disk
hdisk0 Available 09-08-00-3,0 16 Bit LVD SCSI Disk Drive
hdisk1 Available 09-08-00-4,0 16 Bit LVD SCSI Disk Drive
hdisk2 Available 09-08-00-5,0 16 Bit LVD SCSI Disk Drive
hdisk3 Available 09-08-00-8,0 16 Bit LVD SCSI Disk Drive
hdisk4 Available 09-08-01-3,0 16 Bit LVD SCSI Disk Drive
hdisk5 Available 09-08-01-4,0 16 Bit LVD SCSI Disk Drive
hdisk6 Available 09-08-01-5,0 16 Bit LVD SCSI Disk Drive
hdisk7 Available 09-08-01-8,0 16 Bit LVD SCSI Disk Drive
hdisk8 Available 0A-08-02     1722-600 (600) Disk Array Device
hdisk9 Available 0B-08-02     1722-600 (600) Disk Array Device

manage_disk_drivers -d DS4300 -o AIX_fcparray

if DS4300 doesn;t work


manage_disk_drivers -d DS4500 -o AIX_fcparray