FC disk and ioinfo

Phat · February 17, 2020, 6:12am

Hello, I have an issue with detecting the san disk on the OS side. Something looks strange for me.

My environment:

1 LPAR has 2 physical FCs
1 VIO has 2 physical FCs

on VIO, I can see the hdisk5(LUN) and use it as root disk.

Enabled   hdisk5 fscsi0
Enabled   hdisk5 fscsi0
Enabled   hdisk5 fscsi4
Enabled   hdisk5 fscsi4

on LPAR, I can see the hdisk1 (LUN) and use it as root disk.

Enabled   hdisk1 fscsi0
Enabled   hdisk1 fscsi0
Enabled   hdisk1 fscsi2
Enabled   hdisk1 fscsi2

Then, I create the virtual FC on lpar and map it with my VIO physical FC, we can see the port is logged by lpar.

$ lsnports
name             physloc                        fabric tports aports swwpns  awwpns
fcs0             U5802.001.9K81000-P1-C5-T1          1     64     63   2048    2043
fcs4             U78AA.001.WZSHVFA-P1-C5-T1          1     64     64   2048    2046

$ lsmap -all -npiv
Name          Physloc                            ClntID ClntName       ClntOS
------------- ---------------------------------- ------ -------------- -------
vfchost0      U8202.E4C.06337BT-V4-C5                 3 xyzT        FIRMWARE

Status:LOGGED_IN
FC name:fcs0                    FC loc code:U5802.001.9K81000-P1-C5-T1
Ports logged in:3
Flags:a<LOGGED_IN,STRIP_MERGE>
VFC client name:Virtual Fibre Channel - Firmware driverVFC client DRC:U8202.E4C.06337BT-V3-C5

But I cannot see it in the lspath command. However can see it with ioinfo command on the SMS, but marked with "Unrecognized device type: 3f"

FCINFO Main Menu
Select a FC Node from the following list:
  #  Location Code                Pathname
---------------------------------------------------------------
  1. U8202.E4C.06337BT-V3-C5-T1     /vdevice/vfc-client@30000005
  2. U5802.001.9K81000-P1-C6-T1     /pci@800000020000106/fibre-channel@0
  3. U5802.001.9K81000-P1-C6-T2     /pci@800000020000106/fibre-channel@0,1
  4. U78AA.001.WZSHVFA-P1-C6-T1     /pci@800000020000205/fibre-channel@0
  5. U78AA.001.WZSHVFA-P1-C6-T2     /pci@800000020000205/fibre-channel@0,1

  q - Quit/Exit

==> 1

FC Node Menu
FC Node String: /vdevice/vfc-client@30000005
FC Node WorldWidePortName: c0507606437b0000
-----------------------------------------------------------------
1. List Attached FC Devices
2. Select a FC Device
3. Enable/Disable FC Adapter Debug flags

q - Quit/Exit

==> 1

1. 500507680b31d7fe,0                - Unrecognized device type: 3f
2. 500507680b31d7ff,0                - Unrecognized device type: 3f

I even tried to remove 1 physical FC of lpar to check if zoning is working properly, so we can see "missing" below. This means that zoning and lun mapping is working properly.

Missing   hdisk1 fscsi0
Missing   hdisk1 fscsi0
Enabled   hdisk1 fscsi2
Enabled   hdisk1 fscsi2

I could say that nothing wrong at the storage side about zoning & lun mapping, the problem is probably at OS side.

So can someone has the idea to overcome this? I have tried to fix and search for "Unrecognized device type: 3f" but seems not much resources to look into it.

Phat · February 17, 2020, 9:54am

Just thinking, if we can use both physical and virtual FC for same LUN?

Phat · February 18, 2020, 12:51am

no one has an idea on this

RecoveryOne · February 18, 2020, 7:23pm

Taking a massive stab in the dark, but looks like if you removed the physical hba assigned to the lpar from the lpar and then your path for hdisk0 went missing: Your disks are not coming from the VIO. Without going into an in depth look at your system, are you sure you are presenting disks up from the VIO? Are you sure your lpar has virtual adapters presented up from the VIO? I mean, the whole point of running a VIO is so that takes all (or most) of your adapters so you can present them to the lpars as part of the virtualization layer. There should be very few instances where you need physical adapters assigned the a lpar running under a VIO.

Also from ioinfo from the openfirmware, its common to see that message. openfirmware is not aware so to speak to anything not apart of the system.

IF I manage to find time, I will show you how one of my environments are setup. Currently a lot of cruft on my plate that I must address first. Also, you should note that I'm not using NPIV, and my setup may not be the 'IBM recommended' way.
In the meantime, here's some handy links:
Access Denied
And since you are using npiv:
AIX for System Administrators

zxmaus · February 18, 2020, 9:47pm

I have no idea what you are saying - this all sounds confusing.

Maybe start over again. Your frame has 4 fibre channel adapters
you have allocated two to the vio and 2 to an lpar.
And now what - you are trying to map the adapters from the lpar via vio to another lpar?
With NPIV you map to a virtual wwpn, while when you zone to a physical adapter, you map to the physical wwpn. So I do not understand how removing a fibre channel adapter would show anything ?

Phat · February 18, 2020, 11:24pm

Hi Zmaus,
The point 1 and 2 are correct. On the lpar I can see the LUN with 2 physical fibre.
Now I would like to use 1 more virtual fiber adapter to assign to lpar from VIO. And zoning virtual wwpn to the same LUN.

If everything is correct, I can see the LUN with 2 physical fibre adapters + 1 virtual fibre adapter.

--- Post updated at 05:24 AM ---

recoveryone:

Taking a massive stab in the dark, but looks like if you removed the physical hba assigned to the lpar from the lpar and then your path for hdisk0 went missing: Your disks are not coming from the VIO. Without going into an in depth look at your system, are you sure you are presenting disks up from the VIO? Are you sure your lpar has virtual adapters presented up from the VIO? I mean, the whole point of running a VIO is so that takes all (or most) of your adapters so you can present them to the lpars as part of the virtualization layer. There should be very few instances where you need physical adapters assigned the a lpar running under a VIO.

Also from ioinfo from the openfirmware, its common to see that message. openfirmware is not aware so to speak to anything not apart of the system.

IF I manage to find time, I will show you how one of my environments are setup. Currently a lot of cruft on my plate that I must address first. Also, you should note that I'm not using NPIV, and my setup may not be the 'IBM recommended' way.
In the meantime, here's some handy links:
Access Denied
And since you are using npiv:
AIX for System Administrators

Hi RecoveryOne,

I'm checking the links you shared. From my side, I presented the virtual adapter to both VIO and lpar. With the lsmap I can see the client has logged in the fibre channel, from the SAN switch, checked with storage team, I can see the virtual wwpn. Further check with them, I can be quite sure the zoning & LUN masking works properly, that virtual WWPN is the same zone member with physical wwpn. Removing physical wwpn, make the link missing, means that the virtual wwpn(which is same zone) should see the LUN as well.

A bit curious, if you did not use NPIV, so which method you are using now?

Phat · February 19, 2020, 2:26am

I guess you are using vscsi

RecoveryOne · February 19, 2020, 10:27am

Yes, using vscsi.
In the 'planning' phase of moving to npiv, but my infrastructure is a tad behind the times so to speak and my switches don't support vscsi.
When I had the chance to use npiv, due to time constraints I wasn't able to switch a box over to it. One of those projects where they needed it three weeks ago, so I just did it the quick and dirty way that I knew would work, and work well.

So if you are seeing it logged in, did you run cfgmgr after all the zoning?

Phat · February 19, 2020, 11:34am

Yes, I did the cfgmgr and even reboot the box, but no help. I guess that maybe my aix box is very old 5.3 TL 9.
Another possibility, it cannot support both physical and virtual at the same time, just assuming.

zxmaus · February 19, 2020, 10:17pm

ok now I get what you are saying. How to fix this can be found here

Phat · February 20, 2020, 1:03am

I tried your link and what I can but not success

Enabled   hdisk1 fscsi2
Enabled   hdisk1 fscsi2
Enabled   hdisk1 fscsi0
Enabled   hdisk1 fscsi0

I feel 1 thing strange here is that the disk is detected twice by each physical fibre adapter. I think it should be like below:

Enabled   hdisk1 fscsi2
Enabled   hdisk1 fscsi0

zxmaus · February 20, 2020, 12:05pm

I would not be so concerned about the duplicates - we had that a lot on our old boxes with no issues.
Anyways, as your have an OS already, can you see the virtual adapter itself on the VM ?

Phat · February 21, 2020, 6:20am

Hello,

I can see virtual adapter on both VIO and lpar. On VIO, I can see the port logged in and VIO recognize the name of lpar as well.

I decided to stop on this, as I figure out that, when I ask the storage admin to unzone the physical fiber adapter wwpn on VIO, but the LUN is still there, the multipath/link still "Enabled" instead of missing. I concluded there is something wrong at their end