I am planning carving out dual VIO servers on a P750 based and would like to know how to allocate internal disks to the VIOS. This has 4 internal disks and I would to assign 2 disks to each VIO.
This is what I was thinking of assigning . Any advise?
vio1
U5802.001.9K8H757-P1-C1 PCI-E x8 Ext Dual-x4 3Gb SAS Adapter 288 Unassigned Unassigned
U5802.001.9K8H757-P1-C2 Ethernet controller 289 Unassigned Unassigned
U5802.001.9K8H757-P1-C3 Fibre Channel Serial Bus 290 Unassigned Unassigned
U5802.001.9K8H757-P1-C7 Fibre Channel Serial Bus 296 Unassigned Unassigned
U78A0.001.DNWKBTK-P1-C4 Ethernet controller 514 Unassigned Unassigned
U78A0.001.DNWKBTK-P1-T9 RAID Controller 512 Unassigned Unassigned
VIO2
U5802.001.9K8H757-P1-C4 PCI-E x8 Ext Dual-x4 3Gb SAS Adapter 292 Unassigned Unassigned
U5802.001.9K8H757-P1-C5 Fibre Channel Serial Bus 293 Unassigned Unassigned
U5802.001.9K8H757-P1-C6 Fibre Channel Serial Bus 294 Unassigned Unassigned
U78A0.001.DNWKBTK-P1-C5 Ethernet controller 515 Unassigned Unassigned
U78A0.001.DNWKBTK-P1-C2 PCI-E x8 Ext Dual-x4 3Gb SAS Adapter 517 Unassigned Unassigned
U78A0.001.DNWKBTK-P1-C3 Ethernet controller 519 Unassigned Unassigned
I don't see two RAID controllers in your configuration. Is the p750 suppose to come with two? The p740 comes with one in the P1-T9 and P1-C19. If you only have the one RAID controller on vio1 in P1-T9 and all 4 hdisks are attached to it, your vio2 server won't have any disks. You should probably check with your vendor about getting a second RAID controller ordered with it.
That was exactly my confusion to start with. On this P750, there are 3 SAS adapters and using the allocation as mentioned in the previous post, I was able to create 2 VIO partitions with 2 disks assigned to each VIO.
Disks are unique for each VIO. I have verified this. Just , I don't understand how is this working since RAID controller is assigned to 1 VIO.
Find the parent of hdisk0 on both of your vio servers and run a "lscfg -vl sas#" against both of them and that will tell you what slots the hdisks are connected to and you can see what is in those slots.... Here's an example of what I'm talking about:
Yes, DVD is attached to the Raid controller. For installing the second VIO I had to move the Raid controller to the other LPAR to get the DVD rom moved over.
I guess I am confused about how the SAS adapter function is ? My understanding is that RAID controller controls or maintains parity for the i/O to the disk.
So, does this mean that if the RAID controller goes bad, will it affect all the disks.
No, your RAID controller is only connected to your DVD drive. If it does go bad, just your DVD drive won't work. Your disks should remain unaffected. Assuming your LPARs are multipathed and have network connections going through each VIO server, if one of your SAS Adapters does go bad, the VIO server using it will go down till the parts are replaced, but since you have dual VIO servers, your LPARs should move their connections through the other VIO server that is still running and be fine. They'll throw a lot of errors in the errpt, but should continue to run.
This is the best explanation I can give you. Consider a situation where you have a VLAN (100 for this example) then this VLAN will exist as physical and virtual (in the Hypervisor). Your example above would result in ent0 and ent2 would be connected to the physical VLAN and by virtue of the etherchannel construction ent6 is also connected.
the -defaultid 100 flag sets the VLAN id to 100 for this virtual adapter.
would result in the ent7 adapter being created. This is the virtual adapter which would carry the virtual VLAN address for LPAR communication through the VIO servers. Both VIO servers would have the same characteristics set so there would be identical Ethernet adpaters on both VIOS but the trunk priority would be set to alternate values (1, 2 etc).
When creating the Ethernet adapter on the client LPAR one selects the VLAN 100 on the drop down menu on the HMC on both VIO servers. As the adapters on the VIO server have been configured to talk on VLAN 100 nothing else needs to be done except allocating an IP address when the LPAR is powered on. No IP address needs to be added to the VIO virtual ethernet adapter.
All traffic to the outside world from the client LPAR would be as follows:
Client LPAR Ethernet adpater ent0 > along the Hypervisor VLAN 100 > into the VIO server SEA ent7 > Upto the virtual adapter ent4 > through the etherchannel adapter ent6 > finally out of ent0 and ent2 to the real network infrastructure.
This unfortunately is the hardest part of VIO construction to get your head around!
Yes this is the common misconception. Have a look at the MKVDEV command. The virtual adapter ent4 was created using the menu from the HMC or if you are really adventurous keyboard commands on the HMC. I have seen both examples where the mid point adapter is classed as the SEA and the end adapter which connects to the Hypervisor VLAN as the virtual. However for the sake of clarity I feel the opposite is better as the virtual is created on the HMC for each VIO and the MKVDEV command connects the virtual to the each SEA adapter. Very confusing!
Thanks JohnF. I understand that SEA is a Layer 2 bridge between the physical and the virtual interface. But in this case , only ent1 has a I/P. So, without the I/p how is the trafiic send from ent0 or ent2 ?
Attached is a diagram of how I have the VIO networks layed out. IBM use to say to NOT put the VIO IP on the SEA, but I think they changed and now they are saying to put it on the SEA. I think I like it better on its own etherchannel with one virtual adapter connected to the two different internal VLANs. Hopefully the picture will make it a lot more clear. The "ent0" on both VIO servers is the physical adapter. "EC" is "EtherChannel".
Essentially, the VIO server's IP on the etherchannel ends up being configured just like all the LPARs... with one virtual adapter connected to each VLAN.
Thank you for sharing! This was exactly what I was looking for.Very good article. Rated no.vang thank a remark, an honest comment. I like to list on this list. hello nick chat: thienthien_bk
It will never work with the IP address on ENT1! I have no idea where ent1 is, is this on the VIO server, on the client LPAR? If it is on the client LPAR is it a physical adapter on a multi port card? If it is on the client LPAR but not a pysical port then there must be a second virtual adapter defined on the VIO server as logically ent0 on the client LPAR should (SHOULD) be connected to the first virtual adapter on the VIO (ENT7). You need to confirm exactly where ent1 is because at the moment I cannot comment on why ent1 has an IP address on it!
I have had another look at the listing and now fully understand what is going on and your problem!
The VIO server is fitted with a four port ethernet card hence ent0, 1, 2, and 3 which is not listed but must be there as the first virtual connector became ent4 with ent5 as its control channel (These were created on the HMC before the etherchannel adapter was created hence the etherchannel adapter is ent6). The etherchannel was then created probably using the following command on the VIO server:
mkvdev -lnagg ent0 ent2 -attr mode=8023ad
Following this the only virtual SEA was created on the VIO server using the following command:
mkvdev -sea ent6 �vadapter ent4 -default ent4 -defaultid 100 (This is the VLAN ID) -attr ha_mode=auto ctl_chan=ent5
All of this means the adapter ent1 is a physical port on the VIO server which is configured with a management / admin address (whichever your organisation uses).
As I said earlier you will need to connect the client LPARs virtual ethernet adapter to whichever VLAN id was declared when ent4 was created. To find this VLAN id you can either look at the VIO server on the HMC or run the command
entstat -d ent7 | more
which will list the VLAN id in the listing which is quite long!