Virtual Ethernet VIO HMC LPARs

Hi,

I am little confused about the virtual Ethernet configuration on VIO and Client Partitions. There is alot of info on the internet but it gets more confusing....

If I have LHEA, it is very simple. Just assign LHEA (logical host ethernet adapter) to client partition -> run smitty tcpip and do the configuration.

However, if I don't have LHEAs then I have to use SEA (shared ethernet adapter)

My VIO Server has ethernet adapter.

I would like to share this ethernet port on VIO Server for all my client partition traffic going outside ?

How would i create the shared ethernet ?

1) Do I have to create virtual ethernet card both in VIO & Client Partition

2) Map the physical port on VIO with the virtual port ?

3) then smitty tcpip on client partition ?

� Shared Ethernet Adapter (SEA) � A physical adapter or EtherChannel in
a VIO Server can be shared by client partitions. Clients use virtual
Ethernet adapters
Virtual Ethernet � a LPAR-to-LPAR Virtual LAN within a POWER5 Server
� Does not require the APV feature code

I believe there are 3 ways to create Virtual Ethernet on Client Partition

  1. LHEA -- the simplest method. Just assign the LHEA from HMC to the client partition, run cfgmgr and you will see en0

  2. Create Virtual Adapter in HMC on the client partition

  3. In the navigation pane, open Systems Management > Servers, and click the system on which the logical partition is located.

  4. In the work pane, select the logical partition on which you want to configure the virtual Ethernet adapter, click the Tasks button, and click Dynamic Logical Partitioning > Virtual Adapters.

  5. Click Actions and choose Create > Ethernet Adapter.

  6. Enter the slot number for the virtual Ethernet adapter into Adapter ID. {filosophizer: Which slot number is this?}

  7. Enter the Port Virtual LAN ID (PVID) for the virtual Ethernet adapter into VLAN ID. The PVID allows the virtual Ethernet adapter to communicate with other virtual Ethernet adapters that have the same PVID.

  8. Select IEEE 802.1 compatible adapter if you want to configure the virtual Ethernet adapter to communicate over multiple virtual LANs. If you leave this option unchecked and you want this logical partition to connect to multiple virtual networks, you must create multiple virtual adapters by creating additional virtual LAN IDs.

  9. SEA (shared Ethernet Adapter)

  10. Create Virtual Adapter for VIO Server

  11. Run Cfgmgr

  12. mkvdev -sea target_device -vadapter virtual_ethernet_adapters \
    -default DefaultVirtualEthernetAdapter -defaultid SEADefaultPVID
    {filosophizer: what steps do I have to take for SEA on the client side ?}

thanks

P.S: The attachment has many methods to create virtual ethernet , but it is really confusing..

VIO WIKI: IBM Developer

This can be a tough concept to grasp - I'll list the steps I do.

  1. Edit VIO profile from the HMC like you list in #2.
  • I noticed that the steps you listed in #2 go into the 'Dynamic Logical Partitioning' but be careful because if you don't make changes to the profile also then when you reboot the lpar (VIO or client) all changes will be lost.
  • Make sure you put a checkmark in 'Access external network Trunk priority'. Also , make note of which adapter ID you give it (it doesn't matter what the number is, this will be used later when you do the mkvdev command). I used 23 because we have a .23.xxx network segment and a .41.xxx segment (you can guess what I used for the other virtual ethernet adapter id).
  1. Edit the lpar client's profile from the HMC and do all of the steps you listed under #2
  • BUT do NOT put a checkmark in 'Access external network'
  • Use the same Adapter ID you made note of in my Step 1. I also use the same ID for the VLAN ID.
  1. Create the SEA on VIO1 - (on our system the 23 segment cable is plugged into ent1)
    Example from my VIO1:
    # lsdev -Cc adapter |grep ent

ent0 Available 02-08 10/100/1000 Base-TX PCI-X Adapter (14106902)
ent1 Available 0A-00 10/100/1000 Base-TX PCI-Express Adapter (14104003)
ent2 Available 0A-01 10/100/1000 Base-TX PCI-Express Adapter (14104003)
ent3 Available 08-08 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent4 Available 08-09 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent5 Available Virtual I/O Ethernet Adapter (l-lan)
ent6 Available Virtual I/O Ethernet Adapter (l-lan)

## Find 23 segment (C23) virtual adapter (Ex. ent5)
# lscfg | grep ent | grep Virtual

  • ent5 U9117.MMA.10FF11F-V3-C23-T1 Virtual I/O Ethernet Adapter (l-lan)
  • ent6 U9117.MMA.10FF11F-V3-C41-T1 Virtual I/O Ethernet Adapter (l-lan)

## So we will create the SEA using ent1 as the physical adapter and ent5 as the virtual adapter
$ mkvdev -sea ent1 -vadapter ent5 -default ent5 -defaultid 23

  1. Boot the client lpar or run cfgmgr if it's already up and running - then configure the adapter as usual with 'smitty tcpip'

Hope this helps.
Homeyjoe

homeyjoe, thanks for the info. It was helpful.

However, I have few comments:

1) On the VIO SERVER
I did mark YES Access external network & Trunk priority = 2

2) On the VIO SERVER
I did mark YES IEEE.Q Protocol
{ When I marked YES on this one, I couldn't ping from my laptop which is connected to the switch to the VIO SERVER which is also connected to the switch and when I un-marked it, I was able to ping from my laptop to the VIO server... But when it was marked YES, between the AIX servers (LPARS) I was able to ping the VIO Server.....I don't understand this one ? Can anyone explain }

3) To each AIX LPARs.. you mentioned I should not check YES Access external network & Trunk priority = ?
{could you please explain this... will I be able to access outside/external network}

4) As an experiment, I did check YES Access external network & Trunk priority = ? on each LPARs...
But when I give it the same Trunk Priority as VIO SERVER, my LPAR couldn't detect the Virtual Ethernet Adapter. As soon as the Trunk Priority number is different.. I can see the virtual adapter using cfgmgr.
{ -Do I need to check Access outside network for each LPAR ? }
{ -Should I always give different Trunk Priority and what does it mean ?}

Thanks

The trunk priority is not important on the LPAR but is on a dual VIO configuration. Also the access external network MUST NOT be selected on the LPAR as there is no external network to access as it is all logical!

I have removed the Trunk priority and access to external network from the LPARs... and it is working fine.

I am facing another issue.

I would like to create another Virtual Ethernet Adapter and give it a new IP - different segment and different subnet mask..

LPAR 1
My en0 ==> 172.16.101.10
G/W ==> 172.16.101.1
Subnet mask: 255.255.255.0
VLAN ID: 1
IEEE.8 = NO

For my en1 ==> 10.10.10.10
G/W -- No gateway since it is connected to the same switch as above (Cisco Switch) but this IP is going to be used between LPARs only for RAC ( Oracle Real Application Cluster )
VLAN ID: 1
IEEE.8 = NO
subnet mask: 255.255.255.252

LPAR 2

My en0 ==> 172.16.101.11
G/W ==> 172.16.101.1
Subnet mask: 255.255.255.0
VLAN ID: 1
IEEE.8 = NO

For my en1 ==> 10.10.10.11
G/W = No Gateway
VLAN ID: 1
IEEE.8 = NO
subnet mask: 255.255.255.252

MY VIO Server 1
physical Port ==> en1 ==> 172.16.101.2
G/W: 172.16.101.1
VLAN ID: 1
IEEE.8 = NO
Subnet mask: 255.255.255.0

Physical Port
virtual en3 ( mapped with en0 for external ) ==> 172.16.101.3
G/W: 172.16.101.1
VLAN ID: 1
IEEE.8 = NO
Subnet mask: 255.255.255.0

MY VIO Server 2
physical Port ==> en1 ==> 172.16.101.4
G/W: 172.16.101.1
Subnet mask: 255.255.255.0

Physical Port
virtual en3 (mapped with en0 for external) ==> 172.16.101.5
G/W: 172.16.101.1
Subnet mask: 255.255.255.0

But I cannot ping
from LPAR1 to LPAR2 on the IP 10.10.10.11

1) Do I need to add a different VLAN ID in each LPAR for 10.10.10.x SUB-NET ?

2) Do I need to add a different VLAN ID and create a virtual ethernet adapter in VIO_Server ?

3) Do I need to enable the IEEE8 ?

any help would be appreciated.

Guys I need help, any thoughts & comments would be appreciated.

I have two VIOS

VIO_1
en0 = physical
mkvdev -sea ent0 -default ent2 -defaultid 101
en1 = physical / 172.16.101.10
ent0 = adapter with two ports
en3 = virtual / sea / 172.16.101.11

VIO_2
en0 = physical
mkvdev -sea ent0 -default ent2 -defaultid 101
en1 = physical / 172.16.101.12
ent0 = adapter with two ports
en3 = virtual / sea / 172.16.101.13

Lpar_1
en0 = 172.16.101.20
en1 = 10.10.10.20

Lpar_2
en0 = 172.16.101.21
en1 = 10.10.10.20

Lpar_3 ( HACMP )
en0 = 172.16.101.21
en1 = 10.10.10.20
en2 = 172.16.102.21 {boot IP}
en3 = 172.16.103.21 {Admin IP}

in HMC

  • what is the VLAN ID, I should give to VIO_1 ?
    { should it be same as VLAN_ID i give it to LPAR_1 ? }

  • VLAN ID for each LPARs for the subnet 10.x.x.x ?

  • How can the subnet 10.x.x.x & 172.16.102.21 reach external network ?

any help would be appreciated.

Quick Questions:

1) I have two virtual ethernet adapters on my VIO_Server.. Can I create more than one SEA for One Physical Ethernet Adapter

for ENT0
mkvdev -sea ent0 -vadapter ent2 -default ent2 -defaultid 1
en4

then again with a different VLAN ID
mkvdev -sea ent0 -vadapter ent3 -default ent3 -defaultid 2
en5

2) Can I create etherchannel for my physical ports on VIO Server
ENT0 + ENT1 = ENT2
and create SEA for this ENT2 ?

thanks

Answers:
1) I think so but haven't tried it anywhere

2) Definitely yes - I do this on my 2 VIOs and then create a Etherchannel composed of the 2 SEAs (1 from each VIO) on the client LPAR for redundancy.

Thanks for the reply.

The only problem which I face is with the VLANs. May be you can help me out.

on my VIO:
My Virtual Adapter has the following properties
Default VLAN ID: 102
Additional VLANs to accept: 103
IEEE 801.Q ==> YES
Trunk ==> YES
This one is mapped to the physical adapter ENT1 using the command
mkvdev -sea ent0 -vadapter ent3 -default ent3 -defaultid 102
ent4 Available
et4
so I get en4 ==> I give it the IP 172.16.102.40
I can ping my gateway and outside..
No issues what-so-ever.

My Client LPAR has two virtual ethernet adapters
en0 ==> default vlan id: 102 (IP 172.16.102.50)
I can ping outside // to gateway....no issues what-so-ever

en1 ==> default vlan id: 103 (IP 172.16.103.50) {I can't use this yet because SEA has not been configured to route the packets to external... I try pinging.. nothing happens}

IEEE 801.Q ==> NO
Trunk ==> NO

Now I go back to my VIO try to add the 103 VLAN to my SEA
this is how I add
mkvdev -vlan ent2 -tagid 103
ent5 Available
en5
et5

*** However, I cannot give any IP address to en5, it gives me error..
But that's not the issue....
I want my LPAR which is using 172.16.103.50 to go to external network.
But it doesn't go out.

Tried so many methods...really exhausted..

read several documents:
latest one: Redirect Notice
IBM - Create an SEA with vlan tagging and without

but doesn't help..

any info would be appreciated...

  1. I tend not to give IP's to the adapters I create for sea on the vio servers.
  2. Yes, you should use a different VLAN ID for each subnet you create virtual adapters for.
  3. The etherchannel setup is a good idea for failover between the two vio servers, but I would only use the backup adapter feature rather than combining the two adapters.

I know where my problem is now. finding the problem is 50% of the solution.

What I need is here

SEA: Bridging Three VLAN, untagged
VLAN 101,102,103

Command:
$mkvdev -sea ent0 -vadapter ent1 -default ent1 -defaultid ???

$mkvdev -sea ent0 -vadapter ent1 -default ent1 -defaultid 101,102,103

VLAN 101,102,103 frames in the Hypervisor are bridged to the external network as untagged frames

Untagged frames from the external network are tagged with VLAN 101,102,103 and bridged to the Hypervisor switch

Tagged VLAN 101,102,103 frames from the external network are bridged through ent1.

Postscript:

Some Documents:
http://lpar.co.uk/TP/VLAN.pdf
http://learnitfromshiva.wordpress.com/2009/07/08/aix-lpar/
http://www.ibm.com/developerworks/forums/thread.jspa?threadID=130502
http://www.ibmsystemsmag.com/aix/enewsletterexclusive/20127p1.aspx