Can not use zone in LDoms on T4-1

ad101 · July 21, 2013, 8:58pm

I have a T4-1 with Solaris 11.1, LDom ver 3.0.0.1 and I'm encountering two problems with zones.

I'd greatly appreciate help on these, especially the first one.

I have created multiple guest LDoms, all Solaris 11.1, and they all work fine.

The first problem is that I can not boot any zone I create in a guest LDom, the zone installs fine but I get error when trying to boot it, complains it can not create vnic. I get the same error when trying to boot a zone in any guest LDom.

# zoneadm -z tzone1 boot 
zone 'tzone1' failed to create vnic for net0: operation failed 
zoneadm: zone 'tzone1': call to zoneadmd failed

I tried

root@ldm1:/root# dladm create-vnic -l net0 vnic1

but that didn't work, it complained:

dladm: vnic creation over net0 failed, operation failed.

The second problem is that when I try to create a zone on the primary LDom of this machine, I get an error when installing the zone

primary# zoneadm -z tzone3 install
The following ZFS file system(s) have been created:
    data/zones/tzone3
Progress being logged to /var/log/zones/zoneadm.20130722T004401Z.tzone3.install
       Image: Preparing at /data/zones/tzone3/root.
 AI Manifest: /tmp/manifest.xml.RgaW.n
  SC Profile: /usr/share/auto_install/sc_profiles/enable_sci.xml
    Zonename: tzone3
Installation: Starting ...
              Creating IPS image
Startup linked: 1/1 done
              Installing packages from:
                  solaris
                      origin: http pkg.oracle.com solaris release
              Error occurred during execution of 'generated-transfer-7170-1' checkpoint.
              Failed Checkpoints:
              Checkpoint execution error:
                The following pattern(s) did not match any allowable packages.  Try
                using a different matching pattern, or refreshing publisher information:
Installation: Failed.  See install log at /system/volatile/install.7170/install_log
ERROR: auto-install failed.

I get the same error for all zones that I tried to create on the primary LDom.

This is the first time I'm working with a sun4v, I had earlier created zones with Solaris 10 on sun4u without any problem.

I don't plan to use zones on the primary, so that's not a pressing issue, just I'm not sure if the problems are related.

What am I doing wrong?

Thanks for your help.

aeroforce · July 22, 2013, 3:19pm

Quick question...what does your dladm show-phys show? Also did you ipadm ip-create the interface?

ad101 · July 22, 2013, 4:25pm

in the guest ldom,

# dladm show-phys
LINK              MEDIA                STATE      SPEED  DUPLEX    DEVICE
net0              Ethernet             up         0      unknown   vnet0

and no, I did not use ipadm to create the interface

aeroforce · July 29, 2013, 1:56pm

dladm create-vnic -l vnet0 vnic0

On the other issue....do a pkg publisher and see if you are connecting to the oracle package manager. Also you can try to ping oracle. If not then create your own pkg publisher. There are steps on how to do this online.

ad101 · August 2, 2013, 2:08pm

I had tried the create-vnic and it doesn't work:

root@ldm1:/tmp# dladm create-vnic -l vnet0 vnic0
dladm: invalid lower link name 'vnet0'
root@ldm1:/tmp# dladm create-vnic -l net0 vnic0
dladm: vnic creation over net0 failed: operation failed
root@ldm1:/tmp#

On the other issue, in the primary ldom

# pkg publisher
PUBLISHER                   TYPE     STATUS P LOCATION
solaris                     origin   online F http: // pkg . oracle . com /solaris/release/

I noticed another very interesting situation and suspect it could be related: on the primay ldom, simple package searches like

# pkg search /usr/bin/pkg

returns nothing and exits with code 1; even though I can successfully run

wget pkg.oracle.com

btw, the exact same search returns the expected

# pkg search /usr/bin/pkg
INDEX      ACTION VALUE       PACKAGE
path       file   usr/bin/pkg pkg:/package/pkg@0.5.11-0.175.1.0.0.24.0

on a guest ldom

os2mac · August 2, 2013, 4:00pm

In the Primary LDOM do

ldm list -o network

and post the results please?

ad101 · August 2, 2013, 4:08pm

# ldm list -o network 
NAME
primary
MAC
    00:10:e0:23:68:4c
VSW
    NAME             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    primary-vsw0     00:14:4f:f8:f0:aa net0      0    switch@0              1               1                         1500         on
------------------------------------------------------------------------------
NAME
ldm1
MAC
    00:14:4f:fa:a4:c4
NETWORK
    NAME             SERVICE                     ID   DEVICE     MAC               MODE   PVID VID                  MTU   LINKPROP
    vnet1            primary-vsw0@primary        0    network@0  00:14:4f:fb:76:38        1                         1500

os2mac · August 2, 2013, 4:48pm

so in the primary LDOM:

you need to do

ldm add-vnet
Usage:
        ldm add-vnet [mac-addr=<num>] [mode=hybrid] [pvid=<pvid>]
                [alt-mac-addrs=auto|<num1>[,auto|<num2>,...]]
                [vid=<vid1,vid2,...>] [mtu=<mtu>] [linkprop=phys-state]
                [id=<networkid>] <if_name> <vswitch_name> <ldom>

for each zone you want to have an interface for...

Then in the LDOM:

prior to zone startup you need to do

ipadm remove-ip "netname"

That will free up the network interface on the vswitch for the zone.

then either in the zoneconfig set the network physical property or
after boot do

ipadm create-ip "netname"

then do

ipadm create-addr
usage:
        create-addr     [-t] [-T static] [-d] -a {local|remote}=addr[/prefixlen]
                                <addrobj>
        create-addr     [-t] -T dhcp [-w <seconds> | forever] [-h <hostname>]
                                <addrobj>
        create-addr     [-t] -T addrconf [-i {local|remote}=interface-id]
                                [-p {stateful|stateless}={yes|no}] <addrobj>

---------- Post updated at 12:23 PM ---------- Previous update was at 12:18 PM ----------

I would also like to add that if these are Solaris 10 zones in a Solaris 11 LDOM the procedure changes.

---------- Post updated at 12:48 PM ---------- Previous update was at 12:23 PM ----------

I reread your original post, yes the problems appear to be related because you don't have a network adapter configured on your config.

ad101 · August 4, 2013, 10:39am

Thanks for your detailed reply.

In the meantime, I made some progress slightly differently.

First, my original questions were regarding Solaris 11 LDoms under Solaris 11 primary LDOM.

What I did next was I installed a Solaris 10 LDom and within that I could successfully create zones, just as I had done on old sun4u machine.

Next, on the Solaris 11 LDom, I created a zone where I removed the anet, set the ip to shared and did the "add net" just as I did on Solaris 10, and this time, the zone booted fine.

So, yes, something's missing with my networking settings in Solaris 11 that prevents the zone from booting.

I'd like to try what you suggested, but I got to read and understand it carefully because if I make a mistake on the primary LDom then I'll be stuck.

Thanks again for your help.

os2mac · August 5, 2013, 5:37pm

but because it's a shared IP type zone you'll be stuck with doing static IP addresses for all the zone interfaces.

My way allows you to do DHCP like a regular server.