Virtual disks are not showing up in LDom

UNIX Solaris
1

I have an Oracle VM set up with 1 virtual disk. I am trying to add 2 new disks to it. I was able to successfully add 1 (it appears when I run "format" in the VM) but when I add the second and third disks they do not appear in my VM.

Here are the commands I ran:

ldm add-vdsdev /dev/dsk/c0t5000CCA015354CB4d0s0 vol5@primary-vds0
ldm add-vdisk vdisk5 vol5@primary-vds0 ldg1

At this point I booted my VM and saw the new disk:

[root@vm-sol: /]# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0d0 <SUN-DiskSlice-279GB cyl 46874 alt 2 hd 20 sec 625>
          /virtual-devices@100/channel-devices@200/disk@0
       1. c0d1 <SUN-DiskSlice-279GB cyl 46874 alt 2 hd 20 sec 625>
          /virtual-devices@100/channel-devices@200/disk@1
Specify disk (enter its number): ^C

At this point I shutdown the VM and tried to add two new disks, but neither of them show up in my VM:

ldm add-vdsdev /dev/dsk/c0t5000CCA0153542A4d0s0 vol6@primary-vds0
ldm add-vdisk vdisk6 vol6@primary-vds0 ldg1
...
ldm add-vdsdev /dev/dsk/c0t5000CCA0153596A4d0s0 vol7@primary-vds0
ldm add-vdisk vdisk7 vol7@primary-vds0 ldg1

My VM is still showing the same disk selection as before:

AVAILABLE DISK SELECTIONS:
       0. c0d0 <SUN-DiskSlice-279GB cyl 46874 alt 2 hd 20 sec 625>
          /virtual-devices@100/channel-devices@200/disk@0
       1. c0d1 <SUN-DiskSlice-279GB cyl 46874 alt 2 hd 20 sec 625>
          /virtual-devices@100/channel-devices@200/disk@1

Here are my partition tables for the 3 disks I attempted to add:

c0t5000CCA015354CB4d0 (added successfully)
partition> print
Current partition table (original):
Total disk cylinders available: 46873 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 - 46872      279.38GB    (46873/0/0) 585912500
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wm       0 - 46872      279.38GB    (46873/0/0) 585912500
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0

c0t5000CCA0153542A4d0 (unsuccessful)
partition> print
Current partition table (original):
Total disk cylinders available: 46873 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 - 46872      279.38GB    (46873/0/0) 585912500
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wm       0 - 46872      279.38GB    (46873/0/0) 585912500
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0

c0t5000CCA0153596A4d0 (unsuccessful)
partition> print
Current partition table (original):
Total disk sectors available: 585921082 + 16384 (reserved sectors)

Part      Tag    Flag     First Sector         Size         Last Sector
  0        usr    wm               256      279.39GB          585921082
  1 unassigned    wm                 0           0               0
  2 unassigned    wm                 0           0               0
  3 unassigned    wm                 0           0               0
  4 unassigned    wm                 0           0               0
  5 unassigned    wm                 0           0               0
  6 unassigned    wm                 0           0               0
  8   reserved    wm         585921083        8.00MB          585937466

That third disk has a different looking partition table, I'm not sure why. But it looks like the full disk space is on slice 0 so I attempted to add that after my second disk addition failed.

My VM has a ZFS root filesystem, and I was attempting to add these 2 new disks and create a new zpool. Any ideas?

2

I see you are using slice0 of the disk.

You will need to specify the option for it :

ldm add-vdsdev options=slice /dev/dsk/c0t5000CCA015354CB4d0s0 vol5@primary-vds0

Or use the entire disk without the slice option :

ldm add-vdsdev /dev/dsk/c0t5000CCA015354CB4d0s2 vol5@primary-vds0

I would also advice to create vds for every ldom, not use primary for everything.

Remeber if you add slice (any slice), you use <disk>s0 inside ldom.
If you add entire disk you use either s2 or without slice for zpool creation.

The third disk with 'weird' partition table is actualy entire disk added (s2) and during the creation of the zpool inside of ldom no slice as used (c2d0 for instance).

This enables you to import the zpool on both architectures (x86 or sparc).

Regards
Peasant.

1 Like
3

Hm I'm curious, is the options=slice required? Because I was able to add the first disk (or first slice rather?) successfully without it.

Ah yes, this is probably good advice. I am a novice administrator, and I was just learning as I read the instructions. Moving forward I shall try to keep this in mind.

Ah, interesting. I wasn't aware of this, and I just used the format -e option to write an SMI label to disk to it matched the formatting of my other disks. I am not sure the purpose of this, I was simply trying to match the layout of my other disks...

After I labeled "c0t5000CCA0153596A4d0" with an SMI label, I was able to add slice 0 "c0t5000CCA0153596A4d0s0" as a virtual disk to my VM, and it showed up as a third disk. I am still unsure why my "c0t5000CCA0153542A4d0s0" did not show up properly as a virtual disk inside my VM, I will investigate later. But as for now it looks like my task is complete.

Thank you.

4

Slice options is required if you are adding .. well slice.

You can add it without slice, but inside ldom you won't be able to use it as zpool vdev.

I personally like using slices, since it gives me much more flexibility. Slice with Solaris SVM for mirroring (if no external storage) and i use MD devices to add to LDOM(s)

Beware tho, Oracle Solaris 11.1 you must have 17.5 patchset if this to work.
They had the bug from 12.5 forward, which rendered SVM devices on slices unusable inside ldom (it didn't boot with strange messages in syslog).

Of course, because of SVM's great flexibility workaround was to break to mirror, boot from one mirror member, add another, and use mirroring inside ldom.

The downside of this is obvious, if an internal disk fails, you will need to replace and mirror inside every ldom that is using mirrored slices, instead of doing it centrally via SVM on control domain.

Hope that helps.
Regards
Peasant.