Sharing a physical disk with an LDOM

I have a guest LDOM running Solaris 10U11 on a Sun T4-1 host running Solaris 11.4. The host has a disk named bkpool that I'd like to share with the LDOM so both can read and write it. The host is hemlock, the guest is sol10.

root@hemlock:~# zpool list
NAME    SIZE  ALLOC  FREE  CAP  DEDUP  HEALTH  ALTROOT
bkpool  832G   253G  579G  30%  1.00x  ONLINE  -
rpool   832G   666G  166G  80%  1.00x  ONLINE  -

I followed the directions here Using Virtual Disks With Logical Domains but they didn't work.

root@hemlock:~# ldm add-vdsdev /dev/dsk/c0t5000C50070FFBF07d0s2 vdiskbk@primary-vds0 
root@hemlock:~# ldm add-vdisk vdskbk1 vdiskbk@primary-vds0 sol10 root@hemlock:~#

The disk is question is disk 0:

root@hemlock:~# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t5000C50070FFBF07d0 <SEAGATE-ST900MM0006-0001-838.36GB>
          /scsi_vhci/disk@g5000c50070ffbf07
          /dev/chassis/SYS/HDD2/disk
       1. c1t3A0D19F2BE2346CEd0 <LSI-Logical Volume-3000-837.26GB>
          /pci@400/pci@1/pci@0/pci@4/scsi@0/iport@v0/disk@w3a0d19f2be2346ce,0

I stopped and restarted the LDOM, but there is no sign of /bkpool there.

# ls /dev/dsk
c0d0s0  c0d0s3  c0d0s6  c0d1s1  c0d1s4  c0d1s7  c0d2s1  c0d2s4
c0d0s1  c0d0s4  c0d0s7  c0d1s2  c0d1s5  c0d2    c0d2s2  c0d2s5
c0d0s2  c0d0s5  c0d1s0  c0d1s3  c0d1s6  c0d2s0  c0d2s3  c0d2s6
# zpool list
NAME   SIZE  ALLOC  FREE  CAP  HEALTH  ALTROOT
rpool  596G  10.0G  586G   1%  ONLINE  -
#

But according to the host, the disk is there

root@hemlock:~# ldm list-services
VCC
    NAME         LDOM         PORT-RANGE
    primary-vcc0 primary      5000-5100

VSW
    NAME         LDOM         MACADDRESS          NET-DEV   DVID|PVID|VIDs
    ----         ----         ----------          -------   --------------
    primary-vsw0 primary      xx:xx:xx:xx:xx:xx   net1      1|1|--       

VDS
    NAME         LDOM         VOLUME         OPTIONS          MPGROUP        DEVICE
    primary-vds0 primary      vol1                                           /sol10dsk
                              voliso         ro                              /export/home/michele/Downloads/sol-10-u11-ga-sparc-dvd.iso
                              vdiskbk                                        /dev/dsk/c0t5000C50070FFBF07d0s2

I'm stuck. What am I doing wrong?

UPDATE:

Actually it seems like the guest does know about the bkpool disk:

# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0d1 <SUN-DiskImage-600GB cyl 17064 alt 2 hd 96 sec 768>
          /virtual-devices@100/channel-devices@200/disk@1
       1. c0d2 <SEAGATE-ST900MM0006-0001-838.36GB>
          /virtual-devices@100/channel-devices@200/disk@2

So I guess now the question is simply how do I make c0d2 available in the Solaris 10 LDOM? I'd like the final result to be that the LDOM contains a mountpoint named /bkpool that holds the contents of the disk.

Discussion moved to Solaris area, you will find more people with the Solaris specific knowledge to answer this question here.

1 Like

One thing for sure is that only one of the nodes (Solaris 11 Global or Solaris 10 LDOM) can have control of the volume. In any situation, having two operating systems writing to a volume simultaneously is a recipe for instant filesystem corruption. One operating system must control file opening, locking, etc. Even in a cluster scenario using dual tailed storage, a major function of the cluster suite is to control which node has exclusive control of the volume and effect disciplined failover when necessary.

Therefore, like any two nodes, one option is to mount the volume on one node, configure a NFS share on that node, and mount the volume using a NFS client from the second node. The first node then controls ALL activity on the volume.

4 Likes

OK, I'm glad I asked then. So I have to mount the /bkpool disk in the LDOM as an NFS share? Can you give me a pointer on how to do that? Is this what Oracle calls "virtual disk multipathing"? There's an example of that further down in the link in the OP but I'm not quite sure how to do it. Also, do I first need to undo the add-vdsdev and add-vdisk commands I gave earlier? I don't want to mess up my disk.

UPDATE

Well as usual the Oracle documentation was overly complex and ambiguous. I figured it out, thanks to the suggestion above:

On the host, assuming the IP of the LDOM at 192.168.0.78, do:

root@hemlock:/# share -F nfs -o rw,root=192.168.0.78 /bkpool/

Then in the LDOM (with the IP of the host hemlock at 192.168.0.183), do:

# cd /
# mkdir bkpool
# mount -F nfs -o vers=3 192.168.0.183:/bkpool /bkpool

The LDOM now has a mountpoint named /bkpool containing everything on the host's /bkpool disk. The host and the LDOM can both read and write the disk. No rebooting anywhere required. Easy! :slight_smile:

2 Likes