Each slice has a guest domain built. In other words, when the guest domain is built, something similar to /dev/md/dsk/d30 is assigned; the commands look like:
My question is whether there is an issue with having LDoms and RAID1 at the same time? The reason I am asking is, after I am done with the control domain and guest domains cofguration, and metast looks good, once I reboot the control domain I get an error:
Hostname: control_dom
/dev/md/rdsk/d40 is clean
/dev/md/rdsk/d70 is clean
checking ufs filesystems
/dev/md/rdsk/d50: PARTIALLY ALLOCATED INODE I=2
/dev/md/rdsk/d50: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
/dev/md/rdsk/d60: NEGATIVE SIZE -5476372287601639394 I=2
/dev/md/rdsk/d60: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
THE FOLLOWING FILE SYSTEM(S) HAD AN UNEXPECTED INCONSISTENCY: /dev/md/rdsk/d50 (/exp ort/home/sgw), /dev/md/rdsk/d60 (/export/home/mrf)
fsckall failed with exit code 1.
WARNING - Unable to repair one or more filesystems.
Run fsck manually (fsck filesystem...).
mount: Please run fsck and try again
mount: Please run fsck and try again
svc:/system/filesystem/local:default: WARNING: /sbin/mountall -l failed: exit status 1
Reading ZFS config: done.
Feb 4 09:52:41 svc.startd[7]: svc:/system/filesystem/local:default: Method "/lib/svc/method/fs-local" failed with exit status 95.
Feb 4 09:52:42 svc.startd[7]: system/filesystem/local:default failed fatally: transitioned to maintenance (see 'svcs -xv' for details)
After that, ldm commands returns "Failed to connect to logical domain manager: Connection refused". Running fsck does not help; it even makes thing worse. Appreciate your help...
I tend to agree with you but my understanding from Sun documentation is that guest domain should not be affected when the control domain is rebooted or down. The only thing is that we won't be able to manage the guest domain when the control domain is not available. I noticed though when the control domain is down, guest domain response becomes very slow (to typing commands, etc.)!
no, if y have only 1 io/service/control-domain and y reboot this domain, all guest domain lost his virtuell devices (vdisk, vnet ... ). With 1 io/service/control-domain y must shutdown the guest domain before y reboot the control domain.
Y want active guest domains, if y reboot the control domain. Then y need for example T5240 with 2 BUS controler and 2 io-domains.
I did stop all the guest domains (their status was in "bound" state); and got the same error! It is very time consuming cause I have to re-install the whole thing again.
{0} ok boot
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a File and args:
SunOS Release 5.10 Version Generic_138888-03 64-bit
Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hostname: js-client_t5220
/dev/md/rdsk/d40 is clean
/dev/md/rdsk/d70 is clean
checking ufs filesystems
/dev/md/rdsk/d50: PARTIALLY ALLOCATED INODE I=2
/dev/md/rdsk/d50: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
/dev/md/rdsk/d60: PARTIALLY ALLOCATED INODE I=2
/dev/md/rdsk/d60: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
THE FOLLOWING FILE SYSTEM(S) HAD AN UNEXPECTED INCONSISTENCY: /dev/md/rdsk/d50 (/export/home/sgw), /dev /md/rdsk/d60 (/export/home/mrf)
fsckall failed with exit code 1.
WARNING - Unable to repair one or more filesystems.
Run fsck manually (fsck filesystem...).
mount: Please run fsck and try again
mount: Please run fsck and try again
svc:/system/filesystem/local:default: WARNING: /sbin/mountall -l failed: exit status 1
Reading ZFS config: done.
Feb 6 09:29:11 svc.startd[7]: svc:/system/filesystem/local:default: Method "/lib/svc/method/fs-local" failed with exit status 95.
Feb 6 09:29:11 svc.startd[7]: system/filesystem/local:default failed fatally: transitioned to maintena nce (see 'svcs -xv' for details)
js-client_t5220 console login: root
Password:
Last login: Thu Feb 5 14:59:40 on console
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
You have new mail.
root.js-client_t5220# ldm list
Failed to connect to logical domain manager: Connection refused
is js-client_t5220 the guest ldom? you are rebooting the control domain and the control domain is failing to come online?
only thing i can think of is stop the guest domains, tear down d50, d60, boot off cdrom single user, fsck the rdsk's of d50 and d60, run this several times to see if it repairs, mount root fs, remove md's d50 and d60 from vfstab and then reboot. see if you can get a stable os. make sure the guest ldoms don't come up because they will obviously be looking for the metadevices.
i'm not sure about ldom's because i haven't had the pleasure of working with them as of yet but i'm just thinking of this as if a stand alone sol10 install wasn't working after a reboot with metadevices.
js-client_t5220 is the control domain; and I confirm that I am rebooting the control domain and the control domain is failing to come online with the errors reported earlier. I did not spend much time trying to fix it using fsck since this is a new server so I did re-jumpstart the server (control domain and guest domains) and things are up and running well now. I did not reboot yet but I will make sure all guest domains are in "inactive" state before I reboot the control domain.
root.js-client_t5220# cat /etc/vfstab #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options
#
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/md/dsk/d20 - - swap - no -
/dev/md/dsk/d10 /dev/md/rdsk/d10 / ufs 1 no -
/dev/md/dsk/d40 /dev/md/rdsk/d40 /export/home/hlr ufs 2 yes -
/dev/md/dsk/d60 /dev/md/rdsk/d60 /export/home/mrf ufs 2 yes -
/dev/md/dsk/d70 /dev/md/rdsk/d70 /export/home/msc ufs 2 yes -
/dev/md/dsk/d50 /dev/md/rdsk/d50 /export/home/sgw ufs 2 yes -
/devices - /devices devfs - no -
sharefs - /etc/dfs/sharetab sharefs - no -
ctfs - /system/contract ctfs - no -
objfs - /system/object objfs - no -
swap - /tmp tmpfs - yes -
root.js-client_t5220#
root.js-client_t5220# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 4 4G 1.6% 2d 22h 22m
hlr bound ------ 801 8 4G
mrf bound ------ 803 4 1G
msc bound ------ 802 8 4G
sgw bound ------ 804 4 1G
root.js-client_t5220# ldm list-bindings -p
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=4|mem=4294967296|util=0.2|uptime=253340
MAC|mac-addr=00:14:4f:d4:a5:44
HOSTID|hostid=0x84d4a544
VCPU
|vid=0|pid=0|util=2.4%|strand=100
|vid=1|pid=1|util=0.1%|strand=100
|vid=2|pid=2|util=0.2%|strand=100
|vid=3|pid=3|util=0.1%|strand=100
MAU
|id=0|cpuset=0,1,2,3,4,5,6,7
MEMORY
|ra=0x8000000|pa=0x8000000|size=4294967296
VARIABLES
|boot-device=/pci@0/pci@0/pci@2/scsi@0/disk@0,0:a disk net
|keyboard-layout=US-English
IO
|dev=pci@0|alias=pci
|dev=niu@80|alias=niu
VCC|name=primary-vcc0|port-range=801-850
|client=hlr@primary-vcc0|port=801
|client=msc@primary-vcc0|port=802
|client=mrf@primary-vcc0|port=803
|client=sgw@primary-vcc0|port=804
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:e3:84|net-dev=e1000g0|dev=switch@0|default-vlan-id=1|pvid=1|vid=|mode=
|peer=vnet1@hlr|mac-addr=00:14:4f:fb:7e:09|pvid=1|vid=
|peer=vnet1@msc|mac-addr=00:14:4f:fa:63:e4|pvid=1|vid=
|peer=vnet1@mrf|mac-addr=00:14:4f:f8:53:75|pvid=1|vid=
|peer=vnet1@sgw|mac-addr=00:14:4f:f8:3c:08|pvid=1|vid=
VDS|name=primary-vds0
|vol=vol7|opts=|dev=/dev/md/dsk/d70|mpgroup=
|vol=vol5|opts=|dev=/dev/md/dsk/d50|mpgroup=
|vol=vol4|opts=|dev=/dev/md/dsk/d40|mpgroup=
|vol=vol6|opts=|dev=/dev/md/dsk/d60|mpgroup=
|client=vdisk1@hlr|vol=vol4
|client=vdisk1@msc|vol=vol7
|client=vdisk1@mrf|vol=vol6
|client=vdisk1@sgw|vol=vol5
VCONS|type=SP
DOMAIN|name=hlr|state=bound|flags=|cons=801|ncpu=8|mem=4294967296|util=|uptime=
MAC|mac-addr=00:14:4f:fb:41:e5
HOSTID|hostid=0x84fb41e5
VCPU
|vid=0|pid=4|util=0|strand=100
|vid=1|pid=5|util=0|strand=100
|vid=2|pid=6|util=0|strand=100
|vid=3|pid=7|util=0|strand=100
|vid=4|pid=8|util=0|strand=100
|vid=5|pid=9|util=0|strand=100
|vid=6|pid=10|util=0|strand=100
|vid=7|pid=11|util=0|strand=100
MEMORY
|ra=0x8000000|pa=0x108000000|size=4294967296
VARIABLES
|boot-device=/virtual-devices@100/channel-devices@200/disk@0:a vdisk
|keyboard-layout=US-English
VNET|name=vnet1|dev=network@0|service=primary-vsw0@primary|mac-addr=00:14:4f:fb:7e:09|mode=|pvid=1|vid=
|peer=primary-vsw0@primary|mac-addr=00:14:4f:fa:e3:84|mode=|pvid=1|vid=
|peer=vnet1@msc|mac-addr=00:14:4f:fa:63:e4|mode=|pvid=1|vid=
|peer=vnet1@mrf|mac-addr=00:14:4f:f8:53:75|mode=|pvid=1|vid=
|peer=vnet1@sgw|mac-addr=00:14:4f:f8:3c:08|mode=|pvid=1|vid=
VDISK|name=vdisk1|vol=vol4@primary-vds0|timeout=|dev=disk@0|server=primary|mpgroup=
VCONS|group=hlr|service=primary-vcc0@primary|port=801
DOMAIN|name=mrf|state=bound|flags=|cons=803|ncpu=4|mem=1073741824|util=|uptime=
MAC|mac-addr=00:14:4f:fb:d2:01
HOSTID|hostid=0x84fbd201
VCPU
|vid=0|pid=20|util=0|strand=100
|vid=1|pid=21|util=0|strand=100
|vid=2|pid=22|util=0|strand=100
|vid=3|pid=23|util=0|strand=100
MEMORY
|ra=0x8000000|pa=0x308000000|size=1073741824
VARIABLES
|boot-device=/virtual-devices@100/channel-devices@200/disk@0:a vdisk
|keyboard-layout=US-English
VNET|name=vnet1|dev=network@0|service=primary-vsw0@primary|mac-addr=00:14:4f:f8:53:75|mode=|pvid=1|vid=
|peer=primary-vsw0@primary|mac-addr=00:14:4f:fa:e3:84|mode=|pvid=1|vid=
|peer=vnet1@hlr|mac-addr=00:14:4f:fb:7e:09|mode=|pvid=1|vid=
|peer=vnet1@msc|mac-addr=00:14:4f:fa:63:e4|mode=|pvid=1|vid=
|peer=vnet1@sgw|mac-addr=00:14:4f:f8:3c:08|mode=|pvid=1|vid=
VDISK|name=vdisk1|vol=vol6@primary-vds0|timeout=|dev=disk@0|server=primary|mpgroup=
VCONS|group=mrf|service=primary-vcc0@primary|port=803
DOMAIN|name=msc|state=bound|flags=|cons=802|ncpu=8|mem=4294967296|util=|uptime=
MAC|mac-addr=00:14:4f:f8:dc:63
HOSTID|hostid=0x84f8dc63
VCPU
|vid=0|pid=12|util=0|strand=100
|vid=1|pid=13|util=0|strand=100
|vid=2|pid=14|util=0|strand=100
|vid=3|pid=15|util=0|strand=100
|vid=4|pid=16|util=0|strand=100
|vid=5|pid=17|util=0|strand=100
|vid=6|pid=18|util=0|strand=100
|vid=7|pid=19|util=0|strand=100
MEMORY
|ra=0x8000000|pa=0x208000000|size=4294967296
VARIABLES
|boot-device=/virtual-devices@100/channel-devices@200/disk@0:a vdisk
|keyboard-layout=US-English
VNET|name=vnet1|dev=network@0|service=primary-vsw0@primary|mac-addr=00:14:4f:fa:63:e4|mode=|pvid=1|vid=
|peer=primary-vsw0@primary|mac-addr=00:14:4f:fa:e3:84|mode=|pvid=1|vid=
|peer=vnet1@hlr|mac-addr=00:14:4f:fb:7e:09|mode=|pvid=1|vid=
|peer=vnet1@mrf|mac-addr=00:14:4f:f8:53:75|mode=|pvid=1|vid=
|peer=vnet1@sgw|mac-addr=00:14:4f:f8:3c:08|mode=|pvid=1|vid=
VDISK|name=vdisk1|vol=vol7@primary-vds0|timeout=|dev=disk@0|server=primary|mpgroup=
VCONS|group=msc|service=primary-vcc0@primary|port=802
DOMAIN|name=sgw|state=bound|flags=|cons=804|ncpu=4|mem=1073741824|util=|uptime=
MAC|mac-addr=00:14:4f:fb:24:ad
HOSTID|hostid=0x84fb24ad
VCPU
|vid=0|pid=24|util=0|strand=100
|vid=1|pid=25|util=0|strand=100
|vid=2|pid=26|util=0|strand=100
|vid=3|pid=27|util=0|strand=100
MEMORY
|ra=0x8000000|pa=0x348000000|size=1073741824
VARIABLES
|boot-device=/virtual-devices@100/channel-devices@200/disk@0:a vdisk
|keyboard-layout=US-English
VNET|name=vnet1|dev=network@0|service=primary-vsw0@primary|mac-addr=00:14:4f:f8:3c:08|mode=|pvid=1|vid=
|peer=primary-vsw0@primary|mac-addr=00:14:4f:fa:e3:84|mode=|pvid=1|vid=
|peer=vnet1@hlr|mac-addr=00:14:4f:fb:7e:09|mode=|pvid=1|vid=
|peer=vnet1@msc|mac-addr=00:14:4f:fa:63:e4|mode=|pvid=1|vid=
|peer=vnet1@mrf|mac-addr=00:14:4f:f8:53:75|mode=|pvid=1|vid=
VDISK|name=vdisk1|vol=vol5@primary-vds0|timeout=|dev=disk@0|server=primary|mpgroup=
VCONS|group=sgw|service=primary-vcc0@primary|port=804
root.js-client_t5220#
And to answer your question, I have a guest domain for each directory that is assigned to a slice and each is named after an application I will be installing in that domain and each slice is mounted to dx for RAID1 mirroring; e.g. slice /export/home/sgw is mount to d50. d5 is a mirror and has two members as you can see below:
root.js-client_t5220# metastat -c
d70 m 11GB d71 d72
d71 s 11GB c0t0d0s7
d72 s 11GB c0t1d0s7
d60 m 10GB d61 d62
d61 s 10GB c0t0d0s6
d62 s 10GB c0t1d0s6
d40 m 15GB d41 d42
d41 s 15GB c0t0d0s4
d42 s 15GB c0t1d0s4
d20 m 8.0GB d21 d22
d21 s 8.0GB c0t0d0s1
d22 s 8.0GB c0t1d0s1
d10 m 8.0GB d11 d12
d11 s 8.0GB c0t0d0s0
d12 s 8.0GB c0t1d0s0
d50 m 10GB d51 d52
d51 s 10GB c0t0d0s5
d52 s 10GB c0t1d0s5
root.js-client_t5220#
starsol - zfs is a filesystem and svm is a volume manager. they are different things. however, zfs has its own integrated volume management. zfs is free to use with sol10 as well as other flavors of unix.
The control domain is on /dev/md/dsk/d10 and the rest are for logical domains. I did re-jumpstart the control domain and created the guest domains (but did not load OS on the guest domains yet). As you can see below, no problems when rebooting the control domain:
root.js-client_t5220# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4G 0.2% 3m
hlr bound ------ 802 8 4G
mrf bound ------ 804 8 1G
msc bound ------ 801 8 4G
sgw bound ------ 803 8 1G
root.js-client_t5220# cat /etc/vfstab
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/md/dsk/d20 - - swap - no -
/dev/md/dsk/d10 /dev/md/rdsk/d10 / ufs 1 no -
/dev/md/dsk/d40 /dev/md/rdsk/d40 /export/home/hlr ufs 2 yes -
/dev/md/dsk/d60 /dev/md/rdsk/d60 /export/home/mrf ufs 2 yes -
/dev/md/dsk/d70 /dev/md/rdsk/d70 /export/home/msc ufs 2 yes -
/dev/md/dsk/d50 /dev/md/rdsk/d50 /export/home/sgw ufs 2 yes -
/devices - /devices devfs - no -
sharefs - /etc/dfs/sharetab sharefs - no -
ctfs - /system/contract ctfs - no -
objfs - /system/object objfs - no -
swap - /tmp tmpfs - yes -
root.js-client_t5220# reboot
Feb 10 17:40:49 js-client_t5220 reboot: rebooted by root
Feb 10 17:40:51 js-client_t5220 syslogd: going down on signal 15
syncing file systems... done
rebooting...
Resetting...
Sun Netra T5220, No Keyboard
Copyright 2008 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.30.0, 4096 MB memory available, Serial #81044804.
Ethernet address 0:14:4f:d4:a5:44, Host ID: 84d4a544.
Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a File and args:
SunOS Release 5.10 Version Generic_138888-01 64-bit
Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hostname: js-client_t5220
/dev/md/rdsk/d40 is clean
/dev/md/rdsk/d60 is clean
/dev/md/rdsk/d70 is clean
/dev/md/rdsk/d50 is clean
Reading ZFS config: done.
js-client_t5220 console login: Feb 10 17:42:36 js-client_t5220 sendmail[530]: My unqualified host name (js-client_t5220) unknown; sleeping for retry
Feb 10 17:42:36 js-client_t5220 sendmail[531]: My unqualified host name (js-client_t5220) unknown; sleeping for retry
Feb 10 17:43:36 js-client_t5220 sendmail[531]: unable to qualify my own domain name (js-client_t5220) -- using short name
Feb 10 17:43:36 js-client_t5220 sendmail[530]: unable to qualify my own domain name (js-client_t5220) -- using short name
js-client_t5220 console login: root
Password:
Feb 10 17:43:41 js-client_t5220 login: ROOT LOGIN /dev/console
Last login: Tue Feb 10 17:25:30 on console
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
You have new mail.
root.js-client_t5220# metastat -c
d50 m 10GB d51 d52
d51 s 10GB c1t0d0s5
d52 s 10GB c1t1d0s5
d10 m 8.0GB d11 d12
d11 s 8.0GB c1t0d0s0
d12 s 8.0GB c1t1d0s0
d20 m 8.0GB d21 d22
d21 s 8.0GB c1t0d0s1
d22 s 8.0GB c1t1d0s1
d40 m 15GB d41 d42
d41 s 15GB c1t0d0s4
d42 s 15GB c1t1d0s4
d60 m 10GB d61 d62
d61 s 10GB c1t0d0s6
d62 s 10GB c1t1d0s6
d70 m 11GB d71 d72
d71 s 11GB c1t0d0s7
d72 s 11GB c1t1d0s7
root.js-client_t5220#
I always have the problem after rebooting the control domain after I load the guest domains with the OS (by JumpStart)!! What would your suggestion do to this?