Creating metadevices

UNIX Solaris
1

Hi

I am new to SVM , but I would like to configure it in one of my servers. The setup is as follows: I have one server running solaris 10 , and is connected to a NetApp via iSCSI protocol. This is how I intend to do it:

Ask the NetApp admin, to provide me with 3 LUNs of 100G each, them use the format comand.
This is where I am stuck.
After creating disk devices such us c0t1d0s6 , c0t2d0s6 and c0t3d0s6 , I have to create state database using metadb followed by cereating a metadevice using metainit d0 comand. Is this the right procedure?

2

Hi,

The normal way of initialising a device would be;

#> metainit d0 1 1 cntndnsn

This will use the whole slice of the disk, to create a mirror;
Create the mirrored device.

metainit  d16 1 1  c0t0d0s6
metainit  d26 1 1  c1t0d0s6
metainit  d6 �m d16 d26

It is also worth remembering that the state database must have a number of copies, also that if required you can slice the "disk slices" using soft partitioning like;

#> metainit d100 �p d6 10g

Then you have to create the file system on it like;

mkfs �F vxfs /dev/md/rdsk/d100

Regards

Gull04

3

Hi

Thanks for the reply, but I believe that before using metainit I have to use the format , them create state database...

4

The meta database usually sits on a very small disk slice (circa 10MB) so you would need to create that slice, yes.

5

Thanks for the reply hicksd8 , so for the state database replicas, I have to choose a slice not being used in one of the internal disks, them proceed with the metainit command on the 3 different LUNs/disks, such as :
c2t4d0 c2t5d0 and c2t6d0

6

Usually when planning SVM, the sysadmin creates a tiny (separate) slice on each disk for the meta database. You might need to reduce the size of another slice to make way for it but the space needed is tiny (just a cylinder or two).

7

Hi,

Hicksd8 is right in suggesting that normal practice would be to create a single small slice for the metadb, good practice is to have more than one copy of the metadb in each of the slices.

The other point I would make is that when you are using format to initialise your SAN disk, it is common practice if you are using the free hog option to change the selection to slice 0 which would make your disk "c0t1d0s0". But it may be that your standard is to use slice 6.

Regards

Gull04

8

Hi
Thanks again for the replies, here is my internal disk:

 format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 35687 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,25e3@3/pci103c,3234@0/sd@0,0

and if select it in format I get:

partition> print
Current partition table (original):
Total disk cylinders available: 35687 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       1 -   523        4.01GB    (523/0/0)     8401995
  1       swap    wm       1 -   262        2.01GB    (262/0/0)     4209030
  2     backup    wm       0 - 35686      273.38GB    (35687/0/0) 573311655
  3 unassigned    wm     524 -  1555        7.91GB    (1032/0/0)   16579080
  4        var    wm    1556 -  2848        9.90GB    (1293/0/0)   20772045
  5 unassigned    wm    2849 -  4141        9.90GB    (1293/0/0)   20772045
  6        usr    wm    4142 -  5173        7.91GB    (1032/0/0)   16579080
  7 unassigned    wm    5174 - 28120      175.78GB    (22947/0/0) 368643555
  8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
  9 unassigned    wm       0                0         (0/0/0)             0

that means I can only use slice 2 for the database replicas...because acording to :

bash-3.2# df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0t0d0s0      3.9G   438M   3.5G    11%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                    40G   988K    40G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
/dev/dsk/c0t0d0s6      7.8G   3.3G   4.4G    43%    /usr
/usr/lib/libc/libc_hwcap1.so.1
                       7.8G   3.3G   4.4G    43%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
/dev/dsk/c0t0d0s4      9.8G   5.7G   4.0G    59%    /var
swap                    45G   5.4G    40G    12%    /tmp
swap                    40G    32K    40G     1%    /var/run
/dev/dsk/c2t3d0s2      985G   915G    60G    94%    /data2
/dev/dsk/c0t0d0s3      7.8G   6.0G   1.7G    79%    /export
/dev/dsk/c0t0d0s5      9.8G    37M   9.6G     1%    /opt
/dev/dsk/c2t2d0s2      788G   673G   107G    87%    /data1
/dev/dsk/c0t0d0s7      173G   108G    64G    63%    /data
bash-3.2#

it looks like the only slice available...

9

No, you cannot use slice 2. Nothing ever uses slice 2 (unless it's for a raw copy or something like that). On Solaris, slice 2 always represents the WHOLE disk and is used by the system for disk discovery/geometry, etc.

NEVER try to use slice 2 for anything in normal operation!! You will likely corrupt one of the valid slices.

1 Like
10

So I cannot use the internal disk for state database replicas....

11

You can use ISCSI disk slice for metadb.
Or any disk device slice as far as metadb is concerned.

With metadb command (not metainit), you use defined small slices on disks, not the entire devices (cNtNd0 or cNtNd0s2).

After metadb creation, you may use SVM via meta commands for managing metadevices and metasets.

Just be sure that those luns will be present on machine as long as SVM is used.
Or, if old luns are removed, and new one added, add new and put extra database replica on it.

Remove replica from from old LUN, then remove the lun from system.

Having one metadb slice on one disk is a bad idea anyway.
You would want at least two, spanning on multiple disks (not same disk multiple slices).

Only place in which you can or must use entire devices (cNtNd0s2) is
Copying partition table from from same sized devices prtvtoc .. | fmthard ..
Using dd.
Creating zpools.
SCSI,multipath, zfs debug, information and manipulation using luxadm , mpathadm and zdb commands.
Others i cannot remember now.

Good thing to mention is that using the whole disk during zpool creation will enable you to import that zpool on x86 or SPARC architecture due to filesystem endian awareness.

This, of course, provides data moving from one platform to another, not binaries.

Hope that helps
Regards
Peasant.

12

Why using an "old" and "clumsy" tool to manage storage? Solaris 10 has also ZFS and that is way easier to use...

13

Hi

This is how I intend to do:

This are the newly attached LUN�s:

 3. c2t4d0 <DEFAULT cyl 13052 alt 2 hd 255 sec 63>
          /iscsi/disk@0000iqn.1992-08.com.netapp%3Asn.157416847303EC,2
       4. c2t5d0 <DEFAULT cyl 13052 alt 2 hd 255 sec 63>
          /iscsi/disk@0000iqn.1992-08.com.netapp%3Asn.157416847303EC,3
       5. c2t6d0 <DEFAULT cyl 13052 alt 2 hd 255 sec 63>
          /iscsi/disk@0000iqn.1992-08.com.netapp%3Asn.157416847303EC,4
Specify disk (enter its number): 0

create state database replicas:

metadb -a c2t4d0s3 c2t5d0s3 c2t6d0s3

Now create metadevices using metainit and metaattach :

metainit d20 1 1 c2t4d0s6
metattach d20 c2t5d0s36
metattach d20 c2t6d0s6

Now check metadevice and which physical disks are being used:

metastat -p d20

Now create a UFS file system on this metadevice:

newfs /dev/md/rdsk/d20

Create a mount point:

mkdir /data2

Permissions:

chmod 755 /data2

mount the M.P.

mount /dev/md/dsk/d20 /data2

This is how I am planning to do, please is anything wrong or missing

14

You have to create metadevices out of c2t5d0s36 and c2t6d0s6 first. Then attach the new metadevice to d20.

15

Hi fretagi,

When using Solaris Volume Manager you have to remember that things are "event driven", so your steps would be basically as follows.

  • Present the physical storage to the server - this is the work your SAN team would do.
  • Introduce the physical devices to the system - done by the systems admin.
  • If required use format to slice the disk - again done by the systems admin.
  • Use SVM to create the devices - this assumes that the state database has already been created.
  • Using the metainit command, create the required devices.
  • Mount the devices as required.

When it comes to creating devices, there are a number of guidelines that your administrators should advise you on - where you have SAN attached storage or a NAS, let the SAN or the NAS take care of resilience. I would advise that mirroring two SAN or NAS devices would have performance indications for the system.

Where you have internal disk on the server, it makes sense to mirror the physical devices - on SAN or NAS it does not (Unless you have gone for a very cheap solution).

Below is a metastat from one of my systems showing how convoluted the disk planning can become, so it is important to understand what is required before you start and plan accordingly.

$ metastat -p
d10 -m d11 d12 1
d11 1 1 /dev/dsk/c0t5000CCA07023D760d0s0
d12 1 1 /dev/dsk/c0t5000CCA07040B1F8d0s0
d20 -m d21 d22 1
d21 1 1 /dev/dsk/c0t5000CCA07023D760d0s1
d22 1 1 /dev/dsk/c0t5000CCA07040B1F8d0s1
d40 -m d41 d42 1
d41 1 1 /dev/dsk/c0t5000CCA07023D760d0s6
d42 1 1 /dev/dsk/c0t5000CCA07040B1F8d0s6
d270 1 1 /dev/dsk/c0t600507680C808278000000000000024Ed0s0
d300 1 1 /dev/dsk/c0t600507680C80827800000000000001DAd0s0
d202 1 1 /dev/dsk/c0t600507680C8082780000000000000132d0s0
d151 1 1 /dev/dsk/c0t600507680C8082780000000000000134d0s0
d150 1 1 /dev/dsk/c0t600507680C8082780000000000000133d0s0
d873 -p d870 -o 184565856 -b 39845888
d870 1 1 /dev/dsk/c0t600507680C8082780000000000000423d0s0
d872 -p d870 -o 102776896 -b 81788928
d871 -p d870 -o 16416 -b 102760448
d857 -p d850 -o 1289781472 -b 3121152
d850 1 1 /dev/dsk/c0t600507680C8082780000000000000422d0s0
d856 -p d850 -o 1287684288 -b 2097152
d855 -p d850 -o 1283489952 -b 4194304
d854 -p d850 -o 1279295616 -b 4194304
d853 -p d850 -o 1258324064 -b 20971520
d852 -p d850 -o 629178432 -b 629145600
d851 -p d850 -o 32800 -b 629145600
d836 -p d820 -o 524280352 -b 1048576  -o 531620480 -b 1024000
d820 2 1 /dev/dsk/c0t600507680C8082780000000000000420d0s0 \
         1 /dev/dsk/c0t600507680C8082780000000000000424d0s2
d835 -p d820 -o 519037408 -b 4194304
d834 -p d820 -o 481288640 -b 37748736  -o 529523296 -b 2097152  -o 562004704 -b 3145728
d833 -p d820 -o 477094304 -b 4194304
d832 -p d820 -o 468705664 -b 8388608
d831 -p d820 -o 342876512 -b 125829120  -o 525328960 -b 4194304  -o 553616064 -b 8388608
d830 -p d820 -o 334487872 -b 8388608  -o 607093536 -b 1024000
d829 -p d820 -o 145744160 -b 188743680  -o 532644512 -b 20971520  -o 565150464 -b 41943040
d828 -p d820 -o 141549824 -b 4194304
d827 -p d820 -o 137355488 -b 4194304
d826 -p d820 -o 135258304 -b 2097152  -o 523231744 -b 1048576
d825 -p d820 -o 126869664 -b 8388608
d824 -p d820 -o 125845632 -b 1024000
d823 -p d820 -o 104874080 -b 20971520
d822 -p d820 -o 20987968 -b 83886080
d821 -p d820 -o 16416 -b 20971520
d807 -p d800 -o 102777088 -b 1024000  -o 20987968 -b 1048576  -o 26210752 -b 1024000
d800 1 1 /dev/dsk/c0t600507680C8082780000000000000421d0s0
d806 -p d800 -o 81805504 -b 4194304
d805 -p d800 -o 60833952 -b 20971520  -o 85999840 -b 16777216
d804 -p d800 -o 56639616 -b 4194304
d803 -p d800 -o 54542432 -b 2097152  -o 103801120 -b 1023680  -o 22036576 -b 4174144
d801 -p d800 -o 16416 -b 20971520
d730 -p d700 -o 4232101664 -b 4194304
d700 2 1 /dev/dsk/c0t600507680C808278000000000000029Bd0s0 \
         1 /dev/dsk/c0t600507680C808278000000000000029Cd0s0
d721 -p d700 -o 3047210528 -b 16777216
d720 -p d700 -o 3034627584 -b 12582912
d719 -p d700 -o 3214982944 -b 20971520
d718 -p d700 -o 3194011392 -b 20971520
d717 -p d700 -o 3173039840 -b 20971520
d716 -p d700 -o 3152068288 -b 20971520
d715 -p d700 -o 2919284192 -b 115343360
d714 -p d700 -o 1870708160 -b 1048576000
d713 -p d700 -o 1786822048 -b 83886080  -o 3063987776 -b 41943040
d712 -p d700 -o 1782627712 -b 4194304
d711 -p d700 -o 1761656160 -b 20971520
d710 -p d700 -o 1635827008 -b 125829120
d709 -p d700 -o 3131096736 -b 20971520
d708 -p d700 -o 3110125184 -b 20971520
d707 -p d700 -o 1541455136 -b 94371840
d706 -p d700 -o 492879104 -b 1048576000
d705 -p d700 -o 482393312 -b 10485760
d704 -p d700 -o 188792000 -b 293601280  -o 3235954496 -b 52428800
d703 -p d700 -o 146848928 -b 41943040
d702 -p d700 -o 125877376 -b 20971520  -o 3105930848 -b 4194304
d701 -p d700 -o 48224 -b 125829120
d295 -p d290 -o 885014688 -b 125829120
d290 1 1 /dev/dsk/c0t600507680C8082780000000000000250d0s0
d294 -p d290 -o 171982976 -b 713031680
d293 -p d290 -o 151011424 -b 20971520  -o 1010843840 -b 10485760
d292 -p d290 -o 125845568 -b 25165824
d291 -p d290 -o 16416 -b 125829120
d281 -p d280 -o 16416 -b 73400320
d280 1 1 /dev/dsk/c0t600507680C808278000000000000024Fd0s0
d440 -p d430 -o 133185856 -b 8388608
d430 1 1 /dev/dsk/c0t600507680C8082780000000000000253d0s0
d439 -p d430 -o 124797216 -b 8388608
d438 -p d430 -o 116408576 -b 8388608
d437 -p d430 -o 111165664 -b 5242880
d436 -p d430 -o 102777024 -b 8388608
d435 -p d430 -o 100679840 -b 2097152
d434 -p d430 -o 90194048 -b 10485760
d433 -p d430 -o 81805408 -b 8388608
d432 -p d430 -o 67125312 -b 14680064
d431 -p d430 -o 16416 -b 67108864
d421 -p d420 -o 16416 -b 81788928
d420 1 1 /dev/dsk/c0t600507680C8082780000000000000252d0s0
d411 -p d410 -o 32800 -b 1132462080
d410 1 1 /dev/dsk/c0t600507680C8082780000000000000251d0s0
d381 -p d380 -o 16416 -b 71303168
d380 1 1 /dev/dsk/c0t600507680C8082780000000000000247d0s0
d371 -p d370 -o 32800 -b 1310720000
d370 1 1 /dev/dsk/c0t600507680C8082780000000000000246d0s0
d366 -p d360 -o 163594432 -b 1048576
d360 1 1 /dev/dsk/c0t600507680C8082780000000000000245d0s0
d365 -p d360 -o 132137120 -b 31457280
d364 -p d360 -o 73416832 -b 58720256
d363 -p d360 -o 41959520 -b 31457280
d362 -p d360 -o 25182272 -b 16777216
d361 -p d360 -o 16416 -b 25165824
d533 -p d500 -o 2776678112 -b 1090519040
d500 2 1 /dev/dsk/c0t600507680C80827800000000000001E4d0s0 \
         1 /dev/dsk/c0t600507680C80827800000000000001E5d0s0
d532 -p d500 -o 2722152128 -b 54525952
d531 -p d500 -o 2699083424 -b 23068672
d530 -p d500 -o 2625683072 -b 73400320
d67 -p d50 -o 12583008 -b 41943040
d50 -m d51 d52 1
d51 1 1 /dev/dsk/c0t5000CCA07040B1F8d0s7
d52 1 1 /dev/dsk/c0t5000CCA07023D760d0s7
d66 -p d50 -o 10485824 -b 2097152
d394 -p d390 -o 631259264 -b 115343360
d390 1 1 /dev/dsk/c0t600507680C80827800000000000001DFd0s0
d393 -p d390 -o 44056672 -b 587202560
d392 -p d390 -o 41959488 -b 2097152
d391 -p d390 -o 16416 -b 41943040
d261 -p d260 -o 2080 -b 31457280
d260 1 1 /dev/dsk/c0t600507680C80827800000000000001DEd0s0
d257 -p d250 -o 392184032 -b 10485760
d250 1 1 /dev/dsk/c0t600507680C80827800000000000001DDd0s0
d256 -p d250 -o 381698240 -b 10485760
d255 -p d250 -o 88096928 -b 293601280
d254 -p d250 -o 79708288 -b 8388608
d253 -p d250 -o 75513952 -b 4194304
d252 -p d250 -o 33570880 -b 41943040
d251 -p d250 -o 16416 -b 33554432
d330 -p d320 -o 518013248 -b 94371840
d320 1 1 /dev/dsk/c0t600507680C80827800000000000001DCd0s0
d329 -p d320 -o 444612896 -b 73400320
d328 -p d320 -o 436224256 -b 8388608
d327 -p d320 -o 411058400 -b 25165824
d326 -p d320 -o 306200768 -b 104857600
d325 -p d320 -o 289423520 -b 16777216
d324 -p d320 -o 251674752 -b 37748736
d323 -p d320 -o 178274400 -b 73400320
d322 -p d320 -o 115359808 -b 62914560
d321 -p d320 -o 16416 -b 115343360
d311 -p d310 -o 16416 -b 83886080
d310 1 1 /dev/dsk/c0t600507680C80827800000000000001DBd0s0
d55 -p d50 -o 32 -b 10485760
d65 -p d50 -o 359661920 -b 31457280
d64 -p d50 -o 328204608 -b 31457280
d63 -p d50 -o 296747296 -b 31457280
d62 -p d50 -o 191889664 -b 104857600
d61 -p d50 -o 188743904 -b 3145728
d60 -p d50 -o 167772352 -b 20971520
d59 -p d50 -o 142606496 -b 25165824
d58 -p d50 -o 136315008 -b 6291456
d57 -p d50 -o 83886176 -b 52428800
d303 -p d30 -o 10672256 -b 2097152
d30 -m d31 d32 1
d31 1 1 /dev/dsk/c0t5000CCA07023D760d0s5
d32 1 1 /dev/dsk/c0t5000CCA07040B1F8d0s5
d301 -p d30 -o 32 -b 3356672  -o 9648224 -b 1024000
d302 -p d30 -o 3356736 -b 6291456
d106 -p d100 -o 484458688 -b 12582912
d100 5 1 /dev/dsk/c0t600507680C80827800000000000000AEd0s0 \
         1 /dev/dsk/c0t600507680C80827800000000000000ADd0s0 \
         1 /dev/dsk/c0t600507680C80827800000000000000AFd0s0 \
         1 /dev/dsk/c0t600507680C80827800000000000000B0d0s0 \
         1 /dev/dsk/c0t600507680C80827800000000000000B1d0s0
d107 -p d100 -o 497041632 -b 10485760
d108 -p d100 -o 507527424 -b 6291456
d109 -p d100 -o 513818912 -b 2097152
d115 -p d100 -o 1684029920 -b 14680064
d116 -p d100 -o 1698710016 -b 4194304
d117 -p d100 -o 1702904352 -b 20971520
d118 -p d100 -o 1723875904 -b 12582912
d119 -p d100 -o 1736458848 -b 69206016
d120 -p d100 -o 1805664896 -b 31457280
d121 -p d100 -o 1837122208 -b 178257920
d122 -p d100 -o 2015380160 -b 14680064
d123 -p d100 -o 2030060256 -b 71303168
d124 -p d100 -o 2101363456 -b 71303168
d125 -p d100 -o 2172666656 -b 10485760
d126 -p d100 -o 2183152448 -b 8388608
d127 -p d100 -o 2191541088 -b 5242880
d128 -p d100 -o 2196784000 -b 15728640
d129 -p d100 -o 2212512672 -b 67108864
d130 -p d100 -o 2279621568 -b 192937984
d131 -p d100 -o 2472559584 -b 10485760
d132 -p d100 -o 2483045376 -b 35651584
d133 -p d100 -o 2518696992 -b 6291456
d609 -p d600 -o 174080288 -b 2097152
d600 1 1 /dev/dsk/c0t600507680C808278000000000000010Bd0s0
d611 -p d610 -o 32800 -b 1195376640
d610 1 1 /dev/dsk/c0t600507680C808278000000000000010Cd0s0
d601 -p d600 -o 16416 -b 25165824
d602 -p d600 -o 25182272 -b 4194304
d603 -p d600 -o 29376608 -b 8388608
d604 -p d600 -o 37765248 -b 62914560
d605 -p d600 -o 100679840 -b 27262976
d606 -p d600 -o 127942848 -b 6291456
d607 -p d600 -o 134234336 -b 35651584
d608 -p d600 -o 169885952 -b 4194304
d621 -p d620 -o 16416 -b 115343360
d620 1 1 /dev/dsk/c0t600507680C808278000000000000010Dd0s0

Listed above are all the disks on the system, here is the detail on d50 which is an internal slice mirrored to another internal slice.

d50: Mirror
    Submirror 0: d51
      State: Okay
    Submirror 1: d52
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 698047308 blocks (332 GB)

d51: Submirror of d50
    State: Okay
    Size: 698047308 blocks (332 GB)
    Stripe 0:
        Device                             Start Block  Dbase        State Reloc Hot Spare
        /dev/dsk/c0t5000CCA07040B1F8d0s7          0     No            Okay   Yes


d52: Submirror of d50
    State: Okay
    Size: 698047308 blocks (332 GB)
    Stripe 0:
        Device                             Start Block  Dbase        State Reloc Hot Spare
        /dev/dsk/c0t5000CCA07023D760d0s7          0     No            Okay   Yes


Device Relocation Information:
Device                           Reloc  Device ID
/dev/dsk/c0t5000CCA07040B1F8d0   Yes    id1,sd@n5000cca07040b1f8
/dev/dsk/c0t5000CCA07023D760d0   Yes    id1,sd@n5000cca07023d760
$

you
The correct process here would be to create d51 using the metainit command, you would then create d52 again using the metainit command.

Now you would create d50 using the metainit command on d51 followed by the metattach of d52 as below.

This example shows a two-way mirror,  /dev/md/dsk/d50,  con-
     sisting  of two submirrors. This mirror does not contain any
     existing data.

     # metainit d51 1 1 c0t1d0s0
     # metainit d52 1 1 c0t2d0s0
     # metainit d50 -m d51
     # metattach d50 d52

Have you done any of the preparation for the creation of these devices yet, if so what have you done.

Regards

Gull04