Help with mirroring rpool

Hello,

I need some help mirroring my rpool.
I have a 60gb ssd running the rpool alone and want to mirror it for redundancy so I bought a 120gb ssd that I found for a good price. 60gb drives aren't as easy to find at a good price anymore it seems.

I (a bit naively) thought after reading http://docs.oracle.com/cd/E19253-01/819-5461/gkdep/index.html that it would be as easy to mirror as a storage pool using just zpool attach pool drive1 drive2, but I found out it is not.

:~# zpool status rpool
  pool: rpool
 state: ONLINE
  scan: scrub repaired 0 in 10m47s with 0 errors on Sun Apr  2 00:06:47 2017

config:

        NAME      STATE     READ WRITE CKSUM
        rpool     ONLINE       0     0     0
          c3d0s0  ONLINE       0     0     0

errors: No known data errors

:~# zpool attach rpool c3d0 c7d0
Error: c7d0s0 is part of an EFI(GPT) labeled device.
A required boot partition (BIOS Boot) was not found.
cannot attach c7d0 to c3d0: All disk labeling/system requirements are not satisfied.

c3d0 being the existing drive and c7d0 the new drive.

Trying to google the answer and looking in these forums I found some information (ZFS mirroring and http://www.unix.com/solaris/181919-create-boot-disk-mirror-solaris-10-x86.html\) on how to proceed but I just can not get it to work properly. I am probably doing something wrong with partitioning but need help figuring out what.

Here is what I have tried and where I get stuck.

:~# fdisk -B c7d0p0
:~# fdisk -W - c7d0p0

* /dev/rdsk/c7d0p0 default fdisk table
* Dimensions:
*    512 bytes/sector
*     56 sectors/track
*    224 tracks/cylinder
*   18689 cylinders
*
* systid:
*    1: DOSOS12
*    2: PCIXOS
*    4: DOSOS16
*    5: EXTDOS
*    6: DOSBIG
*    7: FDISK_IFS
*    8: FDISK_AIXBOOT
*    9: FDISK_AIXDATA
*   10: FDISK_0S2BOOT
*   11: FDISK_WINDOWS
*   12: FDISK_EXT_WIN
*   14: FDISK_FAT95
*   15: FDISK_EXTLBA
*   18: DIAGPART
*   65: FDISK_LINUX
*   82: FDISK_CPM
*   86: DOSDATA
*   98: OTHEROS
*   99: UNIXOS
*  100: FDISK_NOVELL2
*  101: FDISK_NOVELL3
*  119: FDISK_QNX4
*  120: FDISK_QNX42
*  121: FDISK_QNX43
*  130: SUNIXOS
*  131: FDISK_LINUXNAT
*  134: FDISK_NTFSVOL1
*  135: FDISK_NTFSVOL2
*  165: FDISK_BSD
*  167: FDISK_NEXTSTEP
*  183: FDISK_BSDIFS
*  184: FDISK_BSDISWAP
*  190: X86BOOT
*  191: SUNIXOS2
*  238: EFI_PMBR
*  239: EFI_FS
*

* Id    Act  Bhead  Bsect  Bcyl    Ehead  Esect  Ecyl    Rsect      Numsect
  191   128  199    8      0       254    63     1023    12544      234422272
  0     0    0      0      0       0      0      0       0          0
  0     0    0      0      0       0      0      0       0          0
  0     0    0      0      0       0      0      0       0          0

:~# prtvtoc /dev/rdsk/c3d0s0 | fmthard -s - /dev/rdsk/c7d0s0
Partition 0 not aligned on cylinder boundary: "       0      2    00      16065 117162045 117178109"

If I run format and remove the EFI partition, create a new solaris2 partition and run zpool attach, the drive returns to EFI labels. Should it do that or am I missing something there too?

Any help resolving this is much appreciated.

:~# cat /etc/release
Oracle Solaris 11.3 X86

have you seen this?

How to Create a Mirrored Root Pool (Post Installation)

Yes I have.
I have tried to follow it but zpool attach replies with EFI labeled device and disk labeling requirements not met.
See my first post for exact message.
It does not matter if I use just c3d0 c7d0 or c3d0s0 c7d0s0, I get the same error message.

Hi Guys,

Am I missing something here, shouldn't this be running against slice two (s2)?

Regards

Gull04

Indeed.

---------- Post updated at 06:31 AM ---------- Previous update was at 06:25 AM ----------

So the basic answer here is this, If you are building your zpool with virtual storage (I.E. Luns provided from a SAN, or storage exported from the global zone into the zone) you would use the whole disk I.E. c1d0 . If you are in the global zone you need to provide it with the correct slice (partition). In Solaris the default slice for an OS is slice 2. So therefore you need to create the rpool against slice 2.

Like this:

zpool attach rpool c3d0s0 c7d0s2

Thank you both very much for your help.

Using # zpool attach rpool c3d0s0 c7d0s2 worked and the second disk got added to the rpool. How come the Oracle documentation only shows s0 and guides everywhere only use s0?

On to the next issue, installing grub.

# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c7d0s2
raw device must be a root slice (not s2)
Unable to gather device information for /dev/rdsk/c7d0s2

So I tried s0 instead, as it is being mentioned in the oracle documentation.

# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c7d0s0
cannot read stage1 file /boot/grub/stage1
Error opening /boot/grub/stage1
# ls -l /boot/grub/
total 3101
-rw-r--r--   1 root     bin           57 Oct 21  2011 default_video_modes.txt
-rw-r--r--   1 root     bin         5432 Nov 27 17:53 grub_cfg_net
-rw-r--r--   1 root     bin      1032704 Nov 27 17:53 grub2netx64.efi
-rw-r--r--   1 root     bin       243848 Nov 27 17:53 pxegrub2
-rw-r--r--   1 root     sys       153272 Nov 27 17:53 splash.jpg

So it seems that stage1 is nonexisting, but this is perhaps as it should be? I do not know how installgrub works.
Reading the installgrub manual #man installgrub it says that installgrub is deprecated and that bootadm install-bootloader is to be used instead. Reading the manual for bootadm did not make me much smarter so I tried running it but failed. Just in case I made a snapshot of my rpool before trying.

# bootadm install-bootloader c7d0s2
bootadm: GRUB2 installation failed

I am a bit out of my competence and comfort zone here.
How to proceed?

well as man bootadm specifically said NOT to use s2. I would try again with the other installer but not s2.

Hi,

There is an excellent tutorial on the GRUB boot loader here you should work your way through the page, I believe it will answer most of your questions.

Regards

Gull04

Thanks once again to you both. Will study and try to get it to work and will get back with the results.

I thought I should update this post with how it all ended.

After last time I got stuck with not being able to install the bootloader, so I settled for the time being with at least having the rpool mirrored, providing a little higher level of reliability and safety than before.

Today I dug into the problem again determined to fix it.

The problem from before was that bootadm did not want to install to slice 2.
So I detached the disk from the rpool, applied a default solaris fdisk partition using

# fdisk -B c7t1d0p0

I then checked the partition table using format on both the original rpool disk and the one I wanted to mirror with and noticed that only the original had a slice called root at 0.
So modified the slice to be called root as well, and made it span almost the entire disk.
After that I reattached it to the rpool using

# zpool attach rpool c3d0s0 c7d0s0

which worked.

After that I reread the man pages of bootadm once more and discovered that you should use the -M flag for x86.
So I installed the bootloader using

# bootadm install-bootloader -M c7d0s0

This time it went through without any complaints so I turned off the computer and switched the disk priority in BIOS so that the new disk was first and a storage disk second, to make sure that it would not switch to the original bootdrive.
After a reboot i came to the familiar GRUB menu so it worked with booting!

Shut off, went back to bios and changed the disk priority to the new disk as first and the old bootdrive as second and the storage drives after and restarted the computer and it booted once again all the way!

I know that you in a earlier reply told me to use slice 2, but since bootadm complained about that I decided to try slice 0 instead which worked. Can that cause me any trouble down the road or am I fine as long as the computer boots and zpool status is fine?

Thought I might finish by adding two resources that helped me a bit on the way.
Oracle Solaris 11 Express: Mirroring Your ZFS Root Pool
and