Assuming we have a server with 8 hdd, using raid5. After applying raid5 to the disk, solaris 11 will be install.
In my understanding, 8 hdd will represent 1disk(pls correct me if I'm wrong) and have one pool(rpool).
Using raid5 for 8hdd, could it be possible to have multiple pool? My objectives are:
Thank you
The rpool can only be a mirror (raid1). You can have different pools for what you described. You just have to keep in mind that your raid decision is good for your performance and safety needs!
Hi Sir,
You mean, I can have disk in raid1 for the 2hdd(rpool) and have another disk in raid5 for the remaining 6hdd?
Thank you
If that is what you want to do, yes you can do that. Still, is a raid5 a good performance/safety decision?
Hi Sir,
Raid5 was suggested by our sysad since it was the common used in our windows server. In my understanding, its safe, unless two disk got damage.
Any suggestion Sir?
Hi flexi,
Firstly , we must know the your hardware system ?
Maybe we can determine best choice for your needs to that.
if your system x86 arch maybe you can try "LSI or other Logic Configuration Utility Console " which appears in the boot stages..
You assume 8 disks , but has your system contains these from local or san ?
if your all disks has local ( SAS / Sata .. ) , then what is the your critical data ? ( database )
all disks are at the same or size ?
Yes , Raid5 can be good choice or maybe not for avaliable disk space or security requirements.
That can change to your needs.
For exa : you want to create hard raid 5 vol with 6 disks at size 300 GB. ( security : with "1 disk" failure )
then your avaliable space ( n x size -1 x size ) = 6 x 300 - 1 x 300 ( for total parity size ) = 1500 GB
So at the reality , what is your systems for solaris ( oracle sparc / x86 / non-oracle x86 .. )
in Sparc systems LSI raid controllers does not support for raid 5
if you will be use zfs , then you can use software raid with zfs ( zpool )
For exa :
Now i can say what i do with 6 disk in my sparc hardware.
( note : Raid Controller Support = ( 0, 1, 1e, 10 ) )
We have 6 disks on the two raid controller..
4 disk for RAID 1E ( security two disk in teory but may be "one disk" to controller !! )
-----------------------------------------------------------------------------------------
( * You can create similiar this volume with your 6 disks )
( boot prompt )
{0} ok select /pci@300/pci@1/pci@0/pci@2/scsi@0
{0} ok show-children
FCode Version 1.00.61, MPT Version 2.00, Firmware Version 9.00.00.00
Target 9
Unit 0 Disk HITACHI H109030SESUN300G A31A 585937500 Blocks, 300 GB
SASDeviceName 5000cca01641b6c8 SASAddress 5000cca01641b6c9 PhyNum 0
Target a
Unit 0 Removable Read Only device TEAC DV-W28SS-W 1.0A
SATA device PhyNum 7
Target b
Unit 0 Disk HITACHI H109030SESUN300G A31A 585937500 Blocks, 300 GB
SASDeviceName 5000cca01642c8c0 SASAddress 5000cca01642c8c1 PhyNum 1
Target c
Unit 0 Disk HITACHI H109030SESUN300G A31A 585937500 Blocks, 300 GB
SASDeviceName 5000cca016422b48 SASAddress 5000cca016422b49 PhyNum 2
Target d
Unit 0 Disk HITACHI H109030SESUN300G A31A 585937500 Blocks, 300 GB
SASDeviceName 5000cca0164271b8 SASAddress 5000cca0164271b9 PhyNum 3
{0} ok 9 b c d create-raid1e-volume --> Create RAID 1E ( Mirroring Extending ( + Stripped ) = Striped and Mirrored disk array )
Target 9 size is 583983104 Blocks, 298 GB
Target b size is 583983104 Blocks, 298 GB
Target c size is 583983104 Blocks, 298 GB
Target d size is 583983104 Blocks, 298 GB
The volume can be any size from 1 MB to 570296 MB
What size do you want? [570296]
Volume size will be 1167966208 Blocks, 597 GB --> 4 X 300 /2 = 600 GB disk space
Enter a volume name: [0 to 15 characters] vol1
Volume has been created
{0} ok unselect-dev
{0} ok select /pci@300/pci@1/pci@0/pci@2/scsi@0
{0} ok show-volumes
Volume 0 Target 381 Type RAID1E (Mirroring Extended)
Name vol1 WWID 011ebf1128d9fdcd
Optimal Enabled Volume Not Consistent Background Init In Progress
4 Members 1167966208 Blocks, 597 GB
Disk 3
Member 0 Optimal
Target 9 HITACHI H109030SESUN300G A31A PhyNum 0
Disk 2
Member 1 Optimal
Target b HITACHI H109030SESUN300G A31A PhyNum 1
Disk 1
Member 2 Optimal
Target c HITACHI H109030SESUN300G A31A PhyNum 2
Disk 0
Member 3 Optimal
Target d HITACHI H109030SESUN300G A31A PhyNum 3
{0} ok 0 activate-volume
Volume 0 is now activated
{0} ok unselect-dev
-> Now first Raid volume ( vol1 ) is Ready..
2-) The other 2 disk with RAID1 ( via the other Raid Controller port )
....
....
{0} ok 9 a create-raid1-volume --> RAID 1 Mirror for Operating System ( security one disk )
....
.... same operations..
-> Now first Raid volume ( vol2 ) is Ready..
Raid operations completed..
I installed the Solaris OS on the this volume ( vol2 ) at the Disk Select Menu..
when system is on :
# zpool list rpool
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
rpool 556G 108G 448G 19% 1.00x ONLINE -
# zpool status rpool
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c6t3B347FFD52B27CEFd0s0 ONLINE 0 0 0
errors: No known data errors
# echo|format|nawk '/c6t3B347FFD52B27CEFd0/{a=$0;getline;print a RS $0;exit}'
533. c6t3B347FFD52B27CEFd0 <LSI-Logical Volume-3000 cyl 35701 alt 2 hd 256 sec 128> local02 --> ( vol2 )
/pci@3c0/pci@1/pci@0/pci@2/scsi@0/iport@v0/disk@w3b347ffd52b27cef,0
# prtdiag|nawk '/SASHBA/{a=$0;getline;print a RS $0;}'
/SYS/MB/SASHBA0 PCIE scsi-pciex1000,87 LSI,2308_2 8.0GT/x4 8.0GT/x4
/pci@300/pci@1/pci@0/pci@2/scsi@0
/SYS/MB/SASHBA1 PCIE scsi-pciex1000,87 LSI,2308_2 8.0GT/x4 8.0GT/x4 --> ( raid controller port for vol2 )
/pci@3c0/pci@1/pci@0/pci@2/scsi@0
note : my raid LSI controller(s) does not support other raid levels ( eg : raid5.. )
# raidconfig list all -v|grep Support
Supported RAID Levels: 0, 1, 1E
Supported RAID Levels: 0, 1, 1E
regards
ygemici
Thank you ygemici 
Hi flexi,
Firstly , we must know the your hardware system ?
Maybe we can determine best choice for your needs to that
.
The choice was a HP Proliant server.
if your system x86 arch maybe you can try "LSI or other Logic Configuration Utility Console " which appears in the boot stages..
Yes, this is the plan to raid the disks. Can I have multiple pool using this?
You assume 8 disks , but has your system contains these from local or san ?
From local only.
if your all disks has local ( SAS / Sata .. ) , then what is the your critical data ? ( database )
The main purpose of this new server is for development and disaster recovery testing. We have a separate production/live system for our database.
all disks are at the same or size ?
Yes, same size (1.2GB)
Regards,
flexi
Now , I assume it's a server with 8 discs.
Best practice ,
first 2-disk "Raid1" structure will suffice as you would think for the system ( of course we will monitor the warning LEDs and ILOM )
for the other 6 disks , the most suitable raid config is "Raid 5" ( if your database was production enviroment then maybe 1 disk spare can be considered but not needed for develop )
and your hp tools ( smart storage admin / smart array ) forwards to you for creating raid pools and and advices for some details ( stripe size , raid conf and some defaults... )
good luck
regards
ygemici
RAID(s) aside, consider using Oracle ASM for database instead of ZFS.
Other then that, if you need filesystems, i would strongly recommend using UFS with directio option.
Performance wise, ZFS will be a hassle to tune to achive UFS DIO or ASM performance level.
Also, fragmentation could become issue in long term usage if pool exceeds 80% occupied space.
IF you choose ASM or UFS filesystem for databases, and ZFS for rpool, be sure to limit arc cache (arc_max) to a sane value, not leaving ZFS to eat all the available memory.
I tend to avoid hardware controllers when using ZFS (let zfs handle the protection), but it's your choice.
Hope that helps
Regards
Peasant.
Note that zfs_arc_max has been deprecated since Solaris 11.2 (2014) and newer releases and the recommended tunable is now user_reserve_hint_pct which, unlike the former, can be dynamically set.
First, I'd say that a single RAID5 disk for both the OS and everything else is a bad setup.
As others have mentioned, put the OS on a hardware RAID mirror using two drives. That one drive will be your root ZFS pool (rpool). (And if this were to be a long-lived server under my control, I'd create another two-disk RAID mirror for a second ZFS root pool (rpool2) to be used for OS upgrades and patches - always creating the new boot environment on the other rpool in order to avoid a nasty hell of ZFS rpool clones and snapshots. If the boot environment being updated is on rpool, the new boot environment is created on rpool2)
Then use the other 6 disks for the database - exactly how would depend strongly on what database and what it stores and how it's going to be used.
And yes, in general you will want to limit the ZFS ARC on a DB server - severely (you don't need to read /var/adm/messages very fast...). If your database isn't using ZFS to store data, there's no need for more than a token ZFS ARC, and especially for an Oracle DB not using ZFS storage an unrestricted ZFS ARC can cause severe performance problems. (Oracle DB tends to use large-page-size chunks of memory. ZFS ARC uses 4k pages. On a server with high memory pressure, dynamic Oracle memory demands will force the kernel to have to coalesce memory to create large pages for the Oracle DB process(es). ZFS ARC pressure then breaks those pages up - rinse, lather, repeat as the server unresponsively just sits and spins...)
HP Proliant? Meh. A few years ago, a customer I supported bought new HP servers - because they were "cheaper" than Oracle's servers. Oh? Well, the new servers weren't any faster than the old (so old they still had "Sun" on them...) servers - and it took quite a bit of BIOS tuning just to get the brand-spanking-new "fast" HP servers to even match the old Sun ones performance-wise. As far as "cheaper"? We had to install the HBAs ourselves (labor time is expensive...) and THEN we found out that the ILOM software wasn't part of the basic HP server - it had to be bought/licensed separately - then installed (even more expensive labor hours). Oh, and the HP server didn't come with four built-in 10 gig ethernet ports, so we had to add ethernet cards - more money and more time. When all was done, the customer paid a lot of money and wound up with new HP servers that took a lot of time and effort to make just as fast as the older Sun servers they replaced. Simply buying new servers from Oracle would have resulted in actually getting faster servers - for less money, less time, and a lot less effort.
Slapping a bunch of commodity parts around a good CPU and a decent amount of RAM doesn't make for a fast server. I/O bandwidth, memory bandwidth, disk controller quality? They matter too, and using the cheapest parts you can find in China slapped onto the cheapest motherboard doesn't cut it - especially when you turn around and nickel-and-dime customers over things like ILOM software licenses...
I'm not impressed with HP.</RANT>
Hi All,
Thank you for all the input/advice. Well appreciated.
Regards,
flex