# How to test RAID10 array performance \[Debian Wheezy\]? **URL:** \ **Category:** UNIX for Advanced & Expert Users\ **Created:** [September 24, 2014, 12:22am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602 "2014-09-24T00:22:59Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![gacanepa](https://community.unix.com/user_avatar/community.unix.com/gacanepa/32/8775_2.png) [@gacanepa](https://community.unix.com/u/gacanepa)\ **Post date:** [September 24, 2014, 12:22am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/1 "2014-09-24T00:22:59Z")
I have created a RAID10 array (near layout) with four 8 GiB virtual hard drives, making a 16 GiB array (/dev/md0 formatted as ext4). On the other hand, I have a 16 GiB RAID1 array (/dev/md1, also formatted as ext4). The purpose of these setups is to compare the read and write performances of each array. So far, I have used dd to perform the following tests: dd (sequential write performance): ```plaintext dd if=/dev/zero of=/dev/md0 count=512 bs=1024k 512+0 records in 512+0 records out 536870912 bytes (537 MB) copied, 6.60623 s, 81.3 MB/s dd if=/dev/zero of=/dev/md1 count=512 bs=1024k 512+0 records in 512+0 records out 536870912 bytes (537 MB) copied, 5.74421 s, 93.5 MB/s ``` dd (sequential read performance): ```plaintext dd if=/dev/md0 of=/dev/null bs=4096k 4093+1 records in 4093+1 records out 17168334848 bytes (17 GB) copied, 168.665 s, 102 MB/s dd if=/dev/md1 of=/dev/null bs=4096k 4093+1 records in 4093+1 records out 17170300928 bytes (17 GB) copied, 44.6421 s, 385 MB/s ``` So I changed the RAID10 array layout to f2, and sequential read performance improved but the write performance decreased compared to n2 (which was the previous layout): ```plaintext dd if=/dev/md0 of=/dev/null bs=4096k 4093+0 records in 4093+0 records out 17167286272 bytes (17 GB) copied, 110.424 s, 155 MB/s dd if=/dev/zero of=/dev/md0 count=512 bs=1024k 512+0 records in 512+0 records out 536870912 bytes (537 MB) copied, 6.84386 s, 78.4 MB/s ``` I was expecting that the RAID10 array would have a better read performance that that of RAID1, but it isn't the case as per these tests - even when I have performed them several times to avoid [outliers](http://en.wikipedia.org/wiki/Outlier). I have also used [iozone](http://iozone.org/) for benchmarking the same setups, with similar results. I am also aware that other factors may impact performance, such as the hardware used (perhaps virtual hard drives may not provide the best scenario?) and the filesystem. That being said, what would be the best setup for a RAID10 array that will undergo more reads than writes? (In addition, I am looking for further reasons to use RAID10 instead of RAID1 other than the fault tolerance provided by the latter). Any tips and ideas will be more than welcome. Thanks in advance. ---
**Author:** ![gull04](https://community.unix.com/user_avatar/community.unix.com/gull04/32/14972_2.png) [@gull04](https://community.unix.com/u/gull04)\ **Post date:** [September 24, 2014, 3:35am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/2 "2014-09-24T03:35:10Z")
Hi Gacanepa, You don't really provide enough information about the actual makeup of the disks, what is the actual backend you are using as this can make a significant difference. How are the devices accessed - there are a shedload more questions on this front. In general and it is a generalisation the RAID10 (striped then mirrored) should provide the better performance (true also for RAID01) than RAID1. But it is not really possible to comment further than that without knowing a bit more information about your setup. Regards Dave ---
**Author:** ![gacanepa](https://community.unix.com/user_avatar/community.unix.com/gacanepa/32/8775_2.png) [@gacanepa](https://community.unix.com/u/gacanepa)\ **Post date:** [September 24, 2014, 9:37am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/3 "2014-09-24T09:37:29Z")
> [@gull04](#): > > Hi Gacanepa, > > You don't really provide enough information about the actual makeup of the disks, what is the actual backend you are using as this can make a significant difference. How are the devices accessed - there are a shedload more questions on this front. > > In general and it is a generalisation the RAID10 (striped then mirrored) should provide the better performance (true also for RAID01) than RAID1. But it is not really possible to comment further than that without knowing a bit more information about your setup. > > Regards > > Dave Dave, First off, thank you for taking the time to reply to my post. Here are some other details that I hope will be helpful (if there is any other backend information that you need, please let me know): -Both RAID arrays (1 & 10) were created in the same system. -uname -a: ```plaintext Linux debian 3.2.0-4-486 #1 Debian 3.2.60-1+deb7u3 i686 GNU/Linux ``` -From lshw: ```plaintext *-memory description: System memory physical id: 1 size: 502MiB *-cpu product: AMD Athlon(tm) II X2 250 Processor vendor: Advanced Micro Devices [AMD] physical id: 2 bus info: cpu@0 version: 15.6.3 size: 3GHz width: 32 bits ``` -mdadm --detail /dev/md0: ```plaintext /dev/md0: Version : 1.2 Creation Time : Wed Sep 24 10:02:18 2014 Raid Level : raid10 Array Size : 16764928 (15.99 GiB 17.17 GB) Used Dev Size : 8382464 (7.99 GiB 8.58 GB) Raid Devices : 4 Total Devices : 5 Persistence : Superblock is persistent Update Time : Wed Sep 24 10:04:56 2014 State : active Active Devices : 4 Working Devices : 5 Failed Devices : 0 Spare Devices : 1 Layout : far=2 Chunk Size : 512K Name : debian:0 (local to host debian) UUID : 36b8594f:e14b8caf:df8dedff:99320083 Events : 18 Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 2 8 49 2 active sync /dev/sdd1 3 8 65 3 active sync /dev/sde1 4 8 81 - spare /dev/sdf1 ``` -mdadm --detail /dev/md1: ```plaintext /dev/md1: Version : 1.2 Creation Time : Wed Sep 24 10:07:57 2014 Raid Level : raid1 Array Size : 16767872 (15.99 GiB 17.17 GB) Used Dev Size : 16767872 (15.99 GiB 17.17 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Wed Sep 24 10:14:22 2014 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Name : debian:1 (local to host debian) UUID : 98144751:72e778b7:dc92a91c:fffbfb43 Events : 17 Number Major Minor RaidDevice State 0 8 97 0 active sync /dev/sdg1 1 8 113 1 active sync /dev/sdh1 ``` ---
**Author:** ![gull04](https://community.unix.com/user_avatar/community.unix.com/gull04/32/14972_2.png) [@gull04](https://community.unix.com/u/gull04)\ **Post date:** [September 24, 2014, 10:12am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/4 "2014-09-24T10:12:05Z")
Hi, Can you also post the outpot of the following commands, ```nohighlight pvdisplay lvdisplay cat /etc/mdadm/mdadm.conf ``` Regards Dave ---
**Author:** ![gacanepa](https://community.unix.com/user_avatar/community.unix.com/gacanepa/32/8775_2.png) [@gacanepa](https://community.unix.com/u/gacanepa)\ **Post date:** [September 24, 2014, 10:30am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/5 "2014-09-24T10:30:37Z")
Dave, Thanks again! As for pvdisplay and lvdisplay, I must mention here that these RAID arrays are not on LVMs (but here is the output of the commands anyway): ```nohighlight root@debian:~# pvdisplay --- Physical volume --- PV Name /dev/sda5 VG Name debian PV Size 9.76 GiB / not usable 2.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 2498 Free PE 0 Allocated PE 2498 PV UUID 0VU3Cr-2uLV-68Y8-W9hZ-uwav-APT7-or633a root@debian:~# lvdisplay --- Logical volume --- LV Path /dev/debian/root LV Name root VG Name debian LV UUID Vuyq6i-Ctxr-nz4f-MIYi-x6fB-Fgif-Gjommv LV Write Access read/write LV Creation host, time debian, 2014-08-12 21:27:22 -0300 LV Status available # open 1 LV Size 332.00 MiB Current LE 83 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:0 --- Logical volume --- LV Path /dev/debian/usr LV Name usr VG Name debian LV UUID VIKvTb-Kl4V-l2Uw-K5Zp-vjjG-BCoP-MFO2d5 LV Write Access read/write LV Creation host, time debian, 2014-08-12 21:27:23 -0300 LV Status available # open 1 LV Size 3.41 GiB Current LE 874 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:2 --- Logical volume --- LV Path /dev/debian/var LV Name var VG Name debian LV UUID LPUuFP-JX6Q-ShWJ-otNN-aBvu-DKNg-SxJfcv LV Write Access read/write LV Creation host, time debian, 2014-08-12 21:27:23 -0300 LV Status available # open 1 LV Size 1.66 GiB Current LE 425 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:3 --- Logical volume --- LV Path /dev/debian/swap_1 LV Name swap_1 VG Name debian LV UUID fLMGxe-ksTR-VNbM-rknH-G6OR-Aq1z-Q2RIwP LV Write Access read/write LV Creation host, time debian, 2014-08-12 21:27:23 -0300 LV Status available # open 2 LV Size 560.00 MiB Current LE 140 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:1 --- Logical volume --- LV Path /dev/debian/tmp LV Name tmp VG Name debian LV UUID BgKzko-pArn-LGq9-gLRG-xYFM-clc1-dJkzyN LV Write Access read/write LV Creation host, time debian, 2014-08-12 21:27:23 -0300 LV Status available # open 1 LV Size 300.00 MiB Current LE 75 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:4 --- Logical volume --- LV Path /dev/debian/home LV Name home VG Name debian LV UUID viKzZn-oOmC-JI88-itWv-26vw-eVoG-6RWWGL LV Write Access read/write LV Creation host, time debian, 2014-08-12 21:27:23 -0300 LV Status available # open 1 LV Size 3.52 GiB Current LE 901 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:5 root@debian:~# ``` mdadm.conf (I just realize the definition for /dev/md1 is missing, but I am failing to understand how that would help to the understanding of why the RAID10, with its current configuration, is not performing as well as it should be) ```nohighlight # Please refer to mdadm.conf(5) for information about this file. # # by default (built-in), scan all partitions (/proc/partitions) and all # containers for MD superblocks. alternatively, specify devices to scan, using # wildcards if desired. #DEVICE partitions containers # auto-create devices with Debian standard permissions CREATE owner=root group=disk mode=0660 auto=yes # automatically tag new arrays as belonging to the local system HOMEHOST # instruct the monitoring daemon where to send mail alerts MAILADDR root # definitions of existing MD arrays ARRAY /dev/md/0 metadata=1.2 UUID=d9a83766:cdec2926:53d85d4a:01f535ca name=debian:0 # This file was auto-generated on Wed, 10 Sep 2014 23:25:15 -0300 # by mkconf 3.2.5-5 ``` ---
**Author:** ![gull04](https://community.unix.com/user_avatar/community.unix.com/gull04/32/14972_2.png) [@gull04](https://community.unix.com/u/gull04)\ **Post date:** [September 24, 2014, 10:36am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/6 "2014-09-24T10:36:18Z")
Hi, The missing entry in mdadm.conf is a problem - but probably not the cause of your issue. But to eliminate it you should update the file and re-run the test. You can I think get the correct information by running, ```nohighlight mdadm --detail /dev/md/1 ``` Also a quick question, how have you created the "virtual drives"? Regarda Dave ---
**Author:** ![gacanepa](https://community.unix.com/user_avatar/community.unix.com/gacanepa/32/8775_2.png) [@gacanepa](https://community.unix.com/u/gacanepa)\ **Post date:** [September 24, 2014, 10:54am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/7 "2014-09-24T10:54:06Z")
I created the virtual drives via VirtualBox wizard (select VM --\> Settings --\> Storage --\> SATA --\> Add new disk ---
**Author:** ![gull04](https://community.unix.com/user_avatar/community.unix.com/gull04/32/14972_2.png) [@gull04](https://community.unix.com/u/gull04)\ **Post date:** [September 24, 2014, 11:11am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/8 "2014-09-24T11:11:44Z")
Hi, This will explain things, I think. But it does give rise to a couple of other questions, the first being how many physical drives are there in the system. If the answer is less than five, you will see degradation in the read and write perfomance as it is likely that behind the hypervisor (in your case VirtualBox) there will be less paths than defined devices. The second question is are you in fact running VirtualBox on a Physical or within a Virtual Server. Regards Dave ---
**Author:** ![gacanepa](https://community.unix.com/user_avatar/community.unix.com/gacanepa/32/8775_2.png) [@gacanepa](https://community.unix.com/u/gacanepa)\ **Post date:** [September 24, 2014, 11:15am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/9 "2014-09-24T11:15:38Z")
1) I have only one physical drive in the system. 2) I am running VirtualBox on a Physical server (host). ---
**Author:** ![gull04](https://community.unix.com/user_avatar/community.unix.com/gull04/32/14972_2.png) [@gull04](https://community.unix.com/u/gull04)\ **Post date:** [September 24, 2014, 11:21am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/10 "2014-09-24T11:21:41Z")
Hi, Unfortunately having only a single drive here is going to skew these figures, to test this properly you'll need multiple physical drives. There are multiple impacts here - mostly from the hard drive, the seek time and latency will be major contributors as virtual box only creates a contiguous file if you specify that it should preallocate all the disk space. Other than that it will scatter the writes all over the disk if you specify that it should grow the file as required. I don't think that this is really a suitable way to evaluate the performance of any RAID as it's all on one physical disk. Regards Dave ---
**Author:** ![gacanepa](https://community.unix.com/user_avatar/community.unix.com/gacanepa/32/8775_2.png) [@gacanepa](https://community.unix.com/u/gacanepa)\ **Post date:** [September 24, 2014, 12:32pm UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/11 "2014-09-24T12:32:38Z")
> [@gull04](#): > > Hi, > > I don't think that this is really a suitable way to evaluate the performance of any RAID as it's all on one physical disk. > > Regards > > Dave I agree with you. I actually gave it a try because I didn't have any other setup available to test. But in theory (leaving VirtualBox aside, and if I had the actual physical drives available), reads should be ~twice as fast, and writes about as fast as RAID1, am I correct? ---
**Author:** ![gull04](https://community.unix.com/user_avatar/community.unix.com/gull04/32/14972_2.png) [@gull04](https://community.unix.com/u/gull04)\ **Post date:** [September 24, 2014, 1:45pm UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/12 "2014-09-24T13:45:05Z")
Hi, The RAID10 performance will be faster, but I'm not sure it will be twice as fast. Generally better performance is related to an increase on the number of spindles. There are also benefits to be derived from the committal regime but there can be a certain amount of trial and error, there are a great many factors that influence the performance of a RAID array - that's why there are a great many books and courses on the subject. Regards Dave ---
**Author:** ![achenle](https://community.unix.com/letter_avatar/achenle/32/6_449c533f586939b6e5f246632a46421f.png) [@achenle](https://community.unix.com/u/achenle)\ **Post date:** [September 24, 2014, 2:58pm UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/13 "2014-09-24T14:58:36Z")
> [@gacanepa](#): > > I agree with you. I actually gave it a try because I didn't have any other setup available to test. But in theory (leaving VirtualBox aside, and if I had the actual physical drives available), reads should be ~twice as fast, and writes about as fast as RAID1, am I correct? That depends on what the bottleneck is in whatever configuration you're testing, and how you're testing it. ---
**Author:** ![drl](https://community.unix.com/user_avatar/community.unix.com/drl/32/542_2.png) [@drl](https://community.unix.com/u/drl)\ **Post date:** [September 25, 2014, 9:41am UTC](https://community.unix.com/t/how-to-test-raid10-array-performance-debian-wheezy/346602/14 "2014-09-25T09:41:07Z")
Hi. Here is a report on several disk configurations that was done in 2009 using benchmark code bonnie++ (32-bit 2-CPU Athlon box): ```nohighlight Description-en: Hard drive benchmark suite. It is called Bonnie++ because it was based on the Bonnie program. This program also tests performance with creating large numbers of files. ``` which can be found in the Debian repositories. Currently I use RAID10 with 4 SATA disks with LVM on top of the RAID on a virtual-machine server. I have not run the bonnie++ benchmarks on that system, but perhaps I will now that I see that there is some interest. The only reason I would use RAID10 with less than 4 disks is practice (as with setup, etc), not for performance. Best wishes ... cheers, drl