Re: Software vs Hardware RAID 10?
On Sat, Aug 25, 2007 at 08:37:33AM -0500, Neil Gunton wrote:
> I'm curious as to whether anyone has experience of software RAID in
> Linux giving better overall performance on RAID10 than a RAID card such
> as the Adaptec 2015S.
>
> Server: Dual Opteron 265, 1.8GHz, i.e. 4 cores total, 4GB RAM, 4x10k
> Fujitsu SCSI 73GB, Adaptec 2015S zero-channel RAID card.
>
> Currently I have the four drives in RAID10 using the Adaptec, i.e. it
> appears as one big drive to Linux. Then a few days ago I saw this video
> presentation on scaling from one of the guys who developed YouTube:
>
> http://video.google.com/videoplay?docid=-6304964351441328559
>
> One thing that really caught my attention was around the 34:50 mark. He
> talks about how they were seeing IO wait on their RAID 10 setup, which
> had 5 sets of 2 disk RAID1 arrays, appearing to Linux as one device.
> They thought that since Linux was only seeing one big single RAID 10
> device, maybe the kernel wasn't parallelising disk reads and writes
> aggressively enough. So they changed it to still use hardware RAID 1 on
> the disk pairs, but then exposed these RAID 1 devices to the linux
> kernel (i.e. now 5 distinct RAID 1 arrays) and then used software RAID 0
> to stripe across those. This seems really simple and ingenious, since it
> allows linux now to realize that there is more potential to parallelize
> io. They apparently saw 20-30% increase in performance on the same
> hardware after doing that.
>
> I only have four disks, but now I'm wondering if I might see improvement
> in performance by reconfiguring as two RAID 1 arrays and then doing
> software RAID 0 across those.
>
> Or, why not go even further and do JBOD on the RAID controller, and do
> all the RAID10 in Linux? Would that see better performance?
>
> I have little knowledge of what "zero channel" really means, or how good
> the hardware raid processor is in the Adaptec 2015S. What I am fairly
> sure of is that with the 4 Opteron cores, I have CPU cycles to spare; on
> this server I am much more likely to run out of io throughput before I
> run out of pure CPU cycles. So I am interested in maximizing the io,
> even if it means doing a little more work in the kernel. I'm not doing
> RAID5, so there's no parity work being done; I don't have a handle on
> how much work RAID10 takes for software RAID. Also, really have no take
> on where the real IO bottleneck is here - if putting the RAID into Linux
> would actually have any benefit with this particular type of card. I
> mean, if the actual bottleneck is actually on some bus or other in
> between the disks and the motherboard, then it really doesn't matter
> where the RAID is handled.
>
> Does anybody have any experience of this, or wisdom to impart? I have
> read all the arguments in favor of software RAID, and that's all very
> nice, but what I am primarily wondering about here is if it's likely
> that shifting over to software RAID might allow Linux to improve IO by
> more parallelization of reads/writes, or if it's just shifting
> complexity from one place to another, with the same bottleneck in
> between? (In which case, I guess, might as well go with software RAID,
> since it seems easier to recover from a controller crash).
Linux software raid (at least for raid 0 and 1 and combinations) is
often faster than what a hardware raid card can do, and almost certainly
better than what any fakeraid pulls off (since their drivers are often
crap at doing the raid in software). raid 5 on the other hand can often
be done faster by a dedicated xor engine, and since you don't have to do
as much reading to write you reduce bus bandwidth. Probably the main
advantage of hardware raid is reduced us usage since only the actual
desired data goes over the bus, while with software raid, raid 1
involves twice as much bus traffic to write to both devices.
Certainly many people run 3ware cards as just large driver controllers
and use linux software raid on them to get the best performance.
If you have a fast cpu and you aren't using it for anything else (like
is often the case on a file server) and if you don't cause your pci bus
to be saturated, then software raid will probably be faster than
hardware raid in general.
--
Len Sorensen
Reply to: