# RAID-10 vs 2*RAID-0 then RAID-1

## Moriah

I am wondering what the difference is between RAID-10 and 2 RAID-0 striped arrays which are then combined into a RAID-1 mirror.  Is it only that if a drive dies, then you only need to update the single drive that died, instead of one whole side of the mirror?

----------

## NeddySeagoon

Moriah,

Its a single kernel md driver, so its faster.  

With 4 spindles why not raid 5?

----------

## Moriah

Because of system redundancy requirements, I need the mirror.  Besides, its faster.    :Cool: 

My real question concerns regens.  On a RAID-0 paired with a second RAID-0 to make a mirrored pair, If I pull a drive and replace it, the whole capacity of the RAID-1 mirror must go thru a regen, but with a pair of mirrors striped so it is 2 RAID-1's combined with a RAID-0 on top, then if I pull a drive, it regens twice as fast because each mirrored pair is only half the size.

I was wondering what the regen behavior of a RAID-10 would be.   :Surprised: 

----------

## NeddySeagoon

Moriah,

The rebuild speed is limited by the kernel.

It makes sure that you can use the degraded raid set while its rebuilding.

----------

## HeissFuss

This post is about raid-10 (striping over mirrored pairs) vs raid 0+1 (mirroring over a pair of stripe arrays.)

Imo, raid10 is better for a couple of reasons:

For a single disk failure, raid-10 only needs to rebuild the failed disk from its counterpart.  In raid 0+1, one entire raid0 array needs to be rebuilt which will take a significantly longer amount of time.

In raid10 you can lose n/2 disks provided that you don't lose two disks which are paired together.  In raid 0+1, you can also lose n/2 disks, provided that all disks failures are from only one of the stripe arrays.  If you lose a disk from both stripes, you're screwed.

For a 4 disk setup, it probably doesn't matter which way you go, but it'd be more convenient to use the mdraid10 module rather than build your own 0+1 (you can't use autodetect to build it for one thing, for another you're adding two md layers rather than one.)  However, I have a suspicion that this is what the mdraid10 driver is doing anyway.  Comparing a 4 disk mdraid10 to a 4 disk mdraid5 on the same disks, the raid5 outperforms the raid10 by a fair amount in reads (I haven't yet tested writes, and I haven't tested IOps/random access/simultaneous IOs.)[/quote]

----------

## Moriah

The application is a disk based backup server, ala dervish.  The raid array wil be written to much more than it is read from, and forming hard links is a major part of the work.  Deletions are typically done a month aty a time, and mostly involve removing hard links -- many millions of them.   From a user perspective, deletion time is an important statistic.  

The scenario is a 3-way raid mirror where a single element of the mirror is removed daily and transported to a remote location for offline backup.  The empty nslot in the array is then filled with a spare disk that used to be a remote offline backup disk several generations ago.  For this reason, regen time is also very important.  The 3-way mirror protects against faults when one slot is not online.

So regens will mostly be against an entire mirror element.  If that total RAID size is greater than 1 TB (max disk size today), then each mirror element needs to be a multi-disk arry itself.  This was where the RAID 0+1 arrangement came in.

So given the intended use, which seems both more reliable, and better able to meet the regen and delete time requirements?

Any thoughts?

Thanks!    :Very Happy: 

----------

## HeissFuss

I don't think your raid choice will have any effect on deletion times.  That's really more of a file system issue.

If I understand correctly, you currently have 3 drives in a 3-way mirror, and one drive you take off-site to back up daily.  I'm not really clear on the part of putting an old spare drive back in the array though?

Your question is how does this method scale if your mirror pairs are beyond 1TB in which case you'll need multiple drives for each pair (in which case you'd be taking multiple drives out for backup.)

As you've stated, if you stay with the 3-way mirror, you'll need to rebuild the third leg entirely every day.  In this scenario, raid 10 isn't an option.  You can't take a mirrored pair out of a raid10 without the entire raid failing (and besides, you'd be limiting your backup solution to 2 disks.)

You probably have a couple of options.  Continue with 3-way mirror except replace the single disk elements with striped ones (raid 0+1)

1 1 1

------

0 0 0

0 0 0

0 0 0

Or, you could replace the 3-way mirror with a single raid-0 for the backup disks and a raid-5 or raid-6 for the normal disks with a raid-1 across both.  Either way, your resync will still take about as long as it takes to write the entire raid-0 leg of your mirror.  You should definitely do a bit of benchmarking with your workload on a smaller scale.  What may look good in theory may not actually perform as expected (raid-5 or 6 may actually work better on writes than an equally sized raid 1+0, etc.)

You may want to possibly look at DRDB or another mirror solution which allows for partial mirror resync.  I can't say how well they perform though, or how much space would need to be taken by copy-on-write (or whatever they use) to allow for partial resync.

Good luck.

----------

## Moriah

The requirement is for resync to occur during the business day, when the backup server is only loafing along doing mostly nothing; its only job then is to be available so users can retrieve backed up files from it if they need to.  If the drive (or striped raid-0 set) is pulled every night before they turn out the lights and go home, and another drive -- say from several days ago -- is stuffed into that empty slot the next morning, then regen needs to be complete before the backup kicks off after the lights go out that night.  This should be at least 8 hours allowed for the regen of an entire mirror element.

This approach means you have several days worth of past history off-line and off-site in the event of a flood like in Iowa this year.    :Mad: 

RAIDs are great if a single drive fails, but your backups need to be regularly taken off-line and off-site to provide extra safety.  In addition, the plan is to burn Blu-Ray write-once disks every month for true archival history.

Because the Blu-Ray disks and the pulled mirror elements will be off site, they are subject to loss or theft, so they must be encrypted.  This means that the entire 3-way raid mirror must me encrypted, so when an element is pulled it is already encrypted and ready to go.

As you mentioned, deletion and hard linking performance are more related to filesystem choice than to raid setup, but that is another topic.

----------

