RAID 5 Questions

[cbrunny]

Hi All,

I'm a bit of a rookie when it comes to RAID. Been trying to read as much as I can but its hard to get consistent information. Anandtech has never steered me wrong in the past, so I thought I'd give it a go again here.

I've just recently finished building a home server in anticipation of WHS 2011, and I'm looking at RAID 5 as a storage solution. I've got two questions based on inconsistent information I've read online.

First, is it true that some hard drives do not support RAID, and are therefore more likely to fail? If so, is there anywhere online I can go to find out which is which?

Second, is it possible to add additional drives of the same type and capacity to a RAID 5 array after it has been constructed, say 2 months down the road? I'm working off a software controller built into WHS 2011, which is likely the same as the one in Win7. I'm not concerned with the performance loss due to the software controller.

Thanks!
cb

 

[Emulex]

cheap non-raid edition drives will not be stable with most raid controllers that rely on TLER (or equivalent). QNAP and DROBO disregard all smart/etc and handle errors on their own. you can use an SV/AV drive (no error correction at all), a consumer drive where it can take minutes to return an error or instant smart error, or TLER=8 (enterprise nearline sata) - this driver appears to be not open source or in any windows platform that i know bout - correct me if i am wrong.

The RE4 (2003) WD is fantastic for raid - i pummel the drives for d2d backup. 50 machines to one set full bare metal backup every night. I also run esxi on 6 (soon to be 8) in raid-10 and its almost fast enough for production although i suspect svmotion would fail miserably on a loaded i/o system. (say a few machines running, backups, and storage migration (svmotion).

Good raid controllers can do hot-expansion. These usually require battery/flash back write cache since you are restriping the disks while in operation.

I also have a 10 drive seagate 500gb (7200.11) windows storage server (it was free!) from hp. It is raid-5 and it sucks donkey on write with 512meg battery back write cache at 75% write 25% read with disk drive cache enabled(dangerous for sata imo, ymmv).

Honestly i'd say use a drobo or qnap and stuff cheap drives in but use 2-disk protection or raid-10. Performance is poop with raid-5 and the rebuild times are horrendous with large disks.

keep in mind you need to separate storage from workload for the most part. You should use a server with ECC protection (xeon -> ram -> bus). An old ml110 G5 is pretty cheap these days. I think i paid $200 for the box with 8gb ddr2 ecc and $99 for a p400/512bbwc and replaced the battery for $50. $20 for a cable from ebay. it rocks. slow 1.86ghz core 2 duo based xeon. This plus RE4 style drives would be good for storage not gaming. just separate the roles

 

[dac7nco]

cbrunny, start off by reading this: http://www.servethehome.com/raid-reliability-failure-anthology-part-1-primer/

decide for yourself if you want to trust your data to a software RAID-5 array (hint: you don't). I highly recommend you read up on ZFS.

One of the things that is often overlooked by those beginning to play with RAID, is that you MUST have an available hot-spare drive with which to begin immediately rebuilding the array. The cheapest way you can even begin to implement a fairly safe RAID is to invest in a hardware RAID-6 controller, battery backup, the discs in the array, plus two online hot-spares. You've just spent $500 for an 8-port LSI controller & battery, and $1,200 for eight Ultrastars. Let's not forget the two SAS backplanes you'll want with fail LEDs, so you know which disc to pull.

Looking at that, you'd probably be well off listening to Emulex, and getting a Drobo or a Qnap.

Daimon

 

[Cerb]

First, is it true that some hard drives do not support RAID, and are therefore more likely to fail?

No. Typical consumer drives, upon finding a sector hard to read, will keep trying and trying and trying, and likely remap it. Power saving settings may also not play well with RAID controllers. The drives are perfectly fine. It's an incompatibility issue, and everybody followed suit, after WD capitalized on making different drives, where they previously gave out a utility that would reconfigure a drive for RAID.

I don't buy that the RAID drives are mechanically superior. IMO, you're paying for a firmware flash...but an often necessary one. Also, while I don't know about Samsung and Seagate, WD's RE3 and RE4 seem to have a performance edge in some cases w/ RAID, v. desktop counterparts.

If so, is there anywhere online I can go to find out which is which?

They all have a slightly different series name, and a higher price.
Samsung FnR (R=RAID?)
WD RE (RE=RAID Edition?)
Seagate ES (ES=Enterprise Storage?)
Hitachi Ultrastar

 

[Emulex]

yeah while the drobo or qnap isn't cheap - it makes up in the fact you can use cheap-azz drives (3tb?) and it just handles the business like nothing else can. I got SV (surveillance GP drives) nobody wants them because they have no error recovery (TLER=0seconds) - no good for desktop - no good for raid controller. But QNAP is cool - they ship them with these drives even. if you know the know and someone bought these incorrectly you can SCORE cheap. 2TB drive for $50 used (lightly) because of someone elses mistake or 4x1tb for $100 used (i've seen these deals pop up now and then) that would defer the cost of the raid unit itself. drobo is slow. qnap is hella fast (the atom ones).

so you could go $250 a 2TB RE4 and raid controller or $50-100 1-3TB drives and a (qnap?) - honestly i prefer a real raid controller inside the box. I picked up 4 of those nice LSI ibm raid controllers and they do raid-10/10E (uneven drive raid-10) but require nice drives. $25 raid card + $10 cable(x2) + RE4's and raid-10 is pretty darn fast since it requires no computation power and it is esxi compliant.

the drobo interface is less elegant and slower. the QNAP excels with its dual gigabit ethernet but that is far far slower than Direct Attach Storage.


Good reliable storage isn't cheap. you don't play games or crash your file server. you prefer ecc ram (helps prevent random bit flips/crashes), you use a ups with shutdown capability and make sure it has enough runtime to do the shutdown. and raid is safe.

that said. JBOD works pretty good for most folks.

 

[nk215]

I have 5 Samsung F4 drives on my raid5 setup. No problem so far. I tested them by creating checksum for all the files in the array and keep checking the files weekly for about 3 months or so. Zero issue. I did update the firmware on the Samsung before using them.

A raid 5 setup is not that slow. You will saturate the gigabit connection before bottom out the array. ~70 MB/sec is what you'll get over the Gbit network. Raid5 is 3x that in read and ~2x that in write.

 

[Cerb]

Unless you're using a POS file system, the drive will have ECC on data written to it, the RAID controller will have parity via RIAD 5's striping, and the filesystem will have crc data for either its data blocks, whole files, or both (usually just blocks, though, I think).

RAID 5 generally won't give silent data loss, unless there's a controller/driver bug. The problem is that when/if it happens (RAID 5 w/o plenty of cache, and preferably battery backups for everything), it may very well have borked enough data to make file or file system recovery impossible, leaving you with more downtime than you bargained for, even with all healthy hardware. If downtime needed to rebuild the array, and copy data back over is no big deal, and you have regular backups, don't sweat it (IE, home user with a good backup plan).

A raid 5 setup is not that slow.

That's not true (nor is it strictly false). It varies quite a bit, and it's typically difficult to guarantee good performance without a decent bit of knowledge of the controller, drives, and workload. Out of curiosity, are you getting this 140MB/s from the Dataoptic SPM393?

 

[Emulex]

the cpu has ecc? the cpu to bus has ecc? the ram has ecc? otherwise a silent bitflip and boom corruption begins.

 

[nk215]

CrytalDiskMark 3.0.1x64 gives me this

Seq 187.5 / 236.5
512K 22.49 / 52.42
4K 0.204 / 0.473
4KQD32 0.318 / 0.501

This is on an array that's 37% full (about 3TD out of 8TB). I have mostly DVD/BluRay ISO on this array so the 4K slow read/write doesn't bother me.

Raid5 is not the absolute best option out there but it's significantly better than nothing. I do have another setup with Xeon/ECC ram to backup critical data (and to the amazon S3 also).

 

[dac7nco]

I have never, ever had a problem with Hitachi drives. Even their consumer drives (deskstar) do splendidly in RAID. I have a 24-drive Areca/Hitachi RAID-60 array at an office that hasn't wavered in 19 months.

Daimon

 

[cbrunny]

Hmm, interesting. Plenty to read and to think about.

I'm not nuts about using JBOD. Just lost my entire storage array from one drive going down a few weeks ago using that. That's the appeal for RAID 5 for me at this point - I can lose one and won't lose it all. Performance issues and rebuild times aren't that important for what I'll be using it for, though it is good to know that I'll need a spare drive ready to go in case of failure. I hadn't considered that.

cb

 

[dac7nco]

I think you're having a misunderstanding re: what JBOD is. JBOD is single discs (f:, g:, etcetera). JBOD isn't striped at all; losing a single disc would mean losing only the data on that single disc. JBOD with incremental backups would actually be a lot safer for you than RAID, as it doesn't require any attention, and all discs aren't spinning simultaneously. If you're on vacation and your RAID-5 loses a disc, the odds of another disc falling out are pretty scary. The only reason for you to use RAID is if you need the performance (implying a hardware controller/$$$) of if you need a single huge storage pool.

Daimon

 

[cbrunny]

Sorry, should have checked my statement. I meant JBOD as in a JBOD of spanned disks rather than independently addressed.

 

[hammer256]

I like how RAID disks are more expensive, while the "I" in RAID stands for inexpensive.


By "like", I mean really annoyed.

So far my two Hitachi drives in RAID 1 works pretty well with the LSI 1068E controller.

 

[greenhawk]

I like how RAID disks are more expensive, while the "I" in RAID stands for inexpensive.

it is one of the reasons the meaning of RAID is being slightly changed depending on who you talk to.

The "I" is described as "Independant" now instead of "Inexpensive".

 

[Emulex]

no the concept is its cheaper to buy a bunch of RE4's or SAS drives and raid them than to build a monster (think 70's-80's core storage) "san-like" system that costs millions. or core memory.