Setting up a 3-5TB RAID Fileserver

[imported_vor]

I've been ripping my DVD collection to my PC, so that I have an easy movies-on-demand interface for my home theater setup. So far I have it on multiple 500gb drives, however these are begining to fill up and also there is no redundancy in the event of a disk failure. Backups have been limited to the occassional full copy to expensive external 500gb drives.

I'm looking to set up a 3 to 5 terabyte RAID 5 fileserver. I've been looking at the Areca RAID cards as they seem to be the most fully featured. I'm going to initially use 500GB hard drives until the 1TB drives become cost-effective.

I need some suggestions on what to use for brand of hard drives, and also for what type of case to use. The case has to be large enough to handle 10+ hard drives, plus be easy to work with so it is not too difficult to swap out drives. I've seen some nice full height Lian Li models on Newegg - does anyone have experience with these?

What type of Power supply is required to power so many hard drives?

For cooling, what provisions have to be made so that these hard drives operate at acceptable temperatures?

Once the array is set up and operating, what is the recommended procedure for backing up the array? Tape drives? This would be incase of a virus/raid card failure/electrical problem that damaged the array.

I'm looking to spend around 2k to 3k.

 

[RebateMonger]

There are really three alternatives for backups:

1) Tapes - I doubt you'll find tapes economically feasible. A suitable tape drive for this much storage will likely run as much as your entire server budget and tapes could easily be $100 apiece.

2) SATA drives in housings - This is what I use on most of my client's Servers and on my own Servers. I normally put a single housing into the Server's case and buy removable trays which have large SATA drives mounted. Trays are $30 with built-in fans, fan alarm, and temperature alarm. You'll need a SATA HotSwap-compliant controller card, which costs $10 to $50.

3) Online backup - While it'll take near-forever for the initial backup, you might find online backups useful. They require no investment and are simple to implement. The monthly costs may be prohibitive for this much storage.

 

[SuperNaruto]

Just go with 1tb.. you'll end up spending time rebuilding the array 6 month down when you buy new drives, or use the 750s

Also make sure the areca you get supports over 2tb volume, i think older one might need firmware and make sure your OS supports 2tb volume or multiple volumes adding to 3-5 tb..

I have the full height lian li and had all 12 drives filled on a 350w psu with dual xeon 2.2 before, so you do not need to go and get a 1000w, a high efficient 400w would do.

You can also get the 3.5 drive cages for 5.25 drive and add more 3.5 drives..
http://www.cwol.com/serial-ata/images/satabay4-250.gif

I have tape drive sdlt320, the drive it self (2 years ago) was 1500, almost your budget, and it cost too much to back to 160/320gb tapes or 200/400 ; 300/400 lto..

 

[erwos]

Originally posted by: vor
I'm looking to set up a 3 to 5 terabyte RAID 5 fileserver. I've been looking at the Areca RAID cards as they seem to be the most fully featured. I'm going to initially use 500GB hard drives until the 1TB drives become cost-effective.

Can you really do that with RAID 5? Seems like you'd need to start a new array once you started using 1TB drives, since RAID 5 is limited by the smallest disk in the array.

 

[imported_vor]

Originally posted by: erwos

Originally posted by: vor
I'm looking to set up a 3 to 5 terabyte RAID 5 fileserver. I've been looking at the Areca RAID cards as they seem to be the most fully featured. I'm going to initially use 500GB hard drives until the 1TB drives become cost-effective.

Can you really do that with RAID 5? Seems like you'd need to start a new array once you started using 1TB drives, since RAID 5 is limited by the smallest disk in the array.

One way to do it, and I'm sure there might be a better way, is to swap the 500gb drives for the 1tb drives one at a time.

 

[Matthias99]

Originally posted by: vor

Originally posted by: erwos

Originally posted by: vor
I'm looking to set up a 3 to 5 terabyte RAID 5 fileserver. I've been looking at the Areca RAID cards as they seem to be the most fully featured. I'm going to initially use 500GB hard drives until the 1TB drives become cost-effective.

Can you really do that with RAID 5? Seems like you'd need to start a new array once you started using 1TB drives, since RAID 5 is limited by the smallest disk in the array.

One way to do it, and I'm sure there might be a better way, is to swap the 500gb drives for the 1tb drives one at a time.

Depends on the controller. Higher-end ones will let you expand the array online by swapping out one drive at a time -- although you will need to either create new partitions in the new space, or use some method of expanding your existing partitions if you want them to be bigger.

 

[erwos]

I think my point was more along the lines of "you'll wind up with a bunch of spare 500gb drives" more than anything else. I guess you could sell 'em and use the proceeds for bigger drives, too.

 

[imported_vor]

Originally posted by: SuperNaruto
Just go with 1tb.. you'll end up spending time rebuilding the array 6 month down when you buy new drives, or use the 750s

Also make sure the areca you get supports over 2tb volume, i think older one might need firmware and make sure your OS supports 2tb volume or multiple volumes adding to 3-5 tb..

I have the full height lian li and had all 12 drives filled on a 350w psu with dual xeon 2.2 before, so you do not need to go and get a 1000w, a high efficient 400w would do.

You can also get the 3.5 drive cages for 5.25 drive and add more 3.5 drives..
http://www.cwol.com/serial-ata/images/satabay4-250.gif

I have tape drive sdlt320, the drive it self (2 years ago) was 1500, almost your budget, and it cost too much to back to 160/320gb tapes or 200/400 ; 300/400 lto..

Does WinXP 32-bit w/ SP2 support volumes larger than 2TB, or do I have to split it into multiple volumes?

 

[RaiderJ]

Having an 8-port RAID 5 card that will allow you to swap one drive out at a time to upgrade in size will save you hours in work down the road if you go that route.

 

[Atheus]

Have you thought of... erm... COMPRESSING YOUR VIDEOS?!?!

Honestly man, as much fun as it sounds to build this machine, you surely don't need it. How many DVDs do you have? More than 500? You can fit brilliant quality at 480p into 1GB with H.264 or similar codecs so you would only need one of those 500GB drives.

 

[MerlinRML]

Originally posted by: vor
Does WinXP 32-bit w/ SP2 support volumes larger than 2TB, or do I have to split it into multiple volumes?

Nope. Anything over 2TB will be wasted space. You'd have to create multiple arrays, each one being smaller than 2TB. This will cost you parity space for each array.

Or switch to an OS that supports disks that large - XP 64-bit, Windows 2k3 (32-bit requires SP1), and I think Vista. Then there's the pick of the week in the Linux world.

 

[TallBill]

Originally posted by: Atheus
Have you thought of... erm... COMPRESSING YOUR VIDEOS?!?!

Honestly man, as much fun as it sounds to build this machine, you surely don't need it. How many DVDs do you have? More than 500? You can fit brilliant quality at 480p into 1GB with H.264 or similar codecs so you would only need one of those 500GB drives.

He'd still have to buy 2. Its pointless to do all that work and not have a backup.

 

[Madwand1]

I think that consumer-affordable backup solutions at this volume level don't really exist, so in the meanwhile the best we can do is drive-based solutions. When you exceed the capacity of a drive or two, it might make sense to build another server altogether for backups.

So I suggest building 2 inexpensive servers, one possibly cheaper than the other to host the backup. Optionally stage the purchase to get into two different streams of HD purchases, delaying the backup capacity until there is a need for it with that much data accumulation.

3 TB = 6 * 500 GB.
5 TB = 10 * 500 GB.

So 7-11 500 GB drives would be needed in RAID 5 for each.

Prices are still dropping and features are improving, etc., so I wouldn't really recommend getting 22 of these to start. I'd suggest going more conservative to start, and having a good expansion plan, so that your old primary drives become backup drives, or vice versa. E.g. Drop the backup, and re-define the array with new drives. Re-populate it from primary. Expand the primary with old drives from backup. Sell / re-deploy any extras.

Getting 2 Areca 16-port controllers would blow your budget, so I wouldn't recommend doing that. Get one 8-port perhaps; consider 3ware or LSI, etc., for price advantage perhaps. Or consider dropping that idea altogether and going with *nix-based OS RAID. You could get a dumber multi-port adapter for this or motherboards with a large number of usable on-board SATA.

Edit: Updated somewhat for 3-5 TB instead of 2-3 TB.
Edit2: If this is a real backup of DVDs, then consider dropping the idea of backing up the backups, and only backup original data.

 

[beatle]

I don't really see the point of backing up if you're going to run RAID-5. It's just extra cost. As others have said, it also complicates your upgrade plans.

I have a backup server that has as much storage as my primary box which is backed up. I run spanned volumes on each box and they are mirrored via ViceVersa software that runs nightly. Spanned volumes don't offer any redundancy, as you will lose the whole volume if you lose one drive in the span. BUT you won't care about this, as you'll have a copy of everything on your backup server. Spanned volumes are also very easy to expand, as you can add a drive of any size and just add it to the span. This is how I back up my data, and it works very well. FWIW, I'm running 6 drives and two CPUs on a very old 300w PSU without the slightest hiccup. Case selection, storage, and a couple of good gigabit NICs would be things to spend money on.

 

[Matthias99]

I don't really see the point of backing up if you're going to run RAID-5. It's just extra cost. As others have said, it also complicates your upgrade plans.

Redundant RAID modes don't replace backups. All they do is provide redundancy in case one drive fails.

RAID1/5/6 doesn't help if you:

Get a virus that corrupts your data.
Have bad/unstable hardware that corrupts your data.
Commit user error (accidentally delete something you later realize you need, format the wrong partition while installing an OS, etc.)
Have a catastrophic hardware failure (e.g. PSU or motherboard blows up and takes out all the hard drives, or at least a few of them).
Experience a true 'disaster' situation (fire, flood, theft, lightning strike, etc.) that wrecks the hardware.
Suffer two drive failures either before you realize that one failed or before you can replace/rebuild it.

 

[imported_vor]

Originally posted by: beatle
I don't really see the point of backing up if you're going to run RAID-5. It's just extra cost. As others have said, it also complicates your upgrade plans.

I have a backup server that has as much storage as my primary box which is backed up. I run spanned volumes on each box and they are mirrored via ViceVersa software that runs nightly. Spanned volumes don't offer any redundancy, as you will lose the whole volume if you lose one drive in the span. BUT you won't care about this, as you'll have a copy of everything on your backup server. Spanned volumes are also very easy to expand, as you can add a drive of any size and just add it to the span. This is how I back up my data, and it works very well. FWIW, I'm running 6 drives and two CPUs on a very old 300w PSU without the slightest hiccup. Case selection, storage, and a couple of good gigabit NICs would be things to spend money on.

Spanned / JBOD configurations would make me nervous

 

[Evadman]

Originally posted by: Matthias99
Depends on the controller. Higher-end ones will let you expand the array online by swapping out one drive at a time -- although you will need to either create new partitions in the new space, or use some method of expanding your existing partitions if you want them to be bigger.

I had a Highpoint 2220x that supported that at $250. I had 2 2220x cards and 14 250 GB drives, and it supported spanning arrays across cards.

As for the origianl quesiton of what drives to use, everyone is going to say something different as everyone has had a drive fail and hate that manufacturer forever. But I will toss my 2 cents in. My array had drives from Maxtor, Seagate, Samsung and WD (all 250's though but a mix of SATAI and II) They all worked fine. But I prefer Seagate server (ES) drives as they are marketed for servers and have some of the server hardware features (internally) that you would need in a 24x7 enviroment. Most desktop drives are made for 10 hours on 10 hours off (AFAIK, don't quote me) not 100% on until failure.

 

[RebateMonger]

Originally posted by: beatle
I don't really see the point of backing up if you're going to run RAID-5. It's just extra cost. As others have said, it also complicates your upgrade plans.

You probably wouldn't say that if you saw as many failed RAID 5 arrays as I do. Recovery from a dead RAID 5 array with no backups is a MAJOR expense and involves major downtime.

 

[TSCrv]

lots of valid points seen above, i have taken on building these arrays for clients and theres a few ways to do this...

a) some1 suggested a lian li case that would work, build a reliable box and either do hardware or software raid5... i wouldnt load boot stuff on the array, but i would mirror 2 80s for boot...

b) get a norco 12bay rackmount drive bay enclosure with the accompanying card, flash the cards bios, (cheap method but norco is a cheap company) and fill that puppy up with 500gbs, normally i dont suggest cheap stuff but since i have set a few up i can say it does work.

c) umm? backup solution. yea, to be cost effective you would almost need a completely separate array, tapes backups are fine and dandy, but you would need a tape library, not a tape drive. (unless you like to change tapes every hour)... a 1tb backup using veritas 10d takes roughly 18 hours.... (the size might be off, but the time is right).... if you do make 2 arrays, make sure they are on completely different systems, and i wouldn't run the backup server 24/7, maybe turn it on once a week and run the backup, but some may argue that.

 

[TSCrv]

oh, and for myself, i will be setting up a 3tb array in sept when the seagate 1tb drives come out, 4x 1tb raid5... at the current moment i can get the Hitachi ones for 355 (impressive thermals but i still wanna wait for some seagate benchies)

 

[Venomous]

Originally posted by: TSCrv


b) get a norco 12bay rackmount drive bay enclosure with the accompanying card, flash the cards bios, (cheap method but norco is a cheap company) and fill that puppy up with 500gbs, normally i dont suggest cheap stuff but since i have set a few up i can say it does work..

Thats fine and dandy if you plan to use strictly IDE and not SATA. He will need this bad boy...


http://www.newegg.com/Product/Product.aspx?Item=N82E16816133001

Now, i was in the same boat. I needed a lot of storage and it had to be external because the case can only hold so many drives. I was eyeballing the Gigabyte MBs with the 10 SATA ports, a new case and drives.. Between the board and the case, i was already at the price of this enclosure... So, it made since to go that route. Im in the process of building mine right now. Waiting on 4 of the 1TB drives to ship... I have 8 500gbs already.

 

[TSCrv]

Originally posted by: Venomous


Thats fine and dandy if you plan to use strictly IDE and not SATA. He will need this bad boy...

eh?? its a sata enclosure im talking about, the one you linked that comes with the HBA as well...