Need to store 5 tb+ of media, tell me why i shouldnt UNraid

[Midnight]

My head is spinning from the pros and cons of different options.

Ive spent the last few days reading tons of posts and doing research.

To begin, I have about 5 tb of data storage needs. Its mostly movies and I need to serve it over the network. Im going to be using either windows 7 MCE or XBMC but thats another post!

I have a flexible budget but want to keep it as cheap as possible. I have 3x 1.5tb disks, 1x 1tb, 1x .75 tb.

Id also like to employ redundancy into any solution. This has ruled out WHS because I dont like the cost of full disk duplication. I do like a lot of the other benefits of WHS but i find it too costly to backup.

That leaves me with two solutions. Either raid 5 or something like unraid.

First I thought I was going to go the freenas route but im turned off from the lack of array expansion. Its my understanding that with software raid you often need to destroy the raid to add capacity. The second thing that turns me off raid is the use of same sized disks.

Finally, I dont like the idea of striped data with all disks spun up to use the array and I dont like the idea of losing 2 disks and losing the whole array.

One option I see is hardware raid 5, that would at least allow me to expand the array with same sized disks.

For all the above reasons, I like the idea of unraid.

It seems like this is the best of both worlds. I can add drives of any capacity as long as I use one of the 1.5tbs as a parity. Drives arent spun up if the arent used, if i lose two disks i still can pull data off the other drives.

Is unraid a bad idea? Why wouldnt I chose this solution?

My budget is flexible but Im trying to keep it to necessary hardware.

Another choice I guess is the intel 4 disk raid 5 box.

http://www.newegg.com/product/produc...82E16859117003

its a cheap solution and seems easy.

if i need to expand i can just get another or something.

right now if i price out an unraid/freenas box i see it costing about $300

please help me out with suggestions!

thanks!

 

[JSt0rm]

jbodx2 in 2 locations if its of value

 

[DominionSeraph]

What did you do, pirate the entire Netflix library?

 

[pjkenned]

NetApp's Raid 4 implementation is 7 drive (1 parity, 6 data) for raid 4 even using NVRAM cache and their own proprietary WAFL file system. Currently they support up to 1TB drives. They have over $3.5 billion in annual revenue. Lime tech (unraid folks) I'm guessing have less than 0.1% of that.

From my 3 disk test box, performance was <50MB/s in raid 4, versus 102MB/s for windows w/ hardware raid 5 using the same 3 drives. With raid 4, performance doesn't increase with more disks.

With the drives you have, you really should be looking at WHS's duplication. You can literally throw whatever drive you want in the thing which is a great feature.

 

[Midnight]

i just feel like whs is a bad choice for full back up. it seems like a waste to be forced to do full duplication.

 

[Midnight]

pj hows your experience with free nas alternatives?

im still looking into freenas or unraid. i really dont want to do full duplication. id rather only lose one disk to parity.

 

[RebateMonger]

Well, I think that avoiding striping is a good idea.

I have zero experience with UnRAID. About all I've done is read the Wikipedia description. Maybe I'll create one in Hyper-V with some small virtual disks and take a look at how it works.

About all you can do is ask around and read UnRAID user forums and see what issues they have. I'm looking at the lime-technology user forum and there's certainly lots of activity there.

When you get into large data storage systems, there is no perfect solution. Well, there is, but it costs lots of money. 1 TB and larger disks have tiny heads and tiny data bits and today's disk error-correction systems have to fight to keep up.

As always, be sure to thoroughly read the manuals on the storage system and understand the steps for recovery in case of disk failure.

 

[VirtualLarry]

pj hows your experience with free nas alternatives?

im still looking into freenas or unraid. i really dont want to do full duplication. id rather only lose one disk to parity.

You keep saying "full duplication". WHS's data-duplication feature does not work on a full disk basis. Rather, it works on a folder basis. Only what you specify is duplicated in two locations.

 

[Midnight]

yes but i would duplicate the entire collection. so for all intents and purposes i would need twice the disks to use whs as a redundant backup.

 

[pjkenned]

So my basic experience has been that the free alternatives are good, but there's usually more troubleshooting/ looking how to do things/ fix little bugs on the internet. My three favorite "free" versions are FreeNAS, OpenFiler, and EON. None are as easy to setup/ maintain as WHS for an everyday Windows user.

BTW Rebatemonger: unRaid in Hyper-V is bad news. I've tried it: http://www.servethehome.com/8-nas-servers-virtualized/ It is stable, but the support for the Hyper-V SCSI/ LAN controllers is bad. TBH, it isn't worth the time.

 

[queequeg99]

I use unraid for the same application (serving movies). I was drawn to it for the same reasons highlighted in the original post (don't have to spin everything up at once, flexibility in adding drives, and some level of redundancy).

The only times the box has been down in the past two (and maybe more) years has been to update to a new version (super easy) and to add new disks.

Potential things to be aware of with unraid:

1. Writing to the array can be slow. You can either use a cache drive (which will copy things to the array in the middle of the night) or simply deal with it. I just deal with it (it isn't THAT slow).

2. When I got into unraid, the selection of supported hardware was pretty thin. Some of the most popular network controllers weren't supported and, in general, it was advised that new users buy the exact same components used in the pre-built boxes Lime Tech sold. This eventually became a problem back in the day because the most widely used motherboard was some asus model that had been discontinued and was sold by a single tiny shop down in California somewhere. I haven't done too much lurking around their forums (because I've had no problems!) so it would be very well worth your time to investigate your hardware options.

3. It will also be well worth you time to learn something about file management in unraid. For example, if you want to move large amounts of data from one disk to another, dragging and dropping the files from within Windows will take forever because unraid will run the entire copy process through your desktop. A far better method is to telnet into the unraid box and run a copy command there. There are a number of tricks like this. If you get unraid, you will learn them quickly (especially after you try drag and dropping a folder of ripped DVDs and get the message that the operation will take 86 hours to complete).

 

[Mark R]

I'm planning to build such a system. However, I got rather put off unraid for a couple of reasons.

1. It seems to use some sort of proprietary redundancy algorithm - this could be troublesome in the event of serious system failure (e.g. 2 drives). An HD in a RAID 5 array at work died, with a bearing failure. The vibration was so severe it actually crashed a 2nd hard drive - result: total data loss. There was a 24 hour old backup, but there were a lot of pissed people who had lost a whole days work.

2. You have limited flexibility of redundancy. See above. Not all data needs double redundancy, but I'd quite like it for some of mine.

3. It doesn't stripe data like RAID 5 and 6 do. Reading from a RAID 5 or 6 is a bit like reading from a RAID 0 - the performance is very impressive. Especially, if more than one PC/streamer is accessing it simultaneously. Despite it's puny CPU, my 4x5400 rpm drive NAS box, out performs a decent PC with 1.5 TB 7200 drive when streaming multiple files.

As I'm a bit of a geek, I've been testing a basic Linux system (Ubuntu) and actually it works very satisfactorily for file serving - and RAID-5/6. So my plan is to use a linux installation that can boot from a USB or CF card - and use a 6 drive RAID array. 1 set of paritions would be RAIDed as RAID 5 and another set as RAID 6 - giving me a choice of redundancy. RAID 6 for backups of work, photos, videos, etc. RAID 5 for storage of media, temporary files, etc.

 

[RebateMonger]

BTW Rebatemonger: unRaid in Hyper-V is bad news. I've tried it: http://www.servethehome.com/8-nas-servers-virtualized/ It is stable, but the support for the Hyper-V SCSI/ LAN controllers is bad. TBH, it isn't worth the time.

Thanks for the hint. Maybe I can toss a few IDE disks in an old case/motherboard and try it there. But my Seagate IDE drives are disappearing fast. Most are Seagate 7200.7 and more and more are developing bad sectors.

 

[pjkenned]

Yea :-/ unRaid's OS disk was made to work with a flash drive, but nothing else. So it isn't like FreeNAS or OpenFiler where they can find some productive use for the remainder of the install drive. On one hand unRaid "works", but it costs money and is basically inferior to FreeNAS/ OpenFiler/ EON even non-virtualized. Novelty of Raid 4 and the associated relatively poor performance aside, it is a really tough choice to go unRaid once you have used something like WHS (since they are similarly priced) or a higher-end open source solution.

My FreeNAS on Hyper-V installation Part 1 is going to get published tomorrow morning, Part 2 the following day, and followed by OpenFiler, EON, OpenSolaris, some flavor of Linux (Ubuntu most likely), then... maybe... unRaid. The one real positive is, of course, that the three data drive limitation of free unRaid actually goes hand-in-hand with the IDE 3 drive + IDE OS drive of Hyper-V since you can't boot a Hyper-V VM from USB ATM.

 

[Midnight]

ive read a few things about WHS and raid 5. some people say its an awful idea others have said its ok.

why cant i set up a hardware raid 5 array and run WHS on top of that?

another idea im toying with is hardware raid in freenas.

its important to me to be able to increase the size of the array without destroying the array.

i think i will still go for an unraid build. i have mixed size HDs and would prefer not to just lose them. if i went raid 5 id wouldnt use my 1tb and .75 tb.

ive been meaning to get an unraid box setup. and probably freenas. i have the hardware around.

i was going to setup a 3 disk and then remove a disk and rebuild it etc.

thanks for the feedback guys!

 

[pjkenned]

why cant i set up a hardware raid 5 array and run WHS on top of that?

No reason at all. My "old" WHS (Lasted 14 months before I needed to supersize as it had <20TB of storage) used an Adaptec 31605 with all 16 ports filled. I was running raid 6 and raid 1 with the drives and passing them through to WHS. Do that, turn off duplication for all of your folders (unless you really want something duplicated for some reason) and you are using Raid 6 redundancy with the awesome WHS features like deduplication, add-ins, non-add-in cool toys (it IS windows afterall so there's really just a ton of stuff you can have work).

The new WHS now uses HP SAS Expanders + Areca raid. The big difference of course is that I moved from 16 hotswap drives to 40.

That being said, if I wanted decent speed, decent redundancy, and I had a bunch of randomly sized disks, I would skip hardware raid, and just use WHS duplication. Data recovery is great, and one really nice thing is that when a drive fails, WHS will make sure the data gets spread to other disks in short order. If you think about it, a decent hardware raid solution from Adaptec/ Areca is going to be at least $400, not including another $100 for the BBU. With 2TB drives in the $130 range, you can buy redundancy for all of your disks, and get an extra .5TB usable storage.

As for HW raid in FreeNAS... if you aren't going to use Raid-Z (or Raid-Z2) there is little reason to use FreeNAS over another appliance. In fact, I would suggest going with a Linux based box so you have generally better driver support.

 

[Red Squirrel]

Setup a MD raid 5 or raid 6 using Linux. It's rock solid, easy to add more drives in the future, and not proprietary like WHS.

Just remember raid is not a backup, so if this data is important enough to backup, you will still need to do backups. I would recommend a drive dock where you just insert a drive into it. It's hot swap. Think of it as a tape drive, but 10x cheaper and media is bigger and only a bit more expensive.

It will also be very fast. I stream movies from my Linux box all the time over gigabit. It's pretty much like if it was local, I don't find any major speed difference between my mapped drives, and local disk. I'm sure it is a bit slower if I was to do benchmarks, but as far as every day use goes, no major noticable difference unless I actually sit there and time a transfer. Can definably just open a movie directly off a mapped drive and watch it with zero issues. I have about 10 VMs running on that same raid array too.

There's also freenas and open filer, but I have no experience with these. They sound interesting though.

 

[pjkenned]

Setup a MD raid 5 or raid 6 using Linux. It's rock solid, easy to add more drives in the future, and not proprietary like WHS.

WHS uses NTFS. So in a sense it is proprietary and you could potentially not be able to read data... if of course you could no longer use Windows XP, Vista, or Windows 7. Then again, there's probably only one machine that can read that "proprietary" NTFS volume per 3 people on earth.

And just as a FYI OpenFiler = Linux based (so it does do software raid except with a WebGUI). FreeNAS is FreeBSD based and primarily uses ZFS/ Raid-Z or Z2... unless you want triple redundancy and opt for Raid-Z3, again with a decent WebGUI.