Help configuring storage for home server/HTPC

[Valantar]

I'm planning on upgrading my media server to a combined server+HTPC (it's already hosting all my media, but I'm having some playback issues (unsure whether they stem from the server or the laptop currently serving as a HTPC).

Currently the server is running an old Intel Atom D510 ITX board, with 2 2TB WD Greens for storage and a 750GB Samsung as the boot drive. I had a Seagate 2TB in there as well, but it started having issues and was removed. It's connected by GigE to my home network, and running W7 Pro.

My planned upgrade is to an AMD A8-7600 in 45W mode (or its Godavari replacement, I want the ability to run some light games on it too), with the Asrock FM2A88X-ITX+, 8GB of DDR3 and a 128GB SSD (MX100/BX100) as a boot drive.

My WD Greens have racked up 15000/13500 hours of Power On Time according to their SMART data, so they're used, but still going strong.

What I'm thinking storage wise is this:
-Put my 2TB Greens into a RAID 1, for "cold storage" (photos, backups and the like)
-Buy a 4TB WD RED for media storage, torrents and other non-important stuff. I have had some serious bad luck with HDDs over the years, so I want something reliable here, but with decent performance.
-And as I said, use a cheap SSD as a boot drive. I might transplant my old 80GB X25-m from my laptop and upgrade that instead, but I'm a bit skeptical of that when it comes to reliability, as this machine will be running more or less 24/7 (not in terms of wear, just that the 5+-year-old drive might up and die on me).

My main concern is the RAID - how reliable and stable are integrated RAID controllers these days (specifically the ones on AMD motherboards)? I used to boot from a 4-drive RAID 5 on my main desktop a few years back, which was an utter nightmare (30-minute rebuilds at least once a week, among other issues). I'm hoping things have improved, and that I'll see fewer issues since I won't be booting from the RAID.

Anu thoughts?

 

[XavierMace]

If you want proper RAID, buy a RAID controller.

 

[Valantar]

Not really within my budget, unfortunately. From what I've read, the cheap ones are usually quite bad (or at least no better than what's already built in), and the good ones cost about as much as my whole upgrade.

 

[Cerb]

If you want proper RAID, buy a RAID controller.

Or, use an OS that supports RAID well (like 8.1 Pro).

If you want integrated RAID, you do not want to go with AMD, plain and simple. For that, use Intel.

My main concern is the RAID - how reliable and stable are integrated RAID controllers these days (specifically the ones on AMD motherboards)? I used to boot from a 4-drive RAID 5 on my main desktop a few years back, which was an utter nightmare (30-minute rebuilds at least once a week, among other issues). I'm hoping things have improved, and that I'll see fewer issues since I won't be booting from the RAID.

That's not AMD. That's RAID 5, outside of a well-supported, well-maintained, server environment (where nobody sane uses it for important data anymore, either). Stick to mirrors and stripes, or use ReFS (Storage Spaces), which will at least do all that crap with a live mount (but is not bootable). Even so, AMD's is a pain to manage, compared to Intel's.

 

[Essence_of_War]

My main concern is the RAID - how reliable and stable are integrated RAID controllers these days (specifically the ones on AMD motherboards)? I used to boot from a 4-drive RAID 5 on my main desktop a few years back, which was an utter nightmare (30-minute rebuilds at least once a week, among other issues). I'm hoping things have improved, and that I'll see fewer issues since I won't be booting from the RAID.

OP, as far as RAID goes, what OS are you going to be using? If you're not booting from the drives that you're RAID'ing, this should be very straightforward and modern OS's have excellent software RAID tools (win 8.1 storage spaces - mirror space ; linux mdraid mirror).

 

[Valantar]

I'll be upgrading the W7 Pro to W10 as soon as it's available, and my guess is that the hardware upgrade will happen around the same time.

My previous raid was an Intel build, btw. Still, it was a nightmare, and completely sold me on SSDs 😛

But to sum things up, I should ditch any kind of hardware raid and use Storage Spaces? I don't mind that at all, unless it's a resource hog. Also, if a mirrored drive in a storage space fails, are the data still accessible?

 

[Essence_of_War]

But to sum things up, I should ditch any kind of hardware raid and use Storage Spaces? I don't mind that at all, unless it's a resource hog. Also, if a mirrored drive in a storage space fails, are the data still accessible?

I wouldn't describe intel's chipset RAID as hardware RAID, it's more of...firmware RAID. Software RAID should be portable if your motherboard dies, and it should be able to be recoverable from ANY motherboard, not just one with an identical chipset. On an older atom CPU, a parity-type RAID might be a bit of a resource hog, but a mirrored raid should have very low resource overhead. On an A8, even a parity RAID shouldn't require many resources.

If a single drive in a mirrored storage space fails, your data should still be accessible, but windows should report the storage space as degraded.

 

[Cerb]

But to sum things up, I should ditch any kind of hardware raid and use Storage Spaces? I don't mind that at all, unless it's a resource hog.

Not necessarily, but...

If you want proper RAID, buy a RAID controller.

Not really within my budget, unfortunately.

Intel nor AMD RAID is truly hardware RAID. Just like the cheap add-on cards, it creates a bootable device for the pre-OS environment, and then when the OS takes over from the firmware, it's using a software RAID driver. A hardware RAID controller, which today basically means an LSI one, is its own entire computer on a card, that hides all the inner working from the OS, so the OS just sees block devices.

Software or hardware RAID doesn't really matter. The implementation of each does. Intel's RAID is fundamentally the same technology as a $30 Silicon Image or Marvell RAID card, just implemented better.

Storage Spaces is good, just a bit complicated to plan out, sometimes, because MS made the boneheaded decision to not make ReFS capable of being an NTFS replacement, and still limits OS editions. So, there's bootable Windows RAID with NTFS, with varied support for features in the different OSes, and then Storage Spaces, with ReFS, and varied support for features in the different OSes. On top of that, you have your mobo's RAID features...

With a mirror or parity volume, you can get your data from Storage Spaces with a failed drive. The main advantage of Storage Spaces/ReFS is that a CHKDSK usually doesn't need an unmounted drive. It can handle that sort of thing online, unless the FS gets totally borked. They've been making that happen on NTFS, too, but ReFS was built for it, sort of like ZFS, so you have a much better chance of an FS anomaly being fixed actually preserving your data (and if not, still not having to deal with ambiguity over what data may be good or not), and very little chance of bad shut downs causing problems that can't be easily recovered from (including array inconsistencies).

 

[fleshconsumed]

If you cannot buy hardware RAID card (which I think is actually a complete overkill for home), then I would advise you to look into StableBit DrivePool. I use it and I think it's great.

It's a software solution that allows you to "pool" your hard drives (kind of like Storage Spaces) so that they appear as one drive but it does it without involving RAID striping, so if one of your hard drives goes down, you do lose data on that drive, but the rest of the data on other hard drives is going to be accessible just fine. That is a downside of using DrivePool, however DrivePool does allow you to duplicate selected folder, so you can use that to safeguard your most crucial data in the event of a drive failure.

I'm not sure if I'd want to go with Storage Spaces. It was a year or two since I've last evaluated that option, but last time I looked into it, it just did not seem like a good choice.
http://arstechnica.com/information-...es-explained-a-great-feature-when-it-works/1/

 

[BonzaiDuck]

If you cannot buy hardware RAID card (which I think is actually a complete overkill for home), then I would advise you to look into StableBit DrivePool. I use it and I think it's great.

It's a software solution that allows you to "pool" your hard drives (kind of like Storage Spaces) so that they appear as one drive but it does it without involving RAID striping, so if one of your hard drives goes down, you do lose data on that drive, but the rest of the data on other hard drives is going to be accessible just fine. That is a downside of using DrivePool, however DrivePool does allow you to duplicate selected folder, so you can use that to safeguard your most crucial data in the event of a drive failure.

I'm not sure if I'd want to go with Storage Spaces. It was a year or two since I've last evaluated that option, but last time I looked into it, it just did not seem like a good choice.
http://arstechnica.com/information-...es-explained-a-great-feature-when-it-works/1/

I've got to give a :thumbsup: and +1 to that.

I had a 4-HDD RAID5 in a workstation for years, and an old Pentium-III SE440BX system with an IDE RAID controller for a server. I'm ashamed of what I spent on the controller cards -- given my actual NEED.

I really LIKE the idea now of running all the controllers in AHCI mode, and the Stablebit (CoveCube) DrivePool software really makes things manageable. You can duplicate data at the folder and file level without creating a whole-drive RAID1 or similar solution. If a drive goes bad, you can still read data directly at the file level from the remaining drives, or the pool will rebuild itself for complete recovery of duplicated data -- with only partial loss of the unduplicated files.

My server mobo is an old 680i with an nForce controller (non-compliant with AHCI) -- which I don't want to use for anything. So I picked up two of these:

http://www.newegg.com/Product/Produ...arTech_Sata_Controller-_-15-158-365-_-Product

They even have port-multiplier capability, so one controller could -- in principle -- cable 7 HDDs. But that would mean extra hardware. There are a lot of these low-end controllers available with Marvell chips.

I did the up-front research on the controller, and if you want to run drives in AHCI mode with the native MSAHCI driver, you don't even bother with the controller BIOS -- otherwise capable of RAID0, 1 and 10. You leave them alone so they appear as "Unconfigured" at boot time!! They're also capable of "JBOD" if you configure them, but that's not what you want for using StableBit.

 

[Valantar]

Pooling storage is not my goal, data loss prevention is. 2TB is already far more than I need for photos and documents, so I'll probably be using this volume for backup images of two laptops + possibly my desktop as well. That should still leave me with plenty of space for years to come with 2TB.

So what I want is a simple, stable, safe and non-resource intensive way of mirroring the same data on two identical drives. It does not need to be bootable (I have at one point tried running an OS from a WD Green - never again).

Storage spaces still seem like the best option.

Cerb: which features of Storage Spaces differ between OS versions?

Fleshconsumed and BonzaiDuck: is it possible to implement mirroring at drive level in this software? I'd much rather have the whole drive mirrored than just some folders, both for ease of management and safety. I'm not interested in micromanaging this (unlike how I normally configure my computers). I just don't see what I have to gain from it.

 

[smitbret]

Pooling storage is not my goal, data loss prevention is. 2TB is already far more than I need for photos and documents, so I'll probably be using this volume for backup images of two laptops + possibly my desktop as well. That should still leave me with plenty of space for years to come with 2TB.

So what I want is a simple, stable, safe and non-resource intensive way of mirroring the same data on two identical drives. It does not need to be bootable (I have at one point tried running an OS from a WD Green - never again).

Storage spaces still seem like the best option.

Cerb: which features of Storage Spaces differ between OS versions?

Fleshconsumed and BonzaiDuck: is it possible to implement mirroring at drive level in this software? I'd much rather have the whole drive mirrored than just some folders, both for ease of management and safety. I'm not interested in micromanaging this (unlike how I normally configure my computers). I just don't see what I have to gain from it.

Your answer is SnapRAID.

http://snapraid.sourceforge.net/

It is a software RAID that runs under Windows. Depending on the size of your array, you can run multiple parity drives. I suggest 1 for the first 3-4 data drives and then add a 2nd when you add the fifth.

It is perfect for home media storage where there isn't a lot of frequently changing data. I have been using the paid cousin, FlexRAID for more than two years and I have been very pleased.

unRAID is another option that is very popular and very well supported. www.lime-technology.com

 

[BonzaiDuck]

Pooling storage is not my goal, data loss prevention is. 2TB is already far more than I need for photos and documents, so I'll probably be using this volume for backup images of two laptops + possibly my desktop as well. That should still leave me with plenty of space for years to come with 2TB.

So what I want is a simple, stable, safe and non-resource intensive way of mirroring the same data on two identical drives. It does not need to be bootable (I have at one point tried running an OS from a WD Green - never again).

Storage spaces still seem like the best option.

Cerb: which features of Storage Spaces differ between OS versions?

Fleshconsumed and BonzaiDuck: is it possible to implement mirroring at drive level in this software? I'd much rather have the whole drive mirrored than just some folders, both for ease of management and safety. I'm not interested in micromanaging this (unlike how I normally configure my computers). I just don't see what I have to gain from it.

I think you can do that, with this caveat: If using an OS like WHS-2011, the OS creates a primary folder for sharing named "ServerFolders." You would simply duplicate the whole enchilada. But you also have the capability to duplicate in "triplicate" or greater to make loss impossible if two or more drives just up and die at the same time.

Once they'd started marketing DrivePool for WHS, with the writing on the wall that WHS would be orphaned, CoveCube doubled-down to make the software compatible for Win7, Win 8, Server 2012 and so on.

I can see how you'd want a RAID1 with 2x 2TB drives, but then you'd be committed to a duplication of unused space. You're also doubling wattage draw, without increasing storage-size potential. Maybe there are counter-arguments to these I've made "off the cuff." Can't say at the moment.

There's another benefit that was always "there:" You can add drives of any size to the pool, whereas -- with RAID -- you want drives of identical size and at least preferably same manufacture. Stablebit will balance the distribution of files across a mix of different sizes. It provides some five or so balancing priorities and methods which can simply be re-arranged on the fly.

Of course, you need two or more drives. If you only have -- or want to use -- a single drive, you can't even have RAID1. But if you want redundancy, it's more flexible than RAID1 in ways I've mentioned.

Further -- suppose you pool two drives whatever the respective sizes. And suppose you acquire a third disk -- of any size -- hoping to add it to the pool. With RAID, (correct me) I think you have to rebuild the array within the controller BIOS. If so -- your server is "down" for some period of time. If not, then I'm wrong -- my mistake if you can rebuild a RAID array with the existing drives still "useable" in a Windows session.

If you add the third drive with stablebit, it will continue providing access while re-distributing the data in a background process.

But these choices all depend on what you want to do NOW -- TOGETHER with what you MIGHT want to do later as a matter of expansion.

 

[fleshconsumed]

Your answer is SnapRAID.

http://snapraid.sourceforge.net/

It is a software RAID that runs under Windows. Depending on the size of your array, you can run multiple parity drives. I suggest 1 for the first 3-4 data drives and then add a 2nd when you add the fifth.

It is perfect for home media storage where there isn't a lot of frequently changing data. I have been using the paid cousin, FlexRAID for more than two years and I have been very pleased.

unRAID is another option that is very popular and very well supported. www.lime-technology.com

Heh, I actually use SnapRAID in combination with StableBit DrivePool. DrivePool pools my drives into a single drive, and SnapRAID protects the data.

Unfortunately SnapRAID has its drawbacks, a) it's a command line tool so there is a learning curve that's required to set up configuration file and to learn all the commands, b) it does not offer real time parity because it's a snapshot based raid, so theoretically you can lose any data you have written since the last sync, and c) SnapRAID requires that the parity drive to be at least as big as the biggest drive in the "array", in which case the OP's 4TB WD red would have to be the parity drive leaving him with only 2x2TB Greens for usable space.

As great as SnapRAID is, I'm not sure it's the best solution for the OP.

 

[fleshconsumed]

Pooling storage is not my goal, data loss prevention is. 2TB is already far more than I need for photos and documents, so I'll probably be using this volume for backup images of two laptops + possibly my desktop as well. That should still leave me with plenty of space for years to come with 2TB.

So what I want is a simple, stable, safe and non-resource intensive way of mirroring the same data on two identical drives. It does not need to be bootable (I have at one point tried running an OS from a WD Green - never again).

Storage spaces still seem like the best option.

Cerb: which features of Storage Spaces differ between OS versions?

Fleshconsumed and BonzaiDuck: is it possible to implement mirroring at drive level in this software? I'd much rather have the whole drive mirrored than just some folders, both for ease of management and safety. I'm not interested in micromanaging this (unlike how I normally configure my computers). I just don't see what I have to gain from it.

I'd still caution you about using Storage Spaces
https://helgeklein.com/blog/2012/03/windows-8-storage-spaces-bugs-and-design-flaws/
http://www.smallnetbuilder.com/nas/...es-recovering-a-really-big-storage-space-pool

It just does not seem like a product I would trust my data to.

Now, if you only have two drives you want to mirror you might be fine with Storage Space, but I'd still be cautious.

IMO pooling your drives into one with DrivePool with selected folder duplication would be your best, easiest and safest option, that's what I would still recommend. I don't think it supports entire disk duplication, only folder and file level, but it shouldn't be that hard to set up a few selected folders such as "photos" and "documents" to duplicate.

 

[BonzaiDuck]

IMO pooling your drives into one with DrivePool with selected folder duplication would be your best, easiest and safest option, that's what I would still recommend. I don't think it supports entire disk duplication, only folder and file level, but it shouldn't be that hard to set up a few selected folders such as "photos" and "documents" to duplicate.

"[doesn't] support entire disk duplication" -- if it did, it's advantages over RAID solutions would vanish. You could SAY that it will duplicate all DATA on one drive to another. The pool exists whether data is duplicated or not. The pool is a "virtual disk" combining all the physical disks. The physical disks are still "visible" and "readable" as such from the OS tools available to the user. The redundancy with any advantage occurs at the folder level.

I'm only trying to support the points you make in your statement. I don't think there's any performance-hit with the pool, and StableBit tech support seems to think there's a mild boost in speed as with RAID1 when reading duplicated data.

The only problem with it arises with a need for "imaging". You can't image the virtual drive; you must instead image the component drives -- each one. But if the pool is dedicated only to data, you can image the boot drive in automatic scheduled backups, and you can use a tool like RichCopy (Robo with GUI) with the virtual-drive/pool which backs up -- incrementally as desired -- at the folder/file level, uncomplicated by the duplication. And I think RichCopy allows for scheduled backups.

You can also create more than one pool, and I think they can be mounted or dismounted as desired. I just have no need to explore those more elaborate possibilities -- not at this time.

 

[Valantar]

I'll have to look into DrivePool - it doesn't sound quite like what I'm looking for, but it might work. And of course software with recommendations from skilled users is always a plus 🙂

Fleshconsumed: I don't mind the part about SnapRaid being snapshot based - I'll not be writing data to these drives very frequently. It being command line based is kind of a bummer, though - guis are just easier to get into. But I'll figure that out as well.

The links about Storage Spaces are slightly disconcerting, but at the same time they're both from 2012, and one of them was for a usage scenario I'll never approach. At the very most, this PC can fit six drives, so it'll never get to those kinds of sizes 😛

BonzaiDuck: I'm not really concerned about duplication of unused space, as I'll not be putting anything on these drives that isn't worth duplicating. Anything else will go on the boot SSD or the media drive. Also, power consumption isn't really an issue either, as this will more or less be cold storage - photos will be copied intermittently, and backups of the PSU will be run perhaps monthly (more frequently won't be necessary as any documents and other stuff will be synced with either Dropbox or One Drive. But yeah, RAID is off the table. Just have to figure out what to do instead.

Edit: odd autocorrect error in the middle there, guess that's what comes from posting from your phone.

 

[Essence_of_War]

OP, you've gotten a lot of advice, some of which I would suggest are excessively complicated solutions to problems that you don't have.

If all you want to do is mirror a pair of drives for redundancy on windows 8.1, storage spaces is an easy and transparent way to do this, and is available out-of-the-box without additional software.

I would just do that.

 

[XavierMace]

While I am not discounting Storage Spaces (I went that route on my backup NAS), I will point out that you can get a GOOD used hardware RAID controller for $100 on eBay (IBM M1015).

 

[desitube]

I'd still caution you about using Storage Spaces
https://helgeklein.com/blog/2012/03/windows-8-storage-spaces-bugs-and-design-flaws/
http://www.smallnetbuilder.com/nas/...es-recovering-a-really-big-storage-space-pool

It just does not seem like a product I would trust my data to.

Now, if you only have two drives you want to mirror you might be fine with Storage Space, but I'd still be cautious.

IMO pooling your drives into one with DrivePool with selected folder duplication would be your best, easiest and safest option, that's what I would still recommend. I don't think it supports entire disk duplication, only folder and file level, but it shouldn't be that hard to set up a few selected folders such as "photos" and "documents" to duplicate.

All storage designs have their strengths and weakness. ReFS (Storage Spaces) is like any other and requires some thoughts into what you need and have to design correctly. I have had data lost on multi-million dollar tier 1 systems in my years as a storage admin. Rebuild times can be very long with new super high capacity drives (6TB+). Things that the above links talk about are true with many systems. Striping will always have the same issues regardless which system or technology you use.

At home, I have setup a server with ReFS and love it. The ability to add pools and grow storage is great along with redundancy. A few things to keep in mind when using ReFS:
1) Use smaller storage pools
2) Don't use Thin Provisioning unless you want to actively monitor and manage the system.
3) Don't allocate over 97%. This give you a margin of error.
4) If possible, have a spare drive available or add it to your system.

Do a bit of reading and don't be afraid of ReFS.

Hope that helps

 

[BonzaiDuck]

Actually, I had to (briefly) come up to speed about "Storage Spaces."

That feature is apparently part of Windows 8 and Windows 2012 R2 server.

And it sounds very much to be synonymous with drive pooling -- but without the extra $30 software. Of course, if you already have Win 2012 or Win 2012 Essentials, that would be fine. I assume that the feature is bundled with the "Essentials" release?

The reason I have my own little "hijack-ey" questions: sooner or later I'll have to replace WHS2011 with "Essentials." It helps to get your ducks lined up!

 

[Valantar]

OP, you've gotten a lot of advice, some of which I would suggest are excessively complicated solutions to problems that you don't have.

If all you want to do is mirror a pair of drives for redundancy on windows 8.1, storage spaces is an easy and transparent way to do this, and is available out-of-the-box without additional software.

I would just do that.

Yeah, I've pretty much concluded that that's what I'll do. No reason fiddling with "HW" raid, investing in an actual RAID controller would be ludicrous for this build, and buying (or even just adding) software do do work that Windows already does pretty well is usually not a great idea.