General NAS overview

[Michael]

I wrote a top level overview for my blog on why owning an NAS makes sense. This forum helped me with a few questions in NAS in the past and I thought I would share a ling to the blog.

I personally have an Asustor AS5004T which does quite well for me.

http://mgpotter.com/what-is-network-attached-storage-and-why-should-you-use-it/

Michael

 

[SeanFL]

Michael, a nice write up. If you're open to improving it, I think the "Ks" in a sentence was meant to be "is"

"Let me assure you, what RAID to use Ks almost a religious decision."


Have been a synology user for the past 10+ years, and their DS214 is currently serving me well. Good to see you remind people to further backup the NAS. I've encouraged people to not only backup the NAS, but keep it offsite in cold storage so a cryptowall type infection won't propagate through their backup system.

Sean

 

[frowertr]

From your blog:

My general advice is this. RAID 5 or no RAID. I personally use RAID 5 and sacrifice the space for a little pice of mind. No matter what you choose, back up the files on your NAS. You need at least 3 disks to run RAID 5, so if you go with a 2 disk model for your NAS you need to make another choice. Let me assure you, what RAID to use Ks almost a religious decision. There are quite a few people that strongly dislike RAID 5 and they would disagree with me advice here.

Well we will have to agree to disagree to either use RAID 5 or no RAID at all. RAID 5 stopped being relevant over 5 years ago if we are talking about mechanical disks.

And I'm confused over the statement:

The most commonly used RAID is RAId 5 and it basically allows a 4 disk array to keep working even if one disk goes bad at then penalty of losing 25% of your storage.

You drop one disk in RAID 5, you still have access to all your data. Not 75% of it. But you are running unprotected (degraded). Lose another and youre toast.

 

[Anteaus]

It should really be RAID1, RAID6, RAID10, or no RAID. Your odds of losing a second drive during rebuild is higher than what your odds were of losing the first drive. The ever growing size of hard drives simply adds to your chances of failure. For home use, many people are shifting from RAID to drive pooling with parity such as FlexRAID, mhddfs/SnapRAID, or Unraid.

RAID or no RAID, every user should have two complete backups. If that isn't financially feasible, then they need to seriously question what they are getting into. RAID is about up time and for the most part only adds to the cost of the system. Once upon a time, RAID5 was one of the more affordable ways of pooling drives. That is no longer the case. In my opinion, RAID5 is a liability. If you want RAID parity, RAID6 is the minimum I would go with.

 

[frowertr]

It should really be RAID1, RAID6, RAID10, or no RAID. Your odds of losing a second drive during rebuild is higher than what your odds were of losing the first drive. The ever growing size of hard drives simply adds to your chances of failure. For home use, many people are shifting from RAID to drive pooling with parity such as FlexRAID, mhddfs/SnapRAID, or Unraid.

RAID or no RAID, every user should have two complete backups. If that isn't financially feasible, then they need to seriously question what they are getting into. RAID is about up time and for the most part only adds to the cost of the system. Once upon a time, RAID5 was one of the more affordable ways of pooling drives. That is no longer the case. In my opinion, RAID5 is a liability. If you want RAID parity, RAID6 is the minimum I would go with.

+1. Good advice.

We will split hairs over RAID 6 vs 10 so I'd rearrange your order. I'm ALWAYS on the RAID 10 bandwagon with any mechanical array at 4 drives or more. But that's just my preference and feeling.

 

[Anteaus]

+1. Good advice.

We will split hairs over RAID 6 vs 10 so I'd rearrange your order. I'm ALWAYS on the RAID 10 bandwagon with any mechanical array at 4 drives or more. But that's just my preference and feeling.

I'm in complete agreement. The order was a stylistic choice. The problem with these types of conversations is that RAID 1+0, 1, or 6 scare people because of cost of acquisition so they talk themselves into using RAID5. I know plenty of people who have had good success with RAID5 over the past 10 years and so they push it, but math doesn't lie. You can create a RAID5 array today and have it be fine for a while, but if you use very large hard drives...say over 3-4TB, your chances of a successful rebuild should it be necessary drop sharply.

If money is no object and you want RAID, RAID 10 is where its at. Just be ready to buy a tall stack of drives and a place to install them. $$$

 

[sao123]

From your blog:



Well we will have to agree to disagree to either use RAID 5 or no RAID at all. RAID 5 stopped being relevant over 5 years ago if we are talking about mechanical disks.

And I'm confused over the statement:


You drop one disk in RAID 5, you still have access to all your data. Not 75% of it. But you are running unprotected (degraded). Lose another and youre toast.

1)it basically allows a 4 disk array to keep working even if one disk goes bad. 2)At the penalty of losing 25% of your storage.

Re-read his sentence... he is saying that you get redundancy at the cost of 25% of your total storage... 3 fail safe data drives for the cost of 1 parity drive.

 

[frowertr]

Yeah, the grammar is so broken there that i couldnt tell what he was saying.

 

[Michael]

Thanks for the comments. I fixed the K that should have been an i (I had fixed that before I thought). I will look at the grammar of the explanation of raid 5 a little closer, I meant to say you keep going even if one disk fails.

There is no perfect raid choice, like I said, it is almost a religious discussion. I am not sure that giving up so much storage is worth it for some of the different RAID choices, especially for a NAS at ho,e that is not mission critical.

I said in my post that many people will reject raid 5 out of hand. . There are pretty good reasons for not liking it and liking other choices. I don't disagree with the reasons why other raids are better, but I don't think that raid 5 is a bad choice.

I did make it really clear that you need back-up.

Thanks again for the good comments.

Michael

 

[sao123]

Everyone writes good articles about using NAS, and what RAID you should use etc.
And then everyone says and MAKE SURE YOU BACK EVERYTHING UP...

Yet I have not seen anyone write an article giving a good methodology for backing up a 20TB NAS applicance...

 

[Michael]

Everyone writes good articles about using NAS, and what RAID you should use etc.
And then everyone says and MAKE SURE YOU BACK EVERYTHING UP...

Yet I have not seen anyone write an article giving a good methodology for backing up a 20TB NAS applicance...

I think you have three choices

1) external USB or eSATA drives. If you have a lot of media, you probably only need one or no back-up depending on if the original is easily accessible. That can cut down the size needed. Then you just do incremental back-ups. Free software is available

2) an external service, usually using something like Amazon cloud in the background. Can be expensive and if a lot of the files change could really strain your home connection. Free software is available and the major NAS companies offer software to link to most of the main services.

3) another NAS that mirrors the production NAS. Obviously as far away from the original NAS as practical. Most NAS come with software that automates this.

Michael

 

[frowertr]

Everyone writes good articles about using NAS, and what RAID you should use etc.
And then everyone says and MAKE SURE YOU BACK EVERYTHING UP...

Yet I have not seen anyone write an article giving a good methodology for backing up a 20TB NAS applicance...

Everyone writes good articles about using NAS, and what RAID you should use etc.
And then everyone says and MAKE SURE YOU BACK EVERYTHING UP...

Yet I have not seen anyone write an article giving a good methodology for backing up a 20TB NAS applicance...

For local backup, you would need to backup to an additional NAS or storage server. 20TB isn't that large. So once the initial backup takes place then only subsequent changes are backed up after that. With 10TB drives nowadays (but 6TB being more common) you can stuff 12 bay servers with tons of local storage.

If the data never changes and you just need it for archival purposes than tape is still a popular option. It's just slow as molasses.

 

[beginner99]

+1. Good advice.

We will split hairs over RAID 6 vs 10 so I'd rearrange your order. I'm ALWAYS on the RAID 10 bandwagon with any mechanical array at 4 drives or more. But that's just my preference and feeling.

I'm thinking about getting a NAS but still in conflict with how much I want to spend on it and on drives and if I should use RAID at all. So for a 4-bay NAS and RAID I guess RAID10 makes the most sense as I get the same amount of space at a trade-off for speed and reliability.

Since this is for home-use I also though about just buying a 2-bay NAS and use JBOD. With 6 TB hdds that gives me the same amount of storage than wit a 4-bay NAS and RAID6/10 at +/- half the price or at the same price but with a proper back-up (USB drives) and if I keep them say at work it's even an off-site back-up.

What do you lose with JBOD? Only the drive that failed?

I don't need redundancy. We are talking about BluRay-rips here. With RAID10/6 I would probably not make a backup. Starts to get very costly. Maybe I'm naive because I have not been burned yet but chances of 2 drives failing (RADI10) within days seems tiny? Yeah I now, RAID ain't a backup but the value of these rips isn't huge. In case of 2 or more hdd failures I can remake them or in case of fire and such events I have insurance anyway and could rebuy them. Important docs of course I have on different places incl. cloud.

What would be your general advice for such a case?

 

[frowertr]

With JBOD you won't necessarily lose everything on the remaining disks but you may not be able to perform the recovery on the NAS either. You may have to transfer the disks to a computer to fix the issues. No easy answers on recovery with JBOD.

If you configured LVM during setup with a single volume group on JBOD you would probably lose everything if a single drive failed in that group.

If you don't care about redundancy I'd probably just pick JBOD here. If you want redundancy than RAID 10 on a 4 drive array makes the most sense.

 

[Michael]

The fact that RAID 6/10 takes storage down 50% is why I am not fond of it and prefer RAID 5. This is for a home set-up, not for some mission critical business file server.

Michael

 

[frowertr]

The fact that RAID 6/10 takes storage down 50% is why I am not fond of it and prefer RAID 5. This is for a home set-up, not for some mission critical business file server.

Michael

Only with 4 disks will RAID 6 cost you 50% capacity.

Sure, if you dont care about redundancy than it doesnt matter what you choose.

I run a mdadm managed 6TB RAID 1 array for my home NAS. Real simple and safe. That gets backed up via rsync to an external 6TB WD drive. Pretty easy setup.

 

[Anteaus]

I'm thinking about getting a NAS but still in conflict with how much I want to spend on it and on drives and if I should use RAID at all. So for a 4-bay NAS and RAID I guess RAID10 makes the most sense as I get the same amount of space at a trade-off for speed and reliability.

Since this is for home-use I also though about just buying a 2-bay NAS and use JBOD. With 6 TB hdds that gives me the same amount of storage than wit a 4-bay NAS and RAID6/10 at +/- half the price or at the same price but with a proper back-up (USB drives) and if I keep them say at work it's even an off-site back-up.

What do you lose with JBOD? Only the drive that failed?

I don't need redundancy. We are talking about BluRay-rips here. With RAID10/6 I would probably not make a backup. Starts to get very costly. Maybe I'm naive because I have not been burned yet but chances of 2 drives failing (RADI10) within days seems tiny? Yeah I now, RAID ain't a backup but the value of these rips isn't huge. In case of 2 or more hdd failures I can remake them or in case of fire and such events I have insurance anyway and could rebuy them. Important docs of course I have on different places incl. cloud.

What would be your general advice for such a case?

If you don't need redundancy and you maintain proper backups I would just use RAID 0 instead of JBOD. You will lose the entire array on drive failure, but recovery will simply mean replacing the drive and copying from backup. Many people immediately throw RAID 0 out as an option because it has zero ability to survive a drive failure, but that isn't necessarily bad if you are prepared. A major upside is great performance and the least strain on your drives as far as arrays go.

If money is tight, it is far better to give up redundancy and use the extra money on backups. The bottom line is that you will deal with drive failure at some point. That failed drive might be in the array or it could be one of the backup drives. This is why 2 backups are always recommended. My point is that Raid 6 and 10 are fine and dandy, but they require a not so minor investment in money and capacity.

If this is media that you can recreate, then one backup may suffice. I would still go with two because backups fail, but I don't blame you for not wanting to burn cash on capacity that will sit on a shelf.

 

[Michael]

The average small NAS is 2 or 4 bays. So the storage penalty is big for RAID 6. RAID 5 is not the "best" but it gives you some redundancy for only 1 drive out of 4.

That is why I recommended it in my blog post. Or no RAID at all if you can easily replace what is on the drives.

Michael

 

[frowertr]

Hopefully you wouldn't buy your NAS before deciding what level of redundancy/capacity you need thus locking you into a specific level before you even get off the ground.

 

[VirtualLarry]

And then everyone says and MAKE SURE YOU BACK EVERYTHING UP...

Yet I have not seen anyone write an article giving a good methodology for backing up a 20TB NAS applicance...

That's why you have redundant NAS units... LOL. (No, really, I have several.)

 

[Anteaus]

Everyone writes good articles about using NAS, and what RAID you should use etc.
And then everyone says and MAKE SURE YOU BACK EVERYTHING UP...

Yet I have not seen anyone write an article giving a good methodology for backing up a 20TB NAS applicance...

It's because there really isn't a good methodology outside of the enterprise community. The most efficient way to backup a single 20TB data pool is to another 20TB data pool, which is very costly. Many home users can barely justify the cost to build a single array let alone duplicate it.

The not so sexy answer is that you'll need to span multiple external drives, and it will take an effort to keep it all organized and updated. Media files are easier because they rarely change. If you are like myself and have a small group of files (<1TB) that changes regularly, dedicate a smaller external drive to syncing it daily and keep it attached to the machine.

The solution should match the data type. If you are a professional photographer or videographer for example, you may need a more robust (and more expensive) solution. If you are just running a media server, just be slow and methodical about it.

The process is the same whether you are backing up a single 1TB HDD or a 20TB NAS. The saying is that if your data isn't in three different places, it doesn't exist.

 

[Essence_of_War]

It's because there really isn't a good methodology outside of the enterprise community. The most efficient way to backup a single 20TB data pool is to another 20TB data pool, which is very costly. Many home users can barely justify the cost to build a single array let alone duplicate it.

I'd suggest that if a home user can't afford to back up their 20 TB array, then unless the 20 TB array is itself backups, the home user cannot actually afford to build a 20 TB array.

 

[grimpr]

Personally i trust only raid10 especially on zfs with ecc ram for serious stuff and stablebit drivepool for home users with plain old ntfs on gpt disks formatted at 64k cluster size.

 

[grimpr]

Whats the news on ReFS on Windows 10? is it stable enough? there were horror stories in the Windows 8 days.

 

[Anteaus]

Whats the news on ReFS on Windows 10? is it stable enough? there were horror stories in the Windows 8 days.

If you are referring to Storage Spaces with parity (quasi Raid5), it supposed to be solid if a bit slow.