Question Need to set up a solid backup system

[kellyoh]

Hi guys. I stumbled across this website via a Google search in desperate search to understand and formulate a backup system for my work drives.

If anyone could help me in layman terms, that would be much appreciated. I am not much of a computer geek, however I do understand the necessity to learn how to establish a proper working set up and have backups. I work in marketing, in a company surrounded by people very much not in marketing and very much not computer geeks either. So I kinda need to strike out on my own and learn the lines on this and set myself up so I don't experience the "failure without backup" horror story I've heard from so many others.

It seems to me that people who worked here before me seem to know a lot more about this stuff, but now I have a set of what seems to be 2 external drives, and an "IcyDock" that has 4 bays for hard drives, and 3 in use. They seem to be very inconsistent in their setup, and I suspect it was set up by someone who pretended to know a lot, but didn't.

I do photography and designing, so I have an external Seagate drive for photo storage, and external LaCie Quadra drive for design documents and "links", including a Shutterstock Library.

In the IcyDock, I have 3 hard drives (I think they are called SATA drives): 1 is an old backup drive from my old computer -- don't need this. Another states "Untitled" and is supposed to be a backup drive, but is definitely not from my understanding, and the third doesn't seem to be readable except on occasion, and it's old backups from my previous iMac.

What I want to accomplish is:

1.) Maintain separate storage units for types of media (Photos / Design Storage / Current Projects)

2.) Consolidate the storage unit into a single body, so my desk isn't cluttered

3.) Have all data in both drives being backed up

4.) Have all data in my iMac Pro being regularly backed up

Could someone help me with a step-by-step instruction guide in layman terms? Lol.

Thank you.

 

[VirtualLarry]

IMHO, if you want to do this RIGHT, look into an Asustor NAS unit with 8-10 hot-swap bays, and some 10-14TB HDDs. Probably cost you around $2000-3000.

They've got a "MyArchive" feature, that will let you clone the data on the "fixed" drives on the NAS, to a drive in a hot-swap bay, that can then be removed and archived / stored off-site / in a safe-deposit box, etc.

You're dealing with a lot of data, so I suggest the 8-bay or 10-bay Asustor NAS units. (I have a 6104T 4-bay myself, and I have yet to use the MyArchive feature for archiving backups. I currently have 2x 10TB WD Red drives in mine, in a RAID-1 mirror configuration.)

Asustor 10-bay NAS unit $800:
https://www.newegg.com/Product/Product.aspx?Item=N82E16822225044&Description=Asustor 8 bay&cm_re=Asustor_8_bay-_-22-225-044-_-Product

WD Red 10TB 5400RPM HDD $280:
https://www.newegg.com/Product/Product.aspx?Item=N82E16822231546&Description=WD Red 10TB&cm_re=WD_Red_10TB-_-22-231-546-_-Product

You'll need at least 6 of these to start with. (4x for RAID-5 for "fixed" storage", 2x for "backup" / MyArchive drives.)

If you've got your whole Workgroup LAN accessing the NAS (it can do that), then consider this model for $1700, it has an i5 CPU in it instead:
https://www.newegg.com/Product/Product.aspx?Item=376-0040-00001

Edit: And if this at first blush seems "expensive", then consider how expensive it would be for your design house if you lost your assets, or downtime at least from have a system down.

If you need ultimate uptime, consider a PAIR of these NAS units, real-time mirroring one another. (I believe that they can do that.)

 

[Insert_Nickname]

First off, this is a very deep rabbit hole to go down. It's also a question of just -how- paranoid you want to be. In my experience, more paranoid = better.

Your data does not exist unless you have three copies of it. This is the first principle. Some general advice would be to keep in-flight data (what you're working on currently) on your work system, then have an external (x minutes, what works for you) backup, and a daily offline backup.

A fourth tier "secure storage" can also be a good idea. Avoid overwriting known good backups with unknown copies. You don't want to suddenly find out your whole backup tree has been corrupted by mistake.

So summing up, basic setup could be:

1st tier: This is what you working on right now. Depending on uptime requirements a RAID1 array might be useful.
2nd tier: Some sort of NAS (NetworkAttachedStorage) or DAS (DirectAttachedStorage) for incremental backups.
3rd tier: NAS or DAS backup. Can be another NAS, or an external drive connected to the NAS. If you're using DAS, this could be a daily updated external drive.
4th tier: Secure Storage. What this means is reliant on what your conditions are.

It's important to consider how easy it is to get data back out of your backup, but that is reliant on your specific setup. The Golden Rule is always test your backup system before you commit any important data to it, and this familierises you with how your setup actually works. It's not fun to find out the hard way you've made an error somewhere.

 

[VirtualLarry]

It's important to consider how easy it is to get data back out of your backup, but that is reliant on your specific setup. The Golden Rule is always test your backup system before you commit any important data to it, and this familierises you with how your setup actually works. It's not fun to find out the hard way you've made an error somewhere.

Yeah. My personal rig, uses Macrium Reflect, to do system image backup every day, to a NAS unit. I have the backups directory encrypted, and one time, I had a power glitch or something, or updated the NAS firmware, and the backup directory wasn't "unlocked", so my backups were failing for a week or two, and it wasn't clearly notifying me at all. Not too happy with Macrium about that.

 

[kellyoh]

Alrighty, so two hours and watery, red eyes later, I think I now have an inkling of understanding of the suggestions so far. After having my nose in the web dictionary and explanatory forums trying to come to grips with all the terms and the terms within the terms...

I agree that this is a deep rabbit hole, which is why it took me about 2-4 years of neglect before I witnessed a close friend lose an immense amount of data. By realizing "that could be me" at any point, I finally pulled myself up by my bootstraps and rolled up my sleeves to confront the endless sea of information on computers in the interwebz to understand and formulate a working system for myself. I am also, judging by my actions, assuming I have a safely high level of paranoia.

As a point, I am definitely not worried about how much I will be spending, as long as luxury does not exceed practicality. I want to invest in the best setup, regardless of price, however not in such a way so as to be extravagant. However, I don't want to find myself investing in some NASA space technology, when something at a much affordable rate works just as well.

Also there is no need for LAN accessibility, as I am the only one that will be having access to this.

So from what I understand, here's what I'm being suggested so far:

0.) Set up a RAID1 array for my current work space, which the purpose seems to be to speed up my workflow and simultaneously back up my in-flight data? Is this something I can set up with what I have?

1.) Purchase the Asustor 10-bay NAS unit (Do I need internet for this? If I do, that complicates things...)

2.) Purchase a minimum of 6 drives (4 for RAID5 setup / 2 for backup)

And then based on the second answer above, I should...

3.) Use the "MyArchive" feature which, I would use those two drives out of the six I buy?

That's the extent of my understanding so far. Which brings up my further questions:

1.) Can I set up a RAID1 configuration with my IcyDock? This just means I need to have 1 empty hard drive to pair or whatever with my MacHD? Does the HDD need to be the same size?

2.) What do I do with all the data on all my separate drives, once I do get the Asustor setup?

3.) Holy shit on pairing NAS units...that seems to be the ultimate of ultimate backups...

Whew, I'm feeling a little safer and like I am taking a step in the right direction. Thank you for being so helpful thus far.

 

[VirtualLarry]

There's actually several ways to go about this, if you buy the 10-bay Asustor unit. I haven't actually used the MyArchive functionality yet. I plan to when I get another couple of 10TB HDDs. The advantage of that feature is that you can supposedly treat one or several of the hot-swap bays in the unit as a removable drive, and simply eject the caddy and store the drive for safekeeping. As opposed to having to purchase an external desktop HDD and then plug in the AC adapter and the USB3.0 plug, and pray that it doesn't disconnect off of the USB bus when you least want it to.

You may also wish to keep your important categories of files, each on their own RAID-1 mirror array, rather than all of them as shares on a single massive RAID-5 or RAID-6 array. That might require a few more drives in the end, but it might be more secure in terms of drive rebuilds and potential UREs.

I don't think that you NEED internet to set up the Asustor unit, but it helps ease the set-up process. (Edit: I think that when I set mine up, it offered to auto-update the firmware when I booted it and configured it for the first time, while it was connected to internet. For security, you may wish for this to be unconnected to the 'net. It does sport some "Cloud" features.)

 

[VirtualLarry]

1.) Can I set up a RAID1 configuration with my IcyDock? This just means I need to have 1 empty hard drive to pair or whatever with my MacHD? Does the HDD need to be the same size?

2.) What do I do with all the data on all my separate drives, once I do get the Asustor setup?

I'm not sure what an IcyDock is, other than some form of SATA HDD dock? You can run SATA drives in RAID-1, yes, they should be the same capacity.

For 2, you would connect and set up the NAS via LAN, then plug in your HDD via dock to the host system, access the filesystem on the host, and copy your files to the NAS share via the LAN.

 

[Insert_Nickname]

So from what I understand, here's what I'm being suggested so far:

I'll try to answer at least. Welcome to the data archivist club.

0.) Set up a RAID1 array for my current work space, which the purpose seems to be to speed up my workflow and simultaneously back up my in-flight data? Is this something I can set up with what I have?

RAID1 only protects against drive failure. It's no substitute for backup.

1.) Purchase the Asustor 10-bay NAS unit (Do I need internet for this? If I do, that complicates things...)

2.) Purchase a minimum of 6 drives (4 for RAID5 setup / 2 for backup)

And then based on the second answer above, I should...

3.) Use the "MyArchive" feature which, I would use those two drives out of the six I buy?

RAID5 shouldn't be used since it's very vulnerable when (not if) you're rebuilding an array. Because it can only tolerate a single drive failure. RAID6 is better since it can tolerate two drives failing, but it's not widely supported.

An alternative would be RAID10. It's a combination of RAID1 and RAID0, essentially a RAID0 array of RAID1 drives.

If you want to go completely overboard, there is RAID-Z with ZFS. But that requires a home-brew setup, and is not for the casual.

I can't say I'm familiar with Asustor NAS features, though I'm hearing good things about their products. Someone more knowledgeable will have to step in there.

That's the extent of my understanding so far. Which brings up my further questions:

1.) Can I set up a RAID1 configuration with my IcyDock? This just means I need to have 1 empty hard drive to pair or whatever with my MacHD? Does the HDD need to be the same size?

2.) What do I do with all the data on all my separate drives, once I do get the Asustor setup?

3.) Holy shit on pairing NAS units...that seems to be the ultimate of ultimate backups...

1) Only if the device in question supports RAID1. Some external dual drive cases do, but that's down to the specific model in question. You can do RAID1 with different size drives, but you'll only get the lowest capacity for the array.

2) If you plan on using a NAS, be warned all drives you put in it will be reformatted and thus blanked. The NAS sets up arrays it's own way, so it has to start from scratch. I'd get new drives too, since you obviously have a lot of data on your in-use drive. Just as a precaution too, so you don't loose anything when setting up the new system.

3) Yup. But NAS units have their own vulnerabilities. It's not as safe as it may sound, but that'd really require a longer explanation.

You may also wish to keep your important categories of files, each on their own RAID-1 mirror array, rather than all of them as shares on a single massive RAID-5 or RAID-6 array. That might require a few more drives in the end, but it might be more secure in terms of drive rebuilds and potential UREs.

I agree. RAID5 should be treated as unreliable with multi terabyte drives. RAID6 is slightly better due to it's two drive fault tolerance, but it's not something I'd use privately.

KISS principle applies when you're working privately. Simplicity has it's own merits.[/QUOTE]