Need a very reliable 5TB backup solution

[imported_wired247]

I did not have much luck getting responses in General Hardware.

A friend of mine works for an inventor, who requires about 5TB of backup storage space and has plenty of money to throw at the problem, in the ballpark of $3000 USD but can go above this value, much much above this value, if necessary. To give you an idea, the inventor is a multi-millionaire and can afford whatever you think is necessary.

This is not a competition to build the cheapest solution that gets the job done. It should be as reliable as possible since this will be backing up very important designs for his inventions and basically his entire life. Feel free to help spend all $3000 or even more if it's necessary.


1) NAS or File Server? If NAS is used, then what OS and software must accompany the NAS unit to enable automatic backups?

There are several computers that need to be backed up. Would this require an additional computer be installed to control the NAS or should a backup software simply be installed on all of the computers that need protection?

2) If a File Server is used, then what OS and backup software should be used? Windows Server 2003? Windows XP? Acronis?

3) Are there good NAS units with RAID-6 that you could point me towards? Or should he get 2 NAS units to do software RAID 5 + 1 (mirrored raid 5) ?

4) Off-site backup? Can someone point me in the direction of a good company to deal with?



Your *specific* answers are appreciated! This is not a B.S. request, I'm really looking for some help here.




Edit: Right now I'm thinking about something like this to tell him about, and then they will probably get some off-site backup as well.

http://www.newegg.com/Product/...x?Item=N82E16822102022

 

[Fullmetal Chocobo]

Just felt like adding my $.02, for what it's worth.

You may find this thread worth reading. The OP is asking for help with an enterprise backup solution.

If your friend is looking for an explicit backup configuration, then tape would be hard to beat it seems. If he is looking for data redundancy / storage, then a file server might be more applicable.

Windows Home Server would be a good candidate as an OS, so long as it is understand that files can't be edited directly with certain applications until that bug is fixed (they are currently advising June as a rough estimate for release). Backups are not affected by the bug. Also, WHS is currently 32-bit only and thus has a <4gb memory cap and is limited in volume size to 2tb (per volume).

I am building a home server consisting of the following... If you want detailed specs, let me know. And just FYI, I'm not a professional at any of this--I do everything in my spare time and for fun--so there are others here that are far more qualified on stuff like this than I am.

File Server
Supermicro MBD-X7DCL-I motherboard
Kingston 2gb DDR2-667 (2)
Intel Xeon E5405 2.0GHz 12mb (2)
Windows Home Server
Ultra X3 1kw PSU
Western Digital 500gb RE2 hard drive (24)
Areca ARC-1230 12-port RAID controller (2)
Areca ARC-6120 battery module (2)
Supermicro SATA 5-in-3 enclosure (5)

When the 64-bit version of WHS comes out, I will be adding 4 more 2gb sticks for a total of 12gb RAM (1.5gb per core).

Notes on config
[*]The array will be configured with 3 4-disk RAID 5 arrays. Each volume will be 1.5 terabytes. WHS will setup data across the 3 volumes providing another layer of security. More disks can be added, but I do not believe that disk will be able to be added to the arrays if the disk are in the WHS pool (I can't migrate a 1.5tb disk to a 2.0tb disk).
[*]The Xeon E5405 is the slowest 45nm Xeon with 12mb L2 cache. Faster ones could be used, although that isn't really necessary for a file server.
[*]I will be getting one 12-port controller now, and another later when I need more room; at that point I will likely add another PSU to handle the extra load. Too much power in a file server is far better than too little.

 

[imported_wired247]

If at all possible, they do not want to do tapes. (I think they'd rather rely on the in-house backup, and do some offsite backups once in a while)

Fullmetal as usual you're prolific with your information. Thank you!

What would be the best alternative to WHS if they wanted to avoid any possible issues and do not want to wait for a patch? This inventor guy has very nice contacts for getting microsoft software FWIW.

I think we're honing in on a solution here. My friend is contacting a professional about this solution but I am insisting on double-checking the specs before he implements anything.

 

[Fullmetal Chocobo]

If you didn't want to use Windows Home Server, you could do a Windows Server 2003 config with a backup software solution like Veritas. I would use Server 2003 simply for stability reasons.

BTW, here is information on the WHS corruption issue. It only affects machiens with more than one hard drive and when programs are used to edit files directly. Evidently backups aren't affected.

Also, it wouldn't be a bad idea to get something like a Drobo or other external device for making secondary backups of essential data and taking it offsite.

 

[Syzygies]

If "extinction of data" and not speed is your concern, then multiple solutions in multiple locations is your only viable option. For one of your options, I would build a Linux software raid box. If at all possible, locate it somewhere else, but bandwidth to reach it then becomes an issue.

A crucial reference is Linux Administration Handbook (2nd Ed), one of the best technical books I've ever seen. They give seasoned advice to an audience that will potentially get canned from their jobs if things go south. Forums should play only a supporting role, here: Would you treat cancer depending solely on forum advice? On the other hand, would you treat cancer without consulting forums?

A synopsis, here: Motherboard RAID is a bogus front end for proprietary software RAID. Hardware solutions all have the same failing: If the hardware controller fails, one needs to replace it identically in kind, down to the exact firmware that may no longer exist, to recover the data. Software RAID is reasonably fast (not your concern) and Linux open source software RAID is the most flexible package out there, hands down.

Buy seven terabyte drives for $1400 to build into a $2000 total Linux box, and arrange them into a 5 TB RAID 6 array, so you can recover the data if any two drives fail. Grab five of the drives as the house burns down, install them into a new Linux box, and you've got your data back.

This should be one of several solutions, so $3K isn't going to do it. Never let one fox guard all of your hens. I'd build two of these in separate locations, and pay for off-site backup.

I like Ubuntu 8.04 Server, for ease of installing cross-platform support, e.g. SAMBA. But immediately install the graphical interface.

Since you didn't say what OS is sourcing the data, I'd have to assume the inventor is a white male in a larger U.S. city, running Windows. That would be the demographic that thinks the answer is obvious, not necessary to state. Can't help with the Windows end, but the Linux Administration Handbook (2nd Ed) describes cross-platform support.

I wouldn't trust Windows with my life's work.

 

[Knavish]

I posted in your thread in General hardware. Sorry that my reply sounded very negative. I'll try to elaborate more here.

I think for a big project like this you have to spend some time in the planning stage to find the best solution rather than just getting a list of parts at Newegg. It sounds like you're doing this, which is good! Most of us here are computer guys/girls and enjoy playing with computer hardware. For this type of person it makes a lot of sense to build our own file server. We'll be playing with it / looking at it frequently enough to know if there's a problem (Like a RAID losing a drive and becoming non-redundant). If you're talking about a user who wants a server that "just works" that he never has to worry about, this is a different case. If he really wants enterprise level-redundant storage and full scale backups *AND* he wants it to "just work", he needs some kind of network administrator to keep track of the system over the entire lifetime of the system. If this is the case, you should probably look at contracting some local network administration firm to handle it for him.

---

A NAS probably makes sense for someone who doesn't want to administer his own server, but I don't think it will be as reliable as a server in the long run. You'd still need some kind of backup plan with the NAS. Offsite backup would be the easiest for the user, but I believe it's very expensive. I'm no expert in offsite backup, but according to a couple google searches, it can cost about $1+ per GB per month. This means the user will be spending $1000 PER MONTH to backup 1 TB of files. Did I make a mistake here???!

FYI for almost the same price as that NAS box from newegg, you could get a Dell Poweredge 2900 III (with no OS), 4 GB Ram, dual quad core Xeons, RAID card, and 3 year warranty for $1700 and add 6x 1TB drives for ~$1200.

 

[Knavish]

Originally posted by: Syzygies
I wouldn't trust Windows with my life's work.

:thumbsup:

 

[RebateMonger]

You never did say what you meant by a "backup solution".

Does this mean that the "primary" data is on other servers or PCs, and that the data we are talking about is used for backup purposes only?

Also, assuming this is really "backup", and not the main data store, how much data has to be transferred to the backup system on a daily or weekly basis? Are there five Terabytes of changes each day? Or five Megabytes of changes that need to be copied to the backup system?

 

[imported_wired247]

I've been discussing this with my friend and I'm fairly certain they're going to go with an NAS solution for the time being.

This forum is way paranoid when it comes to data security, and when I tried to pass along your paranoia to my friend, it wasn't too welcome (heh). The NAS is going to be used for periodic backups of up to 3-5TB, not each day. I highly doubt most of the members who recommend an offsite backup even use one, but it's still a solid recommendation of course. Also, my friend is not the IT guy for the inventor, he is a personal assistant. So it's his job to get this up and running, and Linux boxes and whatnot are just not going to happen: he has other things to do than just this.

The main reason for this is some of the data is backed up currently, some is not. More and more data is accumulating, and more and more is not backed up. What they need is a backup solution that can store everything. Since there is no single external hard drive that can do this amount of data, NAS seems to make sense. It is like a gigantic flash drive.

As long as the data gets backed up before Murphy's law knocks on the front door and wipes out an un-raided hard drive, that is sufficient for now, and they will look into offsite backup in the future.

The source hard drives is simply not going to fail at the exact same time as the backup (NAS) device. It's just not going to happen. If the house burns down that's another story.

This isn't really a discussion anymore, I'm just relaying back to y'all what the result of the discussion was.

 

[Syzygies]

Originally posted by: wired247
I highly doubt most of the members who recommend an offsite backup even use one,

I highly doubt various sexual practices have much of a following, but that would make me wrong!

One of my tasks tonight is to set up a Remote Power Switch, so I can turn my NY server on and off from CA, rather that simply leaving it on.

I remember the 60's. We used to say "You're not paranoid if you're right." Many of us have lost data in impossible-to-anticipate ways. Many of us have friends who have fared far worse, because their single backup plan was untested and failed.

We're not saying your buddy should learn to overclock. Just get him to buy two NAS boxes.

 

[MerlinRML]

Originally posted by: wired247
The source hard drives is simply not going to fail at the exact same time as the backup (NAS) device. It's just not going to happen. If the house burns down that's another story.

While I will very rarely criticize the solutions people choose, simply because I feel that they know more about their situation than I do, I do take complete opportunity to attack flawed thinking. Now with that being said, I will grant you that people here (myself included) tend to be a bit overprotective of their data.

Hard drives are guaranteed to fail over time. When, where, and how they fail is impossible to predict. You're saying an unlikely event is impossible. That's just plain wrong. I've gone through enough of those "unlikely" situations and learned how painful they are to deal with that it's just easier to do everything in your power to avoid them.

My favorite example to use is power outages. Most of the hardware failures I see on hardware that runs 24x7 happens after a power outage. During a power outage all of your hardware goes down (maybe gracefully, maybe not). Who's to say you won't get multiple drive failures during that time? What's to prevent your NAS (which most are just low-powered integrated computers anyway) from having a complete hardware failure that isn't disk related? What is your plan if that happens?

Now, I'm not going to rant/rave about your decision or what you choose to do as a result. I just want you to be aware of the flawed concept on which you are basing some of your decisions. Good luck with whatever you choose.

 

[imported_wired247]

and according to statistical thermodynamics, once in ~ a Googolplex years a box full of air molecules will all be preferentially found on one side of the box rather than evenly distributed as they will be found the other 99.9999999999% of the time.

I didn't mean it was impossible, but the likelihood is lower than the average anandtech user would make it out to be, and if it were to happen, it's not the end of the world. And hence, it is not worth worrying about that much. If you are thinking in terms of end of the world situations then some of this paranoia is deserved.

I think a singly-redundant NAS backup should suffice for the next few years at minimum, but like I said they will look into offsite backup of the NAS as well.