Enterprise Backup to Cloud solution

[DHT]Osiris

Lifer
Dec 15, 2015
17,189
16,342
146
All,

Hoping someone in this thread has some experience with small -> medium sized enterprise backup stuff. My environment (relatively small, 80ish servers) is making some changes, among which is shifting to a better off-site backup solution. Presently we have a combination of vmware VDP for VM backups (unfortunately on-host, so in the same rack as the VM), Microsoft DPM, and large scale tape backup system that we basically rent access to.

The tape system is phenomenally slow and access is shared among a few dozen other systems so access is a concern. VDP has the aforementioned issue of being on-host so we can't easily offsite the data, and DPM (where I'm prodding at right now) has a very specific issue in that in order to truly offsite the storage (to Azure, conveniently included in the GUI for DPM) you have to back it up to disk first, which means you need XXTB of storage dedicated to something you're then trying to offsite, which I find phenomenally unhelpful. We're also fully on-prem so built-in cloudy mechanisms aren't helpful to me

So basically, does anyone have thoughts on higher-end backup solutions that meet the following criteria:
  • No stupid home-brew solutions (buy this PCIe card from Latvia and it makes your s3 instance look like a hard drive! Use robocopy plus scripting to do API hooks into your vmware instance!). I won't be the only one managing this and we have enough other garbage on our plate.
  • Something vaguely centrally-managed. Trying to not create a half dozen new tools to do a single job, or require components a, b, and c to do what one component should be able to do
  • Needs to not require every system to hit the internet. We have some internal systems without internet access and no true proxy deployed (limitation of our environment, not by choice) so it needs to hook into something that can hit the internet, see centrally-managed point.
  • Have some capability of incremental/block change based backups (excludes most tape options), and/or dedupe. We're probably looking at 15-30TB right now of full + delta changes, and that will be expanding probably double in the near-future. I'd rather not require hundreds of TB of stuff for a few weeks of snapshots.
  • Multi-OS environment would be superb, as we have a few *nix boxes. Also VMware integration for just yanking VM snapshots instead of doing OS level stuff would be neat, I like VDP for this.
The above criteria has excluded local backups -> cloud (Microsoft's Azure Backup System) because everything has to hit the internet, DPM because it cannot do disk -> cloud directly (requires local storage), and other solutions haven't been vetted fully due to not having trials for them/not having ample time to test, hence me asking here! Right now we're leaning towards buying a crapton of storage on a plain old server (like 50ish TB, priced around $10k? I think) and just doing DPM to disk and bypassing cloud, which would give us 'offsite-enough' if we just co-locate it somewhere.

Thoughts?
 
Last edited:
Feb 25, 2011
16,991
1,620
126
Well... it sounds like you need a SAN to centralize your VM host storage, and a second one offsite, and a replication tunnel. Ideally. That would let you do all your block snapshots, dedup, etc., and most of them will let you compress/dedup your replication traffic, so that should save pipe.

Managing on-host storage is a pain in the butt (unless you're abstracting it with a vSAN or something. But it's still a pain in the butt.)

But assuming dual SANs are too pricey (likely), it's seems to me that you're on the right track with a centralized server to host your DPM backups, if it were also hosted offsite. If Microsoft DPM works anything like Crashplan, it's doing compression and encryption on the client side, so bandwidth use is minimized too.
 

thecoolnessrune

Diamond Member
Jun 8, 2005
9,673
583
126
Who's your Storage Vendor? Do you have a good VAR who could do the solution hunting for you? You shouldn't have to test a ton of options, as that's exactly what the VAR is there to do.

Simpana's Commvault (the heavyweight), EMC's Avamar, and Veritas' NetBackup solutions will check all the requirements you've put forward here. They all have cloud ability with all the major offerings (Microsoft Azure, Amazon AWS, IBM Bluemix), they all let you install expensive, large proxies to act as go-betweens, they all allow combinations of baremetal, and virtual infrastructure with various snap-ins for your Storage Vendors. They all are centrally managed. They all work with even some of the more eclectic OS and Database solutions out there (AIX, DB2, Lotus snap-ins, etc.).

They all cost a fortune, and you'll still need a Backup admin for patch control, troubleshooting with failed backups, and issues with the backup system itself. Depending on cost constraints and personnel, there are also Managed Services providers who can leverage these systems and manage them for you. We currently do that for several large businesses.
 
Last edited:

[DHT]Osiris

Lifer
Dec 15, 2015
17,189
16,342
146
Well... it sounds like you need a SAN to centralize your VM host storage, and a second one offsite, and a replication tunnel. Ideally. That would let you do all your block snapshots, dedup, etc., and most of them will let you compress/dedup your replication traffic, so that should save pipe.

Managing on-host storage is a pain in the butt (unless you're abstracting it with a vSAN or something. But it's still a pain in the butt.)

But assuming dual SANs are too pricey (likely), it's seems to me that you're on the right track with a centralized server to host your DPM backups, if it were also hosted offsite. If Microsoft DPM works anything like Crashplan, it's doing compression and encryption on the client side, so bandwidth use is minimized too.

We already have a SAN for VM centralization, very recently set up (but running smoothly). Buying a second san for replication purposes doesn't really fit the bill since management looking for no-kidding long-term backups (ransomware/long-term malware has become a concern, especially with what happened over the weekend). Buy yeah to buy another of what we have would be outside of the financial envelope we're aiming for as well.

Not sure about Crashplan, but DPM (rather notoriously) does *not* do dedupe innately. I have to build a very obtuse and convoluted system of 'vhdk's on an NTFS volume with dedupe enabled' garbage, which is really frustrating. It can do some form of 'compression over the wire' but there's like zero documentation on it.

Who's your Storage Vendor? Do you have a good VAR who could do the solution hunting for you? You shouldn't have to test a ton of options, as that's exactly what the VAR is there to do.

Simpana's Commvault (the heavyweight), EMC's Avamar, and Veritas' NetBackup solutions will check all the requirements you've put forward here. They all have cloud ability with all the major offerings (Microsoft Azure, Amazon AWS, IBM Bluemix), they all let you install expensive, large proxies to act as go-betweens, they all allow combinations of baremetal, and virtual infrastructure with various snap-ins for your Storage Vendors. They all are centrally managed. They all work with even some of the more eclectic OS and Database solutions out there (AIX, DB2, Lotus snap-ins, etc.).

They all cost a fortune, and you'll still need a Backup admin for patch control, troubleshooting with failed backups, and issues with the backup system itself. Depending on cost constraints and personnel, there are also Managed Services providers who can leverage these systems and manage them for you. We currently do that for several large businesses.

EMC, who we've had considerable issues with already (see other thread by me), so frankly I'd rather not even reach out to them about this. Having said that, if we can just plug in to one of those vendor-supplied solutions, I can throw it on my plate to set it up (gotta do that with something anyhow). The cost might be a concern though as we've got other things to dump scratch on. Now, having said that, we're gonna have to spend money on something so whether it's a storage server (8-10k, depending on our discount) or yearly Azure/AWS costs + backup solution, it's gotta go somewhere.

How expensive/reasonable would those three be on a small/medium environment? Single purchase or yearly support costs? Not looking to have you do research for me but if you have any experience with them I appreciate the info.