Real time data sync recommendation.

[nk215]

Hi guys,

Do you guys/gals have any experience with real time data sync? Basically I want a RAID1 over local network.

Here’s the situation:
I have my critical data base backed up rather well, on site, off site, cloud etc. so if the power supply on my “server” blows up and take out all the attached HDs, my most recent out-of-server backup is half-day old.

There are many software packages that promised the same thing but I am looking for user recommendation.

 

[Data-Medics]

I use a Synology NAS with their Cloud sync feature to sync folder shares across the network and keep a copy on the NAS. Other options you might consider are Bittorrent Sync, and Symform both of which support this type of thing.

 

[Data-Medics]

Are you just looking to sync the files? or do you need sector level sync to be able to build a bootable clone?

 

[gea]

You have basically four options

- Sync data in a continous manner
You can use any sync tool (ex rsync or robocopy)

Problems:
Open files are not synced
depending of the amount of data, you have a delay up to half an hour

- blockbased data replication
ex with ZFS and ZFS send
This replicates a filesystem over your network. It works with snaps so
open files are included. It transfers only modified data and keeps all attributes.
You can run a replication update every minute

- external SAS enclosure
you can use external SAS disks for mirroring. This helps in case of a PSU problem but not for fire or theft.

- Mirroring over the network
This is realtime. You need a second computer that offers iSCSI targets that you can use to mirror your local disk. If you use a ZFS system, your mirror is checksummed and data is validated.

Problem: Your local disk is fast (SSD, to 500 MB/s) while a 1Gb/s network can only deliver about 1oo MB/s max. In a mirror, your overall performance is reduced to the slower part. This is ok with 10 Gb/IB/FC networks.

 

[hasu]

I bought a Windows utility called "Mirror folder" a few years ago and at that time that was the best mirroring software I could find. Supposed to act as software raid-1 for selected files/folders. Basically it is a windows driver which will replicate changes to a file in system level. IIRC, you could use it for real-time backup over the network. I never tried it with database or other live contents, though. Later I switched to real hard drive mirroring (e.g, mdadm in Linux and software raid in Windows pro versions).

 

[Carson Dyle]

How often has a hardware failure taken out all of the local HDDs on a system? I've certainly never seen or even heard of this happening, and I've seen plenty of PSU failures.

Keep in mind that RAID and your real-time "network RAID" proposal are NOT backups. A far more likely failure scenario is that someone fat-fingers something and wipes out a few hundred critical files, which (in real-time) will promptly be deleted from your network mirror as well, leaving you to rely on last night's backup anyway.

 

[Data-Medics]

How often has a hardware failure taken out all of the local HDDs on a system? I've certainly never seen or even heard of this happening, and I've seen plenty of PSU failures.

I've seen it dozens and dozens of times. In fact, it's half the reason my data recovery business is in existence.

 

[Carson Dyle]

I've seen it dozens and dozens of times. In fact, it's half the reason my data recovery business is in existence.

Every hard drive on a system trashed by a hardware failure, dozens and dozens of times?

How many times has this ever happened to anyone else here? I've administered hundreds, perhaps thousands of desktop and server systems over the past 20 years and have never seen it once. I've certainly seen hard drives fail, but it's seldom due to another failure in the system and never all drives in a system.

 

[nk215]

Hello everyone

Thanks for the input. I maybe paranoid about the ps taking out HDs. I have not experience this fortunately. I've seen ps taking out motherboard but not HDs.

There are lots of good information here.

 

[Data-Medics]

I have seen it quite a number of times. But to be fair, I do work in data recovery and one of my specialties is RAID recovery. So I probably do see more of this than most people ever will. But it does happen. Once it was a computer of mine that did it. Heard a loud pop, computer shut off, all hard drive PCB's fried. Fortunately we're smart enough to back up our data here so it wasn't a big loss.