scheduled dd backup

[jae]

I was wondering if anyone has a script to schedule a dd backup. On my ProxMox, I was planning to run RAID1 but found out they no longer support it in the 3.X series (well unless I install Debian first as raid1 then install proxmox). So I'm looking to set up a cronjob i guess, but ive never done this before.

 

[Cerb]

No. Not just as in not having something to do it, but as in don't do that.

If you're backing up changing data, just find an rsync front-end app, or script, that you like, and go from there.

RAID is a whole other issue.

 

[jae]

Well I wanted to make a 1:1 copy of the disk incase it died for some reason.

Right now I just run dd once a week.

 

[Elixer]

While it does depend on the distro, this will get you started.
http://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/

 

[Essence_of_War]

Well I wanted to make a 1:1 copy of the disk incase it died for some reason.

Right now I just run dd once a week.

I'm fairly certain that 'dd' will not correctly clone/image an on-line filesystem. You'll have to boot with a live-cd or something like that to correctly do what you want to do.

If you're just trying to back-up your data, you could set-up an rsync cronjob for say, your home directory, and that should work fine.

 

[Fallen Kell]

I'm fairly certain that 'dd' will not correctly clone/image an on-line filesystem. You'll have to boot with a live-cd or something like that to correctly do what you want to do.

If you're just trying to back-up your data, you could set-up an rsync cronjob for say, your home directory, and that should work fine.

You are correct, dd will not work properly on an active filesystem.

If you are trying to make images, look into using mondo.

 

[Red Squirrel]

Yikes, no raid at all, for VMs? Run a NFS server with mdadm raid at very least. I'm not aware of ways to backup VMs live though. You can't just DD it. You could maybe find a way to script the shutdown of a VM, do a file copy then fire the VM back up. If you do file level backups then you can probably get away with doing the VMs themselves like once a month.

 

[Cerb]

Yikes, no raid at all, for VMs? Run a NFS server with mdadm raid at very least. I'm not aware of ways to backup VMs live though.

Snapshots (make snapshot, backup, delete snapshot). But, the details will depend on the system used to handle the VMs and those snapshots (I'm not familiar with ProxMox). Also, the VM might be unresponsive during the creation of the snapshot, so scheduling it when it won't be serving files could be important.

You can back up a whole file system and restore it, so long as your partitions match up, and you use a file system that can have snapshots taken (like ZFS, or BTRFS) (FI, set the UUID or label of your new partition to that of your old one--don't go editing fstab, because something else might be using those same UUIDs or labels, in some distros). But, there isn't a simple generic way to do it, but rather, tools for each file system, then tools for each hypervisor, and so on.

 

[jhu]

Snapshots (make snapshot, backup, delete snapshot). But, the details will depend on the system used to handle the VMs and those snapshots (I'm not familiar with ProxMox). Also, the VM might be unresponsive during the creation of the snapshot, so scheduling it when it won't be serving files could be important.

You can back up a whole file system and restore it, so long as your partitions match up, and you use a file system that can have snapshots taken (like ZFS, or BTRFS) (FI, set the UUID or label of your new partition to that of your old one--don't go editing fstab, because something else might be using those same UUIDs or labels, in some distros). But, there isn't a simple generic way to do it, but rather, tools for each file system, then tools for each hypervisor, and so on.

Bolded part. Absolutely look into this. RAID1 doesn't ensure data integrity the way ZFS and BTRFS do.

 

[Essence_of_War]

RAID1 doesn't ensure data integrity the way ZFS and BTRFS do.

In this vein, I think all distros, and if not all, certainly Debian, support btrfs natively. Also, zfs is supported through the zfsonlinux project (kernel space port, not the older userspace hack) and it is available for Debian.

 

[jhu]

BTW, here's an article on BTRFS and ZFS and why you need them.

 

[manly]

No experience with ZFS. From what I've read on Ars, btrfs is about a year away.

Just do the right thing and install Debian first with RAID1, then ProxMox over it. What's so bad about that?

 

[jae]

Don't know if I trust btrfs yet. Was running it in Fedora 20 months ago and of course ran into the free space issue. Deleting snapshots and rebalancing the drive didn't work. So I don't if btrfs is ready yet, for me at least.

If I reinstalling proxmox on top is the only option, I may play with XenServer first as proxmox has removed a few good features from 2.X to 3.X.

 

[Cerb]

I had a VM that got borked from having gobs and gobs of free space, but ran out of allotted room for metadata--WTF?! Fixed that, and then in short order got but by a bug where it messes up if /boot is in a subvolume. That was maybe a month go. I rebuilt it with an EXT4 /boot and XFS /.

Now, no data was lost, no FSCK needed, hell, not even read-only mounting, so that whole resilience thing works. But, it's not quite there yet, IMO.

 

[MrColin]

I was wondering if anyone has a script to schedule a dd backup. On my ProxMox, I was planning to run RAID1 but found out they no longer support it in the 3.X series (well unless I install Debian first as raid1 then install proxmox). So I'm looking to set up a cronjob i guess, but ive never done this before.

as others have mentioned, dd doesn't do well with currently mounted inputs. I bet you could rig up an automated grub option to boot off of another filesystem like a USB key or even pxe and have that image your drive(s). I think clonezilla or partedmagic installs can be automated this way but running it won't be easy to automate, or even set up. I have no idea what ProxMox is so I could be way off.

I would lean toward rsync myself though its not for imaging.