permission woes when doing remote backups

[Red Squirrel]

This is an issue I'm always fighting with when doing remote backups. I'm wondering if I should just say screw it and do my backups as root... but I don't really like that idea.

For rsync to work properly the user that it's running under has to have access to the files on the remote system, which makes sense.

But how do I get around this whole issue? It's annoying having to have mass chmod / chown scripts just so I can get my backup scripts to work properly. There's got to be an easier way.

In some cases I can't give the backup user permission as some specialized files need to have specific permissions. Short from running all backups under root, what are the other options for painless backups without worrying about permissions?

also what does "skipping non-regular file" mean? My logs are flooded with these all the time.

 

[Nothinman]

I'm pretty sure most people just run their backups as root. There's no special group for backups like there is in Windows.

also what does "skipping non-regular file" mean? My logs are flooded with these all the time.

Just what it says. It's skipping a file that's not normal, i.e. a pipe, device node, etc.

 

[Red Squirrel]

I'm pretty sure most people just run their backups as root. There's no special group for backups like there is in Windows.



Just what it says. It's skipping a file that's not normal, i.e. a pipe, device node, etc.

Hmm so guess I could just run as root. As long as I have a strong password and change the keypair every now and then I'll hopefully be good. I normally don't open up root access for direct ssh. Need to log in as a user then su.

For the unregular files, is this something I should worry about backing up? I often see .so files and what not, which I believe are libraries. If it's not a big deal is there a way to suppress those messages?

 

[Colt45]

If it's not a big deal is there a way to suppress those messages?

2>/dev/null

 

[Red Squirrel]

2>/dev/null

I still want the rest though, like real errors, and the regular stuff. 2> will throw all errors to /dev/null which is not quite what I want.

I suppose I could write a program and pipe the output through that program then use regex to remove those statements.

 

[degibson]

Skipped files are often symbolic links. Libraries are just files -- there's no reason they can't be backed up normally -- but whatever.so is often symlinked to whatevers.so.some.version.junk

 

[Colt45]

could pipe stderr to grep i guess, and have grep print lines that don't match "special" or so.

 

[Red Squirrel]

Thanks, this makes more sense now. And think I'll try using grep. Can probably just do a reverse grep on "skipping special file".

 

[derrick2237]

In some cases I can't give the backup user permission as some specialized files need to have specific permissions. Short from running all backups under root, what are the other options for painless backups without worrying about permissions?

My workaround for that part is to tar up those directories before rsync runs. I think I actually tarred up the whole /etc before running rsync for some odd reason.

 

[Red Squirrel]

My workaround for that part is to tar up those directories before rsync runs. I think I actually tarred up the whole /etc before running rsync for some odd reason.

Oh so do it as a local cron job, then have the remote backup only take the tar? Would take longer though but that's an option.

 

[derrick2237]

Assuming that all your special files are in the same directory, you could just tar them up and write them to a location that will get backed up. Add that tar command to the crontab a few minutes before the rsync job will run.

 

[skyking]

Dirvish does most of what you are going to script for you.
It works like a charm.

 

[Red Squirrel]

Taring takes longer, and also uses more bandwidth (have to transfer the same tar file over and over) but guess for local backups it works.

Though, if I have a very secure root password, is just doing it as root (using key pair) considered bad? That seems like the easiest way, I just feel like it's wrong so I don't do it but suppose if it's a very strong password, and that fail2ban is installed, it should not be that bad right?

 

[weovpac]

Take a look at http://www.rsnapshot.org/. Read the faq, howto and manpage. Using Rsnapshot and SSH http://troy.jdmz.net/rsnapshot/. I run a cronjob as root as explained from the rsnapshot site. The options I use are:

rsync_short_args -azAHX
rsync_long_args --delete --numeric-ids --relative --delete-excluded

Using keys is more secure than using passwords alone. Just don't let anyone near you keys and you should be fine. The troy link explains how to setup keys without a password, it is safe using keys with no password, in your lan and maybe even over the net, but you need to be more careful. Also changing your keys once in a while is a good idea, don't forget to safe guard your keys!

 

[Red Squirrel]

Hmm rsnapshot does look interesting. I might consider reading up further and testing it out and implementing it. I am actually working on a control panel and once I get to the backup portion of it I'll probably include that as an option. The control panel will basically take care of setting up all the key encryption stuff and what not and make things easier.