Any way to make RSync more efficient? (Mac to Debian)

[VinylxScratches]

I got a Macbook with Snow Leopard. I manually Rsync my Music files to my server that's running Debian.

The building file list process takes forever... since this is a one way sync I was thinking about maybe writing a CSH script that watches the Music drive for changes, I guess it would compare to the last scan and see what's the difference by using the diff command, then maybe running a SFTP command on the files that were added since the last scan....

Would this be a good way of doing it?

Current code...
#! /bin/csh -f

rsync -e "ssh" -vrca --delete-after ~/Music/ USERNAME@192.168.255.107:/mnt/hd1/share/media/music/

 

[Nothinman]

Does OS X have nothing like inotify? Something like it would be the best for your script.

As for speeding up rsync, have you tried going over rsyncd instead of ssh?

 

[LuckyTaxi]

Ive been using GIT as of late for my files. Rsync does take forever.

 

[Emulex]

arrange your music into subdirectories

a
b
c
d
e
f

then rsync smaller chunks. i used to have to rsync 400gb of html/jpeg (millions of files) and it was the only way back when rsync only used ram. the use of disk for storing data helps alot but break it down. it really does help.

 

[skyking]

I'd use dirvish myself, if I could get it going in snow leopard. It is a backup package that uses rsync to do the legwork, but it automates all the other stuff nicely.
Looks like there is some info here.
http://dirvish.darwinports.com/

 

[VinylxScratches]

NothinMan, I don't think it exists. I also want it SSHed because eventually I want to do this over the internet.
Emulex, I may have to do that...
LuckyTaxi, git might be cool but I'd like to keep it as "stock" as possible.
Same with Dirvish skyking...

 

[n0cmonkey]

Does OS X have nothing like inotify? Something like it would be the best for your script.

As for speeding up rsync, have you tried going over rsyncd instead of ssh?

I think the closest thing would be kqueue, and it may be a poor substitute in this case.