Linux n00b needs hard drive sync advice

[LoveMachine]

I have an HTPC (hacked Asus Chromebox) running KodiBuntu (14.04 kernel I think) and another Windows7 PC, both with SSDs for the OS and large HDDs for media (an external USB 4TB drive on the HTPC). All connected over wired gigabit ethernet. I've never dabbled in any flavor of Linux before setting up the HTPC, so I'm in unfamiliar territory.

I'd like to be able to sync the media folders of the hard drives in each computer so they both have identical libraries. The Win7 PC is on almost 24/7, but I'd prefer to have whatever server/backup application be on the Ubuntu machine which will always be on. I'm envisioning that at some routine interval, the HTPC scans the appropriate folders on the Win7 PC and pulls over any new files, preferably at the block level (rather than just new file names in case some file gets overwritten with new info) without any user involvement. In case of an emergency (house fire, for example), the external drive on the HTPC serves as an easy to grab back up for all our files, in addition to being a media server.

Any recommendations? Is software RAID what I'm looking for? I don't need instantaneous syncing since this is mostly movies/music/photos, and prefer simple/easy/reliable. I've briefly looked at options like FreeNAS etc, but welcome first hand recommendations. Something with an easy to use GUI/minimal command line fiddling would be nice.

 

[paperwastage]

I'd like to be able to sync the media folders of the hard drives in each computer so they both have identical libraries.

any reason why you would want to sync?

If you have one machine (eg HTPC) that's online 24/7, you can get other machines to remote-mount the stuff on the HTPC... for media stuff, there shouldn't be any buffering issues especially on wired gigabit

that way, you avoid any need for syncing (and stuff getting out of date/out of sync). Downside is that you can only modify files on one machine at one time, shouldn't be a problem for videos/media. On your HTPC box, set up a proper backup/archiving solution for those files

 

[lxskllr]

If you still want to sync after paper's suggestion, or need it for something else, LuckyBackup provides a friendly frontend to rsync.

 

[LoveMachine]

I'd like to have 2 identical libraries on 2 machines, so if I'm tinkering on the HTPC (rebooting/updating/whatever) the kids can watch that movie that they've already seen 7,004 times with the other computer. Both computers are connected to the TV/AVR. I'm still in the process of learning Linux and setting up the HTPC, and that takes some tinkering. And the only time I have to tinker is when it's movie time for the kids. Eventually, the Win7 machine will rarely be used and the HTPC will function as a NAS and media watching gadget, but I'd like to have the same library on Win7 computer for those times when the HTPC will be down.

The backup applications I'm familiar with (Acronis and Windows backup) make a giant block of inaccessible data for general use. Rather than having a library and a backup of the library somewhere else, I'd rather have 2 identical libraries.

 

[dealcorn]

Unison is my preferred file synchronization tool that does exactly what you want and supports both Linux and Windows. My use has been limited to the Linux side but Windows support is a touted feature. http://www.cis.upenn.edu/~bcpierce/unison/ Rsync provides the back end file transfer engine and Unison is available with both GUI and CLI interfaces. You may need to play with it a bit to get comfortable with your configuration that always syncronizes to the Windows preferred directory. The initial run is slow, and then it is really fast for subsequent synchronizations.

I set up Unison with a cron job using the CLI interface to fully automate the synchronization. I am certain Windows has a command similar to cron (maybe "AT") so you can drive the process from a script (batch file(?)) on your 24/7 Windows server. It may also be helpful to investigate the the Windows version of WakeOnLan so you can wake the Linux box for synchronization then put it back to sleep. The Linux command "shutdown now" requires root/superuser or sudo privileges. That likely drags you into "visudo" which may be deeper into the rabbit hole than you want to go. Initially, it is easiest to just configure the synchronization to run while the Linux box is already turned on. Easy is good.

 

[MrColin]

After setting up samba on the linux box, robocopy is the scriptable windows command line tool to sync from whatever to/from windows. If you have file sharing on windows turned on, you could use rsync, or whatever gui front end to push/pull from/to linux.