Freenas box questions

[SlitheryDee]

So I just built a freenas box and set it up. Everything went pretty well with the help of the manual and online resources, but I have a couple of nagging questions. I'm not sure if this belongs in storage or networking, but the questions seemed primarily networking related.

First of all, whenever my windows 8.1 HTPC goes to sleep, when it wakes back up it can no longer access the CIFS share I have set up on the freenas box. I set up the share using the instructions in the freenas manual for creating a CIFS share accessible by anyone on the network with no username or password. I can provide the specifics of that if necessary. Rebooting the windows pc seems to restore access to the share until the next time it goes to sleep, which causes the share to be inaccessible again. Is there any way to make sure my HTPC retains access even after going to sleep?

The other question has to do with transferring the data from my old nas to the freenas box. I have a readyNAS NV+ with about 6 terabytes of data on it that I want to transfer over to the freenas box. The freenas box is capable of very fast transfer speeds over my gigabit network, but the readyNAS is quite slow. I'm guessing that it would take close to a week to transfer everything at the speeds the readynas is capable of. It's possible to copy/paste everything from a windows pc, but then the process is dependent of the windows pc running the transfer for a week without crashing/glitching, me accidentally canceling it, etc. Obviously it would be preferable if I could have the NASes perform the task between themselves independent of any other devices. My first impression was to use rsync to instruct the freenas box to pull the data off the appropriate readyNAS share and then delete the task when it's finished. I got that up and running fairly easily, but it seems to be performing particularly slowly. It ran all night and only got through about 100gb of data. I was wondering if there might be a better way to do this that isn't terribly troublesome?

Thanks ahead of time for any help you can give me.

 

[dave_the_nerd]

No idea on the first question. That sounds like a problem with your PC though - I've never had anything like that.

The second piece is tougher. Rsync has some overhead, and it checks files for changes. But it will be slower/faster depending on the number and type of files being transferred. 100GB in 8 hours is about 3.5MB/sec. Which isn't fast, at all, but would be "normal" speed for copying, say, 100GB of 4kB source code and HTML files. (It would also make me wonder about your network... are you using WiFi somewhere in that chain?)

 

[SlitheryDee]

No idea on the first question. That sounds like a problem with your PC though - I've never had anything like that.

The second piece is tougher. Rsync has some overhead, and it checks files for changes. But it will be slower/faster depending on the number and type of files being transferred. 100GB in 8 hours is about 3.5MB/sec. Which isn't fast, at all, but would be "normal" speed for copying, say, 100GB of 4kB source code and HTML files. (It would also make me wonder about your network... are you using WiFi somewhere in that chain?)

The files being transferred are mostly blu-ray rips in MKV form. I have no idea what the "normal" transfer speed of those would be, but if I attempt to copy one from the readyNAS NV+ to my windows PC, I get 15-20MB/s transfer speeds. Doing the same thing from the freenas box gets me 50-70 mb/s transfer speeds, so I'm assuming that the readyNAS is the bottleneck here. Both NASes are connected to the same gigabit switch via ethernet cable which is in-turn connected to my gigabit router via ethernet cable. I figure it's going to take a while no matter what, but I should at least be getting faster speeds than I am right?

 

[azazel1024]

Can you pull the drives from one, drop it in the other and copy it locally? Sneaker net is almost always the fastest way to do things*

Yes I'd assume you should be getting faster speeds, but no idea what rsync is going to have for overhead. Depending on what it is doing for file changes/constency, it might at a heck of a lot of overhead. That said, basic file changes shouldn't. My server might be a lot beefier than your gear (no idea), but it is running windows 8. Using synctoy for backups, it takes all of about 5 seconds to scan a directory of sub-directories containing around 10,000 files for changes. General transfer overhead on it seems to be around 10-20% compared to straight SMB/CIFS (transfers seem to run around 180-200MB/sec compared with 210-235MB/sec for a straight network share transfer).

However, I doubt that synctoy is running anything like the same kind of checks or processes that rsync on freeNAS runs (also it is running on a G1610 with 8GB of RAM, a couple of gbe NICs and a RAID0 array).

 

[dave_the_nerd]

The files being transferred are mostly blu-ray rips in MKV form. I have no idea what the "normal" transfer speed of those would be, but if I attempt to copy one from the readyNAS NV+ to my windows PC, I get 15-20MB/s transfer speeds. Doing the same thing from the freenas box gets me 50-70 mb/s transfer speeds, so I'm assuming that the readyNAS is the bottleneck here. Both NASes are connected to the same gigabit switch via ethernet cable which is in-turn connected to my gigabit router via ethernet cable. I figure it's going to take a while no matter what, but I should at least be getting faster speeds than I am right?

Interesting.

Yeah, in that case, you should be seeing close to the ReadyNAS's maximum speed for the transfer.

How are you doing the rsync? (what file sharing method, etc. Rsync daemon on the ReadyNAS? Can you copy/paste the command you're using?)

If that's just not going to work, my fallback would be to enable NFS on the ReadyNAS, mount it on the FreeNAS box, and do local rsync or cp from the console. You can check bandwidth by looking at the network utilization stats on the FreeNAS Web UI.

 

[SlitheryDee]

Interesting.

Yeah, in that case, you should be seeing close to the ReadyNAS's maximum speed for the transfer.

How are you doing the rsync? (what file sharing method, etc. Rsync daemon on the ReadyNAS? Can you copy/paste the command you're using?)

If that's just not going to work, my fallback would be to enable NFS on the ReadyNAS, mount it on the FreeNAS box, and do local rsync or cp from the console. You can check bandwidth by looking at the network utilization stats on the FreeNAS Web UI.

I don't know anything about a script. I just used the freenas web GUI and readynas frontview to set up rsync.

Having no previous experience with rsync, I had to cobble together something that would work by comparing the documentation of each NAS. They didn't seem to use a lot of the same terminology, but I figured it out eventually. As far as I can tell the "module" is the share name on the readyNAS that I simply enabled rsync for in the readyNAS configuration page. Then all I had to do was give rsync read permission and add the ip address of the freenas box as an allowed host.

On the freenas end, I created an rsync task that listed the readynas share name that I enabled rsync on under "module" and the ip address of the readynas under "host". I then I set it to "pull" and pointed it to the folder on the freenas volume I wanted the data to appear in. I set it to run every minute because I figured the frequency didn't matter since I was probably only going to run it once to completion and then delete the task.

A minute later the directory structure and data of the readynas share started to appear in the correct folder on the freenas box. That's pretty much all I did.

I'm at work now, but I'll look into your other suggestion as soon as I have access to the box. Thanks.

 

[SlitheryDee]

Just in case anyone want to know, I think I solved my problems.

What was actually happening in the first issue wasn't me losing access to the share, but actually losing access to the mapped network drive connected to the share. Somehow the drive letter that windows automatically chose for the share was getting reassigned to something else after the computer woke from sleep. Since the letter was in use, attempting to access the drive produced a "this resource is already in use" type of error, which somehow locked up the share in network as well as the mapped drive. Simply choosing a different drive letter seems to have completely solved the problem, as weird as that sounds.

The second issue was harder. I tried enabling nfs, mounting the share, and performing a copy operation, but the process always seemed to copy a part of a file very quickly and then stall for some reason. The best solution to my file transfer woes turned out to be FTP. Configuring FTP in freenas and then instructing the readyNAS to perform a backup to the Freenas using FTP is producing speeds several times faster than rsync did. I'm still not maxing out the connection, but I'll take what I can get.

Thanks for the help!