need help with SSH

[Special K]

I have afs space that I access via SSH Secure Shell in winXP Pro. Normally when I want to transfer files to/from my PC and the afs space, I use this program called SSH Secure File Transfer, which has a windows explorer-like GUI. On one side is the files on your computer, and on the other side is the afs directories. The program also comes with a terminal window application, but that apparently only allows you to copy files from one network location to another, not between the network and a remote PC.

Anyway, I want to use this program to peform nigtly backups of certain folders, and I am wondering if there is a way for me to make it automatically connect to the server, upload the specified files, automatically overwrite duplicates, and then disconnect?

 

[oog]

should be possible. i personally use rsync over ssh to do backups.

 

[Red and black]

install cygwin, then set up a "scheduled task" within windows that kicks off a cygwin shell script that does an rsync over ssh.

 

[drag]

Is directly connecting to the AFS server a option? I know that openafs has a client for windows and the authentication is generally secure... being kerberos 4-style by default (I use kerberos 5 for my afs server though). That's pretty much as good as anything you can do with ssh. The encryption for file transfer is fairly weak, but it's good enough for information that isn't critical.

I know it's fairly common for people to do this with afs. Unlike SMB/CIFS or nfs security is pretty good.

here is a partial list of AFS cells that are configured by default for my openafs client install in Debian Linux.

1ts.org
acm.uiuc.edu
ams.cern.ch
andrew.cmu.edu
anl.gov
asu.edu
athena.mit.edu
atlass01.physik.uni-bonn.de
atlas.umich.edu
bazquux.org
biocenter.helsinki.fi
bme.hu
caspur.it
cats.ucsc.edu
cede.psu.edu
cern.ch
chem.cmu.edu
citi.umich.edu
clarkson.edu
club.cc.cmu.edu
cmf.nrl.navy.mil
coed.org
cs.cmu.edu
cs.pitt.edu
cs.rose-hulman.edu
cs.stanford.edu
cs.uwm.edu
cs.wisc.edu
dapnia.saclay.cea.fr
dbic.dartmouth.edu
dementia.org
desy.de
dev.mit.edu

There are about a 156 in total aviable. Sometimes when I get bored I'll go around snooping on them for interesting information and stuff. Kinda cool.

Otherwise with SSH standard practice dictates using rsync.

Rsync is very cool.. when your dealing with large files updates can happen fairly quickly as only the differences in files are transfered over ssh. You don't have to re-download the entire file. People use it for even doing backups over modem connections.

 

[Special K]

Originally posted by: drag
Is directly connecting to the AFS server a option? I know that openafs has a client for windows and the authentication is generally secure... being kerberos 4-style by default (I use kerberos 5 for my afs server though). That's pretty much as good as anything you can do with ssh. The encryption for file transfer is fairly weak, but it's good enough for information that isn't critical.

I know it's fairly common for people to do this with afs. Unlike SMB/CIFS or nfs security is pretty good.

here is a partial list of AFS cells that are configured by default for my openafs client install in Debian Linux.

1ts.org
acm.uiuc.edu
ams.cern.ch
andrew.cmu.edu
anl.gov
asu.edu
athena.mit.edu
atlass01.physik.uni-bonn.de
atlas.umich.edu
bazquux.org
biocenter.helsinki.fi
bme.hu
caspur.it
cats.ucsc.edu
cede.psu.edu
cern.ch
chem.cmu.edu
citi.umich.edu
clarkson.edu
club.cc.cmu.edu
cmf.nrl.navy.mil
coed.org
cs.cmu.edu
cs.pitt.edu
cs.rose-hulman.edu
cs.stanford.edu
cs.uwm.edu
cs.wisc.edu
dapnia.saclay.cea.fr
dbic.dartmouth.edu
dementia.org
desy.de
dev.mit.edu

There are about a 156 in total aviable. Sometimes when I get bored I'll go around snooping on them for interesting information and stuff. Kinda cool.

Otherwise with SSH standard practice dictates using rsync.

Rsync is very cool.. when your dealing with large files updates can happen fairly quickly as only the differences in files are transfered over ssh. You don't have to re-download the entire file. People use it for even doing backups over modem connections.

I tried openAFS already, but for some reason it's extremely slow. Half the time it just times out when I try to access the mapped nework drives. If you know how to fix that, it would be great, because than I can just use ntbackup to solve my problem

It really puzzles me that openafs is that slow, but ssh is not.

 

[Special K]

Originally posted by: Red and black
install cygwin, then set up a "scheduled task" within windows that kicks off a cygwin shell script that does an rsync over ssh.

Is writing the shell scripts difficult? I have never used cygwin before.

 

[Red and black]

Originally posted by: Special K

Originally posted by: Red and black
install cygwin, then set up a "scheduled task" within windows that kicks off a cygwin shell script that does an rsync over ssh.

Is writing the shell scripts difficult? I have never used cygwin before.

No, writing the shell scripts would be straightforward. And I think I misunderstood your needs, too -- if you just want to manually copy some files over every once in a while, you wouldn't need to do any scripting at all, just run rsync over ssh.

The best part of learning cygwin tools is that the knowledge will mostly apply to any unix system you end up using later.

 

[drag]

Originally posted by: Special K

Originally posted by: drag
Is directly connecting to the AFS server a option? I know that openafs has a client for windows and the authentication is generally secure... being kerberos 4-style by default (I use kerberos 5 for my afs server though). That's pretty much as good as anything you can do with ssh. The encryption for file transfer is fairly weak, but it's good enough for information that isn't critical.

I know it's fairly common for people to do this with afs. Unlike SMB/CIFS or nfs security is pretty good.

here is a partial list of AFS cells that are configured by default for my openafs client install in Debian Linux.

1ts.org
acm.uiuc.edu
ams.cern.ch
andrew.cmu.edu
anl.gov
asu.edu
athena.mit.edu
atlass01.physik.uni-bonn.de
atlas.umich.edu
bazquux.org
biocenter.helsinki.fi
bme.hu
caspur.it
cats.ucsc.edu
cede.psu.edu
cern.ch
chem.cmu.edu
citi.umich.edu
clarkson.edu
club.cc.cmu.edu
cmf.nrl.navy.mil
coed.org
cs.cmu.edu
cs.pitt.edu
cs.rose-hulman.edu
cs.stanford.edu
cs.uwm.edu
cs.wisc.edu
dapnia.saclay.cea.fr
dbic.dartmouth.edu
dementia.org
desy.de
dev.mit.edu

There are about a 156 in total aviable. Sometimes when I get bored I'll go around snooping on them for interesting information and stuff. Kinda cool.

Otherwise with SSH standard practice dictates using rsync.

Rsync is very cool.. when your dealing with large files updates can happen fairly quickly as only the differences in files are transfered over ssh. You don't have to re-download the entire file. People use it for even doing backups over modem connections.

I tried openAFS already, but for some reason it's extremely slow. Half the time it just times out when I try to access the mapped nework drives. If you know how to fix that, it would be great, because than I can just use ntbackup to solve my problem

It really puzzles me that openafs is that slow, but ssh is not.

I went and looked up some stuff about it.

Well the AFS client for Windows sucks, I beleive. It has problems like not being able to keep track of the local file cache on shutdown and restart and stuff like that.

Technically it should be very fast after you initially download the file first, since after that everytime you use that file it will be on your local drive in cache. This is one of the major reason why openafs is usefull over the internet at all.

Also it does a weird thing were it maps it's output to SMB/CIFS (windows file sharing protocol) and that transition is done on your computer.. so everything is downloaded to openafs, but you open it up on smb.

There maybe issues with using it behind certain types of firewalls/routers and such, and it's not usefull over slow connections.

If you were using a 1.2 series client try trying a the newest version and see if that helps, they improved the performance of it considurably. They recently released a 1.4.0 version, it's the new stable version. Maybe that will help.


But if all you want it for is backups and not for doing work over the internet and having the changes sent back and forth between the clients/server then rsync would be much faster. Since rsync only sends parts of files that have been changed since the first big initial download.

 

[oog]

if it helps, i use cygwin and just run: rsync.exe -Rvrltgz --delete --delete-after --stats --modify-window=2 --delete-excluded -e "ssh.exe -i keyfile" --files-from=include.txt --exclude-from=exclude.txt /cygdrive/ user@machine:/targetdir

include.txt is a simple text file that contains a list of directories that i want to back up. they are listed as unix-like directories -- "an entry might be /c/documents and settings/oog/my documents". under cygwin, each drive letter shows up as a directory under /cygdrive/. since i use a source directory of /cygdrive/ on the rsync command line, include.txt should start with /driveletter/.

exclude.txt is a simple text file that contains a list of files that i do not want to back up. for instance, i include Thumbs.db in that file.

 

[Special K]

OK I got rsync working, but how do I get it to store my password so I don't have to type it in at the cygwin terminal every time? I have read several tutorials on that on the web but none seem to work.

here is the command I am using:

C:\cygwin\bin\rsync -e ssh -vrtz --delete /cygdrive/c/test kevinkl@login.engin.umich.edu:/afs/engin.umich.edu/u/k/e/kevinkl/Private/Favback

I am just trying to backup my Favorites folder as a test.

And it keeps asking for my password every time. How do I stop this?