How to setup sshfs/fuse for users?

[xtknight]

Please see posts below this.

(PREVIOUS NFS PROBLEMS)

Here's the situation:

My friend can export any of his regular file system to NFS and I can mount it. The sole exception being stuff on his file system that is a samba mount. For example, /mnt/music on his file system refers to a samba location. However, on his PC, the samba location is mounted in /mnt/music and it appears just as any directory. The files can be listed just fine.

When he tries to export /mnt/music, both he and I get access denied when trying to mount it. I found a similar problem here but no solution: http://lists.samba.org/archive/samba/2006-June/121954.html

The /etc/exports looks like this:
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
/mnt/music *(ro,root_squash,sync)
/home/tom *(ro,root_squash,sync)
/media/hdb1 *(ro,root_squash,sync)

(And I can access /home/tom just fine.)

We do exportfs -vra and it shows it as being exported.
exporting *:/mnt/music
exporting *:/home/tom
exporting *:/media/hdb1

Here is his mounts:
//server/Users on /mnt/music type cifs (rw,mand)
/dev/sda1 on /home type xfs (rw)
/dev/hdb1 on /media/hdb1 type ntfs (rw,nls=utf8,umask=007,gid=46)

I can see the folder as exported with this command:
showmount -a <ip>

*:/mnt/music
*:/home/tom
*:/media/hdb1

Here is the command we use to mount:
sudo mount <ip>:/mnt/music /shares/tom/music
Returns: mount: <ip>:/mnt/music failed, reason given by server: Permission denied

The same thing goes for /mnt/music/ with a slash at the end.

He gets the same error on his own machine. /shares/tom/music does exist on both machines and is empty. There is nothing pertinent in dmesg.

We have restarted nfs-common and nfs-kernel-server numerous times. (sudo /etc/init.d/service restart)

The only folder I can not access is /mnt/music. His other NFS shares are mountable by both me and him (we both mount using same ip which is over hamachi). The whole /mnt/music folder is owned by him (chown -R).

What could it be? BTW, /mnt works fine, but it freezes at ls -al when mounted as an NFS share. music is the only folder under /mnt.

 

[cleverhandle]

Are you sure this is supposed to be possible? It wouldn't surprise me if it's not. Why can't you just smbmount the same share your friend is using?

 

[xtknight]

What is preventing it from being possible? Solely because it was mounted by cifs and not, ext3 for example? Shouldn't I be able to export anything with NFS, regardless of filesystem?

He can reshare /mnt/music using smbfs and it acts like any other folder would. It just doesn't want to 'reshare' with NFS. We had too much trouble getting smbfs to work reliably so we're using NFS. We do have two Linux machines so smb doesn't really make sense.

That's an idea but I don't think I can access his domain remotely.

 

[Nothinman]

What is preventing it from being possible? Solely because it was mounted by cifs and not, ext3 for example?

Possibly, not all filesystems interact with NFS nicely. From what I remember reiserfs had some really big NFS problems with it's first few releases.

 

[cleverhandle]

Originally posted by: xtknight
What is preventing it from being possible? Solely because it was mounted by cifs and not, ext3 for example? Shouldn't I be able to export anything with NFS, regardless of filesystem?

I have an admittedly limited understanding of the low-level details of NFS. But AFAIK one purpose of NFS is to present a filesystem to the client that looks and acts as close to a native filesystem as possible. If the underlying filesystem (cifs in this case) doesn't support the same kinds of permissions or system calls or whatever that client thinks it should, then there could be problems. I could see authentication being a particular issue as well.

We do have two Linux machines so smb doesn't really make sense.

Actually, I would often prefer using CIFS to using NFS, even between Linux clients. NFS is a PITA to get working quite frequently, especially if the two machines don't run identical UNIX flavors. NFSv4 looks promising, and most of it actually works now, but it will still be another year or two before it sees much widespread use.

 

[drag]

I would use SMB in a mixed environment. If it was just pure Linux AND if it was a secure network then I'd use NFS. Also if speed is critical then NFS is usefull.

SAMBA itself is actually fairly nice system. It's a bit slow, but since everybody has no choice but to work with Microsoft operating systems then everybody has SAMBA and supports it.

Personally I am a big fan of sshfs. Simply because it's easy to setup and it has strong options user-centric authentication (which NFS lacks completely)

It's just to bad that Windows is such a PITA to work with and it is ubiquious. There are such nice other network file systems.. such as AFS (usefull over slower/unreliable campus-wide links and such with good security model, advanced cache'ing mechanism and encryption), that are mature (AFS is older then dirt. Originates back with the Athena project and IBM donated OpenAFS which now has a active community behind it) and usefull but are almost worthless when you introduce Windows clients.

As far as NFSv4 I don't realy see it making much headway with clustering file systems and dedicated storage (which is something that is more interesting) taking up the high-speed networking stuff and SAMBA on the workgroup level. Who wants to take 2 sets of incompatable extended ACLs (linux and ntfs) and then add a third incompatable set? Unless of course Microsoft adopts NFSv4, then it will be dominate within a year if that happenned. 😛

 

[n0cmonkey]

Originally posted by: drag
I would use SMB in a mixed environment. If it was just pure Linux AND if it was a secure network then I'd use NFS. Also if speed is critical then NFS is usefull.

SAMBA itself is actually fairly nice system. It's a bit slow, but since everybody has no choice but to work with Microsoft operating systems then everybody has SAMBA and supports it.

Personally I am a big fan of sshfs. Simply because it's easy to setup and it has strong options user-centric authentication (which NFS lacks completely)

It's just to bad that Windows is such a PITA to work with and it is ubiquious. There are such nice other network file systems.. such as AFS (usefull over slower/unreliable campus-wide links and such with good security model, advanced cache'ing mechanism and encryption), that are mature (AFS is older then dirt. Originates back with the Athena project and IBM donated OpenAFS which now has a active community behind it) and usefull but are almost worthless when you introduce Windows clients.

As far as NFSv4 I don't realy see it making much headway with clustering file systems and dedicated storage (which is something that is more interesting) taking up the high-speed networking stuff and SAMBA on the workgroup level. Who wants to take 2 sets of incompatable extended ACLs (linux and ntfs) and then add a third incompatable set? Unless of course Microsoft adopts NFSv4, then it will be dominate within a year if that happenned. 😛

NFS fits the "too small for NAS/SAN, too big for SMB" niche.

I'm planning on playing with AFS as soon as I get kerberos up and running nicely. The biggest problem is kerberos and a lack of a heimdal framework for Windows, and Apple's silly use fo the MIT code base...

 

[drag]

Remember the kerberos in AFS is it's own little world. You'll need a compatability software to get the version 4 tickets AFS uses to go with the Kerberos 5 everybody else uses.

 

[n0cmonkey]

Originally posted by: drag
Remember the kerberos in AFS is it's own little world. You'll need a compatability software to get the version 4 tickets AFS uses to go with the Kerberos 5 everybody else uses.

Heimdal has kerberos 4 support. 🙂

 

[cleverhandle]

Originally posted by: n0cmonkey
Heimdal has kerberos 4 support. 🙂

But, IIRC, you don't need (or want) AFS to actually use Kerb4 tickets or speak that protocol over the wire. There's some kind of hack you can apply to the KerbV tickets so that they'll authenticate properly to AFS without using any kind of 5to4 daemon or equivalent. My memory may be foggy, though, since it was a couple years ago that I set up a cell and the whole process is rather arcane from start to finish. But I'm pretty sure that when you're done, a klist will show only v5 tickets, even in an environment with MIT Kerberos and no 5to4 daemon.

 

[drag]

Originally posted by: n0cmonkey

Originally posted by: drag
Remember the kerberos in AFS is it's own little world. You'll need a compatability software to get the version 4 tickets AFS uses to go with the Kerberos 5 everybody else uses.

Heimdal has kerberos 4 support. 🙂

Well it's more then just that. AFS has it's own built in kerberosv4-style server with it's own users and such. There is a package in Debian that is able to map a kerberos server to server as a AFS authentication-thingy, but I dont' recall it right now. (I used MIT Kerberos btw) You know, for SSO capabilities.

If you want to deploy AFS you don't even need to setup a seperate kerberos server if you don't want to. It's sort of like SAMBA in that respect.

edit:
Oh, the package is openafs-krb5. I guess it's included in the official openafs source code stuff now.. I thought it was a different little add-on for some reason.

edit2:

Oh, and if your doing krb5 integration into openafs the service key or whatnot that openafs would use has to be in a specific format for openafs to be able to use it.

 

[n0cmonkey]

Originally posted by: cleverhandle

Originally posted by: n0cmonkey
Heimdal has kerberos 4 support. 🙂

But, IIRC, you don't need (or want) AFS to actually use Kerb4 tickets or speak that protocol over the wire. There's some kind of hack you can apply to the KerbV tickets so that they'll authenticate properly to AFS without using any kind of 5to4 daemon or equivalent. My memory may be foggy, though, since it was a couple years ago that I set up a cell and the whole process is rather arcane from start to finish. But I'm pretty sure that when you're done, a klist will show only v5 tickets, even in an environment with MIT Kerberos and no 5to4 daemon.

Interesting. I haven't really looked into it at all. The biggest problems are that ARLA doesn't have a stable AFS server and OpenAFS appears to be i386 only on OpenBSD... That and I have almost no time to play with stuff. 😛

 

[n0cmonkey]

Originally posted by: drag
Well it's more then just that. AFS has it's own built in kerberosv4-style server with it's own users and such. There is a package in Debian that is able to map a kerberos server to server as a AFS authentication-thingy, but I dont' recall it right now. (I used MIT Kerberos btw) You know, for SSO capabilities.

If you want to deploy AFS you don't even need to setup a seperate kerberos server if you don't want to. It's sort of like SAMBA in that respect.

edit:
Oh, the package is openafs-krb5. I guess it's included in the official openafs source code stuff now.. I thought it was a different little add-on for some reason.

edit2:

Oh, and if your doing krb5 integration into openafs the service key or whatnot that openafs would use has to be in a specific format for openafs to be able to use it.

Happen to have a doc on getting MIT kerberos clients to use Heimdal servers? I don't want to use MIT. 😛

I haven't looked at OpenAFS much, their documentation makes me want to cry.

 

[xtknight]

So how do I use AFS? Or sshfs? Whichever's easier?

 

[drag]

Originally posted by: xtknight
So how do I use AFS? Or sshfs? Whichever's easier?

Well if you have to deal with Windows clients you just don't. It's not usefull in a mixed environment. You just have to use SMB. 😉

AFS is a pretty hardcore thing. You have to setup a bunch of services and such. It has a strong kerberos based athentication method and volume management scemes. Also it has it's own set of ACLs. It's a very old fasion thing, pretty bizzare but it has a lot of features unmatched by contemporary Linux network filesystems.

It's only usefull if you want to do something 'campus wide'. It's only realy used in universities and some government stuff nowadays.
http://www.openafs.org/pages/doc/AdminGuide/auagd002.htm#ToC

And on a local fast lan it may actually be slower then SMB. It's nice when your dealing with slow networks and such because it has extensive cache'ng mechanisms that mean that when your editing files and such it's all on your local harddrive.

One of hte reasons it's unpopular is due to the administration overhead and how you need to train users to use it. It's great though in some circumstances, basicly unbeatable in situations like I described.

Although the Windows client sucks donkey-balls. It uses a odd AFS to SMB translation to make it work with Windows and the cache'ng mechanism pretty bad. (for instance you loose your cache during a reboot)



Sshfs, on the other hand.. It's trivially easy to setup. But it's usefullness is limited to modern Linux distros and probably FreeBSD. Nothing else is going to support it.. however there are nice sftp clients in OS X and Windows that will accomplish pretty much the same thing.

It's a FUSE-based file system so you need a kernel that supports FUSE and you need to have permissions setup for your user to use fusermount and access /dev/fuse. (the kernel module is just named fuse)

Sshfs client itself is just a small C program. Only a couple hundred lines. Fuse stands for 'File System in UserSpacE", and it is a generic framework for building simple file systems in Linux. Some distros (like Debian) have sshfs packages aviable, but it's pretty easy to compile.

On the server side you have to have OpenSSH running (or a SSH server with SFTP support) and you have to give the user a account on that machine they can ssh into. (and that is the major limitation of it. You can only give it out to users you can trust enough to give them a shell on your server.. there are ways to secure it up more like using scponly or whatnot, but those have problems of their own)

Sshfs basicly transforms file system requests into sftp requests. Pretty simple and works pretty decently.

The cool thing about sshfs is that since your leveraging OpenSSH you have good security with various authentication mechanisms you can use, and strong encryption. It's about the most secure network file system your going to find.

The downside to it is the shell access thing and the encryption overhead hurts performance over 100Mb/s speeds.

http://fuse.sourceforge.net/sshfs.html

The nice thing about FUSE, of course, is that users can mount and umount volumes without administrator intervention.

List of FUSE file systems:
http://fuse.sourceforge.net/wiki/index.php/FileSystems

Some examples are...
smbfs -- mount umount smb shares.
ntfs-3g -- reliable read/write access to NTFS shares.
encfs -- encrypted directories mounted as file system
gmailfs -- use GMAIL account as a mountable filesystem
gphoto-fuse-fs -- mount gphoto access to your camera as a file system
wdfs -- webdav fuse file system. Combining Apache + Webdav + SVN + wdfs + various webdav clients you can have a version control distributed file system.
wikipediafs -- wikipedia articles show up as files
beaglefs -- mount Beagle (Linux desktop search) search queries as file systems.

All sorts of bizzare things. So you get the idea.


You should be able to easily access SMB shares via sshfs as long as you don't mind giving accounts out to your clients and the server is running a decent version of a SSH server with sftp support. (the stability of the file system depends heavily on the server. Recent versions of Openssh are the best)

 

[xtknight]

Thanks for taking the time to write that up. I'll probably settle on sshfs. Thankfully the Ubuntu repos had it.

As long as the "user" option is present in fstab, can't users mount or unmount any volume anyway?

Thanks again. I'll report back later, hopefully with success.

 

[drag]

Originally posted by: xtknight
Thanks for taking the time to write that up. I'll probably settle on sshfs. Thankfully the Ubuntu repos had it.

As long as the "user" option is present in fstab, can't users mount or unmount any volume anyway?

Ya pretty much. But it still requires a admin to set it up.

Imagine you have something like 20 users that would like to setup a encrypted space for important files, or want remote mounts from their homes. With encfs and sshfs they can do it willy-nilly without having to have you set it up for them. (not that the average user would be able to figure out encfs on his/her own.. but that's a different problem)

 

[xtknight]

Ummmm

So, how do users get access to /dev/fuse? For the life of me I can't get permissions to it.

ls -al /dev
crwxrwxrwx 1 andy fuse 10, 229 2006-08-09 16:26 fuse
cat: /dev/fuse: Operation not permitted

mounts (/ and /dev)
udev on /dev type tmpfs (rw)
/dev/sda2 on / type ext3 (rw,errors=remount-ro)

andy is a member of the fuse group.

Wasn't this supposed to make it easy for users? 😉

andy@ubuntu:/dev$ sudo sshfs andy@localhost:/ /sshfstest -C -r -o cache=yes
That works, but this doesn't. I shouldn't need root right?

andy@ubuntu:/dev$ sshfs andy@localhost:/ /sshfstest -C -r -o cache=yes
andy@localhost's password:
fusermount: mount failed: Operation not permitted

I already had to fix it so users could run fusermount at all. andy is the owner of /sshfstest and the group of /sshfstest is andy. The folder is empty.

ls -al /usr/bin
-rwxr-x--- 1 andy fuse 21384 2006-05-11 13:45 /usr/bin/fusermount

BTW I have done sudo modprobe fuse.
lsmod:
fuse 41112 0

 

[xtknight]

I just don't know. Maybe smbfs is best at this point. I just want something to work. I don't want him to have read/write access, only read-only and specified directories.

 

[drag]

Told ya SMB is the way to go.🙂

But to get FUSE working may depend on your distro.

Right now using Debian Sid all you have to do is add your user to the 'fuse' group. Then you have to logout, and log back in to have the changes make effect.. Group changes only happen on login.
adduser username fuse

But for permissions you need to have /dev/fuse setup correctly and you need to have fusermount command setup correctly.

here is the permissions that I use.
crw-rw---- 1 root fuse 10, 229 Aug 3 01:43 /dev/fuse
-rwsr-x--- 1 root fuse 18688 Jun 13 21:53 /usr/bin/fusermount


I think the only real difference is that fusermount has setuid to root.

edit:
oh the command to do that (I know I rarely have to mess with setuid, so I forget)
sudo chmod u=srwx `which fusermount`

 

[xtknight]

One problem after another...this is quite a rollercoaster.

Both SMB and NFS just cause unexpected freezes on my system. (these are known bugs: for example, Rhythmbox freezes when an SMB mount can't be accessed)

I'll just keep it simple and use ftp. (Expect me to have problems with that and then move to http) 😛

 

[drag]

So.. Are you using a slightly older version of Gnome, maybe?

If your using famd daemon to monitor file system changes in your home directory this can cause hangups with nautilus if you have network shares in there. To fix this you either have to disable famd or replace it with gamin (designed to fix famd's limitations).

This is unlikely, but it caused problems for me for a while.

To avoid real hangups (simply caused by I/O trying to access the shares when the network drops out or whatnot, usually) you can configure automounter to work with nfs or smb. Autofs stuff. Rather then using fstab.

That way the system will automaticly mount a share when you try to access it (also works with other mounts such as cdroms or usb thumb drives and such). I've been told that this solves much of the problems with hangs and such although I haven't used it myself.


edit:

Also it allows you to avoid having to teach your users howto to use mount commands. 🙂

here is a powerpoint-style presentation on it. Has a couple of examples of conifguration files and a smb example. (also it mentions Lufs, which Fuse is kinda outgrowth or replacement for)
http://www.greenfly.org/talks/autofs/autofs.html