Server NFS randomly stopped working

Red Squirrel

No Lifer
May 24, 2003
71,312
14,085
126
www.anyf.ca
So I had to reboot my server the other day for a power switch over. Other than having to fight with a few things when I started it back up everything was working within a few hours. Typically when that server is shut down something breaks, so it was to be expected.

Today, it refuses to mount ANY NFS volume. If I try to mount anything it just says:
mount.nfs: access denied by server while mounting

Also if I use showmount on the server and input the clients IP this is what I get:

[root@borg ~]# showmount -e 10.1.2.10
showmount: RPC: Program not registered
[root@borg ~]#

The server refuses to accept any mounts from any other server. I can mount shares off other servers fine from those clients. When I google stuff it tells me to make sure portmap is running but I dont see anything called that. The command is not even recognized and if I use yum to try installing it, nothing happens. I don't understand why something that worked one hour ago suddently would stop working.

I know the raid array is up, as that was originally my first hunch, that I may have lost two drives. I would have also gotten an alert for that anyway. The array is fine, it's just the NFS shares that refuse to allow any client to mount. NFS does not seem to have much logging. The only thing I see is in /var/log/messages:

Jan 18 17:18:01 borg mountd[11115]: authenticated mount request from 10.1.2.10:997 for /raid1/backups (/raid1/backups)

One entry per share. I also tried forcing version 1,2 and 3 on the client, no go. The server is very old, it's running Fedora Core 9, so it does not have 4. Interestingly, if I try to force version 4 I get a different error, saying that the directory is not found (which is not true, it's there). But I don't imagine that server is running 4 anyway. I never had to force any version before the reboot, everything just worked.

I was planing to move that raid array to my new file server anyway, but that is not something I wanted to be doing this weekend so I'm just wondering if there's any way I can make this work, or if I'm out of luck.
 

Red Squirrel

No Lifer
May 24, 2003
71,312
14,085
126
www.anyf.ca
Ended up having to reboot the entire server. I don't know what the hell is up with that... I really need to get stuff off that server as soon as I can... now DNS is hosed up. Probably NTP as well, those were the two things I had to fight with when I rebooted it a few days ago.

SSH logons also don't work very well when DNS is down so this can get pretty serious. I did managed to get in though.

Part of the issue is it seems a lot of distros have lot of crap that overwrites /etc/resolv.conf so I end up having to reconfigure DNS whenever I reboot. I wish those settings would just stick.

Hopefully this old server can last a few more weeks till I can move the array over to the new box. Physically going to be moving all the drives so it's a fairly big move as I also need to document which bay each one is in (so I know which one to pull out when I get a failure).
 
Last edited: