My story about package management.

[n0cmonkey]

Why package management matters:
Just recently (today, if you happen to be reading it now) I ran into a small little problem. I needed to enable tcp-wrapper support in SSH on about 40 boxes. Last week. Not a problem, I've got scripts that do half the work for me. I just had to test the configuration and I'm set. Or so I thought.

I spent a couple of hours (in between getting my butt kissed and performing my regularly scheduled duties) testing out simple configurations for tcp-wrappers. The syntax is simple enough even my manager could set it up. hosts.deny was filled with a simple "ALL: ALL" and hosts.allow allowed SSH from one single system. I attempted to SSH in from a non-allowed system, pre-basking in my elite glory, and it connected just fine.

I was shocked. Had I done something incorrectly? It was possible, rare, but possible. I rechecked all configurations, added a few lines, and modified files. Shift 4, r, esc, shift 6, x, :w were all flying across my screen. If one of the other techs was watching over my shoulder he would have had a siezure.

I tried to SSH again, a bit less over confident this time. I connected. A wave of doubt crossed my face as I brought up man page after man page. Was this an error with Linux? Could I rant and rave about how much I dislike the system? Could I praise OpenBSD for not having these issues while Linux users slobber over their keyboards trying to defend their favorite system?

I checked RPM: rpm -qa| grep ssh

I checked the dependencies of the output: rpm --requires openssh-version

libwrap.so.0 was there. This should be working. I double checked the configuration files, they were fine. I checked log files, nothing was there.

I sought help from the Anandtech community, and cleverhandle came to my rescue. ldd /usr/sbin/sshd revealed that libwrap.so.0 was not a requirement for the sshd we were running. Confusion replaced all confidence. Why was RPM lying to me? Linux sucks! Yeah, that's it!

ssh -V Wait, why is this version number different than the one reported by RPM? Of course! Someone replaced the sshd from the RPM with a custom compiled one. Of course, options like tcp-wrappers weren't enabled.

It wasn't Linux, Red Hat, the RPM system, or my perfect configuration skills. No, it was someone that apparently didn't believe in package management.

Not only did I was several hours of off and on work, but now we get to look into a major upgrade process.

If package management is available, use it!





OLD:
hosts.allow and hosts.deny don't seem to affect SSH. I'm wondering if support for tcpwrapper has been included in the SSH port. Anyone know?

EDIT: I figured the above out, now trying to get tcp-wrappers working. I don't remember this being difficult on other platforms...

 

[n0cmonkey]

Answer: rpm -q --requires openssh-server
Output includes libwrap.so.0, so it should be working...

Now, why isn't it working?

 

[cleverhandle]

Hmm... seems strange. It always works in the usual way for me on Linux. You've got something like "sshd: 10.0.0.1" in hosts.allow? Works fine on Debian at least. Maybe RH did something weird on their newer platforms? A PAM module perhaps? They always seem to like those.

 

[n0cmonkey]

Originally posted by: cleverhandle
Hmm... seems strange. It always works in the usual way for me on Linux. You've got something like "sshd: 10.0.0.1" in hosts.allow? Works fine on Debian at least. Maybe RH did something weird on their newer platforms? A PAM module perhaps? They always seem to like those.

I've tried:

/usr/sbin/sshd: ALL
sshd: ALL
ssh: ALL
nonstandard_port: ALL
ALL: ALL
ALL: ALL : DENY




Still googlin' though.

 

[cleverhandle]

Are you certain that a connection is being denied because of tcpwrappers and not, say, because of a built-in iptables firewall?

edit: Oh yeah... "sshd: ALL" is what I have on this machine and it works, so your syntax shouldn't be an issue.

 

[n0cmonkey]

Originally posted by: cleverhandle
Are you certain that a connection is being denied because of tcpwrappers and not, say, because of a built-in iptables firewall?

edit: Oh yeah... "sshd: ALL" is what I have on this machine and it works, so your syntax shouldn't be an issue.

The problem is that the connections are _not_ being denied. I can get in just fine, and that's an issue.

hosts.allow is currently blank, what I have above is from the hosts.deny. I've also tried individual machines, both ip and hostname.

Sorry about the confusion...

 

[cleverhandle]

Seems like we must be missing something simple. ldd /usr/sbin/sshd does show the libwrap dependency, right?

 

[n0cmonkey]

Originally posted by: cleverhandle
Seems like we must be missing something simple. ldd /usr/sbin/sshd does show the libwrap dependency, right?

No, but rpm does.

I just verified from the rpm, and it differs. Someone must have messed with the ssh installation and did it poorly. :|

I hate stuff. :|

Thanks for the help!

 

[cleverhandle]

Definitely a bug in the ssh package, then. Really strange, though. The dependencies in RPM's are found precisely by rpmbuild running ldd on the packaged executables. If sshd was miscompiled without libwrap support, then it must be another binary in the package that depends on libwrap, which seems unlikely. Or maybe the packager put in an explicit dependency, but that's bad form and RH usually has really tight spec files.

 

[n0cmonkey]

Originally posted by: cleverhandle
Definitely a bug in the ssh package, then. Really strange, though. The dependencies in RPM's are found precisely by rpmbuild running ldd on the packaged executables. If sshd was miscompiled without libwrap support, then it must be another binary in the package that depends on libwrap, which seems unlikely. Or maybe the packager put in an explicit dependency, but that's bad form and RH usually has really tight spec files.

rpm gets the dependency lists from the executables instead of the database on the system? That seems unlikely, but I don't know a whole lot about it.

Well the package and the executable are different versions. My former boss didn't uninstall or update the rpm. Instead he compiled a new OpenSSH and installed it on the system. Unfortunately, he didn't use the right configuration. :roll:

 

[cleverhandle]

Originally posted by: n0cmonkey
rpm gets the dependency lists from the executables instead of the database on the system? That seems unlikely, but I don't know a whole lot about it.

That's not what I meant exactly. Yes, when you run rpm -q --requires foo it reads from the database. But that database was created by rpmbuild running ldd on the executables when the packager built them. So the dependencies listed for an rpm are always a superset of the union of the dependencies given by ldd for each executable. Which didn't appear to be the case before because...

Well the package and the executable are different versions. My former boss didn't uninstall or update the rpm. Instead he compiled a new OpenSSH and installed it on the system. Unfortunately, he didn't use the right configuration. :roll:

That would explain the discrepancy. It would also explain why you're using a Red Hat box.

 

[n0cmonkey]

Originally posted by: cleverhandle
That's not what I meant exactly. Yes, when you run rpm -q --requires foo it reads from the database. But that database was created by rpmbuild running ldd on the executables when the packager built them. So the dependencies listed for an rpm are always a superset of the union of the dependencies given by ldd for each executable. Which didn't appear to be the case before because...

Ahh, I misunderstood then.

That would explain the discrepancy. It would also explain why you're using a Red Hat box.

RH is one of the few commercially supported distributions out there. I believe _that's_ why I'm using it.

Ok, mystery solved. I guess I can only perform one miracle per week, and this ain't it.

 

[Sunner]

Bwahahaha!

Yep, people who insist on custom compiling stuff when there are perfectly functional packages available suck.

 

[spherrod]

Package management :thumbsup:

 

[n0cmonkey]

Hehe, today I get to write up some stuff about cleaning this mess up. :beer: