Anyone here have experience with RHEL 7.2?

[XavierMace]

Recently built two RHEL 7.2 VM's at work on different subnets. Downloaded all updates, started prepping for application installs. Found both VM's were having connectivity problems. Couldn't access anything on different subnets, first half dozen pings to the default gateway would fail then start working. At that point I could reach other devices on the same subnet but not other subnets.

No other systems are having this issue, including others on the same host pointing towards this being a VM issue. Restarted network service, connectivity starts working. Reboot server, connectivity broken again. Then I found this thread: https://access.redhat.com/discussions/2173111

Both my VM's are fresh builds from 7.2 installation media, but that otherwise matches my symptoms directly. Anyone else run into this and have a permanent fix?

 

[Fallen Kell]

Might be similar to an issue we ran into, but not quite the same. Under RHEL 7.2 we were not having problems, but upgrading to 7.4 on the physical and all the VMs had issues. Rolling back to 7.2 resolved our issues at least.

Surprisingly, this problem only affected the VMs, we have plenty of systems running 7.4 on the physical box with no problem at all (they do not have any guest VMs on those systems).

 

[mv2devnull]

RHEL 7.2 was released Nov 2015, RHEL 7.3 was released Nov 2016, RHEL 7.5 was released Apr 2018.

I run CentOS and do have VM's created during the 7.2 period, without real issues. IIRC, there have been significant (for me) changes on the point updates. At least the NetworkManager has got more features.

Was there a specific reason to install RHEL 7.2 and not RHEL 7.5?

 

[XavierMace]

RHEL 7.2 was released Nov 2015, RHEL 7.3 was released Nov 2016, RHEL 7.5 was released Apr 2018.

I run CentOS and do have VM's created during the 7.2 period, without real issues. IIRC, there have been significant (for me) changes on the point updates. At least the NetworkManager has got more features.

Was there a specific reason to install RHEL 7.2 and not RHEL 7.5?

That was the ISO I was provided at the time. I found a 7.3 ISO, built another box using that, and it's having the same issue. Unlikely they'd go with something as recent as April as they consider that too big a risk of running into unexpected bugs.

That said, it seems I've found the issue, it's just not clear why it was the issue. I did straight GUI installs as Linux isn't my thing. GUI only shows the one network adapter. Command line had a different story. It looks like it created bridge adapters for using the box as a virtualization host. No idea if hypervisor functionality is part of the default install, but we're not using it. But it appears it likes sending out traffic using the bridge adapter instead of the actual active NIC. Disabled the virtualization service and the networking problem seems to have gone away.

 

[mv2devnull]

GUI only shows the one network adapter. Command line had a different story. It looks like it created bridge adapters for using the box as a virtualization host. No idea if hypervisor functionality is part of the default install, but we're not using it. But it appears it likes sending out traffic using the bridge adapter instead of the actual active NIC. Disabled the virtualization service and the networking problem seems to have gone away.

The installer shows only one adapter, if there is only one adapter. Network even on it might remain disabled, unless explicitly enabled. (Funny default that.)

Regular "Desktop" package set (at least on CentOS) includes libvirt (aka Virtualization). The install of libvirt configures "Default" virtual network (that is "routed, with NAT") for VM's to use. The libvirtd service is enabled by default and the Default network is set to start when the libvirtd starts. Yes, stop and disable them (or preferably uninstall / never install), if you don't need them.

The Default network is indeed a bridge that is not connected to any "physical NIC". It is a virtual network switch for the VM's to link to. The host runs a 'dnsmasq' process to act as DHCP and DNS for the VM's. There should not be default route on the host via that bridge.


RHEL 7.4 was released Aug 2017.

 

[XavierMace]

Yeah, I understand the point of the virtual network. I just don't understand why it would be trying to spontaneously trying to route traffic over it, especially when there are no VM's and why restarting the network service would fix it until the next reboot. It's a 100% repeatable behavior on our boxes (and in that linked thread).

I personally would still expect the bridge adapter to show up in the network GUI however.

 

[mv2devnull]

I just don't understand why it would be trying to spontaneously trying to route traffic over it ...

It should not. The only route entry that results from setting up the bridge by libvirtd ought to be the link-local route. The bridge interface is essentially "LAN-port of home router", and the VM's are hooked to that "home LAN". (I don't generally use those libvirt-defined networks, but where I had one, it was behaving.)

It would actually be interesting to see what a system has when it is in that error state.

systemctl status network.service
systemctl status NetworkManager.service
nmcli
nmcli d s
nmcli c s
ip -4 ad
ip ro

I personally would still expect the bridge adapter to show up in the network GUI however.

GUI of the Anaconda installer, or GUI of already installed system? Personally, I don't use GUI (except 'nmtui' on rare occasion).

There are both network.service and NetworkManager.service. Both can in principle be active simultaneously, i.e. some connections managed by network scripts and others by NM. IMHO that is asking for trouble. I have disabled network.service and do everything via NM (mostly with 'nmcli', but installer's kickstart does most of the work). The other option would be to uninstall all NetworkManager packages that you can and remain with the network.service. (That's what I do in CentOS 6, where NM does not cut it yet.)

restarting the network service would fix it until the next reboot.

The "restart" runs "stop" and "start". If that service wasn't properly up by boot, you got it started now. Obviously, that does not solve the problem.

 

[XavierMace]

I know one of the other guys who was looking at it checked the status, but I'll try to check it next time I'm working on it. I'm currently waiting for the powers that be to give me an updated ISO so I can try rebuilding one of the boxes.

I'd expect the Network Adapter GUI on the installed OS to show the bridge adapter. I know not using GUI is the Linux norm, but the GUI should still be providing you the same info as the CLI.

 

[mv2devnull]

the GUI should still be providing you the same info as the CLI.

Indeed.

Oh my, just peeked into Gnome settings -- network (CentOS 7.5).
Three physical NIC's listed. Check.
Five VLAN's listed. Check.
.. and that's it. Properties of those connections might have, hidden in dropdown menu's, something.

So much for GUI, enter CLI:
* The three NIC's are enslaved to a bond (whose name you can see in the GUI, if you know what you see)
* Four bridges ... wait 5 bridges and one more VLAN; one of the bridges and VLAN's are configured, but disabled.


That is the nature of GUI: someone, somewhere decides what is relevant to show. They have to. And then they choose to not provide views to everything (or do their best to hide). That is not a Linux-specific issue. That is GUI; Microsoft, Google, and Apple all do that too. CLI, on the other hand is not "choose from these". It is "type what you will and I'll parse what I can".

I've seen the opposite on a HP ProCurve switch. The (Java-based) GUI did allow changing some details. The CLI (via ssh/telnet) did not. You could see, but not touch -- and it was trivial data.