Ubuntu Server 5.10/Linux Question...

[GTaudiophile]

I have Ubuntu Server 5.10 installed on a machine and it works just great.

Uptime: 172 days, 19:06 hours.

What I want to do is install a faster (gigbit) NIC into it. As I have never does this before, what does Linux do when you take out one piece of hardware and install a new piece? I mean, how does it react in terms of drivers? What configuration changes would I need to make?

In the past, all of my hardware upgrades have been between OS installations, but I would prefer not to do that this time.

 

[n0cmonkey]

At most you'll just have to load another module. There should be a modules.conf or something similar in /etc.

 

[GTaudiophile]

Originally posted by: n0cmonkey
At most you'll just have to load another module. There should be a modules.conf or something similar in /etc.

Hmmm. Never done that before. What does that do? And how would one run it?

 

[n0cmonkey]

Originally posted by: GTaudiophile

Originally posted by: n0cmonkey
At most you'll just have to load another module. There should be a modules.conf or something similar in /etc.

Hmmm. Never done that before. What does that do? And how would one run it?

I haven't done it in a long time. You just add the module to the modules.conf (or whatever ubuntu calls it) file and it should be loaded the next time you boot. You can test it out with modprobeing for the correct module.

 

[GTaudiophile]

Originally posted by: n0cmonkey

Originally posted by: GTaudiophile

Originally posted by: n0cmonkey
At most you'll just have to load another module. There should be a modules.conf or something similar in /etc.

Hmmm. Never done that before. What does that do? And how would one run it?

I haven't done it in a long time. You just add the module to the modules.conf (or whatever ubuntu calls it) file and it should be loaded the next time you boot. You can test it out with modprobeing for the correct module.

Are you 100% sure I'll have to do this or just in an extreme case?

 

[Brazen]

In Redhat derivatives, you just run kudzu and it detects and configures your new hardware automatically. Ubuntu may have a similar program, but I'm almost positive kudzu is Redhat-only.

 

[n0cmonkey]

Originally posted by: GTaudiophile
Are you 100% sure I'll have to do this or just in an extreme case?

I'm not 100% sure. It could depend on the chipset of the gigabit card. It could depend on Ubuntu's kernel driver choices. I haven't used Ubuntu, and I haven't had to add things to a modules.conf file in a long long time. Sorry.

It shouldn't be too tough though. What chipset is your new gigabit card using?

 

[Nothinman]

Kudzu is crap, udev/hotplug will detect and load the driver without any need for extra programs. You shouldn't have to edit anything as long as the NIC has a driver. The only problem you might run into is interface naming, if you have another interface (something odd that presents a network interface like a firewire port) the order of the names might change, I'm not 100% sure if Ubuntu will work around that.

 

[n0cmonkey]

Originally posted by: Nothinman
Kudzu is crap, udev/hotplug will detect and load the driver without any need for extra programs. You shouldn't have to edit anything as long as the NIC has a driver. The only problem you might run into is interface naming, if you have another interface (something odd that presents a network interface like a firewire port) the order of the names might change, I'm not 100% sure if Ubuntu will work around that.

Unless you add or remove a network device, that should never happen.

 

[Nothinman]

That's what he's asking about, he specifically says "what does Linux do when you take out one piece of hardware and install a new piece?".

But it can happen during normal operation too if you have more than one NIC. Module loading is asynchronous so the names aren't guaranteed to be consistent across reboots. udev has the ability to rename the interfaces before they're brought up and I know the Debian (and I assume Ubuntu) udev packages set that up on installation, but they use the MAC so the rules won't apply to the new NIC and I'm not sure if it updates the rules so some manual intervention might be required.

 

[n0cmonkey]

Originally posted by: Nothinman
That's what he's asking about, he specifically says "what does Linux do when you take out one piece of hardware and install a new piece?".

Which is different from what I said, but not really by much.

But it can happen during normal operation too if you have more than one NIC. Module loading is asynchronous so the names aren't guaranteed to be consistent across reboots.

Sounds like a bug. That should never happen.

udev has the ability to rename the interfaces before they're brought up and I know the Debian (and I assume Ubuntu) udev packages set that up on installation, but they use the MAC so the rules won't apply to the new NIC and I'm not sure if it updates the rules so some manual intervention might be required.

Sounds like a hack.

 

[Nothinman]

Sounds like a bug. That should never happen.

No, it's how the module loader was designed.

Sounds like a hack.

From the perspective of the people who designed the module loader I'm sure that making the module loader synchronous just to avoid that is seen equally as hacky and it would slow down boot by quite a bit.

 

[n0cmonkey]

Originally posted by: Nothinman

Sounds like a bug. That should never happen.

No, it's how the module loader was designed.

Sounds like a hack.

From the perspective of the people who designed the module loader I'm sure that making the module loader synchronous just to avoid that is seen equally as hacky and it would slow down boot by quite a bit.

Sounds like a bad design. Things should be consistant. I haven't noticed any slow boot times on systems that have a consistant design, in fact I was recently surprised at how fast one of these systems booted on an older machine I installed it on.

 

[Nothinman]

Sounds like a bad design. Things should be consistant. I haven't noticed any slow boot times on systems that have a consistant design, in fact I was recently surprised at how fast one of these systems booted on an older machine I installed it on.

In 99% of the cases the design is fine. Ethernet and hard disks are the only things that might get name changes that you would care about and there are other things that might cause them to change too, so even with a synchronous module loader you're not guaranteed 100% consistent names. And there are identifiers for each of those mediums (MAC address, filesystem labels, etc) that should be used to protect against that.

 

[n0cmonkey]

Originally posted by: Nothinman

Sounds like a bad design. Things should be consistant. I haven't noticed any slow boot times on systems that have a consistant design, in fact I was recently surprised at how fast one of these systems booted on an older machine I installed it on.

In 99% of the cases the design is fine. Ethernet and hard disks are the only things that might get name changes that you would care about and there are other things that might cause them to change too, so even with a synchronous module loader you're not guaranteed 100% consistent names. And there are identifiers for each of those mediums (MAC address, filesystem labels, etc) that should be used to protect against that.

A better design wouldn't require labels or naming cards based on their MAC addresses. Proper probing should keep the names from changing, except when something is added or removed (not switched).

And I haven't gotten labels to work except in the default install of RHES Linux. When I update the kernels on some of these systems I have to change the LABEL back to the proper drive/partition. It's probably a RH thing, using their updater to update the kernel should "just work." Just thought it was funny. 😛

 

[Nothinman]

A better design wouldn't require labels or naming cards based on their MAC addresses. Proper probing should keep the names from changing, except when something is added or removed (not switched).

In the kernel's view something is added, on bootup udev 'coldplugs' all of the hardware. And how should the kernel keep the names from changing? It can't keep a cache anywhere with what they were called last time. USB drives are a good example because they show up as SCSI devices, if you have a SCSI controller in your machine and you bootup with the USB drive plugged in as well /dev/sda will be given to the drive that's found first. If the USB stuff loads ahead of the SCSI HBA driver you're screwed. It's not really likely to happen because if you have a SCSI controller chances are you're booting off of it so it's driver will be loaded in an initrd before anything else or statically linked into the kernel so it'll always load before the USB stuff, but it is possible.

Same thing for NICs, booting up with a PCMCIA or USB NIC plugged in could cause the devices to be seen in a different order than if you plugged them in after the system was up. How would you propose that the kernel defend against that?

And I haven't gotten labels to work except in the default install of RHES Linux. When I update the kernels on some of these systems I have to change the LABEL back to the proper drive/partition. It's probably a RH thing, using their updater to update the kernel should "just work." Just thought it was funny.

Yea, it's a RH thing but I've only ever seen the boot loader root=LABEL=/ part break. IIRC the root=LABEL=/ thing is done in their initrd, not the kernel, so if you use a non-RH kernel that'll probably stop working. The rest of the label parsing of /etc/fstab should all work fine. One nice thing that udev does (at least on Debian) is it creates links in /dev/disk/by-label so you can mount the filesystems by their label without using the LABEL= syntax.

 

[n0cmonkey]

Originally posted by: Nothinman

A better design wouldn't require labels or naming cards based on their MAC addresses. Proper probing should keep the names from changing, except when something is added or removed (not switched).

In the kernel's view something is added, on bootup udev 'coldplugs' all of the hardware. And how should the kernel keep the names from changing? It can't keep a cache anywhere with what they were called last time. USB drives are a good example because they show up as SCSI devices, if you have a SCSI controller in your machine and you bootup with the USB drive plugged in as well /dev/sda will be given to the drive that's found first. If the USB stuff loads ahead of the SCSI HBA driver you're screwed. It's not really likely to happen because if you have a SCSI controller chances are you're booting off of it so it's driver will be loaded in an initrd before anything else or statically linked into the kernel so it'll always load before the USB stuff, but it is possible.

Same thing for NICs, booting up with a PCMCIA or USB NIC plugged in could cause the devices to be seen in a different order than if you plugged them in after the system was up. How would you propose that the kernel defend against that?

As I said before, as long as no hardware has changed it should always boot up with the same name. If you switch out one NIC for another NIC (putting it in the same slot) it should appear to be the original NIC. If you start adding PCMCIA or USB devices you're changing the hardware which will obviously change things.

Just use a different chipset for the USB/PCMCIA device and you don't have to worry about it changing the name of your PCI devices. Oh, wait, it's all eth on Linux... Too bad. 😉

Yea, it's a RH thing but I've only ever seen the boot loader root=LABEL=/ part break. IIRC the root=LABEL=/ thing is done in their initrd, not the kernel, so if you use a non-RH kernel that'll probably stop working. The rest of the label parsing of /etc/fstab should all work fine. One nice thing that udev does (at least on Debian) is it creates links in /dev/disk/by-label so you can mount the filesystems by their label without using the LABEL= syntax.

Yeah, that's the part that breaks, but I'm using official kernels downloaded through the RHN. 😕 The fstab labels still work just fine. It's weird, but easy to fix.

 

[drag]

The way you keep interface names from changing would be to edit/create udev rules.
http://reactivated.net/writing_udev_rules.html

Otherwise device names can change on bootup for odd reasons. Maybe it's slightly random. Also different kernels or changes system boot up scripts may detect hardware in different ways, causing changes.

I understand that with OpenBSD and such you get different device names based on the driver, which is helpfull if you have networking cards with different manufacturers/chipsets, but I don't think it would be helpfull if you had, say, 3 Intel multi port PCI nic cards. It would get into a big mess quickly.

And Linux is being used in fairly large machines.. You could have a half a dozen or more drive controllers in a machine nowadays with many harddrives and many different I/O interfaces of various types.

Also you have freaks like me that have at least 4 different USB input devices that I use and plug in at random intervels.

 

[n0cmonkey]

Originally posted by: drag
The way you keep interface names from changing would be to edit/create udev rules.
http://reactivated.net/writing_udev_rules.html

Otherwise device names can change on bootup for odd reasons. Maybe it's slightly random. Also different kernels or changes system boot up scripts may detect hardware in different ways, causing changes.

That's just bad though. Device naming should be consistant through reboots.

I understand that with OpenBSD and such you get different device names based on the driver, which is helpfull if you have networking cards with different manufacturers/chipsets, but I don't think it would be helpfull if you had, say, 3 Intel multi port PCI nic cards. It would get into a big mess quickly.

Why would that be messy?

 

[drag]

That's just bad though. Device naming should be consistant through reboots.

That's HOW it stays consistant between reboots. You should be able to take the cards out, move them around and they'd still stay consistant.

Why would that be messy?

Because device names are generally granted on a ad-hoc basis. First come first serve. Now this is computer hardware and software and it will stay consistent between reboot just because thats how it works. But between kernel changes, driver updates, userland updates, bios updates, and changing hardware this stuff can't be expected (as in 90-100%) to always be a reliable way to determine which port gets the paticular 'eth*' address over the long term.

So if you have 3 4 port intel nic cards what does that get you with OpenBSD (for instance). int* devices or something?
int0
int1
int3
int4
int5
int6
int7
int8
int9
int10
int11
int12

Something like that? How do you keep track of those with a BSD system?

I know with modern Linux I just write udev rules for device naming based on their mac addresses (or other mechanism depending on what I want)... That way if I add or remove cards or have to work on the computer and put them back together differently or whatnot then they'll remain consistant..
KERNEL=="eth*",SYSFS{address}=="00:0e:2e:57:22:23",NAME="eth0"
KERNEL=="eth*",SYSFS{address}=="00:50:b4:01:93:3f",NAME="eth1"
KERNEL=="eth*",SYSFS{address}=="00:13😀4:ef:4B:Ec",NAME="eth2"
etc etc.

This sort of thing is based on hardware information from /sys. (which is lowercase, which is different from ifconfig, kinda stupid)

I know it's not perfect, but it's better then it was just a years or so ago.

So If I have this big firewall/router thingy built with all these pf or iptables and dynamic routing stuff then it should, theoreticly, work consistantly for the life of the OS/hardware. Otherwise if I decide to add or remove a card or change something in the kernel or drivers that affect hardware detection then that would be a mess to fix.

 

[Nothinman]

As I said before, as long as no hardware has changed it should always boot up with the same name.

Generally that's what happens. But since they all get added asynchronously they 'race' for the names and there is a slim chance that something odd will happen and the names will get swapped.

Just use a different chipset for the USB/PCMCIA device and you don't have to worry about it changing the name of your PCI devices. Oh, wait, it's all eth on Linux... Too bad.

Actually the device naming in Linux good thing, I know that my ethernet card is designaged eth# no matter what hardware I have in the system. But if you want to name them something else you can with ifrename or udev rules. My wifi card comes up as ra0 but I have udev rename it to wlan0 because that's what I was accustomed to with my old card.

Yeah, that's the part that breaks, but I'm using official kernels downloaded through the RHN. The fstab labels still work just fine. It's weird, but easy to fix.

Well I don't think any of our RHEL boxes have had that problem so I don't know what to say other than PEBCAK. =)

That's just bad though. Device naming should be consistant through reboots.

The Linux kernel guys don't want device naming policy in the kernel, hence udev. And with udev it's possible to have consistent device names through more than just reboots.

Why would that be messy?

It wouldn't necessarily be messy. At least no more messy than having 3 NICs in a Linux box named eth0-2.

 

[n0cmonkey]

Originally posted by: drag
Because device names are generally granted on a ad-hoc basis. First come first serve. Now this is computer hardware and software and it will stay consistent between reboot just because thats how it works. But between kernel changes, driver updates, userland updates, bios updates, and changing hardware this stuff can't be expected (as in 90-100%) to always be a reliable way to determine which port gets the paticular 'eth*' address over the long term.

Switching hardware will cause issues, but changing the kernel shouldn't (unless they change the oder things are probed in, which would be stupid).

So if you have 3 4 port intel nic cards what does that get you with OpenBSD (for instance). int* devices or something?
int0
int1
int3
int4
int5
int6
int7
int8
int9
int10
int11
int12

Something like that? How do you keep track of those with a BSD system?

em for gigabit and it'd be 0-11, but you have the idea. 😉

You keep track of them by what network they are assigned to. You can also assign the interface to a group, but I don't know how useful that is for human identification tasks.

I know with modern Linux I just write udev rules for device naming based on their mac addresses (or other mechanism depending on what I want)... That way if I add or remove cards or have to work on the computer and put them back together differently or whatnot then they'll remain consistant..
KERNEL=="eth*",SYSFS{address}=="00:0e:2e:57:22:23",NAME="eth0"
KERNEL=="eth*",SYSFS{address}=="00:50:b4:01:93:3f",NAME="eth1"
KERNEL=="eth*",SYSFS{address}=="00:13😀4:ef:4B:Ec",NAME="eth2"
etc etc.

You shouldn't have to move cards around. Just put them back in the same order you'll be fine.

This sort of thing is based on hardware information from /sys. (which is lowercase, which is different from ifconfig, kinda stupid)

I know it's not perfect, but it's better then it was just a years or so ago.

So If I have this big firewall/router thingy built with all these pf or iptables and dynamic routing stuff then it should, theoreticly, work consistantly for the life of the OS/hardware. Otherwise if I decide to add or remove a card or change something in the kernel or drivers that affect hardware detection then that would be a mess to fix.

The hardware detection in the kernel shouldn't change that much really. The order is fairly set, and with the BSD evolution philosophy it shouldn't change.

If you add a card you should put a little thought into it. Start with your interfaces at the beginning of the PCI bus, add new stuff after those ones. They should appear after the established pieces of hardware.

 

[n0cmonkey]

Originally posted by: Nothinman

As I said before, as long as no hardware has changed it should always boot up with the same name.

Generally that's what happens. But since they all get added asynchronously they 'race' for the names and there is a slim chance that something odd will happen and the names will get swapped.

I like the idea of probing the devices in order so they always appear the same.

Actually the device naming in Linux good thing, I know that my ethernet card is designaged eth# no matter what hardware I have in the system. But if you want to name them something else you can with ifrename or udev rules. My wifi card comes up as ra0 but I have udev rename it to wlan0 because that's what I was accustomed to with my old card.

I still hate that.

Well I don't think any of our RHEL boxes have had that problem so I don't know what to say other than PEBCAK. =)

Not a surprise. Linux frustrates me constantly. I think Linus gets a kick out of it. 😛

 

[drag]

If you add a card you should put a little thought into it. Start with your interfaces at the beginning of the PCI bus, add new stuff after those ones. They should appear after the established pieces of hardware.

"Should" is the correct word for it.

PCI cards usually work out that way.. But what about USB?

Theoreticly Linux should be able to support hotplug everything. From disks (SATA hotplugs) to PCIe cards (which are hotpluggable to). (as well as other things like CPU's and RAM, but obviously they don't have device names.)

Now you don't even have consistant naming during reboot because now you don't even have to reboot to add hardware!

This is relatively new stuff. Libata (linux sata driver stuff) didn't support hotplug until very very recently. (and it still isn't merged into mainline yet)

 

[n0cmonkey]

Originally posted by: drag

If you add a card you should put a little thought into it. Start with your interfaces at the beginning of the PCI bus, add new stuff after those ones. They should appear after the established pieces of hardware.

"Should" is the correct word for it.

PCI cards usually work out that way.. But what about USB?

USB is a bit trickier apparently, but I can't think of a single USB device I'd consistantly use on a server. Maybe a GPS device.

Theoreticly Linux should be able to support hotplug everything. From disks (SATA hotplugs) to PCIe cards (which are hotpluggable to). (as well as other things like CPU's and RAM, but obviously they don't have device names.)

Now you don't even have consistant naming during reboot because now you don't even have to reboot to add hardware!

This is relatively new stuff. Libata (linux sata driver stuff) didn't support hotplug until very very recently. (and it still isn't merged into mainline yet)

I like OpenBSD's hotplug. It's independant of the other subsystems. I had it setup to mount a CF card and bring up my wireless CF card in my zaurus. Haven't messed with it much lately though. 😛

Oh well, to each their own. :beer:😀