How many driver "levels" are there?

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
Hello everyone,

Let me explain what I mean by detailing an experience I had a few weeks ago. Bear with me.

I had a new laptop that came with Windows 8.1 from the factory. I wanted to run Windows 7 on it instead. So, I took a spare laptop HD I had and plugged it into another desktop computer (I had to do this because the motherboard in the new laptop that I ultimately wanted to use prevented me from installing another OS. But, that's besides the point.) I had a Windows 7 image burned to a bootable USB, and installed Windows 7 on my spare HD through the motherboard of my second desktop computer.

There were no problems. I then detached the HD and placed it into the laptop that I wanted to use. I booted up, and the splash screen pops up. Maybe 2 seconds later it blue screens on me. I tried again and same thing.

I learned from asking members on the OS sub-forum that you can't just swap HDs that way. During installation, bundled with the OS comes a set of "low level" drivers that are designed to communicate with the chip set on the motherboard. I'm assuming that these drivers allow the OS to communicate with low level hardware like the interrupt controller, the DMA controller, the memory management unit, etc.

I came up with a clever work around to my dilemma by using the second desktop computer to copy the installation files from the USB to the HD and then plugging the HD into the laptop and continuing with the installation.

Finally, when everything was setup, I had to hunt down the drivers for "high level" ancillaries e.g. the wireless network device, the SD card reader, the Bluetooth transceiver, etc.

OK. I offered that long-winded account to say that I realized that there is more than one driver "level". (That's probably not the right term, but bear with me.)

Then, I started thinking about the BIOS. So, I opened up my computer, detached my HD, and started up my computer. I went into BIOS and everything worked just fine. The keyboard, monitor, and mouse worked (my computer has a GUI bios).

As the HD was detached, there is no way that the drivers and interrupt handlers associated with the keyboard, monitor, and mouse operation were being evoked from the HD. Further, the BIOS gives you the option of booting from the USB, ODD, or NIC. So, there must be drivers and interrupt handlers associated with all those devices stored somewhere on the motherboard.

So, with all that said, my final conclusion is that there are 3 levels of drivers: the low-level stored somewhere on the motherbaord, the mid-level that comes bundled with the OS upon installation, and the high-level that my also come with OS, but not bundled with it, from the factory but may need to be downloaded by the user.

What do you think about my perspective? I'm pretty sure that my view of things constitutes a gross over simplification or is quite possibly completely wrong.

I would appreciate any input from you guys to help me better understand this seemingly multi-tiered I/O paradigm.

Thanks.
 

Ken g6

Programming Moderator, Elite Member
Moderator
Dec 11, 1999
16,660
4,602
75
So, with all that said, my final conclusion is that there are 3 levels of drivers: the low-level stored somewhere on the motherbaord, the mid-level that comes bundled with the OS upon installation, and the high-level that my also come with OS, but not bundled with it, from the factory but may need to be downloaded by the user.
I think you're probably pretty close. When an interrupt happens, it probably calls the BIOS. (When you say you "went into the BIOS", what you mean is you went into the BIOS setup utility. Which is software running on top of the BIOS.)

The BIOS calls the OS, and then the OS calls all the programs that have registered with the OS to get calls when that particular interrupt happens.
 

Gryz

Golden Member
Aug 28, 2010
1,551
204
106
I don't think there are several levels of drivers.

There are several layers of abstraction. That you should never forget.

I think what you encountered is the difference between standard minimal guaranteed functionality of hardware, and the expanded capabilities. Inside a computer, the different components use protocols to communicate. Just like computers use network protocols to communicate with other computers. Examples of these internal protocols are SATA to talk to disks, VGA to talk to monitors, USB to talk to mice and keyboards, etc.

There internal protocols are not written in stone. They are not forever. They change all the time. SATA went to SATA2 and then to SATA3, etc. All HDDs will do SATA, but not all will do SATA3. VGA assumes that all monitors can do at least 640x480/16-bit. Most monitors can do better, but not all. All regular hardware in a PC will be able to communicate in this basic mode for their protocol.

When a PC boots, software is ran that is loaded from the BIOS. This software will have minimal support for the hardware that is needed (monitor, keyboard, hdd, maybe mouse). This software only uses the minimal functionality that is guaranteed to work. This is enough to get going.

But if the OS wants to make full use of all capabilities of the hardware, and wants to use all new features of new protocols (like SATA3, USB3.0, etc), then the OS needs to use drivers that know about all those latest features of new protocols. But also those drivers need to know about all kinds of intricacies of the chipsets involved. And those intricacies are probably very vendor-specific.

So what happens when you boot your PC ? Let's look at the HDD. The BIOS might be able to talk only SATA. It will use SATA to look at the HDD, find the OS installed, and "boot" the OS (in other words: it will load the initial part of the OS in memory, and then it will give control to the OS). The OS will maybe come with SATA2 drivers for many SATA-chipsets. It will look at the indentifier for the SATA-chip, and if it recognizes it (from a list that comes with the OS), it will load the SATA2-driver for that chipset. If it doesn't recognize the SATA-chipset, it might load a generic SATA1-driver with minimal capabilities. Note, it doesn't load 2 drivers. Only 1. The best (most efficient) driver it can find.

You as user might know that the HDD can do SATA3 too. So you download the drivers from the vendor's website. And install it in your OS. Next time you reboot your PC, when the OS looks for a better driver than the minimal one, it will find both the standard SATA2 that came with the OS, and the SATA3 that you installed yourself. It will then load the SATA3-driver, and forget about the SATA2-driver.

Now why did you get the problem you got ?

When you install an OS, and then install drivers for particular pieces of hardware, those drivers will be specific for the chipsets on your motherboard, and the hardware installed in your PC. Those drivers are tailored to that specific hardware. And they use more features than the minimal feature-set. Now you take that HDD out of your PC, and put it in another PC, with another motherboard and other devices. Result: the chipset-drivers might try to do stuff that is not supported by the hardware. They might try to access registers on specific memory-locations on the bus to access the hardware. But if those registers are not there, you will get an error (e.g. bus error, or memory violation).

Many drivers for devices like mice, keyboard, monitors, etc, will check to see what devices are connected. So that won't cause problems. The OS will detect at boottime that stuff has changed. Newer protocols like USB have been designed to be able to do this.

But for chipsets on the motherboard it is different. The OS is probably capable of detecting changes in HDDs, SSDs, maybe even videocards. But chipsets are a different thing. Chipsets are not supposed to change. Only when you change the mobo, which hardly ever happens. And I guess it's harder to check for motherboard-compatibility. And therefor it often goes wrong when you swap in another mobo.

So no, there are not layers of drivers. It's just that during boottime, simple drivers get replaced by more elaborate drivers. Once the OS has fully booted, only 1 driver (per device) is running.

From the BIOS wiki-page:
In modern personal computers running modern operating systems the BIOS is used only during booting and initial loading of system software.
 

Cogman

Lifer
Sep 19, 2000
10,284
138
106
Gryz has it right.

But just to add, it is really hard to speak about driver "levels" on an OS because different OSes do different things when it comes to drivers.

Take DOS, for example, it pretty much didn't have drivers. Rather, there were understood protocols which every application and device was expected to follow. You printed a document by writing directly to the serial/parallel port, it was up to the individual application to do that "right", often by pulling in libraries from the manufacturers (drivers) but sometimes by rolling their own.

Another classic example of this is the setup process for old dos games. Often, you would have to, per game, pick out the sound card, the PCI port, and everything from within the game itself!
 

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
I think you're probably pretty close. When an interrupt happens, it probably calls the BIOS.

So, you're saying that the BIOS runs continuously throughout the computer's operation and handles all I/O?

I think that used to be the case, but nowadays I think that the BIOS plays a more limited role after the OS has been booted.

(When you say you "went into the BIOS", what you mean is you went into the BIOS setup utility. Which is software running on top of the BIOS.)

OK.

So, the BIOS runs two programs: one that performs a check on I/O devices attached to the computer and then pulls the boot-loader into RAM and one, the setup utility program, that lets you set the parameters for the first program.

Yes?

This code is obviously not stored on the HD.

It's all stored on the BIOS chip, right?

The BIOS calls the OS, and then the OS calls all the programs that have registered with the OS to get calls when that particular interrupt happens.

Again, is this how it stills works?


I'm familiar with the paradigm, but the paradigm only makes sense once the OS has been loaded.

In BIOS world, there is no kernel.

Take DOS, for example, it pretty much didn't have drivers.

At that time, didn't the BIOS have control over I/O even after the OS was loaded?
 

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
I don't think there are several levels of drivers.

There are several layers of abstraction. That you should never forget.

I think what you encountered is the difference between standard minimal guaranteed functionality of hardware, and the expanded capabilities. Inside a computer, the different components use protocols to communicate. Just like computers use network protocols to communicate with other computers. Examples of these internal protocols are SATA to talk to disks, VGA to talk to monitors, USB to talk to mice and keyboards, etc.

There internal protocols are not written in stone. They are not forever. They change all the time. SATA went to SATA2 and then to SATA3, etc. All HDDs will do SATA, but not all will do SATA3. VGA assumes that all monitors can do at least 640x480/16-bit. Most monitors can do better, but not all. All regular hardware in a PC will be able to communicate in this basic mode for their protocol.

When a PC boots, software is ran that is loaded from the BIOS. This software will have minimal support for the hardware that is needed (monitor, keyboard, hdd, maybe mouse). This software only uses the minimal functionality that is guaranteed to work. This is enough to get going.

But if the OS wants to make full use of all capabilities of the hardware, and wants to use all new features of new protocols (like SATA3, USB3.0, etc), then the OS needs to use drivers that know about all those latest features of new protocols. But also those drivers need to know about all kinds of intricacies of the chipsets involved. And those intricacies are probably very vendor-specific.

So what happens when you boot your PC ? Let's look at the HDD. The BIOS might be able to talk only SATA. It will use SATA to look at the HDD, find the OS installed, and "boot" the OS (in other words: it will load the initial part of the OS in memory, and then it will give control to the OS). The OS will maybe come with SATA2 drivers for many SATA-chipsets. It will look at the indentifier for the SATA-chip, and if it recognizes it (from a list that comes with the OS), it will load the SATA2-driver for that chipset. If it doesn't recognize the SATA-chipset, it might load a generic SATA1-driver with minimal capabilities. Note, it doesn't load 2 drivers. Only 1. The best (most efficient) driver it can find.

You as user might know that the HDD can do SATA3 too. So you download the drivers from the vendor's website. And install it in your OS. Next time you reboot your PC, when the OS looks for a better driver than the minimal one, it will find both the standard SATA2 that came with the OS, and the SATA3 that you installed yourself. It will then load the SATA3-driver, and forget about the SATA2-driver.

Now why did you get the problem you got ?

When you install an OS, and then install drivers for particular pieces of hardware, those drivers will be specific for the chipsets on your motherboard, and the hardware installed in your PC. Those drivers are tailored to that specific hardware. And they use more features than the minimal feature-set. Now you take that HDD out of your PC, and put it in another PC, with another motherboard and other devices. Result: the chipset-drivers might try to do stuff that is not supported by the hardware. They might try to access registers on specific memory-locations on the bus to access the hardware. But if those registers are not there, you will get an error (e.g. bus error, or memory violation).

Many drivers for devices like mice, keyboard, monitors, etc, will check to see what devices are connected. So that won't cause problems. The OS will detect at boottime that stuff has changed. Newer protocols like USB have been designed to be able to do this.

But for chipsets on the motherboard it is different. The OS is probably capable of detecting changes in HDDs, SSDs, maybe even videocards. But chipsets are a different thing. Chipsets are not supposed to change. Only when you change the mobo, which hardly ever happens. And I guess it's harder to check for motherboard-compatibility. And therefor it often goes wrong when you swap in another mobo.

So no, there are not layers of drivers. It's just that during boottime, simple drivers get replaced by more elaborate drivers. Once the OS has fully booted, only 1 driver (per device) is running.

From the BIOS wiki-page:

What chip-set capabilities exist while running the BIOS setup utility?

I'm assuming that you don't have full access to what the chipset offers in the BIOS.

Is the interrupt controller functioning? What about the DMA controller?

Are BIOS programs loaded into RAM? Or, does the processor run the code directly from the BIOS chip?

I'm fascinated with the computer's self awareness. How does it check to see what hardware is connected?

How does it decide which drivers should be loaded?

Thanks.
 

sm625

Diamond Member
May 6, 2011
8,172
137
106
There were no problems. I then detached the HD and placed it into the laptop that I wanted to use. I booted up, and the splash screen pops up. Maybe 2 seconds later it blue screens on me. I tried again and same thing.

What often works is before moving the HDD to the new computer, go into device manager and uninstall all of your IDE ATAPI controllers. Just uninstall everything in that section. It wont hurt anything, it will reinstall the right driver. This has never not worked for me on Win7 even when switching from an intel motherboard to an AMD motherboard.
 

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
What often works is before moving the HDD to the new computer, go into device manager and uninstall all of your IDE ATAPI controllers. Just uninstall everything in that section. It wont hurt anything, it will reinstall the right driver. This has never not worked for me on Win7 even when switching from an intel motherboard to an AMD motherboard.

Wow.

You've actually found the exact drivers that are causing the transfer problem?

How did you figure that out?
 

Merad

Platinum Member
May 31, 2010
2,586
19
81
AFAIK the main reason for this behavior on windows is that officially a windows license is tied to the PC it was activated on. MSFT considers changing motherboards to a different model to be a new PC, which requires (in theory) reinstalling windows with a new license key. Official support for motherboard swaps would make it rather easy to bypass the license terms MSFT desires.

The BIOS does not contain any drivers used by the main OS.

As for your other questions, they could fill a couple of graduate level CS/CE/EE courses.
 

Gryz

Golden Member
Aug 28, 2010
1,551
204
106
AFAIK the main reason for this behavior on windows is that officially a windows license is tied to the PC it was activated on. MSFT considers changing motherboards to a different model to be a new PC, which requires (in theory) reinstalling windows with a new license key. Official support for motherboard swaps would make it rather easy to bypass the license terms MSFT desires.
I doubt it.

The reason PCs crash when you change the combination of motherboard and harddisk, is because of the drivers for the chipsets on the motherboard.

When you deal with a device that is accessed over USB, or SATA or any other inter-component protocol, those protocols can often do device discovery. So when a PC boots, the drivers can discover which USB or SATA devices are attached, and deal with them.

Chipsets on a motherboard are different. Usually the only way to deal with them is by writting and reading directly over DMA over the memory bus to specific memory locations. If you write to those locations, and indeed there is hardware listening, everything is fine. However, if you write to a memory location, and nothing is there, the bus-hardware will recognize that you are writing to memory that you're not supposed to touch. And that will make the PC crash. And as it is a crash inside an important driver, this means it will be a full OS crash (aka Blue Screen of death). The basic problem is, you can't probe for which chipsets are installed in the PC, without just trying to write stuff to those memory locations.
 

Crusty

Lifer
Sep 30, 2001
12,684
2
81
I doubt it.

The reason PCs crash when you change the combination of motherboard and harddisk, is because of the drivers for the chipsets on the motherboard.

When you deal with a device that is accessed over USB, or SATA or any other inter-component protocol, those protocols can often do device discovery. So when a PC boots, the drivers can discover which USB or SATA devices are attached, and deal with them.

Chipsets on a motherboard are different. Usually the only way to deal with them is by writting and reading directly over DMA over the memory bus to specific memory locations. If you write to those locations, and indeed there is hardware listening, everything is fine. However, if you write to a memory location, and nothing is there, the bus-hardware will recognize that you are writing to memory that you're not supposed to touch. And that will make the PC crash. And as it is a crash inside an important driver, this means it will be a full OS crash (aka Blue Screen of death). The basic problem is, you can't probe for which chipsets are installed in the PC, without just trying to write stuff to those memory locations.

Linux manages to do this just fine. I can take a drive out of a machine and boot it up on another machine without a complete reinstall of the Operating System.
 

Gryz

Golden Member
Aug 28, 2010
1,551
204
106
What chip-set capabilities exist while running the BIOS setup utility?
I'm not a hardware specialist. And know very little about drivers. But it is fair to assume that the drivers that are inside the BIOS are the most basic ones that are necessary to get the system going. (In other words, the minimum driver to access a harddisk, find the file-system, manouver into the file-system to find the boot-file, copy the boot-file from hdd to RAM, and give control to that boot-file). The smaller the better, so nothing more than the minimum. Also note that performance isn't an issue during boot, so the drivers inside the BIOS can do the oldests slowest stuff. As long as compatability with as much old hardware as possible is achieved.

I'm assuming that you don't have full access to what the chipset offers in the BIOS.
Exactly.

Is the interrupt controller functioning? What about the DMA controller?
I don't know. That is such specific stuff, it only matters to the people designing chipsets, and the guys who write the BIOS.

Are BIOS programs loaded into RAM?
Most likely.
There are forms of flash-memory that can act as memory for programs to run from. Usually it is quite a bit slower to run from flash than it is to run from RAM.

Or, does the processor run the code directly from the BIOS chip?
The flash-chip on the motherboard, that holds the BIOS-image, is mapped into memory. So the CPU could run from that location, without copying into RAM first. I have no idea if that actually happens.

I'm fascinated with the computer's self awareness. How does it check to see what hardware is connected?
There are protocols on busses. How do devices communicate with the CPU, how do they communicate with memory, with each other, with the bus-controller, etc. I only know the broad ideas, not any details. The bus-protocols often contain ways for the bus-controller to send out probes to figure out what is attached to the bus. (E.g. the SATA-chip on the motherboard is basically the controller for the SATA-busses).

How does it decide which drivers should be loaded?
When probing for devices on the bus, those devices will answer with a bunch of information about themselves. The OS can then just looks up those identifiers in a table and sees if it has drivers for those devices.

I guess there will be very few people who know the details about all of this. You should just know the global idea, and don't bother about the details. If you want to know the details, you will soon find out that there are more details than any individual can know.
 

Merad

Platinum Member
May 31, 2010
2,586
19
81
All x86 code is executed from memory. It's possible that the BIOS has some ability to map its flash storage to RAM, but most likely executable code is just loaded into memory.

Some CPUs do not execute code from RAM. The most common being the Harvard architecture used on many microcontrollers.
 

exdeath

Lifer
Jan 29, 2004
13,679
10
81
The BIOS setup utility is it's own isolated piece of software and has nothing to do with anything else. If I recall the standard PC BIOS memory mapping only allows for 128kb below 1 MB so anything above that, esp some fancy GUI BIOS and such, are going to be stored compressed or in stored in a vendor proprietary way or location in Flash. The code responsible for decompression that BIOS image to RAM and jumping to it will be a stub accessible in the 128kb that the CPU can access on power on in real mode.

As for booting... To keep it as simple as possible, it's primary job is to install basic software to interact with the motherboard devices via well known very simple real mode interrupt handlers used to boot the system such as int 15h (memory config), int 10h (video), int 13h (disk), int 16h (keyboard), etc.

Early on the OS loader calls these to read sectors from a disk, discover the memory map, etc. Once the kernel is booted and protected mode is enabled, the BIOS is fairly useless, and from that point on the OS is using basic hardware abstraction layers to interact directly with the hardware I/O without using the BIOS. At a very basic level, all video cards are VGA compliant, and all SATA storage adapters share some basic IDE I/O interface compatibility despite being radically newer architectures that are nothing alike.

General compatibility is provided at this point by various standards and abstraction layers (ATAPI standard is a good example). Things like SATA and PCI Express still fundamentally provide some level of minimal compatibility with IDE and PCI protocols for example so it will work good enough until the OS loads it's real vendor provided drivers then everything else is pretty much discarded.

PC architecture is a history of layers upon layers of extended and expanded specifications built around existing standards and backward compatibility. An i7 to this day still boots acting like it's an 8088.

I haven't been programming or keeping up with architecture changes lately and things have changed greatly in the last 10 years (replacements for old school BIOS, MBR format, finally ditching a large amount of legacy baggage, etc). But an example is I'm fairly certain basic PCI framework functions still function on PCI Express, despite being completely different hardware, not that you can do much else after you've enumerated something. (This introduces the concept of "classes" of drivers and hardware interface layers) PCI Express despite being as different from PCI as a car is from a watermelon, still implements the basic PCI framework model, or language specification. You might not be able to use your Geforce 980 as a GPU in DOS but after all these years it's still backward compatible with VGA and has a framebuffer mapped at 0x000A0000-0x000AFFFF in real mode. This is one of the things set up by the BIOS ROM on both the motherboard and VGA card before boot sector load. This is ignored and discarded altogether once you go into 64 bit long mode which made strides in getting rid of many old PC conventions that have existed since the 8086.

Where this starts to diverge is, with video cards only as one example, is VGA is to this day the last true universal standard for all graphics cards at a hardware level. ALL graphics cards from 1980 (?) to 2015 are VGA cards and are fundamentally and universally compatible down to every last mode and register location. SVGA and beyond is vendor proprietary (or VESA 2.0). It's no coincidence that hardware level compatibility across vendors stopped at the same time it became no longer necessary. At this point Windows 3.x was mainstream and device drivers became the norm so there was never a need to implement hardware level compatibility across vendors anymore. They didn't have to be register compatible in hardware, only Windows compatible in that a device driver took care of the vendor proprietary hardware access to implement "DrawRect".

Most of the time an OS crashes when an OS config and motherboard are mismatched is because the OS has assumed a particular device config that is necessary for booting, eg: a vendor specific SATA driver and when that driver fails to init on another platform you end up with most commonly unmountable boot volume BSOD or something along those lines. BIOS -> MBR -> PBR -> ntldr -> ntoskrnl via basic raw disk access, but once the kernel tries to init the drive controller and NTFS it can't.

Why can't it just simply PnP and load a new driver? It wasn't prepared for it, and any PnP modules and drivers are stored... in the file system it cannot access now. Also note that a BSOD does not actually mean a crash or failure of the OS. It's a controlled and deliberate voluntary step back hands off response once something has been determined to be in an unreliable state to avoid possible corruption or damage. Something is interacting with the display to write that message out for you right?

9 times out of 10 it would probably work. One of the first things a kernel does after the loader finishes is load a proper file system driver so it can start loading more things, which relies on a storage adapter driver, which will fail init simply as a failsafe for something as simple as identifying that the ID of the hardware isn't the correct vendor even if it could work in theory. The kernel now having the file system and storage controller drivers returning an init failure, is done.

There are numerous tasks the BIOS does from ROM that the OS does not have the luxury or ability to perform in RAM like program the memory controller, enumerate DIMM slots, change timing parameters, etc.

There's a bit more here than needed but I've observed your other posts showing your interest in systems programming and it's happening at a time that I'm getting interested in getting back to x86 hacking.
 
Last edited:

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
Why can't it just simply PnP and load a new driver? It wasn't prepared for it, and any PnP modules and drivers are stored... in the file system it cannot access now.

That makes sense.

Though, how does the installation program query the chipset?

When the OS is being installed, the installation program queries the chipset and bundles the right drivers in with the OS's kernel.

What does that mechanism look like?

There are numerous tasks the BIOS does from ROM that the OS does not have the luxury or ability to perform in RAM like program the memory controller, enumerate DIMM slots, change timing parameters, etc.

When you say program the memory controller, do you mean for memory mapped I/O?

It's my understanding that in memory mapped I/O, all registers and memory locations, specifically those on I/O hardware, are seen by the processor as having memory addresses as if they were RAM locations, right?

It's then up to the memory controller to say: Oh, physical address a10fee66, that's not in RAM, that's the control register on the DMA controller.

Is this what you mean by programming the memory controller, somehow telling it what physical, addresses map outside of RAM?

Why can't this be done from RAM?

-----------------------------------------------------------------------------------------------------------------------------------------------

Thanks, guys.

But, I have some new questions:

Why bother with minimal capability at all during boot up? Why not just launch the system with full capability immediately?

Is it a size constraint?

Is it because the BIOS can only contain so much code burned onto its ROM and so you require simple, basic, small drivers?

Also, how much of this paradigm is specific to x86 architecture?

Specifically, how much of this carries over to ARM?

Thanks.
 
Last edited:

exdeath

Lifer
Jan 29, 2004
13,679
10
81
That makes sense.

Though, how does the installation program query the chipset?

When the OS is being installed, the installation program queries the chipset and bundles the right drivers in with the OS's kernel.

What does that mechanism look like?



When you say program the memory controller, do you mean for memory mapped I/O?

It's my understanding that in memory mapped I/O, all registers and memory locations, specifically those on I/O hardware, are seen by the processor as having memory addresses as if they were RAM locations, right?

It's then up to the memory controller to say: Oh, physical address a10fee66, that's not in RAM, that's the control register on the DMA controller.

Is this what you mean by programming the memory controller, somehow telling it what physical, addresses map outside of RAM?

Why can't this be done from RAM?

The installation program is running at a more generic level. It's not depending on the higher level drivers. Once it selects the higher level driver and discards the lower levels in the config and the OS hardware abstraction layer is defined, it's placed in the config external to the OS kernel. The kernel simply boots from that config. If you change these critical items, the kernel boot process isn't prepared to be it's own setup.exe every time. PNP features in the OS aren't capable of functioning that early in the boot process. Part of this is just reasonable expectations, just like you wouldn't design a car with a user swap-able chassis, it's just not something that's expected to be done on a regular basis to build a setup into the boot process. A change this early on invalidates everything else in the config rooted from this point. Say for example, ntldr and ntoskrnl.exe can't install it's own hal.dll or boot.ini or config\SYSTEM, these are created externally by a setup program.


The x86 traditionally has two memory buses. These use the same address and data lines but there is a pin that specifies memory access vs I/O access. The difference in toggling that pin is the use of a MOV instruction (memory bus) vs a IN or OUT instruction (I/O port bus). I/O port addressing is mostly legacy at this point. Straight flat memory mapped I/O registers in the CPU address space using standard MOV memory addressing is also mostly dated (?), most modern paradigms use single destination address FIFO buffer addresses that you DMA vendor defined hardware command packets into.

By program the memory controller I mean reading the SPD EEPROM on the DIMMs, setting up the banking and DRAM refresh cycles and timings, interleaves, etc. The RAM is being reset and unstable in this state, so this process can only happen from a ROM.


-----------------------------------------------------------------------------------------------------------------------------------------------

Thanks, guys.

But, I have some new questions:

Why bother with minimal capability at all during boot up? Why not just launch the system with full capability immediately?

Is it a size constraint?

Is it because the BIOS can only contain so much code burned onto its ROM and so you require simple, basic, small drivers?

Also, how much of this paradigm is specific to x86 architecture?

Specifically, how much of this carries over to ARM?

Thanks.

Everything in a computer is built around well known layers of generic abstraction and interchangeability from general to specific. View it like an upside down pyramid. A BIOS is only supposed to provide Basic I/O (the definition of BIOS). It knows how to read sectors from a storage device and present them in a universal interface, it is intentionally not designed to know anything further about file systems or Windows or DOS or Linux. From there a user provided boot loader has to load a kernel and filesystem driver, etc. It get's built up from general to specific.

Without working in layers like this upside down pyramid, you don't have an open platform family of compatibility and interchangeability.

If you provided a 32 GB ROM that had everything built in, it wouldn't be a Windows PC, it would be whatever OS was in the BIOS and compatible with only itself like say, a Xerox copier or other embedded system. There would be no need for drivers or a driver API since it's all built in and at that point you wouldn't need expansion slots either. Software written for it would only function on that one system. If the BIOS dealt with filesystems then you would be forever fixated to whatever OS supports that filesystem, and so on.

These are pretty standard computer science and engineering concepts and applies to all PC (as in personal computer, not just x86) oriented microprocessor based systems and operating systems, at least those meant to be open platforms. If you are working on a closed non user expandable embedded system like a microwave oven or a television or an automotive EFI computer, you can do whatever you want. But even then these devices will be designed and maintained by 100s or 1000s of people and need to last for the duration of their intended life cycle so they too will maintain some level of layering and module interchangeability internally, just not open to the consumer.
 
Last edited:

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
The x86 traditionally has two memory buses. These use the same address and data lines but there is a pin that specifies memory access vs I/O access. The difference in toggling that pin is the use of a MOV instruction (memory bus) vs a IN or OUT instruction (I/O port bus). I/O port addressing is mostly legacy at this point. Straight flat memory mapped I/O registers in the CPU address space using standard MOV memory addressing is also mostly dated (?), most modern paradigms use single destination address FIFO buffer addresses that you DMA vendor defined hardware command packets into.

Sounds very interesting.

What does a command that sends bytes to the NIC, sound card, video card, etc. look like?

Is it just a standard mov instruction?

If so, what do these addresses look like?

So, you're saying instead of just sending a raw bit stream to these registers, you utilize some sort of internal protocol with packets like an internal version of TCP?

Do have any good books/documents that you can direct me to?

How did you acquire this knowledge?

most modern paradigms use single destination address FIFO buffer addresses that you DMA vendor defined hardware command packets into.

Your language is a bit confusing here. Tell me if my understanding is correct:

You send packets of information adhering to some internal protocol, like an internal TCP for example, through DMA to FIFO buffers located on the pertinent peripheral, correct?

If so, then the DMA controller reaches into RAM to grab the packets and needs an address to send those packets into, correct?

Who decides what those address are? I mean how do you actually assign an address to a buffer?

Also, the DMA controller itself can't be DMA'ed into. It needs an address on the address bus, yes?

By program the memory controller I mean reading the SPD EEPROM on the DIMMs, setting up the banking and DRAM refresh cycles and timings, interleaves, etc. The RAM is being reset and unstable in this state, so this process can only happen from a ROM.

Yes, but can it be reprogrammed from kernel space after the OS boots?

Also, connected to my questions above, how does the memory controller differentiate between addresses in RAM and addresses on peripheral I/O devices e.g. NIC buffers, interrupt controller control registers, etc.?

Do you have any good reference materials on the specifics of the memory controller?

Everything in a computer is built around well known layers of generic abstraction and interchangeability from general to specific. View it like an upside down pyramid. A BIOS is only supposed to provide Basic I/O (the definition of BIOS). It knows how to read sectors from a storage device and present them in a universal interface, it is intentionally not designed to know anything further about file systems or Windows or DOS or Linux. From there a user provided boot loader has to load a kernel and filesystem driver, etc. It get's built up from general to specific.

Without working in layers like this upside down pyramid, you don't have an open platform family of compatibility and interchangeability.

If you provided a 32 GB ROM that had everything built in, it wouldn't be a Windows PC, it would be whatever OS was in the BIOS and compatible with only itself like say, a Xerox copier or other embedded system. There would be no need for drivers or a driver API since it's all built in and at that point you wouldn't need expansion slots either. Software written for it would only function on that one system. If the BIOS dealt with filesystems then you would be forever fixated to whatever OS supports that filesystem, and so on.

These are pretty standard computer science and engineering concepts and applies to all PC (as in personal computer, not just x86) oriented microprocessor based systems and operating systems, at least those meant to be open platforms. If you are working on a closed non user expandable embedded system like a microwave oven or a television or an automotive EFI computer, you can do whatever you want. But even then these devices will be designed and maintained by 100s or 1000s of people and need to last for the duration of their intended life cycle so they too will maintain some level of layering and module interchangeability internally, just not open to the consumer.

Ah, I see. If you wish more than one OS to run on a given hardware platform, then you need to provide some basic interface that they all understand.

The more you have the basic hardware do, the more OS specific the situation becomes.
 
Last edited:

exdeath

Lifer
Jan 29, 2004
13,679
10
81
As far as what the commands and registers do, these are all vendor proprietary and inside secrets. IBM PC/XT and VGA era were the last time everyone shared a compatible hardware interface. You're best bet is going to be vendors that provide driver source files for Linux.

This is one reason I prefer old consoles for playing around on. They are fixed and well documented if you can get ahold of the programming manuals, so you don't have 1000 diff vendors doing their own thing and don't need drivers).

Re: FIFO communication with hardware.

What I mean is you don't typically have directly memory mapped registers anymore.

You'd have something like (I'm going to make stuff up, this doesn't correlate to any real system. This sorta thing is proprietary vendor inside knowledge and potentially trade secrets and not shared, so it's hard to be specific without talking about X platform from Y vendor and having vendor's programming manuals)

0x8000F000 = memory mapped address of a piece of hardware's command buffer, lets say this is the only thing memory mapped. Maybe a status register @ 0x8000F004 to read if busy, done, ready, error, etc.

Command format might be something like 1 QWORD wide, and a minimum of 2 QWORDs or 4 DWORDs.

The first QWORD would be the header, among other things it might contain the type of command we are sending, the length of the packet, how many sub commands, etc. If this is the first, or last command in a chain so the hardware knows what to expect and when to stop the transaction, etc. Then the payload with your parameters and commands.

So something like this, except you'd be setting bits:

QWORD 1 = [this is a start of command header, we are sending a register set cmd, this packet is 2 qwords including this header and this is the only one]

QWORD 2 = [I want to set your internal RGB_2 register to a value of 0x000000FF]

If it's a 32 bit bus, the vendor will determine what order you write the 64 bit QWORDS (high or low DWORD first), and we'd have to successively write 4 DWORDs to 0x8000F000.

so the packet would be like this (and im just making up my own bits and fields, this is all vendor defined):

.mypacket:
.dw 0xCC0F0003 ; header low 0xCC command delimiter, function 15 (F) = set register, some other bits, blah blah blah
.dw 0x00000002 ; header high ; 0000 first and last command for this transaction, count = 2 qwords long including header, so only 1 command
.dw 0x00020004 ; command low ; 0002 function set register 0004 = RGB_2 register
.dw 0x000000FF ; command high ; 32 bit value to set RGB_2

Simple movs to illustrate writing 4 consecutive dwords to my make believe command FIFO @ 0x8000F000, you'd use DMA for very large chains and asynchronous (eg parallel) operation or a loop with special string repeat instructions and more advanced addressing modes (eg: base+index*elementsize).

mov edx, 0x8000F000 ; dest ptr to FIFO
mov ebx, .mypacket ; base pointer to packet

mov eax, [0+ebx]
mov [edx], eax ; write header low dword

mov eax, [4+ebx]
mov [edx], eax ; write header high dword, open transaction, count-1 qwords to follow

mov eax, [8+ebx]
mov [edx], eax ; write command low dword

mov eax, [C+ebx]
mov [edx], eax ; write command high dword, command and packet kick on last dword of packet


The hardware would accept these and form and verify the packet and parameters internally. This is a preferred architecture to straight flat register I/O or memory mapping because you can have command queues and completely asynchronous operation of the hardware , eg you can shove it commands non stop even if it's busy and it can accept them and queue them up internally and prioritize however it wants. Peripherals like GPUs are far more complex than a couple register bits to read/write directly with the host CPU anymore as with CGA/EGA/VGA. Even VGA already started to implement internal registers to minimize the number of host I/O addresses used by using only index/data registers externally, as it has over 300 registers internally IIRC. We have 64 bit now, so it's not so much an address space problem, but there is still the issue of being able to queue things to a front end and allowing the hardware to work asynchronously. You can create a display list for example, kick it off with DMA, then start working on the next display list while the DMA and GPU are working. It's not as simple as "set bit, get immediate result" anymore.

Another way would be building the packet in host RAM or memory mapped peripheral RAM then writing to a handful of mem mapped registers to tell it where to pull it from on it's own (the old PowerVR on the Dreamcast was a good example of this, you set up tile bins and display lists in 8MB VRAM mapped into the SH4s address space then told the PVR to go work on them).

As far as who decides what all these bits and commands and addresses are? The hardware vendor. The OS vendor. The platform architecture. Many factors. Nothing is set in stone.

Low level systems programming in general requires you to know 3 things to effectively program anything on any platform:

1) your platform's memory map - where do we put stuff to make things happen
2) your hardware communication protocols, be it packets or memory mapped registers - what do we put there
3) your CPU instruction set and OS/BIOS support if required - how do we put it there
 
Last edited:

exdeath

Lifer
Jan 29, 2004
13,679
10
81
If this kind of thing is exciting to you I strongly recommend you start learning these concepts on an older game console. The Gameboy Advance is excellent beginner platform. ARM7 is clean and easy to understand, its a nice flat 32 bit memory map, documentation is plentiful and easy to understand and the hardware and flash carts are cheap, and it's all simple bitmap graphics and memory mapped I/O.

VisualBoyAdvance is an emulator that can show you everything in memory visually and has a built in single step debugger, etc.

However it doesn't have an OS, but you get to work directly with the hardware and there are ROM syscalls built in to assist with common functions.

The reason I say this is PC is big ball of mess with multiple proprietary vendors, intellectual property activism and hoarding = no public hardware documentation, etc. The multi vendor open platform nature of PC combined with it's legacy of backward compatibility for almost 4 decades makes it a very confusing place to start learning systems programming So when you start getting into the questions of "where is this address or how do I make this device to that" nobody here can really answer that since Windows 3.1 and beyond. It's obfuscated by OS and driver layers and direct metal access isn't much of a thing on PC anymore except to driver writers (nVidia, etc).

In fact this is what got me started on consoles. I wanted to learn how DirectX and OpenGL worked at the metal to produce 3D graphics, and well... good luck getting programming manuals for anything past SVGA VESA BIOS extensions or maybe an old incomplete reverse engineered Voodoo 1 or something.

And even then you'd have to either write your own OS, or spend years familiarizing yourself with WDM/DDK (or whatever the Linux equivalent is)

If you want to stick with PC, it's best to start learning with 8088 real mode and VGA back when there were universal hardware standards when "IBM PC compatible" was a thing. Then we can definitely tell you specifics without a doubt the VGA framebuffer is @ address A0000-AFFFF or the palette index and data read/write registers are at IO address 3c7-3c9, etc. Even if you work your way into Windows/Linux kernel hacking and protected/long mode, you will still have a hard time as a hobbyist doing anything more than a simple RAM drive or USB driver, lacking detailed documentation of vendor chips and protocols.
 
Last edited:

exdeath

Lifer
Jan 29, 2004
13,679
10
81
Yes, but can it be reprogrammed from kernel space after the OS boots?

The CPU would be executing code from the DRAM it's messing with. If you did something that caused a DRAM reset or significant change it could cause any sort of errors and crashes (most likely freeze requiring hard reset)

I suppose you could abuse the cache and prefetch buffers but you'd be limited to doing something small like jumping back to the soft reset vector in ROM to reload everything if you did something to corrupt DRAM contents.

Also since you can likely manipulate banks individually (I wouldn't know I haven't messed with that part) you can have your active code shuffling around to banks you aren't messing with at the moment? /shrug Similar to how a memory test program currently executing would have to relocate itself as it's running to make the memory the program is using available to test.

Also, connected to my questions above, how does the memory controller differentiate between addresses in RAM and addresses on peripheral I/O devices e.g. NIC buffers, interrupt controller control registers, etc.?

Ok we are going all out ECE 101 here.

At the most fundamental level? Digital logic. Specifically a topic called address decoding which uses logic gates on the address lines to form a physical memory map using combinations of not, and, nand, or, etc gates with the address lines being the inputs, to enable the chip enable pin of the device that is supposed to respond at that address and connect it's data lines to the data bus for the duration that address is active.

Easy example:

16 bit address space = 64k

To keep it simple, no mirroring, banking, etc, lets say we have a 32k ROM and a 32k RAM you want to map into that 64k.

The ROM and RAM chips are going to have 15 address lines and 8 data lines.

The CPU is going to have 16 address lines and 8 data lines.

How do we connected it all?

Lets say this is a 65C02. And we'll use 32k static RAM so we don't have to worry about DRAM controllers and all that. Hardware definition dictates we need a zero page @ $00xx and a stack at $01xx, these must be writable. In addition, reset and other interrupt vectors are hard wired at $FFFx.

This means we would want the ROM on the top half and RAM on the bottom half of the 64k map.

$0000-7FFF = 32k RAM
$8000-FFFF = 32k ROM

The first 15 address lines A0-A14 of the CPU would be would be ganged together in parallel with the only 15 address lines of the RAM + ROM chips. Same for data bus lines D0-D7. We have an extra address line on the CPU that will be used to determine when we cross the 32k mark and use that pin to determine which chip to talk to.

Now use the 16th address line from the CPU, A15 and observe that any time the CPU accesses an address $8000 or higher this line will go active and below that it will be inactive.

I'd have to look up specs, but lets say the CPU A15 line is a active high (drive to +5v @ $8000+) and the ROM/RAM chip enable /CE is active low (0v turns it on 5v turns it off).

If A15 is on (address $8000-$FFFF) we want to toggle ROM's /CE and if A15 is off ($0000-7FFF) we want to toggle RAM's /CE.

But we want the inverse because the CPU is active high and the chips are active low. In addition, we want the ROM and RAM /CE to complement each other so if one is on the other is off and vice versa, all determined by A15 from the CPU.

So we do CPU A15 > inverter gate > to ROM /CE and A15 directly to RAM /CE.

When CPU hits $8000 or higher, A15 will turn on and drive RAM /CE high disabling it, but be inverted to low at ROM /CE and enable it. Below $8000 RAM /CE will be active low, while ROM /CE is active high.

From the programmer's perspective we see a memory map that looks like $0000-7FFF is RAM and $8000-FFFF is ROM.

Note that byte 0 in the ROM chip is byte 32768 ($8000) to the CPU and that byte 32767 in the ROM chip is byte 65535 ($FFFF) to the CPU.

In our ROM lets say our startup code starts at the first byte (0) in the ROM chip. To the CPU this is location $8000. CPU location $FFFC and $FFFD are defined by the CPU maker to contain the power on reset address, and this maps to third and forth to last bytes in the ROM. ROM would look like

ROM is burned in a programmer like this:

$0000 boot code
... xx xx
$7FFC 00 80
$7FFE xx xx

To the CPU it would look like:

$0000
...
$8000 boot code
...
$FFFC 00 80 <- on power on goto address $8000 (stored in little endian), eg the reset vector in ROM points to boot code located within itself at an address that doesn't exist when you are burning the ERPOM but is valid in the CPU memory map once the ROM is mapped in!

Devices are the same way. They have address and data lines and a chip enable or chip select pin. Interrupt controllers, DMA controllers, timers, serial UARTs, other microcontrollers (the original 8042 keyboard controller itself is it's own self contained microcontroller with it's own RAM and ROM and instructions). The same buses the CPU uses to access the registers on a DMA controller to start a transfer are the very same buses the DMA controller itself manipulates to do the transfer. Data and address lines are all wired together, that's why it's called a bus. A device that needs only 4 memory mapped registers would have mere 2 address lines A0-A1 but will still have 8-32 data lines depending on the word width of it's registers.

A DMA controller is tricky. You need to have all address lines for the ranges of memory the DMA can access so the DMAC can manipulate the A/D buses itself, but for the purposes of memory mapping the DMA controller itself being addressed by the CPU to program it when it's /CE is enabled would only respond to a few addresses to map it's configuration registers, probably multiplexed over the same address lines? When the CPU is talking to the DMAC at the address it's decoded to map to, it's /CE is asserted and it might listen only on A0-A3 (for 16 I/O register locations or whatever it needs), but when /CE is released after the CPU is done talking to it directly, it would use A0-A15 to actively drive addresses of other devices to perform the transfer. It's probably wise to map the DMAC programming register to a location that the DMAC itself can't reach to avoid DMA to itself to prevent undefined bus states. Or something :p Lets use a fake example DMA controller that can reach 32k with 15 address lines A0-A14. We would use the previously discussed address decoding logic to enable it's /CE somewhere in the upper half of 64k such that we make /CE and A0-A3 respond at say $B000-B00F, but the DMAC itself only having 15 total address lines wired A0-A14 on a 16 bit address bus would never be able to address itself because it could only DMA from $0000-7FFF. CPU writes source, destination, direction, count etc to $B000-B00F to tell it to copy 1k from $1000-$1FFF to $2000-$2FFF. And since it's a 15 bit DMAC the src/dest regs are 15 bits ignoring the 16th bit so you couldn't program it for $B000 as a src/dest (it would ignore the highest bit and $B000 becomes $3000 anyway). DMAC cannot access itself accidentally or maliciously. It' doesn't have an A15 line that is needed to interact with it's own /CE. Easy enough.

As you start adding multiple devices of multiple address block sizes at varying addresses, this decoding logic gets complex very quickly. If you recall old motherboards and expansion cards (and jumpers and DIP switches) with dozens of discrete 74xxx DIPs on them before all in one VLSI chipsets became common, all these served the purpose of address line decoding to put all the devices into the memory map of the CPU at unique locations.

I chose the 6502 and static RAM because it's very simple to understand. DRAM controllers and the x86 memory map are far more complex but the principle is identical. The x86 power on reset vector is 0xFFFF0 16 bytes from the end of the 1MB mark. Guess what, this is where the BIOS ROM is mapped. 6502 the reset vector is an actual vector, the CPU starts its first instruction at the address indicated at the reset vector location. The x86 actually starts execution AT FFFF0 so what is contained there is not just an address but a JMP instruction to a startup address , also located in the same ROM.
 
Last edited:

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
The x86 actually starts execution AT FFFF0 so what is contained there is not just an address but a JMP instruction to a startup address , also located in the same ROM.

Why have a jump instruction at the initial address?

Why not have the code you actually want to execute at the initial address?

Is this some sort of support for a legacy protocol from back in the day?

Thanks.
 

Ken g6

Programming Moderator, Elite Member
Moderator
Dec 11, 1999
16,660
4,602
75
Why have a jump instruction at the initial address?

Why not have the code you actually want to execute at the initial address?
Because there's only 16 bytes there that you could use.
 

chrstrbrts

Senior member
Aug 12, 2014
522
3
81
Because there's only 16 bytes there that you could use.

But the address jumped to is also on the ROM.

Wouldn't those memory restrictions apply to the jumped to address as well?

I'm sorry, I don't know what you mean by there's only 16 bytes available.
 
Last edited: