Making System Calls from User Space using Linux Assemblers

[chrstrbrts]

Hello,

I recently asked a question about talking to A/D converter chips in the sound sub-system on the motherboard. Here's the link:http://forums.anandtech.com/showthread.php?t=2463388
Ken closed the thread as it went astray at some point, so I can't ask this question there.

Anyway, at some point I asked the following:

But, at the hardware level, what you're actually doing is making the 'int'
hardware call with a direct integer argument and setting parameters in
certain registers.

This is what I want to do.

What's the proper 'int' argument for fopen()?

To which Merad responded:

Like many of your recent questions, the answers is: you can't do that.

A modern OS like Linux/Unix or Windows is a managed playground.
Teacher is watching you to make sure you don't do something you aren't
supposed to. And AFAIK, directly interacting with hardware resources from
a user mode program is very high on "not allowed" list.

If you want to tinker with things at that level, you need to either start
writing your own OS, or start playing around with microcontrollers.

Now, this has me questioning what I've learned up to this point about OS. It was my understanding that you could interact with hardware from user space if you asked the OS to intervene. This is called a system call, kernel trap, or software generated interrupt. This is a big part of what an OS does; it provides an interface for user space code to talk to hardware.

OK. So, in Linux assemblers, if you call the 'int' instruction and give the instruction an integer argument, that tells the processor to dive into kernel space and check the integer value's offset in the vector interrupt table that usually lies at the very bottom of physical RAM. This is a table of pointers that point to code somewhere else in kernel space.

A lot of these pointers are for when the hardware fails or the CPU throws a fault like divide by zero. But, there should be a number that codes for software interrupts. That pointer points to another table of pointers that point to the functions that you want to ultimately call. By setting parameters in certain registers in the CPU, you tell the computer which of those software interrupt functions you wish to use.

OK. That was my understanding. But, when Merad said what he said, I began questioning my understanding.

Is there something special about the fopen() function in C?

Is it not just a simple wrapper for some OS API function?

I thought that high level I/O functions in languages like C were just wrappers for OS I/O functions.

Perhaps, the fopen() function wraps many OS API I/O functions into one bundle?

Still, though, shouldn't you be able to call the required low-level OS API functions from assembly language using the 'int' hardware instruction and get the same result as if you had called fopen()?

Thanks.

 

[Exophase]

fopen isn't a Linux syscall. The syscall is just called open.

fopen is part of the standard C library, the stdio (standard I/O) FILE stream part. This file library adds an extra layer to perform things like buffering. And it abstracts out whatever differences exist between OSes.

You don't have to use fopen, fread, fwrite, etc. On POSIX systems you can use their lower level equivalents where available, like open, read, and write. Those turn into simple syscalls.

 

[chrstrbrts]

fopen isn't a Linux syscall. The syscall is just called open.

fopen is part of the standard C library, the stdio (standard I/O) FILE stream part. This file library adds an extra layer to perform things like buffering. And it abstracts out whatever differences exist between OSes.

You don't have to use fopen, fread, fwrite, etc. On POSIX systems you can use their lower level equivalents where available, like open, read, and write. Those turn into simple syscalls.

Yes, that's what I thought.

But, then why did Merad say what he said?

He said that I couldn't make those calls from user space in assembly.

Are we missing something?

 

[Exophase]

Merad didn't say that in response to a question of calling open syscalls, he said that in response to this:

"But, at the hardware level, what you're actually doing is making the 'int' hardware call with a direct integer argument and setting parameters in certain registers.

This is what I want to do."

These are very different things.

The whole point of syscalls is to allow user space programs to communicate with the kernel, so naturally you have to be able to make them.

 

[exdeath]

Can't speak about Linux too much, but if you trace into a standard C fopen call in Windows you'll end up in kernel32.dll::OpenFile, which in turn calls ntdll.dll::NtCreateFile which will in turn might call ntdll.dll:KiIntSystemCall.

KiIntSystemCall is ultimately where you would need to know the ordinal number of the service you want. ntdll.dll also handles the best way to call the kernel on a given platform, eg, it might use instead KiFastSystemCall using sysenter instead of int 2e.

There is a TON of user mode stuff along the way as the parameters for OpenFile are far more extensive than the simple C fopen. Part of this library code can also determine if you need a syscall, for example the file is already open or fread() returning data from a buffer from the last syscall block read when you are reading tiny bits at a time.

It's really pointless to want to call syscalls in assembly though. It's ultimately going to do the same thing, or nothing at all, maybe even get your program canned if you don't format the user mode stack correctly.

TLDR: just because you are using assembly, in user mode, and calling syscalls from assembly doesn't mean you are interacting with the hardware directly or even in a more efficient manner.

I consider myself to be a very advanced and experienced systems programmer but even I don't know, don't want to know, and don't care to understand or memorize the entire Windows API with every syscall ordinal and parameter format. It's just rather pointless and especially has nothing to do with what you are seeking, which is learning how software controls hardware.

You cannot do this in a modern operating system without years of experience with that particular operating system and the knowledge to write and debug drivers in a paged memory multi processor pre-emptive multitasking environment. It's just way too complex.

I've suggested before you are better off working from DOS, an old game console, or even an Aurduino or Pi or some type of microcontroller that allows you to interact with and learn how hardware and software integrate. Trying to learn these concepts for the first time with an OS as large as Linux and Windows with no prior expertise on their internals is practically impossible.

How IRPs and DPCs and IRQLs and locked pages all that stuff work is well beyond just wanting to write some data to an IO port and see what happens to the voltage on a pin. And the thing is even if you were able to pick that up fairly quickly, scheduling an IRP still isn't "writing directly to the hardware" like you want anyway.

Get an old DOS PC already! You will have LOADS of fun and learn a lot. Trust me.

 

[Merad]

Aurduino or Pi or some type of microcontroller that allows you to interact with and learn how hardware and software integrate

I second this. Modern processors are mind-bogglingly complex. Even something like the "simple" MCU on the Arduino Uno (Atmel 328) has a datasheet on the order of 500 pages. That's a lot, but you can wrap you head around all of it with enough time and effort. You get to dick around with all kinds of low level stuff that the OP seems to like, and IMO the hardware side of it is pretty fun as well. From there you can upgrade to much more advanced ARM based MCUs like what you'll find on the Arduino Due (Atmel SAM3X8E) and eventually go all the way to a full SOC like the Raspberry Pi's BCM2835.

 

[chrstrbrts]

Get an old DOS PC already! You will have LOADS of fun and learn a lot. Trust me.

Yes, you've made similar suggestions in the past.

Namely, getting an old GameBoy (with special system's ROM cartridge I presume).

I respect your advice, but here's the issue:

Tech moves so quickly.

I'm interested in learning how modern computers work.

My fear is that I will invest appreciable time and effort into studying an old DOS based desktop or GameBoy only to learn that 95% of my newly acquired knowledge was supplanted 20 years ago.

I do indeed see the wisdom of your advice, but the above scenario remains my big fear.

How would you suggest that I take from studying older systems only what's relevant today while discarding what's been rendered obsolete?

Thanks.

 

[Merad]

I'm interested in learning how modern computers work.

You have the wrong impression of what constitutes modern computers. Most of the topics I recall you expressing interest in are things that haven't changed significantly in 20-30 years.

 

[chrstrbrts]

You have the wrong impression of what constitutes modern computers. Most of the topics I recall you expressing interest in are things that haven't changed significantly in 20-30 years.

OK. So, the low-level memory management: paging, segmentation, kernel protection, virtual addressing, etc. haven't changed since the early 80's or so?

So, everything I learn about memory management on an old DOS machine should remain valid today?

Then, what has changed appreciably?

Obviously, memory capacities and speeds have grown by several orders of magnitude.

Bus technology has radically changed as well, right?

I/O memory access too has changed, yes? From polling based programmable I/O to interrupt driven DMA.

But, what else?

Did the old DOS era machines have MMUs and TLBs?

 

[Exophase]

OK. So, the low-level memory management: paging, segmentation, kernel protection, virtual addressing, etc. haven't changed since the early 80's or so?

At the level you're looking at it, yes. More or less.

So, everything I learn about memory management on an old DOS machine should remain valid today?

No, DOS was never a proper OS, it wasn't ever implemented with or updated to include most of those features

Then, what has changed appreciably?

In terms of CPU architecture, not that much. SIMD and virtualization come to mind, but those aren't really core to what you're doing.

I/O memory access too has changed, yes? From polling based programmable I/O to interrupt driven DMA.

Interrupt driven DMA I/O is very old.

Did the old DOS era machines have MMUs and TLBs?

386s and newer have TLBs. 386 came out in 1985 and it and clones were widely used by the late 80s. So it really depends on what you mean by "DOS era."

While DOS didn't mess with any of this stuff itself there were third party memory managers and extenders that did and they became pretty commonly used.

 

[Merad]

In terms of CPU architecture, not that much. SIMD and virtualization come to mind, but those aren't really core to what you're doing.

You could arguably put superscalar and SMT/Hyperthreading on the list. Both technically date back to the 60s and were released in some products prior to the 90s, but both went "mainstream" when Intel released products featuring them (the Pentium in '93 and and P4/Xeons around '02-03).

 

[exdeath]

OK. So, the low-level memory management: paging, segmentation, kernel protection, virtual addressing, etc. haven't changed since the early 80's or so?

So, everything I learn about memory management on an old DOS machine should remain valid today?

Then, what has changed appreciably?

Obviously, memory capacities and speeds have grown by several orders of magnitude.

Bus technology has radically changed as well, right?

I/O memory access too has changed, yes? From polling based programmable I/O to interrupt driven DMA.

But, what else?

Did the old DOS era machines have MMUs and TLBs?

Yes. DOS is in real mode with no protection in place. Make sure you aren't confusing items on your checklist as being associated with the OS and thus not supported in DOS. These are features of the x86 hardware platform that are unclaimed by DOS allowing you to experiment freely. You can do whatever you want and reboot in 2 seconds when you hang for the 10,000th time while experimenting.

You can use cpuid, lgdt, use CR0 and CR3 registers, in/out, etc. It's all unprotected with the CPU booted in x86 real mode. DOS would give you a working dev platform to learn to write your own DOS extender and explore protected mode, paging, etc. without the overhead of having to write your own loader from scratch or having to be a Linux/Windows kernel expert.

You'd need to be careful with interrupts to avoid triple faulting and write your own protected mode <> real mode handoff mechanism (for debugging and user interaction unless/until you want to reinvent the wheel in protected mode). In other words write your own DOS extender. Far more manageable than writing an OS from scratch. There should be a bajillion "how to go to protected mode" tutorials from real mode DOS that talk about how to preserve your real mode interrupts, thunk back to real mode for DOS and BIOS calls, etc. Once you get this functional, then you can play in protected mode and be the boss, and play with descriptors and page tables and IO addresses to your heart's content.

When I said "old DOS PC" I meant more a PC capable of booting DOS with a standard MBR BIOS able to boot DOS, we are still talking 386-Pentium+, not an 8088/286. I'm not sure to what extent a modern PC can even run DOS but it's mostly because of BIOS evolution. I can't say I've tried recently.

All of the concepts you are talking about have remained fundamentally the same for the past 30 years, even for x86 which is considered a late adopter on top of that. It may not seem that old because mainstream PC operating systems and software (eg: Microsoft) took a LONG time to arrive where we are now. Example, we were still gaming in DOS in the 90s and using Windows 9x when Pentium was new, a decade after paging was added to the 386 in like 1985, coincidentally for the same reasons you want (it was easier to bypass the protection and complexities of dealing with a modern OS and have exclusive control of the CPU environment with a DOS DPMI 4G extender).

 

[chrstrbrts]

OK. Do you have any specific, old PC models in mind?

Hopefully, I can find some of your suggested models on eBay.

 

[Exophase]

You could arguably put superscalar and SMT/Hyperthreading on the list. Both technically date back to the 60s and were released in some products prior to the 90s, but both went "mainstream" when Intel released products featuring them (the Pentium in '93 and and P4/Xeons around '02-03).

I'm making the distinction between architecture and microarchitecture, so I wouldn't count pipelining, superscalar, OoOE, etc as architectural advancements.

SMP has architectural implications, but it's also been around for quite a long time, even in x86. In the case of Hyperthreading I would count that as architecturally mostly built off of SMP (so that there simply appear to be more cores), but in other architectures SMT has become more of its own thing.

 

[Exophase]

DOS would give you a working dev platform to learn to write your own DOS extender and explore protected mode, paging, etc. without the overhead of having to write your own loader from scratch or having to be a Linux/Windows kernel expert.

Starting with real mode in DOS doesn't give you an awful lot more than starting with a bootloader from a floppy disk. I guess the hard part is actually being able to boot from a floppy, but compared to getting a system that boots DOS...

Either way it's a really bad idea to not develop this in an emulator first. I learned this the hard way in undergrad when I did an OS for an independent study course. I wasted so much time on reboots from floppy disk. When I finally started using Bochs I fixed much more in much less time. The only downside was that when I finally did move back to the real system I had an issue that Bochs didn't catch, because I had a floppy driver using the high speed mode that no 2MB drive supports.

 

[Fox5]

X86 and DOS are really the wrong place to start if you want to learn how modern computers work.

You'd be better off getting a guide on embedded ARM development and one of the low end ARM boards, like a raspberry pi. Much less baggage and simpler to learn.
Even simpler still would be something MIPS based, while still being pretty modern.