Writing OS code

[chrstrbrts]

Hello,

When writing kernels and OSes in general, whether it be in assembly or something higher level, you need to assemble or compile your code in flat binary, right?

You can't assemble it or compile it to anything like ELF format or anything like that, right?

If you did, the processor would misinterpret the formatting as code and start executing unintended instructions.

After all, you format executable binaries so that the OS knows where code and date segments start and stop and then can load them into the GDT and add them to the paging structures.

But if the program you're writing actually is an OS, then it won't running over an OS like a user app will as it IS the OS, right?

That is, the OS runs on the metal, not over any other software.

Am I correct here?

 

[Cogman]

Yes.

Though generally there is a bootloader + OS. The bootloader is that flat binary and it simply serves the purpose of choosing which version of the OS you want to load up.

I'd look into Grub to learn more about how semi-modern boot loaders work.
https://en.wikipedia.org/wiki/GNU_GRUB

 

[chrstrbrts]

The bootloader is that flat binary...

Yes, I know what a bootloader is.

But, you're telling me that only the bootloader is assembled into flat binary?

The OS is not?

How can the OS be assembled / compiled into some kind of format?

There is no software to interpret that format and load things properly into memory.

The OS isn't running over any software the way that a user app runs over the OS.

The OS runs right on the metal.

Right?

 

[Merad]

How can the OS be assembled / compiled into some kind of format?

You write code to load your OS, of course.

The bootloader really just does the bare minimum amount of setup necessary to hand control over to the kernel's startup code. That code has to do a lot of set up, such as doing some very basic memory initialization and taking the CPU from 16 bit mode to 32 bit and ultimately 64 bit mode. It loads the kernel into memory, and will eventually call the kernel's main function. Only then does the OS itself really begin to boot.

You will probably find this interesting: https://0xax.gitbooks.io/linux-insides/content/Booting/linux-bootstrap-1.html

 

[chrstrbrts]

You write code to load your OS, of course.

The bootloader really just does the bare minimum amount of setup necessary to hand control over to the kernel's startup code. That code has to do a lot of set up, such as doing some very basic memory initialization and taking the CPU from 16 bit mode to 32 bit and ultimately 64 bit mode. It loads the kernel into memory, and will eventually call the kernel's main function. Only then does the OS itself really begin to boot.

You will probably find this interesting: https://0xax.gitbooks.io/linux-insides/content/Booting/linux-bootstrap-1.html

Yeah, I know what a bootloader does.

But answer me this: to what format do I assemble or compile OS source code?

Flat binary or something else?

 

[mxnerd]

All computers can only run native binary codes. No matter what OS or platforms. Binary code in different formats only means how it's laid out in a file.

 

[chrstrbrts]

All computers can only run native binary codes. No matter what OS or platforms. Binary code in different formats only means how it's laid out in a file.

Thanks, but I already knew that, and that doesn't answer my question.

My question stands: to what format do I assemble or compile OS source code? Flat binary or something else?

 

[mxnerd]

If Windows, usually it's PE (Portable Executable) IIRC.

I have no idea for Linux.

And I don't get the meaning of "flat" binary term. Binary is binary, why "flat" binary?

Are you studying computer science courses?

 

[chrstrbrts]

If Windows, usually it's PE (Portable Executable) IIRC.

I have no idea for Linux.

You're not interpreting my question properly.

I'm not asking what format to assemble code to run over a pre-loaded OS.

I'm asking what format in which to assemble my own OS code.

Example: I wrote my own kernel source code.

What format do I tell the assembler to use for the binary that it produces?

I think it's flat binary, that is no format at all.

Am I right?

 

[chrstrbrts]

And I don't get the meaning of "flat" binary term.

Jesus......

 

[chrstrbrts]

Flat binary means no formatting. No tables or table headers. Just your code in machine language instruction after instruction.

 

[mxnerd]

Hahaha, if you want to write your own OS, why do you care what format it is.

You can lay it out any way you want. Just remember to load them correctly.

Jesus Christ...

If your "flat" binary file is big enough, across several disk sectors, then when you use hex viewer/editor to view executable files, you absolutely will see chunks and chunks of flat binary code after headers. Like what you said, instruction after instruction.

But I believe most OS will have some kind of header section for binary files. Care to share your experience?

 

[chrstrbrts]

Hahaha, if you want to write your own OS, why do you care what format it is. You can lay it out any way you want.

I'm not talking about the format of the OS itself. I'm sure there are an infinite number of ways to organize the data structures and control mechanisms that constitute an OS.

I'm asking about the format of the binary that constitutes the OS's machine code.

If I tell the assembler to output my source code in ELF format, for example, then when the bootloader loads the kernel into memory and sets the instruction pointer to the head of my kernel's binary, the CPU will interpret the header information and formatting info as instructions to be run even though they're not meant to be treated as such.

Get my point?

 

[mxnerd]

If CPU is set to execute the code from the code section, why will it execute the header portion? header portion is data, not code.

x86 CPU has an instruction pointer to tell it where the code address is and where to start.

 

[chrstrbrts]

Care to share your experience?

I have none. I'm brand new and am still learning.

I just learned, though, from another site *cough* stackoverflow *cough* that bootloaders can be sophisticated enough to decipher formatting on the fly and load formatted binary into RAM to be run properly.

Though simplistic bootloaders would require flat binary.

 

[chrstrbrts]

If CPU is set to execute the code from the code section, why will it execute the header portion? header portion is data, not code.

Yes, but only after the formatted code has been deciphered. You have to know the format of the code and set the IP register to the right place. You have to have a bootloader sophisticated enough to do this.

 

[mxnerd]

Even if a file is a complete flat binary as you described. Always there will be data in between instructions.

 

[chrstrbrts]

Even if a file is a complete flat binary as you described. Always there will be data in between instructions.

What?! How?

 

[mxnerd]

I admit I'm not familiar with bootloaders. Need to investigate.

 

[mxnerd]

What?! How?

You always have CPU registers to add values, move some values, point to some addresses, those are data, aren't they?

 

[mxnerd]

I think the decipher thing you mentioned is only needed if the system / disk file is encrypted in a way that the CPU does not recognize the code .

If it's not encrypted, there is absolutely no need to decipher or decrypt the code on the fly.

If the system is encrypted, then you absolutely need a small chunk of decrypting code loaded into memory first before loading anything else.

 

[chrstrbrts]

You always have CPU registers to add values, move some values, point to some addresses, those are data, aren't they?

Man, you have an unusual way of interpreting things. Yes, obviously the above is true, but data is not pushed into your code when your formatting is flat. When your formatting is flat, the assembler just spits out exactly what you've written, nothing more nothing less.

I think the decipher thing you mentioned is only needed if the system / disk file is encrypted in a way that the CPU does not recognize the code .

If it's not encrypted, there is absolutely no need to decipher or decrypt the code on the fly.

If the system is encrypted, then you absolutely need a small chunk of decrypting code loaded into memory first before loading anything else.

I didn't mean decipher literally. I meant the bootloader needs to be able to make sense of the formatting.

 

[mxnerd]

Well, if you think my interpreting is unusual, probably someone else can interpret what you really mean.

You can't even give an example what flat binary format is in your mind.

 

[chrstrbrts]

Well, if you think my interpreting is unusual, probably someone else can interpret what you really mean.

You can't even give an example what flat binary format is in your mind.

Is English your first language?

First of all, I can't believe you don't know what flat binary means; it's a very common term.

Second, I've given several examples of flat binary in the messages above.

You didn't even look it up, did you?

Go here specifically: If a binary file does not contain any headers, it may be called a flat binary file.

 

[mxnerd]

You are correct, English is not my first language.

2nd, I know exactly what you really mean flat binary file is.

What I don't understand is that you kept saying bootloader will misinterpret the code, why? And need to make sense of the formatting? IF it's that flat binary file format, why won't it make sense of it?

By the way, I learned x86 assembly 30 years ago, beginning with 8008 microprocessor board that only has bitswitches and LEDs.

I went through APPLE II 68000 processor, 8080, 80286, 80386, Pentium,... till now.

My environment is almost Windows exclusively after I started working.

Although I don't write programs anymore, I have used debugger/disassembler tools like Olldbg / PE Explorer, etc. before.

If you are talking about Linux, sorry I wasted your time.