Buffer overflow exploit detection?

CTho9305

Elite Member
Jul 26, 2000
9,214
1
81
I figured this would be the best place to ask...

I'm not familiar with the memory layout for windows, but I know the setup under linux. Do exploits generally spend a fair amount of time running the code from the stack? If it is longer than 1 time slice... the kernel could do a quick check of $eip and if it is higher than $esp, kill the process. I can't imagine an exploit getting back into the .text section (or some other location) that fast, unless it spawns a child ftp downloader or something and then returns (?). Even then, I don't know if there will be faster than a time slice. The penalty would be about 1 instruction more per context switch.
 

Matthias99

Diamond Member
Oct 7, 2003
8,808
0
0
I think (correct me if I'm wrong) it would be pretty trivial for the exploit code to completely trash the stack pointers and frame, thereby invalidating your detection mechanism. Also, context switching is something that gets optimized like crazy, so even a small overhead on each one could hurt performance significantly. However, given the number of problems coming out of this sort of bug, it would probably be worth it if it worked 100%. Better would be to insist on not using unbounded string copies in places where you're getting data from the outside world in the first place.
 

sao123

Lifer
May 27, 2002
12,656
207
106
In order to understand the way buffer overflow expoits works you must understand how the stack works.

When a function(C++ term for subroutine) is called the following things happen...(without going into excessive elaborite detail)
1)All the data in local variables, passed parameters, & the registers - including the CPU Instruction Address register is put on the stack.
2)The jumpto location is loaded into the CPU Instruction Register and code begins executing at this new address.
3)The function body is executed.
4)After completion the data on the stack is popped back into their original places. Including the CPU instruction Address (this provides the return to address)
5)Executing resumes at the return to address.

Buffer overflow exploits flood the passed parameters with data to overwrite the location in the stack containing the return to address. Thus if a valid address is supplied here, it can point to any place in memory and run any piece of code desired at the new address.

The problem with your theory is that the kernal wouldnt know if a particular return address is correct or not, since it cannot actually see and modify the running code, it merely controls the time slice available to this particular code.
Code actually never runs from the stack. Only the piece of data that stores the return address within the stack is overwritten.

What has to happen is that the input and passed parameters must be checked to be sure the size matches the type of data expected, if there is more data than will fit in the variable location the excess must be discarded.
Example: this information can be obtained using the sizeof() function.
bool = 1 byte
char = 1 byte
short = 2 bytes
int = 4 bytes
long = 4 bytes
float = 4 bytes
double = 8 bytes
long double = 8 bytes
int * = 4 bytes
etc....
now if i was using int variables and flooded it with a integer larger that can be held in 4 bytes
(n > 4294967296 which would occupy 5 bytes) then it would cause a buffer overflow. Now to be realistic, it would have to be more than 1 byte of data to be effective...but you get the idea. It could take many bytes depending on the number and size of parameters your dealing with that were pushed onto the stack. You have to get to exactly the location which stores the return address and overwrite it with another valid address containing your code and that could be very complicated.
 

CTho9305

Elite Member
Jul 26, 2000
9,214
1
81
Originally posted by: Matthias99
I think (correct me if I'm wrong) it would be pretty trivial for the exploit code to completely trash the stack pointers and frame, thereby invalidating your detection mechanism.
If the exploit code wants to use more than 4 registers, it's going to need a reasonably valid stack pointer. I guess it could move the stack pointer up above its own code... but doing that would break the stack enough that the victim app (say IIS) would crash, rather than serve up copies of the worm (like Nimda did).

Also, context switching is something that gets optimized like crazy, so even a small overhead on each one could hurt performance significantly.
Yes, but I think you have much higher penalties already (a context switch takes multiple us), so ~one instruction is a very small extra penalty. At 2000 context switches per second, the penalty would be 2000 instructions, out of 2 billion.

However, given the number of problems coming out of this sort of bug, it would probably be worth it if it worked 100%. Better would be to insist on not using unbounded string copies in places where you're getting data from the outside world in the first place.
Of course. And that in combination with a nonexecutable stack would be ideal. However, lazy programmers will be lazy programmers.


Originally posted by: sao123
In order to understand the way buffer overflow expoits works you must understand how the stack works.
....
After doing this homework, I have a pretty good understanding ;).

The problem with your theory is that the kernal wouldnt know if a particular return address is correct or not, since it cannot actually see and modify the running code, it merely controls the time slice available to this particular code.
Right. When the timeslice ends, the kernel saves all the registers. Among the registers it saves are EIP and ESP. It can see whether the instruction pointer is in the stack.

Code actually never runs from the stack. Only the piece of data that stores the return address within the stack is overwritten.
Read over your explanation of buffer overflows. The exploit code starts out on the stack. To get that code to execute, you set the return address to point to your code on the stack. Getting your exploit code into the heap would be more difficult.

now if i was using int variables and flooded it with a integer larger that can be held in 4 bytes
(n > 4294967296 which would occupy 5 bytes) then it would cause a buffer overflow.
No, that would cause an overflow, which would either be caught or ignored. If your data type is specified as int, no matter what you do, the CPU isn't going to generate more than 32 bits of data.
 

sao123

Lifer
May 27, 2002
12,656
207
106
Quite an interesting assignment you got there.... As a one time instructor in the areas of C++, Data Structures, with particular interest in also the areas of Assembly Language, and System Architecture id be interested to see the solution to it. :sun:



Right. When the timeslice ends, the kernel saves all the registers. Among the registers it saves are EIP and ESP. It can see whether the instruction pointer is in the stack.

Read over your explanation of buffer overflows. The exploit code starts out on the stack. To get that code to execute, you set the return address to point to your code on the stack.

Thats not what i said...
The instruction pointer (EIP) never points to an address in the stack (in the range EBP, ESP). What i said was... the return instruction pointer contents (an address) are pushed in the stack, and then popped off the stack.

When a function jump is called the address in the EIP is incremented to the next and then the EIP contents (current instruction address) is pushed (stored) as DATA in the stack which eventually becomes the return address when it is popped. If this DATA (return address) is overwritten by a different valid address, when the return instruction executes: the return address is popped from the stack back into the EIP, and code will continue to execute at the new return address. Code is never stored or executed in the stack.



No, that would cause an overflow, which would either be caught or ignored. If your data type is specified as int, no matter what you do, the CPU isn't going to generate more than 32 bits of data.

what your suggesting to me here is that buffer overflow usually must take place on unformatted data...interesting. Now I see, arrays of any type though formatted by a particular type...are very much subject to overflow because there is no true boundry... I missed that linking concept last time i studied this. Thanks for the insight.
Pass by value parameters are passed in array form inside the stack which facilitates buffer overflow during function calling.
 

CTho9305

Elite Member
Jul 26, 2000
9,214
1
81
Originally posted by: sao123
Quite an interesting assignment you got there.... As a one time instructor in the areas of C++, Data Structures, with particular interest in also the areas of Assembly Language, and System Architecture id be interested to see the solution to it. :sun:



Right. When the timeslice ends, the kernel saves all the registers. Among the registers it saves are EIP and ESP. It can see whether the instruction pointer is in the stack.

Read over your explanation of buffer overflows. The exploit code starts out on the stack. To get that code to execute, you set the return address to point to your code on the stack.

Thats not what i said...
The instruction pointer (EIP) never points to an address in the stack (in the range EBP, ESP). What i said was... the return instruction pointer contents (an address) are pushed in the stack, and then popped off the stack.

In all of my solutions to the assignment I linked to, my exploit code most certainly resides on the stack. The return address is overwritten with the expected location of the exploit code - somewhere on the stack. When the vulnerable function returns, EIP is pointing to the stack.

When a function jump is called the address in the EIP is incremented to the next and then the EIP contents (current instruction address) is pushed (stored) as DATA in the stack which eventually becomes the return address when it is popped. If this DATA (return address) is overwritten by a different valid address, when the return instruction executes: the return address is popped from the stack back into the EIP, and code will continue to execute at the new return address.
Right.

No, that would cause an overflow, which would either be caught or ignored. If your data type is specified as int, no matter what you do, the CPU isn't going to generate more than 32 bits of data.

what your suggesting to me here is that buffer overflow usually must take place on unformatted data...interesting. Now I see, arrays of any type though formatted by a particular type...are very much subject to overflow because there is no true boundry... I missed that linking concept last time i studied this. Thanks for the insight.
Pass by value parameters are passed in array form inside the stack which facilitates buffer overflow during function calling.
buffer overflows happen with arrays of anything. If the array is statically defined (e.g. char mystring[10]), it will be on the stack, and thus a buffer overflow can result in manipulation of the return address. If the array is dynamically allocated (e.g. char* my string = (char*) malloc(10*sizeof(char))), it will live on the heap, and worst case, your data will be overwritten.

edit:

Let's say we have the following function:
void crackMe()
{
char myBuf[12];
printf("enter your name\n");
Gets(myBuf);
}

Right after crackMe is called, the stack looks like:
eip - where crackMe's caller will return to
ebp - saved ebp
...
caller's local variables
...
eip - where crackMe should return to
ebp - saved ebp
The next step is to create room for the buffer on the stack:
eip - where crackMe's caller will return to
ebp - saved ebp
...
caller's local variables
...
eip - where crackMe should return to
ebp - saved ebp
_ _ _ _
_ _ _ _
_ _ _ _

Note that we now have a 12 byte buffer on the stack.

Ignore the printf.

Now we're at the Gets. We push the address of that buffer and call our Gets function. Let's say the user enters 11 characters.
The stack when Gets returns will be:

eip - where crackMe's caller will return to
ebp - saved ebp
...
caller's local variables
...
eip - where crackMe should return to
ebp - saved ebp
h e r 0 - note terminating zero.
s t o p
C h r i

All is well. What if, instead they enter something longer?
eip - where crackMe's caller will return to
ebp - saved ebp
...
caller's local variables
...
n j 0 0 - where crackMe should return to
x I p w - saved ebp
3 t h a - note terminating zero.
m y l 3
F e a r
Now, the return address has been overwritten. In this case, the call will return to address 30304A4E (I think... 0 0 J N ascii codes), and the program will most likely crash (segfault, protection violation, illegal instruction, whatever).

Instead, let us enter a string which contains some machine language, and the 4th set of 4 bytes will be specially crafted to point somewhere on the stack:
eip - where crackMe's caller will return to
ebp - saved ebp
...
caller's local variables
...
22 a3 ff bf - where crackMe should return to
# # # # - saved ebp
# # # # - note terminating zero.
# # # #
# # # # - let us assume that this is location 0xbfffa322, for convenience

Obviously that return address is somewhere on the stack in linux... it would have to be crafted to fit the program you're exploiting. The #s represent machine language I don't feel like generating for this post. Now, when CrackMe returns, it will find that return address, (0xbfffa322) and jump there - now it is executing our exploide code off the stack. If the code is complex enough, or we get lucky timing, a context switch will occur while we're executing exploit code off the stack. The kernel sees EIP pointing in the stack, and uh oh, we know we've been exploited.

Exploits that jump to existing code (let's say IIS had a downloadAndExecuteFile function somewhere) could escape detection most of the time of course, because they'd just jump to an allowed location, but if your exploit code itself does anything fancy, it could be caught.

This is what my prof said when I asked:
Chris,
This would help with that particular exploit. However, some
OS's such as Linux actually depend on being able to run
user code on the stack in order to process signal handlers,
so somehow you'd have to allow for this. Also, there are
other exploits that execute existing code and others that
work by overwriting heap and .data buffers that would slip by.

edit2: I won't post my exploits because I know at least a few schools use this same assignment... but if you want to talk about it, pm me or AIM/ICQ me.
 

sao123

Lifer
May 27, 2002
12,656
207
106
ok i see we're now saying the same thing...What I was saying is that you could could have put your code anywhere in the available memory page and used that address as a return segment. This is especially useful if you can replace an existing DLL with an exploited one, you could return to an address in your dll directly in this case.

but just for convenience you placed your 1st exploit instruction on the stack and made the return pointer point to the stack, a very sensible thing to do if your exploit code is small...even using just an additional jump statement to the real exploit code block. Very well done.
I will PM you soon, as I wish to seek further details about this assignment. I may want to include it in my curriculum for next semester.
 

kylef

Golden Member
Jan 25, 2000
1,430
0
0
Your professor is correct: some obscure algorithms depend on being able to run code in the stack. It has been proposed that the stack should be marked no-execute and these algorithms be rewritten, but it seems like backwards compatibility always wins out. (and it doesn't help that Intel x86 has no capability to mark pages no-execute in hardware unlike Itanium)

VS.NET offers a new C/C++ compiler flag (/RTCs) that performs stack frame error checking. If you look at a disassembly of the code it generates for function calls, you'll see something like the following:

1. Push the return address on the stack
2. XOR the return address with some kind of key and store the resulting hash in a different location (not the stack)
3. Execute the function call
4. Before returning, compute another hash of the stack's return address; compare this new hash to the one computed and stored by the caller; if the two match, return normally. Otherwise, throw an exception.

That's basically it in a nutshell. Obviously, this does nothing for heap attacks, but there are other checks you can do for heap overrun detection.