[Digging back into my memory .. ]
Here's what I recal from my introductory Operating Systems class, I can't apply it directly to your overclocking scenario (although, like mentioned, that could cause minor errors which would create some of the following conditions ... )
There are four conditions for deadlock, which is when processes are effectively locked against eachother, preventing any work being done. Now, depending on the scheduling policy, this may cripple or disable the system.
The first condition is hold and wait, i.e., a process "holds" a resource, and is waiting for some other condition. The resource may be a space in memory, or access to some other shared resource. The second is not pre-emption, saying that a process may not forcefully take a resource away from another process (imagine if any running process could grab ANYTHING from any other, at any time it wished, the system would cease to function). The third condition is Circular Wait, i.e. Process A is waiting for a resource Process B holds, while Process B is waiting for something that Process A holds. This may continue on for N processes, as long as they are waiting on eachother. The last is that there is mutual exclusion, i.e. [certain] resources may only be held by a single process at a time (picture a single variable, you can't have two threads modifying/reading it simultaneously and expect deterministic behavior).
Those are the standard conditions that must exist, I cannot speak for what happens when a machine is overclocked (All Bets Are Off). But following this logic, like Capybara said, the CPU will be running full bore, trying to do "something". It may be an infinite loop (SuperTool's message ), or it may just be occupying all it's time with contect switches (switching from one process to the next, trying to get some work done), or doing something else equally useless.
I hope this make some sense (and is correct =) ).
-Josh