What's the application? Is it indeed a Windows app, or is it a console app?
Do you know what DLLs it loads? Does it load COM components? Are the COM components out-of-process? Do the out-of-process COM servers exist once the "parent" process is killed? Is it a .NET app?
There are many, many reasons why this could happen, but if you can answer all of the above, I can tell you what's wrong. If it's simply a normal app then everything loaded is removed once the process is destroyed. The only way any state of disorder would be persisted is if the process serialized its memory to disk, then deserialized it once it was opened again...