it depends on the motherboard... I've used computers with no heatsink all the time. Not by choice, but sometimes, when all you need to do is see if a motherboard posts, finding and installing a heatsink is just too much of a pain in the ass. either way, were you to remove the CPU mid-operation, I'd expect to see an instant crash, and possibly a motherboard-frying power surge.
if you're talking about a PCI card... again, I've removed PCI cards when a computer is running. I've never had a problem because of it (thank god), but then again, it was only once and I had a total senior moment and forgot that the server I was working on was powered on. worst case scenario, you're looking at another possibly motherboard-frying power surge.