I think to some extent that it comes down to sloppy, quick designs.
For example, if you remove the heatsink of a standard 4850 card, you can see that there's a lot of empty space on the PCB. Also, I think it was Sapphire that made a modified 4850 that was much shorter than the reference design, proving that it's possible.
I can understand that a high-end, power hungry card like the GTX285 or 5870 might have to be longer, but why mid-range and low-end cards? The 4770 is a bit shorter, but even the 4830 is as long as most high-end cards.
Maybe all the machinery etc. is standardized for the high-end cards and then they re-use the bare circuit boards for the lesser cards?
My 4850 sits right up against the hard drives, leaving one 3.5" bay unusable. With a dual-slot card, two bays would probably be blocked... maybe it would work if I turned the hard drives around so the connectors pointed to the front of the case but I'm not sure. Also with my previous mobo, the card blocked two SATA ports - a dual slot card would have blocked all four.
I can't see myself ever buying a full tower case..so ugly and big... but the way video cards are evolving, I might have to take the plunge eventually. At least CPU manufacturers had the sense to stop at 140W TDP's - video cards just keep using more and more power, running hotter and hotter...It's only a matter of time before the first triple or quad-slot card appears with a 400W TDP. Too bad. My previous system was a Shuttle mini-PC and it fit the most high-end components at the time, 3500+ CPU and the X800XT video card. Loved the portability and stylish design.