AMD cards you listed are slower than a 9800GT. No amount of "driver overhead" is going to make an HD5670 faster than a 9800GT.
New optimized drivers, new game engines, and "vendor bias" can certainly put the 5670 ahead of the 9800GT.
Also here the 5670 and 9800GT trade blows.
I don't really think CPU driver overhead comes into play with the 9800GT, but if it did then the two cards are close enough in performance for overhead to certainly put the 5670 ahead.
I have a question in this regard. I am running an E4500 at stock, and a 9800GT. It seems like I am CPU limited in a lot of games. How much extra CPU performance would I get if I went to something like a HD5670 or HD6670? Would it be worth the switch?
It's not worth the effort for you to swap out video cards. All three (5670, 6670, and 9800GT) are close in performance and trade blows with each other. You likely wouldn't even notice the difference at all, because your CPU is going to bottleneck you especially in multithreaded games. You would be better off researching if you can drop in a CPU upgrade to your system than switching video cards.
Also the extra CPU overhead for Nvidia drivers has only been shown on higher end Fermi cards as far as I know. The 9800GT's drivers may not have any more overhead than Radeon drivers. So again, this is just another reason to stand pat.