It's really tough to guage the effect of cache on performance. If you look at the original Sempron 3100+ "Paris," it was a pretty big step down from the Athlon 64 2800+, although the only differences were a 256 KB L2 (instead of 512 KB) and no 64-bit extensions. On the other hand, the Sempron 2800+ and 3100+ Palermos, with 256 KB L2 caches, are very similar in performance to the 2600+ and 3000+, respectively, which have 128 KB L2 caches.
I just bought the 64-bit Sempron 2800+ with an Epox EP-8KDA3J motherboard, and I couldn't be happier. With a simple retail box cooler, I've been running at 2.28 GHz (as opposed to the stock 1.6 GHz) for over 14 hours straight, now, without a hitch. It's possible I may need to back off, later, but I'm sure you'll be able to reach 2.0 GHz without a problem.
Also, the 64-bit Semprons all have the latest instruction sets, namely SSE3. The 32-bit Semprons do not... except maybe the 3100+ and 3300+ Palermos...do they have SSE3? I don't think so, but I could be wrong.