As SynthDude said, all games will have better performance with an Athlon64 running a 32-bit OS. However, upgrading to x64 does give you the advantage of being able to apply the FarCry 64-bit patch. Having it applied myself I have to say that the performance difference is
striking. You can really see farther and with more detail, and framerates even seem to go up a bit.
But as has been mentioned, very few programs support x64 yet. (
in the sense that they work, but have no improved performance) Drivers can also be a nightmare, particularly for "lesser" companies. If you have all major-company stuff, like an nForce-based board, a legitimate Creative Labs SB, etc., it isn't very hard to find drivers at all though. Daemon Tools doesn't work with x64 at all, and the next version -- which promises it will -- isn't out of Beta yet. That's a bit of a downer. On the plus side, there are a number of 64-bit builds of popular open-source software, like Firefox. (
as if the speed boost would be that noticable 
)
Personally, I'm rather happy I went to x64.
All my programs work with it -- though Deus Ex has a goofy resolution cap -- and the only device I can't get drivers for is my Chinese-made TV tuner. Seeing as I used it all of twice before, I'm not that broken up about it. Daemon Tools is probably the biggest inconvenience I've encountered so far, and that will supposedly remedy itself soon. I'm also in the process of acquiring the build tools necessary to compile my own 64-bit apps, which means I'll be able to make 64-bit Mozilla (
not Firefox) and whatever else I can find the source code for. A lot of my 32-bit apps seem to run faster on x64 as well. The hacked OS X theme doesn't hurt things either.
In short, I'd suggest trying x64 first if at all possible. It may work out to your liking, or it may have limitations you aren't ready to deal with yet.