I dual booted x64 for 2 days. After that I never went back to 2k, except to install x64 when it went out of Beta status.
I have not had any problems with games and x64, at least compared to problems I've had with 32-bit XP. Everything runs just as fast, sometimes a little faster. It helps that my onboard devices are all nForce-based, as nVidia had 64-bit drivers out instantly. Same for my video card, as it's an x800 Pro and ATI did the same thing. My self-powered micro-LCD even works, and has built-in support with x64. (Chips 65550-based) The only things that chafe me with x64 are the fact that my TV tuner doesn't have driver support, and that there's nothing like Daemon Tools out for it yet. I've had to limp along with IsoBuster in the meantime.
I'd recommend going to x64 if you have major brand equipment only. Otherwise you may want to hang back.