I haven't had any major problems with software compatibility that I can think of (and games run fine), but the situation with drivers (or more accurately, the lack thereof) is enough to drive anyone mad. Unless you are absolutely 100% sure that you can locate x64 drivers for every single piece of hardware in or attached to your computer, don't bother.
Since as far as I know there's yet to be any mainstream 64 bit app that gets real performance gains, that pretty much makes the hassle not worth it for most people. I didn't even install it until I sold my HDTV tuner (which doesn't have 64 bit drivers and probably never will), and now whenever I buy a new one I have to make sure it's one that does have 64 bit drivers. With Vista coming out though and x64 hopefully becoming somewhat more mainstream, I hope more hardware manufacturers will provide support.
