When x64 came out I switched over to it and never looked back. I've had no serious problems with x64, barring a temporary lack of Daemon Tools. (Now fixed, thankfully) Here's my thoughts:
First off, the only software you really have to worry about not working is antivirus software. At the moment there's only a handful of free AV solutions for x64, the best of which probably being Avast! Antivirus. However, Avast is a bit of a resource hog at times; certainly if you do any significant amount of downloading you'll notice some issues with other programs running while you're downloading if you have Avast running. Personally I threw in the towel and bought a 2-year license of NOD32, which is much less CPU-intensive. The only downside to it I've noticed is that all the executables run in 32-bit mode, as the only 64-bit specific modules are dll files. It's not really a problem, just vaguely irritating if you're like me and would like as much stuff to run in 64-bit mode as possible. Also, any 16-bit programs you have will not run on x64. Then again, if you're using 16-bit programs you should probably just build a legacy gaming system. On that note, check out DosBOX.
Secondly, unless you have some really weird hardware in your computer you shouldn't worry about drivers. I know of no 64-bit compatible motherboard that doesn't use a chipset with 64-bit drivers. Most of them use nForce-based chipsets, and nVidia has good (though not excellent) 64-bit driver support. The only thing that I couldn't find drivers on the site for was my onboard IrDA, and x64 had built-in drivers for it. On that note, x64 has built-in drivers for a lot more hardware than XP does. The drivers aren't necessarily the most efficient, but they work. What you may run into problems with are some lesser-used or non-business related pieces of hardware, particularly webcams, TV tuners and "special" joysticks. At the moment there's only one decent x64-compatible TV tuner card I know of, and webcam and steering wheel/flight control/etc. support is spotty at best. (USB-based ones anyway) However, I should point out that if you have a TV tuner card using the BT848/878 chipset there are custom third-party drivers for that chipset (a huge number of cards, all analog input though) which work pretty well, though the included capture program has issues.
Third, while some programs offer 64-bit support, the number is not that large. Currently there are only a few games which have 64-bit versions available (the most notable probably being Half-Life 2 and FarCry) The games that take advantage of it are definitely worth seeing in 64-bit mode though; very pretty. Unfortunately it looks as though 64-bit gaming will only take off once Vista launches, since Vista won't have any 64-bit specific version; apparently it will simply detect whether or not you have a 64-bit processor and install the appropriate 32/64-bit files accordingly. I will say that 32-bit games run just as well under x64 as they do in XP, sometimes slightly better. While I can't say for sure, my instinct is that there's slightly more processing time for the games since almost all background processes on my machine are 64-bit, hence the tiny tiny tiny bit of improvement in performance.
As long as you have no weird hardware in your computer and you aren't going to be running any obscure bits of obsolete software I think x64 is worth running.
