the joystick port is also a midi port, if you ever need to hook up to a midi device... but assuming you won't, the other problem is that the joystick port is routed through the sound card, so a monster joystick could possibly cause distortion in your sound, also it will eat up PCI interrupts...
also an analog joysticks needs a thicker cord, and plugs into the back of the computer with a thicker connector - with usb the connection is serialized and smaller, more universal, won't have to calibrate it in the control panel, allows you to plug into the front of the computer or keyboard if they have open USB ports...
and then there's the fact that many sounds cards, if they haven't already, are trying to retire the midi/joystick port. My Sound Blaster X Gamer uses an optional second slot modul for the midi/joystick port, if I so choose to use it, otherwise it has builtin firewire port instead.
I would go with USB. Unless you planned running old dos games that needed an analog joystick. Why go retro when you can go proactive. I've seen computer systems without even ps2/serial/parallel all in favor of usb...