In addition to those already mentioned.
Software has better support for all the buttons with USB. It is much easier to write a game that supports all the USB buttons. With non-USB you may have to map some of your buttons to keys with included software.
In addition, USB being digital generally has a lot less calibration issues with joysticks and wheels (I know you are asking about joypads), one does not need to recalibrate the USB versions as often.
If you can disable your game port you probably free up an I/O range at a minimum, possibly an IRQ.