Well actually you can boot from a USB drive, but it depends on a lot of things. How good BIOS support for it is one thing. Make sure you have the latest BIOS installed for your mobo.
Some mobos support booting from USB better than others. Make sure Legacy support for USB is enabled in BIOS and that the you have the boot order set correctly. There's rarely a setting in the boot order for a USB drive, but there's usually an "other" choice of some kind. Make sure it's configured in the boot order before IDE or SATA.
Also, did you do a clean install of WinXP onto the USB drive? You can't clone your internal drive to the external USB drive and expect it to work - Windows won't find the boot partition on the same hardware address.