You need to find the .inf and .pnf files for your particular hardware once you've uninstalled it and BEFORE rebooting and AFTER uninstalling/removing from Device Manager. These files tell Windows what drivers to install.
Once you know the .inf and .pnf file for your particular device and delete them, you can reboot and Windows will tell you that it's detected a new device, but it won't install any drivers automatically because it has no information on how to do that. That's when you can point it to the new drivers.
XP normally saves these .inf and .pnf files in the windows\inf folder. Make sure you've got the right .inf/.pnf file. Notepad will let you view an .inf file. .pnf files are a little harder to read with notepad because they are not just text, but frequently you can pick out text in them. But just find the .inf file for your device and the .pnf will have the same name.
Example: vidtek.inf will have a corresponding vidtek.pnf (vidtek is just a madeup name for this).
You can use the search program if you can't find an obvious match for your .inf file.
Use XP's search (make sure Advanced Options are checked, and check that System Folders, Hidden Files, and Search Subfolders are checked).
Then, search for files named ".inf" (no quotes)
Containing text: (whatever your device is called, like Radeon, for example).
But honestly, I usually do NOT uninstall devices or drivers before updating them. I simply choose the Update Drivers option and point XP to the new drivers. It's a simple matter to check the file details after a reboot to ensure that the driver files are all current, vs. a mismatch of old and new. If there's a problem at that point, I'll go delete the .pnf and .inf files and uninstall the device. I'll also track down the various driver files in the drivers folder and the driver cache folder and get rid of those before updating. But 99 percent of the time, that's not an issue.