The answer is, "it depends."
It depends to a large extent on how the device's .inf was written.
You can safely assume that the registry will be modified so that W2K doesn't attempt to load the driver. However, it's a good bet that the driver files will remain on your hard drive.