Assuming nobody else is using the XP licenses elsewhere (which is a violation of the EULA), the Dell OEM licenses are stuck with these PCs forever. So, yeah, you can re-install XP. The COA Key on the side of the computers won't actually be used in the re-install, assuming you are using Dell OEM media for the installation.
As far as documentation, you can install one copy and and read the EULA.TXT file that is installed. Or you can take a look at Microsoft's Piracy web sites for OEM licensing requirements.