The p2 systems will likely have PC66 SDRAM and small/slow hard drives. but they may well be ATX case & power supply (my p2-266 was). The power will almost certainly not be good enough for duron, but probably good enough for celerons.
Another thread just recommended the Asus TUSI-M mobo for $60 with onboard audio/video/LAN, there's also the TUVX for $75 (both at newegg.com), pair them with a celeron 1.1a / 1.2a and 128 MB of PC133 (256 MB if running Win2000/WinXP).
Given the cost of removing / installing mobos and possible power supply problems it might be better to buy a barebones PC -- newegg has them starting at about $150, and Googlegear has the tiny little Asus barebones fairly cheap.