I use Gentoo on two of my systems (for about 10 months), and while I really like it for some things, I'm really considering moving over to Debian. Compiling everything is a slow PITA, even on my Barton core 2400MHz.... and while some programs really do run faster, alot do not, and I really dont care that much for the extra trouble involved.
Overall it does provide for a clean system and allows you to customize alot from the get go, rather than going back and changing it after the install like alot of distro's. I think the main thing bugging me is that I built a Redhat 9 box for freevo on a Celeron 1100/512MB/80GB 7200rpm, and its pretty damn fast, and only took about 25 minutes to get up and running.... I really like playing around with Gentoo, however I often get annoyed when I need to get something done and need something else installed and have to wait for a compile (even if its only 5-10 minutes, its still really annoying when you need to get work done)