Actually... 
If you ask me, buying a Linux box to use remotely is a bit of a mistake. What you want to do is to run the Linux box natively and use the Windows XP box remotely.
I've seen a couple threads here and there that end up with a guy asking how to use the Linux box remotely on his Windows box.
You have a few choices..
SSH using Putty.exe --this is the 'best', but it's command line only.
Use Cygwin enviroment and ssh and X --this is good and provides a GUI, but X servers for Windows suck.
Use VNC -- which is ok.
Now I use ssh all the time, but I prefer using Linux over Windows generally because then I get the nice command line on the box that I am using. And going from Linux to Linux is easy because ssh rocks and X over ssh is easy and fairly fast. For slow links you have FreeNX which compresses X well and makes it fast over the internet and such. SSh encryption keeps it secure.
Now if your going the other direction.. from Windows to Linux, then it's easy. You use Windows XP Pro and setup remote desktop and have the window's remote desktop on Linux. It works quite nice and if you configure stuff nicely you can get a full screen (minus the toolbars) Windows with 24bit color support and it runs fairly fast. There isn't much difference between running it remotely vs natively over a lan.
Between OS X and Linux.. it's pretty much the same. OS X has good X windows support and vnc and I suppose rdesktop (never tried that though) and all that.
The difference between OS X and Linux is that although OS X has a better desktop experiance.. linux still makes a better server setup.
Now with Debian Stable, if you can manage to get a good Gnome or KDE desktop setup on the thing, (debian's default install can't rely be considured newbie freindly) then it's very easy to manage and install different things... You have a couple different versions of Apache to choose from, different databases (mysql, postgresql, firebird, etc), different languages like PHP, perl, Python, ruby, etc.
All if is 'officially supported'. All of it has it's dependancies taken care of and it all works together with all the different permutations and whatnot. All the software has been tested, bugs are known fixed or workarounds documented. You just have to install them with apt-get and configure them to your specific setup.
Also security is very good.
There is a good reason why Debian is rapidly becoming one of the most popular OSes for web serving.
Now with OS X you have things like Fink or Gentoo for Mac to choose from, which do a ok job.. much better then Windows does, but it's not as nice as Debian.
Also there is Fedora, which does a better job at managing packages then OS X can, and is quite popular also. Also if your planing on working within a 'enterprise' enviroment then Redhat is the most common Linux OS your going to run into and Fedora is very close to that.
Now aside from doing that and investing in a second machine there are some alternatives...
One is to use Cygwin. Cygwin is a Linux/Posix-like enviroment for Windows were most GNU software works well. You can do perl, run a ssh server and things like that. Lots of web developers use cygwin and it's quite popular.  It's Free software, originally from Redhat, I beleive.. and is good on the resources.
Also another is to use a virtual machine like Vmware. Just run the entire Linux OS as a application on your Windows OS. Basicly. It's also very popular. This costs money and requires a bit more machine.
For Linux/OS X hardware with a 600 dollar budget and small and quiet formfactor...
Apple's Macmini makes a good Debian box, and it supports OS X. In fact you can run OS X inside Linux if you realy wanted to. Booting and setup Linux is a bit more confusing then it is with x86, though. Probably stick with OS X if you go this route. It's nice that it's inexpensive. 
Shuttle XPC is a good Linux route as long as your choosy..
For these you'd probably want to go the Intel route for less headaches.
The 915G* chipset is ok supported. Everything should work out of the box.. sound, video, networking, etc. 3D graphics it may be a bit new for Debian Stable, I am not sure. I think it will work. If not go with Debian Testing, Fedora, or Ubuntu. If that matters to you.
There is one model, I beleive, with a 'Sound Blaster Live! 24bit' sound card. Those sound cards are not well supported in Linux... they sound like the normal Sound Blaster Live! 5.1 cards or Audigy 1/2/etc.. which are very well supported, but are internally completely different. 
Also the Audigy LS is a similar thing to SoundBlaster 24bit. These should be avoided.
Audigy 1 or other Audigys are well supported as well as the Soundblaster Live 5.1 stuff. All these use the Emu10k1 chipset or derivative, which are very well supported.
The intel-onboard and such are well supported, but their capabilities are limited.
Avoid the ATI motherboards. Not good at all with Linux.
Go with onboard Intel for video. Avoid ATI. You can upgrade to propriatory Nvidia drivers in the future for more performance if you need to.. (but you have to deal with drivers, which kinda suck)
Pentium-Ms are easy to keep quiet and are fast, but they are expensive...
Also there are some Asus barebones that are small and quiet. Similar to the shuttle.
Another option if you want is to go with a Laptop.
Intel everything.. Intel Video, Intel Wireless, etc.. Well supported by Linux. Older 'centrino' style laptops with the 855 chipsets have subpar performance with Video, but are well supported non-the-less. Same thing with 915 chipsets.
Avoid broadcom wifi. Anything with the official 'centrino' lable will have all Intel stuff in them, except for some with ATI video, which is so-so, but you have to deal with the drivers.
You can find these guys new or refurbished from Dell and HP and such for around and under 600 bucks.
That's pretty small and quiet. As well as having a built-in UPS.
And the final option that I can think of is going Mini-ITX formfactor.
This is nice because it's less propriatory then Shuttle or Asus minipc stuff. It's upgradable and such.
Check out Logic supply.
http://www.logicsupply.com/
I haven't seen these up close or used them before but they seem pretty nice and are low-energy.
There are Via-based items... with slow cpus and slow video, but make great servers non-the-less. Also have some Celeron/P4/P-M systems.
Fanless, rackmount, firewalls, carputers.