I use gnuchess. It's free, open source, and built in such a way that you can play it by itself, or with any of the multitude of frontends. Supports you vs. ai, ai vs. ai, and multiplayer by hotseat, e-mail, or network.
And I'm pretty sure it works on windows. I will attempt to find a link.