If you just want a quick router and don't care about firewalling, etc., lrp is probably a good choice, then. Fast, easy, and painless.
I'm the wrong person to ask about anything debian specific.
(I'm also the wrong person to ask about fast, easy, and painless solutions-being a computer masochist and all), but here goes:
-Boot your lrp disk.
-It'll probably ask for IP address/netmask/gateway info that you got from your ISP, so enter that info.
-On your windows machines, give them IP addresses of 192.168.0.10 & 192.168.0.20 (192.168.x.x, rather), netmasks of 255.255.255.0, and default gateway of whatever the firewall IP is.
-Plug everything into a hub
-type 'ping 192.168.0.10' (& 'ping 192.168.0.20') from the lrp box to check if they can see each other.
That should give you what I think you want, unless I'm forgetting something.
(edited for command line clarity)