What ports does your router have on it?
You might want to consider getting a switch, pluging the router into the uplink on the switch, and pluging you computers into the rest of the ports.
I guess if you really didnt want to get a switch, you could always put two nics in one computer and use it as a router of sorts. The dsl router would directly go into the computer with two nics, the second computer would plug into the second nic on the computer with 2 nics. This should work, but im kinda rusty on setting this up.
How far apart are your two comps?