As others have mentioned, check the MTU of each computer and or the router. Like PPPoE connections (usually DSL) need 1492 often.
Another thing to check is play with the half vs full duplex setting of the computer.
Most routers an handle anything your modem can dish out. Since something like 99% of all DSL/cable modems has 10 mb network equipment inside they can't go over that. The routing capabilities of the linksys WRT54G for example at something like 25mb-35mb, which is way more than enough to handle almost any consumer internet connection.
I have had an old router before though, a linksys NR041 that could NOT handle my connection. I have a 6 mbit connection and at best I could only get 3.5mbit or so worth through that old router. But you are talking 4 out of 5 routers not handling your connection so I dont think thats the issue.
What speed is your internet supposed to be? 3mbit? 6mbit? 8mbit?
Also start checking with download speed tests at
http://www.broadbandreports.com/stest
see what results it gives when you switch routers around. Make sure to run it a few times (a differant location each time) with each router.
If the results are all within a few percent of each other then I'd say everything is fine. If some routers are VERY SLOW compared to the rest than I'd say check the MTU of all the equipment.