If you have RJ45 jacks, they must lead back somewhere.
Generally, two computers can be directly connected together using a "crossover" cable - A special Category 5 (twisted pair) cable that crosses the send and receive pair. If you have more than two computers, you need to connect them all into a hub or a switch (A fancy hub) using standard Cat5 (straight through) cables.
You have wired your house with an RJ45 jack at the stations, but what's at the other end? Typically, you'd see all the wires run in your house terminated at a single point. This is where you'd put your hub and have everything connected together.
One note - Getting the computers all physically connected is just the first step. What do you plan on doing? Sharing files, sharing Internet access, etc?
- G
Edit: And Jack-The-FAQ-man beats me yet again!