Being absolutely sure is pretty tricky with consumer grade gear. However, there are a few methods that will give you a good idea of what is going on. These won't detect devices that really don't want to be found; but they'll pick up casual conections.
Log into the router and poke around until you find the DHCP client table, or something of similar name(I've an earlier Linksys router, not sure if they've changed the name of that one). This will show all devices that have asked the DHCP server for an address. It won't show devices that have hardcoded IPs; but it is a good start.
Using a tool like Nmap to scan the internal IP range of your network can ferret out hosts that aren't using DHCP. Again, you are not certain to catch everything; but you'll catch anything that isn't trying to hide.
Running Wireshark, or similar, to monitor a wireless ethernet card set to promiscuious mode and receiving packets from your wireless network should be able to catch pretty much anything that is chattering on air. It doesn't promise certainty; but it should catch most anything.
Facebook
Twitter