Communication between private IP addresses over the Internet

[androidoutthere]

Suppose we have messenger like GTalk, Skype etc running on two different PCs(A and B) which have private IP addresses but connected to Internet through some ISP. The two PCs could be located in different parts of the world. Now its possible to send messages and make VoIP calls between A and B.

This obviously uses server for communication establishment. For text messages it could be routed through the server to reach A and B.

But for VoIP calls, I believe initial establishment would be taking help of server and later VoIP data would flow directly between A and B. Now, A and B both have private IP addresses.

So my question is how is it possible to communicate between A and B directly over the Internet who have private IP addresses ?

Is there any specific standard for this ? I am not talking about VoIP standard like RTP etc but the mechanism to communicate over Internet using private IP addresses. In other words, I simply have a TCP client and TCP server running on A and B respectively. How can TCP client reach TCP server ?

I would like to know if messengers like GTalk, Yahoo, Skype and many of the Android applications like Viber, Kakoa talk, WhatsApp etc use any specific standard for this. And I have seen that these work almost always, calls between A and B work, located anywhere in the world.

It would be nice if somebody can share the architecture and the concepts involved.

[I did read about concepts like NAT, STUN and hole punching. But I don't see any specific way. And there is no guarantee that these would work depending on the NAT behavior]

Thank you.

 

[lif_andi]

Its called Network Address Translation.

Two private addresses cannot communicated over the internet, period. Somewhere there is always a routable IP address that you have to go though.

 

[androidoutthere]

I know about NAT.
I am talking about a case where a TCP connection needs to be established from private IP to private IP over the Internet. I know NAT is involved in this process. My question is how does this happen altogether. Because NAT doesn't allow incoming connection from unknown source. NAT should have a port mapping created etc.

 

[lif_andi]

There is port forwarding involved, sometimes central servers, sometimes each side needs to accept or initiate a connection from their side... there are several ways to do this.

In short, a private address is never used on the internet. You could set up a VPN tunnel between two LANs and connect to private numbers, but again, that is through routable IP addresses.

 

[androidoutthere]

I am aware about port forwarding.
Also hole punching technique is used. Protocol like STUN could be used.
But all these rely on the NAT behaviour.
And there are some protocols involved.
I was wondering if there is any specific generic solution which will work most of the time.
Do we have any specific generic solutions which is used by GTalk, Skype etc ?
Thank you.

 

[drebo]

In this case, where endpoints are behind NAT, there must be a mutually accessible media termination point (i.e. skype's servers.) The data itself does not flow directly between endpoints.

 

[serpretetsky]

In this case, where endpoints are behind NAT, there must be a mutually accessible media termination point (i.e. skype's servers.) The data itself does not flow directly between endpoints.

Im fairly certain most VOIP data actually DOES flow between the endpoints. The server is only used to initiate the connection.

This is an interesting topic. VOIP application either use UDP or RTP is that correct? I'm not familiar with how NAT treats UDP or RTP.

 

[drebo]

Im fairly certain most VOIP data actually DOES flow between the endpoints. The server is only used to initiate the connection.

This is an interesting topic. VOIP application either use UDP or RTP is that correct? I'm not familiar with how NAT treats UDP or RTP.

The media path can only be directly connected between endpoints if the endpoints are able to directly communicate with each other. Any introduction of NAT will affect that.

The reason for this is that the SIP proxy and the phones themselves have no knowledge of the outside vs. inside addresses or the ports involved.

If the endpoints are directly able to communicate and the server is configured to allow them to, yes, the media will not traverse the server. This is only possible in instances where NAT is not present, however.

Source: I run an ITSP and implement VoIP phone systems for a living.

 

[serpretetsky]

The media path can only be directly connected between endpoints if the endpoints are able to directly communicate with each other. Any introduction of NAT will affect that.

The reason for this is that the SIP proxy and the phones themselves have no knowledge of the outside vs. inside addresses or the ports involved.

If the endpoints are directly able to communicate and the server is configured to allow them to, yes, the media will not traverse the server. This is only possible in instances where NAT is not present, however.

Source: I run an ITSP and implement VoIP phone systems for a living.

but like android mentioned, there's the concept of hole punching.

There's tcp hole punching and udp hole punching.
http://en.wikipedia.org/wiki/UDP_hole_punching

Furthermore, if p2p communications are not possible when both endpoints are behind NAT, how does p2p filesharing (such as bittorrent) work?

edit: oh, and sorry to give you so many questions. I'm not familiar with this. I'm also know absolutely nothing about professional/business VOIP systems. I'm not sure if they are any different from regular applications like Skype, ekiga, etc

 

[SecurityTheatre]

I am aware about port forwarding.
Also hole punching technique is used. Protocol like STUN could be used.
But all these rely on the NAT behaviour.
And there are some protocols involved.
I was wondering if there is any specific generic solution which will work most of the time.
Do we have any specific generic solutions which is used by GTalk, Skype etc ?
Thank you.

In short, no.

All of these services have servers that tunnel the traffic when both clients are behind a NAT and do not punch a hole in it.

So your choices are...

1) Punch a hole for a TCP port on at least one end
2) Set up a VPN between the private networks
3) Relay through a server.

Note, #3 could be accomplished through something complicated like a Teredo tunneling IPv6 server - this wouldn't necessarily rely on the provider's tunneling service, but essentially on the NAT-bypass capability of a protocol like Teredo.

 

[lif_andi]

How bittorrents work:

http://www.howtogeek.com/141257/htg-explains-how-does-bittorrent-work/

 

[drebo]

but like android mentioned, there's the concept of hole punching.

There's tcp hole punching and udp hole punching.
http://en.wikipedia.org/wiki/UDP_hole_punching

Furthermore, if p2p communications are not possible when both endpoints are behind NAT, how does p2p filesharing (such as bittorrent) work?

edit: oh, and sorry to give you so many questions. I'm not familiar with this. I'm also know absolutely nothing about professional/business VOIP systems. I'm not sure if they are any different from regular applications like Skype, ekiga, etc

Bittorrent is its own protocol and I'm not entirely sure how it works, as I have not studied it.

In the case of SIP, the SIP proxy relays the information between endpoints and can assist in determining the media path. The source and destination ports for each direction of the RTP stream are determined by the endpoints and sent in SDP information. A NAT translation in a router is not open-ended, and there is no facility in SIP to determine and relay the external IP addresses of endpoints between the endpoints, hence the need for an external proxy. Similarly, SDP has no facility to determine that information, either. A phone cannot send an Invite to a phone that it doesn't know the address of, and that's what's required to set up the RTP stream (in this case, a Re-Invite.)

The NAT translations that are created in the router are the result of outbound communications between the endpoint and the proxy because that's the only address the endpoint knows about. The NAT router in this case is oblivious to the type of traffic.

Yes, if you were to forward all possible RTP ports and the appropriate signalling ports to your phone, then you could make it work...but that's only going to allow you to use one phone at a time behind a NAT connection.

Media has to also be proxied between the two endpoints which are behind NATs because the endpoints don't know how to contact each other any other way. The endpoint only knows about its internal IP address and that is the address that is communicated within the SIP packets. The proxy knows about the external IP address and translated port of the endpoint only because that's in the TCP/IP header of the SIP packet it receives. As such, the other endpoint can only have knowledge of the original endpoint's internal IP because that's the only information in the SIP and SDP packets and proxies do not (cannot) rewrite this information.

So, if you try to establish a direct media connection between two phones not on the same network and not otherwise routable to each other, the media connection will fail.

 

[serpretetsky]

In this case, where endpoints are behind NAT, there must be a mutually accessible media termination point (i.e. skype's servers.) The data itself does not flow directly between endpoints.

again, I don't know much about professional Voip systems, but I found some info on skype
http://www.h-online.com/security/features/How-Skype-Co-get-round-firewalls-747197.html
In most cases, both clients (who are behind NAT) talk directly with each other with no need for port forwarding.

 

[lif_andi]

There is port forwarding involved, and a central server. The article specifically talks about port forwarding.

Also, if you are not logged in (available to the central server) there will be no connection.

 

[Gryz]

Androidoutthere, it is possible to have private IP addresses to communicate directly with each other through NAT. Sometimes, but not always. It depends on the form of NAT that is being used.

Reread about STUN. STUN is a way that allows direct communication between 2 private addresses. But it requires a server for the initial setup of a TCP connection.

The way it works is this:
Suppose two endpoints.
A has ip-address 10.1.1.1 and the public ip-address of its NAT box is 100.1.1.1
B has ip-address 10.1.1.1 (the same, by coincedence, it doesn't matter), and the public ip-address of its NAT box is 200.2.2.2
There is a STUN server with public ip-address 150.150.150.150.

A and B need to know they want to talk with each other. And they need to know the public ip-addresses of the other side. This can be told by the Skype server, or the bittorrent tracker, or some other form of directory services. Or simply configured names that are resolved through DNS.

A creates a TCP socket. Its OS tells the application which TCP portnumber it has. Suppose it got portnumber 1025. A opens another TCP connection to the STUN server. A says: "I want to talk to 200.2.2.2 and I am on port 1025". The STUN server also sees that this request came from 100.1.1.1.
B creates a TCP socket. Its OS tells the application which TCP number it has.
Suppose it got portnumber 2090. B opens another TCP connection to the STUN server. B says: "I want to talk to 100.1.1.1 and I am on port 2090". The STUN server also sees that this request came from 200.2.2.2.
The STUN server then replies to B: "You want to talk to port 1025 on 100.1.1.1". The STUN server then says to A (over the connection that is still open): "You want to talk to port 2090 on 200.2.2.2". The STUN server then closes the 2 TCP connections to A and B, and is done.

A now opens a TCP connection from its existing socket to 200.2.2.2 port 2090. A TCP packet leaves A, goes through its NAT box. The NAT box creates a NAT entry in its NAT table: incoming packet from 200.2.2.2 port 2090 to 100.1.1.1 port 1025 needs to get translated to 10.1.1.1 port 1025. The packet makes it to B's NAT box, and gets dropped.

B now opens a TCP connection from its existing socket to 100.1.1.1 port 1025. A TCP packet leaves B, goes through its NAT box. The NAT box creates a NAT entry in its NAT table: incoming packet from 100.1.1.1 port 1025 to 200.2.2.2 port 2090 needs to get translated to 10.1.1.1 port 2090.

The packet makes it to A's NAT box. There is a valid NAT entry there. Packet makes it all the way to A. A replies. The two necessary NAT entries exist. A TCP connection is set up. A and B connect directly, without any Skype or STUN server, or bittorrent tracker in between.

This trick works on one condition. That all outgoing TCP connections from a particular private ip-address *always* get translated into the exact same external ip-address. And that the port numbers do not change during translation.

This is true for most NAT services that are implemented into small cable and DSL routers for users at home. Larger NAT boxes deployed at campus or company networks also do this.

But there is a new type of NAT that does not do this. It is called Carrier Grade NAT (CGN). A very limited set of public ip-addresses is used for a much larger set of private ip-addresses. This forces CGN to assign only small blocks of portnumbers to specific private ip-addresses. And thus private ip-address 10.1.1.1 might only get the portnumber block 21000-22000. If this happens, then STUN won't work. And in that case, you can't set up direct communication.

 

[serpretetsky]

There is port forwarding involved, and a central server. The article specifically talks about port forwarding.

Also, if you are not logged in (available to the central server) there will be no connection.

I am not sure that we're talking about the same thing. Port forwarding is when you manually setup a static map on the NAT device for a certain port to go to a certain ip. The article i linked does not talk about port forwarding at all. When I skype with someone I am not required to go into my router and setup port forwarding. It just works.

Yes, a central server is required for the initial setup, I never said otherwise. All subsequent connections (assuming the hole punch was successful) go directly between the two clients bypassing any other servers.

 

[lif_andi]

Well... NAT uses port forwarding (dynamic port forwarding), in that it maps ports for return communications when a translation is made. Otherwise, traffic would never be able to return to the same device it was initiated from. The central server becomes a focal point of the initiating traffic, and ports are forwarded with NAT, and afterwards, because of this port forwarding, the communication can continue. Without it, all packets from the peer would be dropped and would not make it through the firewall.

 

[serpretetsky]

Yeah, i agree. I still wouldn't call it port forwarding, because although that does DESCRIBE the process, port forwarding usually refers to static port mappings. I also would not call it dynamic port forwarding, because that actually refers to something else as well.

I suppose PAT (port address translation) or NAT with port overloading could be used pretty safely.

 

[QuietDad]

All the P2P networks like Skype, Bittorrent and such have a server or tracker in the middle you need to sign into and thats where the mapping takes place.

 

[androidoutthere]

Thank you for all the posts here.

 

[androidoutthere]

I think what 'serpretetsky' saying is correct. Just to initiate the connection we would need the server and in case of VoIP call, it would make more sense to send the VoIP data directly between the clients instead of routing it through the server which would add delay depending on the network condition, location of users and server.

 

[androidoutthere]

'Gryz', I think what you have mentioned is actually hole punching mechanism. I think STUN protocol is used to find the NATed public address for a specific host behind NAT.

 

[androidoutthere]

I think given all these:
- STUN
- Hole punching
- Dynamic port forwarding using UPnP
It might be possible to arrive at an algorithm and strategy to do peer to peer communication without use of any server except for STUN servers.

Links:
http://www.howtogeek.com/122227/how...ts-on-your-router-from-a-desktop-application/

If anyone has any other suggestions, please pour in.

Thank you everybody. This was a great discussion so far.

 

[androidoutthere]

I just did a quick experiment and saw that if the NAT is symmetric NAT, then hole punching mechanism fails.

 

[QuietDad]

You only need the server in the middle to establish the initial contact between two IP addresses. Once the communications between the two hosts, be it skype, bittorrent or VOIP is established, the central servers are no longer in the picture and it's all basic packet switching.