I'm trying to set up a connection over TCP between two devices on different networks in node.
I've been reading all over that to connect over TCP two devices I either need to set up port forwarding on the router, ore use ICE connections (good summary here).
For what I know, these limitations make total sense.
So what I don't get is, how can WebTorrent work without either port forwarding or ICE connections. The project Readme explains that In node.js, this module is a simple torrent client, using TCP and UDP.
I've looked at the code but honestly I did not get how it's done, anyone knows how torrent clients are able to seed and download data from other peers over TCP without port forwarding/middleware?
Related
I'm trying to follow the code given here to implement NAT hole punching in Node.js. I'd like to know if the server is strictly necessary. Having read about hole punching, I am under the impression that the purpose of the server is to allow the clients to exchange some information (including but not limited to their addresses and ports they want to communicate on) so that they can proceed to talk directly. Assuming the clients already had each other's information (again, including but not limited to their addresses and ports), would the server still be necessary? If so why and if not, how could this be implemented?
For instance, say one were to build an application where client_A prints out all information that would have been transmitted to the server for user_A to read, who then sends this to user_B, who then submits this info to client_B (this could be done via email for example). Wouldn't this avoid the need for a server?
Here is another explanation of why I think it might be possible to remove the server in the middle:
In NAT hole punching (assuming I understand it correctly), the communications begin when client_A sends a message to the server. The message contains some information that the server then passes on to client_B when client_B contacts the server. After this point, client_A and client_B are able to communicate directly without the need for the server. I am under the impression that once a direct connection between client_A and client_B has been established, the server could go offline and the two clients would still be able to communicate directly with one another. If this is the case, then I would imagine that any information that is being used to maintain this connection (be that addresses, ports, or any other kind of info) could be exchanged through any other channel (eg: email, a handwritten letter, a voice call, etc) at the beginning of the protocol, and then the connection could be established without ever needing the server.
Regarding 'tricking' the router
As manishig pointed out to me in a comment (thanks), NAT hole punching also requires tricking the router. If I understand correctly (please correct me if not) the router is tricked by having the router store the info for directing incoming packets from the server to client_A, however, these packets are actually coming from client_B after the initial phase of the protocol. If this is a correct description of the problem, is there a way to trick the router that doesn't require using a server?
There are ways to communicate between two remote computers over the internet without an intermidiate server, but IMO it is not the preferred way.
Why an intermidiate server is needed?
If client_A and client_B are both in the same LAN (e.g your home/office network) you can make sure (configure on the clients side and/or the router) that they will have a static ip address over this LAN and they can just talk freely.
E.G: If client_A is listening on port 8080, client_B can create a connection to client_A_ip on port 8080
Over the internet any packet sent is passed through NAT usually at least twice. One time after going through your LAN (e.g your home/office router) and at least once over an ISP endpoint. Which means you have no controll over the public ip and port assigned to your packet.
Now not only that you don't have controll over your packet's assigned public ip and port, these are also not static. They won't change while you have an active TCP connection, but you don't have any other guarantee from your ISP regarding your assigned public ip and port.
The intermediate server`s purpose is to dynamically update each client with it's peer info and also keeping the tcp connection open, so that peer to peer comunication will be available.
Alternative solution to an intermidiate server (Not recommended)
If you want your clients to communicate without an intermidiate server you can buy a public static ip from your ISP (if they support it) and then there are ways you can make (with some config) that one of your clients have a public static ip and port that the other client can connect to.
But I wouldn't recommend it, since it requires some understanding in IT and security risks.
Also if both client's are portable and connect to different networks all the time it's not a valid solution
I'm working on a project where some clients (embedded linux systems) needs to connect to a main server using so far at least two protocols: HTTPS and SSH. One of the requirement is that only one flow is allowed from every client to the server, so I've to found a way to make the two services works on the same port.
One solution that I was thinking about is to use the iptables markers: on the client side mark the packets for SSH with 0x1, the packets for HTTPS with 0x2 and then on the server side, based on the marker, redirect the packets to the right service listening on the local interface. Is it an acceptable solution? Are the markers kept by the network routers or is only something working locally on the same machine for iptables?
And anyway, if you can advice about a different solution, of course it's welcome!
More for other users finding this question in the future:
https://github.com/yrutschle/sslh has what you might need. I haven't used it (yet) but planning on it.
From the Github site:
sslh -- A ssl/ssh multiplexer
sslh accepts connections on specified ports, and forwards them further based on tests performed on the first data packet sent by the remote client.
Probes for HTTP, SSL, SSH, OpenVPN, tinc, XMPP are implemented, and any other protocol that can be tested using a regular expression, can be recognised. A typical use case is to allow serving several services on port 443 (e.g. to connect to SSH from inside a corporate firewall, which almost never block port 443) while still serving HTTPS on that port.
Hence sslh acts as a protocol demultiplexer, or a switchboard. Its name comes from its original function to serve SSH and HTTPS on the same port.
I created tcp connection between client and server using nodejs (net module). Server is listening on already predefined port and client is connecting to that port.
As far as i understand port for client is dynamically assigned by node? Is that correct?
What kind of algorithm node is using to assign "random" port for the client? How this works, is this determined by node or by OS?
Is it possible to define static port which client is going to use? Is it possible to define range of ports for the client to use?
NOTE: I think i found discussion/question with similar subject on stackoverflow before, but i cannot find it anymore. I would apprecaite if you can share any reliable resources regarding this subject.
The source port number is usually pretty much irrelevant to your programming unless you have a router or firewall that is somehow restrictive in that regard. It is merely used by the underlying TCP infrastructure to keep track of different TCP connections.
From this article:
A TCP/IP connection is identified by a four element tuple: {source IP,
source port, destination IP, destination port}. To establish a TCP/IP
connection only a destination IP and port number are needed, the
operating system automatically selects source IP and port.
The above referenced article describes how Linux selects the source port number.
As to your particular questions:
What kind of algorithm node is using to assign "random" port for the
client? How this works, is this determined by node or by OS?
It is determined by the OS. That source port number is selected by the originating host at the TCP level before the connection is even made to node.js.
Some other reference articles:
Does the TCP source port have to be unique per host?
how can an application use port 80/HTTP without conflicting with browsers?
Note: there is no security reason I'm aware of for a firewall to limit the source port number or block certain source port numbers. They are a TCP bookkeeping number only, not related at all to security or the type of service being used. Note, this is different than the destination port which is usually correlated directly with the type of service being used (e.g. 80 is HTTP, 25 is SMTP, 143 is IMAP, etc... When you make a TCP connection to a different host, you specify the host address and the destination port number. You don't specify the source port number.
The selected answer is provides a lot of info, but does not deal with the underlying problem. Node does not appear to allow https.request to specify a port for the client. There exist localAddress and localPort options, but they appear to be broken.
I've opened a new question on this issue. Hopefully someone will answer with something other than "just don't do that."
Is there a way to set the source port for a node js https request?
Such as Skype/Team viewer/Logmein etc application, which send audio/video behind NAT (behind firewall). But when i make a small tiny application which send text to another NAT location it failed to do the same.
Example:
Sender:
-> Public ip: 91.1.2.3 My lan ip is: 192.168.1.2 with port 14446 udp
-------> Data format: RTP packets
Receiver:
<------- Data received: 0 packets
-> Public ip: 92.1.2.3 Friend lan ip is: 10.0.0.2 with port 14446 udp
* same in both way
How others does this? What is the way of doing peer 2 peer application development to overcome NAT issues? Always we have public ip's and mostly it has NAT issues.
But how does then Skype works in such cases too? Do we have a audio/video port range for UDP or always UDP is open from anything? But mine does not work above range ports for UDP i also tried. What is the secret? that is making me curious!!.
Note:
My goal is audio packets handling where i believe too much filtering or firewall cause latency and delay and other issues gets involved relatively too. So i would like to know very clearly for my application that some of the ports (which port ranges?) can be used for such purposes, where it really not blocking development stress.
There are a number of types of NATs, which vary in what traffic they'll allow in.
See the Wikipedia article on NATs
For most NATs, STUN will let you open ports AND find out what port you opened (may be different than the port you sent from). In SIP and RTSP you'd typically provide the external IP and port determined by STUN to the other end.
A fully-symmetric NAT means that STUN won't let you use a 3rd-party server to prop ports via STUN, so you'll have to use UPnP (if enabled) or map ports in the router (or set up triggers), or you'll have to play evil games to make both sides think they initiated the connection. (Not easy and not guaranteed.)
See the ICE & TURN specs (RFCs) from the IETF for detailed mechanisms to traverse NATs - though note that in some cases you must use an external proxy to forward packets.
One common solution is that the client program connects outward to the server and thus establishes a connection. Most firewalls allow outward connections - the assumption being that you are trusted and can always connect to the outside. When the server then wishes to send a message to you, it responds on the open connection.
I believe the port that you use is what is usually used to determine if it should be allowed or not. Certain ports are always let through. I'm not sure of the exact ports, but that will be different for all NATs and firewalls.
How can an application be designed such that two peers can communicate directly with each other (assuming both know each other's IPs), but without outgoing connections? That's, no ports will be opened. Bitorrent for example does it, but multiplayer games (as far as I know) require port forwarding.
I'm not sure what you mean by No Outgoing Connections, I'm going to assume like everyone else you meant no Incoming Connections (they are behind a NAT/FW/etc).
The most common one mentioned so far is UPNP, which in this context is a protocol that allows you as a computer to talk to the Gateway and say forward me this port because I want someone on the outside to be able to talk to me. UPNP is also designed for other things, but this is the common thing for home networking (Actually it's one of many definitions).
There are also more common and slightly more reliable ways if you don't own the network. The most common is called STUN but if I recall correctly there are a few variants. Basically you use a third party server that allows incoming connections to try and coordinate a communication channel. Basically, what you do is send a UDP packet to you're peer, which will open up you're NAT for a response, but gets dropped on you're peer's NAT (since no forwarding rule exists yet). Through the connection to the intermediary, they are then told to do the same, which now opens up their NAT, and matches the existing rule in you're NAT. Now the communications can proceed. Their is a variant of this which will allow a TCP/IP connection as well by sending SYN and SYN-ACK messages with some coordination.
The Wikipedia articles I've linked to has links to the relevant rfc's for these protocols on precisely how they work. Essentially it comes down to, there isn't an easy answer, as this is a very network centric problem.
You need a "meeting point" in the network somewhere: the participants "meet" at a "gateway" of some sort and the said "gateway function" takes care of the forwarding.
At least that's one way of doing it: I won't try to comment on the details of Bittorrent... I am sure you can google for links.
UPNP dealt with this mostly in the recent years, but the need to open ports is because the application has been coded to listen on a specific port for a response.
Ports beneath 1024 are called "registered" because they've been assigned a port number because a company paid for it. This doesn't mean you couldn't use port 53 for a webserver or SSH, just that most will assume when they see it that they are dealing with DNS. Ports above 1024 are unregistered, so there's no association - your web browser, be it Internet Explorer/Firefox/etc, is using an unregistered port to send the request to the StackOverflow webserver(s) on port 80. You can use:
netstat -a
..on windows hosts to see what network connections are currently established, including the port involved.
UPNP can be used to negotiate with the router to open and forward a port to your application. Even bit-torrent needs at least one of the peers to have an open port to enable p2p connections. There is no need for both peers to have an open port however, since they both communicate with the same server (tracker) that lets them negotiate and determine who has an open port.
An alternative is an echo-server / relay-server somewhere on the internet that both peers trust, and have that relay all the traffic.
The "problem" with this solution is that the echo-server needs to have lots of bandwidth to accomodate all connected peers since it relays all the traffic rather than establish p2p connections.
Check out EchoWare: http://www.echogent.com/tech.htm