I have a Server socket and a device which uses TCP long-connection mode.
They can connect and exchange data together. The node.js server is something like this:
net.createServer(function (socket) {
console.log('ip:port' + socket.remoteAddress +':'+ socket.remotePort);
socket.on('data', console.log);
}).listen(‘0.0.0.0’, 8888);
The device connects just right and I'm able to receive data from it.
I can send commands to it by using the same process, by just doing socket.write('dothisplease') and this works too.
Now I have another worker process which should be sending commands at regular intervals. I can get ip and port from console.log when the device connects, it looks like: xx.xxx.xx.xxx:63024
I tried using this combination ip:port to create new connection:
var client = new net.Socket();
client.connect(device_port, device_ip, function () {
client.write('dothisplease');
});
... but the result was ECONNREFUSED
Is it right to use the same port to create a second connection to the device?
Why does it work from the same process, but does not work from another?
Eventually, can I pass the socket to another node Worker process. How?
Thanks a lot!
Your server is listening on port 8888. That's the port that all clients need to connect to. The client will also have to know what the server's IP address is in order to connect to it. A client uses a target IP address and target port to specify the destination for a TCP connection.
socket.remotePort is not the port that anyone can connect on. That is the outgoing port that the first client used to connect to your server port. The outgoing port is a client bookkeeping thing that helps the client keep track of which network traffic belongs to which socket and is usually randomly assigned by the client.
You read more about what remotePort is here.
For reference, a TCP connection consists of two endpoints and each endpoint has an IP address and a port. The server will need to have a known IP address and a known port that all clients will use in order to connect to it. The client will already have its own IP address. Then, during the process of making a TCP connection to a server, the client will dynamically allocate an unused port number for the communication for this socket. That port number is used by the client to keep track of which network traffic belongs to which socket. This is not a port number that anyone can connect to.
Is it right to use the same port to create a second connection to the device?
No. You can't create a connection to a client. You can only create a connection to a listening server. And, you couldn't use the client's port that belongs to that other socket either.
Related
I read the docs, concerning the .listen() method, used in express. I can USE the method and setup a server that is listening to HTTP requests.
However, since I am fairly new to coding, I find it difficult to grasp whats really happening when using the .listen() method. The high level explanation "listening for connections" didn't help me.
I think, this could be made easier if I could actually see the function instead of only calling it.
Any help is very much appreciated
In a nutshell, the Express app.listen() method creates an http server object and then configures it to receive incoming TCP connections on a specific port and IP address so that when clients request a connection to that port and send an http request, the server can receive that http request and process it, sending a response. The code in app.listen() is shown below later in the answer - though all it does is call down to one further layer down in the http server object.
Here are the lower level details for how that works.
When a server wishes to start listening for incoming connections, it informs the local TCP stack by creating a socket and binding to a particular port and IP address. That essentially reserves that incoming port for this particular server (no other server will be allowed to also bind to that port). So, for example, on a regular http server on the default port, you would bind to port 80. This type of bound socket is used for incoming connections only, not for two-way communications with a client.
Then, the server informs the TCP stack that it is ready for incoming connections. At the TCP level, this is referred to as listen. Within nodejs, the bind and listen steps are combined into the one step called listen.
From then on, whenever the local TCP stack receives an incoming connecting request whose destination is the IP address and port that the server bound to, then that incoming connection will be accepted and inserted into a queue for the server that is configured for that IP address and port. There will typically be a maximum number of incoming connections that can be queued in this way and, if that number is exceeded, then the connection will be refused. This manages load and protects the host if the server gets "backed up" and is behind on processing incoming connections.
The server will then be informed by the TCP stack for each new incoming connection. Once the server accepts that connection, then it can start reading any data that the client has sent over the socket. In the case of an HTTP server working with the HTTP protocol, this would be the initial request protocol, method, version, headers and any body data. For different types of servers, the data would be in a different format.
Here's a useful diagram of the server:
Source: https://medium.com/javarevisited/fundamentals-of-socket-programming-in-java-bc9acc30eaf4
The server creates a socket used for the server to accept new connections..
It binds that socket to a specific IP address and port so it will only be informed about incoming connections targeted to that IP address and port.
It listens on that port to inform the TCP stack it is ready to accept incoming connections.
When it is notified of an incoming connection, it accepts that incoming connection.
Then it can read and write to that new connection over the new socket.
Then, sometime later, the incoming socket is closed to complete the client transaction.
The app.listen() method in Express encapsulates these steps and a few others. Internally (within Express), the code looks like this:
app.listen = function listen() {
var server = http.createServer(this);
return server.listen.apply(server, arguments);
};
You can see that method here in the open source repository.
To get an http server ready for steps 1-6 above, this creates the http server object within nodejs and then registers the app as the request listener for that server object (so it will be notified of incoming http requests).
Then, the call to server.listen() encapsulates steps 1-3 above.
Step 4 happens inside the http server object implementation and the app object is called when a new connection has been established and a new HTTP request is available. The http server reads the initial request and parses the http protocol and that initial request is already made available to the app for routing to the appropriate handler.
Then, subsequent calls such as res.send() or res.json() write a response back on the http socket and close the socket or res.end() will close it directly (steps 5 and 6 above).
Some other useful references:
Why is bind() used in TCP? Why is it used only on server side and not in client side? - Helps explain how a port and IP address define the TCP endpoint represented by a server. This port has to be known by the client so it can specifically request to connect to that port. The client end of the socket also has an IP address and a port, but its port can be dynamically assigned, thus the client does not have to bind to a specific port itself. The four pieces of data [server IP, server port, client IP, client port] define a specific TCP connection.
How TCP sockets work - has a good section about how new connections to a server work.
Understanding socket and port in TCP - talks about active and passive sockets. Passive sockets are sockets in "listen" mode used to accept incoming connections. Active sockets are two-way communications channels between two TCP endpoints.
Transmission Control Protocol (TCP) - more details on the various aspects of TCP from initiating a listening server, initiating a client connection to that server, through packet transmission to closing the socket.
There are a gazillion other references on the topic on the web. You can probably find 1000 articles on any single aspect of TCP that you might want more info about.
I think, this could be made easier if I could actually see the function instead of only calling it.
The underlying code for listen is inside the operating system's TCP stack and is not part of nodejs or Express. Express relies on the nodejs http server object as its interface to that and the nodejs http server object uses native code (built into nodejs) to call libuv (which is a cross platform C library that nodejs uses for networking and other things). Then, libuv talks to the underlying operating system APIs to reach the actual TCP stack on that target host. All of this is to put the server socket into listen mode so it can be notified of new incoming client connections to that target IP address and port.
Here's some doc on the related portions of the Linux TCP API if you want to see what the underlying TCP interface and description of that interface is:
socket() - https://linux.die.net/man/7/socket
bind() - https://linux.die.net/man/2/bind
listen() - https://linux.die.net/man/2/listen
And, portions of the libuv library that nodejs uses for networking:
TCP handles - http://docs.libuv.org/en/v1.x/tcp.html
Server listen() and accept() - http://docs.libuv.org/en/v1.x/stream.html#c.uv_listen
I thought when a server is started, it creates a specific number of TCP ports on a computer. so whenever a new connection comes in, it assigns a port to that client ('connection'). Recently I opened tutorialsPoint website 'https://www.tutorialspoint.com/socket.io/socket.io_namespaces.htm' and in there is written:
"Socket.IO allows you to “namespace” your sockets, which essentially means assigning different endpoints or paths. This is a useful feature to minimize the number of resources (TCP connections) and at the same time separate concerns within your application by introducing separation between communication channels. Multiple namespaces actually share the same WebSockets connection thus saving us socket ports on the server".
This part i did not understand: "Multiple namespaces actually share the same WebSockets connection thus saving us socket ports on the server". My question is how can all the connections share a single port on the web-server.
Any help will be highly appreciated.
Do all the sockets in a namespace connect to the same port on the server in socket.io?
Yes, they do.
First off socket.io is built on the underlying webSocket protocol. A webSocket connection starts with an http connection which is built on top of a TCP connection and then the two sides agree to "upgrade" the protocol to start talking the webSocket protocol instead of the http protocol.
So, when a socket.io connection comes in, it's initially an http connection.
Second, any TCP server is listening for inbound connections on a known port. The client must know what that port is and the client attempts to connect to the combination of IP address and port. A regular TCP server using only one network adapter will just be listening on that one port. All inbound client connections will arrive on that one port.
I thought when a server is started, it creates a specific number of TCP ports on a computer. so whenever a new connection comes in, it assigns a port to that client ('connection').
That's not how it works. A listening server creates a passive socket listening for inbound connections on one specific port. When a TCP client initiates an outbound connection, that client picks a dynamically selected port number for that outbound connection (that is unique for that client and not currently in use). This source port number is typically not visible in TCP, http, webSocket or socket.io programming (though you can see what is is if you want - you just don't have to use it yourself at the level we usually program at). It's part of the TCP plumbing that helps packets get delivered to the right socket. So, at that point it has a source IP address and a source port number. It then attempts to connect to a target IP address on a target port.
That unique combination of those four parameters:
source IP
source port (dynamically assigned on the client)
target IP (known in advance by the client)
target port (known in advance by the client)
defines a unique TCP connection. No two TCP connections will have the same four parameters. If the same client makes another TCP connection to the same target IP and port, it will be assigned a different source port number and thus it will be a different unique combination.
There's one little (somewhat confusing) aspect here that I'll make you aware of, but not try to overly explain or confuse things by. Many clients are actually on a private network and have a private IP address. That private IP address is not what the server actually sees as the source of the connection. At some point the connection goes through a gateway that connects the private network to a public network. This gateway will do NAT (network address translation). It will swap the private source IP/port for a public source IP/port that corresponds to the gateway itself. It remembers what it swapped so that when packets come back the other directly, it can swap it back. So, the target server actually believes it's communicating with the gateway, but anything the target sends to the gateway is "forwarded" onto the private IP address/port of the original sender. So, you don't really need to understand the details of the gateway except that it's serves as a broker between the private IP address of some computer on a private network and some computer on the public internet that you are trying to connect to. It does what's called "network address translation" to make this all work. For the rest of the discussion, you should forget about this and just pretend that both source and target are both on the public internet with public IP addresses (even though that is almost never the actual case, but the gateway makes it just work as if they were).
"Socket.IO allows you to “namespace” your sockets, which essentially means assigning different endpoints or paths. This is a useful feature to minimize the number of resources (TCP connections) and at the same time separate concerns within your application by introducing separation between communication channels. Multiple namespaces actually share the same WebSockets connection thus saving us socket ports on the server".
In socket.io, when you connect on a namespace, you are creating a new underlying webSocket connection to the same target IP/port. A server can have many inbound connections to the same IP/port. Each is given it's own TCP socket and the four parameters mentioned above uniquely define each one. When an inbound network packet arrives at the lowest level, TCP can tell which source IP and source port it came from and which target IP/port is was sent and that allows the TCP driver to figure out which socket that packet belongs to so that the packet can be delivered to the code that is monitoring that specific socket.
This part i did not understand: "Multiple namespaces actually share the same WebSockets connection thus saving us socket ports on the server". My question is how can all the connections share a single port on the web-server.
To use a namespace in socket.io, you make a new socket.io connection to that specific namespace. You don't use multiple namespaces on a single socket.io connection. But, a namespace operates at a higher level than the TCP or webSocket connection logic. It rides on top of that in the application layer. So, all namespace connections, no matter which namespace you are using, connect to the same server on the same IP and same port. Once the connection has been established, socket.io sends some data that it would like a "logical" connection on this namespace and then the receiving socket.io code is informed that the new connection belongs in this namespace.
Here's a useful article to read on the topic: Understanding socket and port in TCP.
For instance, a tcp server is listening address 10.10.10.1:80. And client 10.10.10.2:555 connect in. Server accept() will create a new connected socket which is identified by (10.10.10.1:80,10.10.10.2:555,tcp). And server create a thread for connected socket and continue to listen socket.
when the client has new data to server, My question is how kernel distinguish the listen socket and connected socket? (of course they have different filehands, but how kernel get the right socket via the target address?) How kernel know this coming data will be handover to APP via connected socket, not the listen socket? Not sure I get my question clear.
It doesn't have to. The incoming segment is addressed to a local IP:port and has a source address of another IP:port. All it has to do is find a socket with that local IP:port and that remote IP:port. The listening socket won't have a remote IP:port at all, and sockets with that local IP:port that are connected to other peers will have a different remote IP:port.
I'm writting a program that simulates nodes in network. Every node is listening to some port on local for incoming requests. If a request is received it replies to sender of the request. The reply is sent after a socket is created associated with the address of the sender of the request. Since sender is using some port on localhost and has used bind to listen to id, trying to bind to the same port with an other process results in a messages that states that the port is already taken (bound).
How should I solve this in order to be able to simulate server/client on the same machine? I am using UDP protocol for this program.
You solve this by using different ports for client and server interaction. A useful example would be how client and server interact during DHCP. The client sends requests via UDP on port 67 and the server sends responses back via UDP on port 68.
Ok this is kind of embarassing but I just have a rather "noob" question.
In a client server TCP communications, where my system is a client accessing a remote server at say Port XX, isnt the client opening a random port YY in its system to talk to remote port XX?
So when we code we do specify the destination port XX right?
For the client, the port YY itself is chosen when the socket is created, isnt it?
Is there anyway I could monitor/restrict/control any client talking to a particular server?(like say clients talking to servers at specific serving ports??)
Is there any IPTABLE rule or some firewall rule restricting the client?
Can this be done at all??
Are destination ports saved in the socket structures? If so where??
Thanks!
First, server side creates a listening socket, with the chain of socket(2), bind(2), and listen(2) calls, then waits for incoming client connection requests with the accept(2) call. Once a client connects (socket(2) and then connect(2) on the client side) and the TCP/IP stacks of the client and the server machines complete the three way handshake, the accept(2) returns new socket descriptor - that's the server's end of the connected socket. Both bind(2) on the server side, and connect(2) on the client side take server's address and port.
Now, the full TCP connection is described by four numbers - server address, server port, client address, and client port. The first two must obviously be known to the client prior to the connection attempt (otherwise, where do we go?). The client address and port, while could be specified explicitly with the bind(2), are usually assigned dynamically - the address is the IP address of the outgoing network interface, as determined by the routing table, and the port selected out of range of ephemeral ports.
The netstat(8) command shows you established connections. Adding -a flag lets you see listening sockets, -n flag disables DNS and service resolution, so you just see numeric addresses and ports.
Linux iptables(8) allows you to restrict where clients are allowed to connect to. You can restrict based on source and destination ports, addresses, and more.
You can get socket local binding with getsockname(2) call, remote binding is given by getpeername(2).
Hope this makes it a bit more clear.
Yes you can create a firewall rule to prevent outbound TCP connections to port XX. For example, some organizations prevent outbound TCP port 25, to prevent spam being sent from network PCs to remote SMTP servers.