I want to build a DSR load balancer for an application I am writing. I wont go into the application because it is irrelevant for this discussion. My goal is to create a simple load balancer that does direct server response for TCP packets. The idea is to receive all packets at the load balancer, then using something like round robin, select a server from a list of available servers which are defined in some config file. The next step would be to alter the packer received and change the destination ip to be equal to the chosen backend server. Finally, the packet will be sent over to the backend server using normal system calls for sending packets. Theoretically the backend server should receive the packet, and send one back to the original requester, and then the requester can communicate directly with the backend server rather than going through the load balancer.
I am concerned that this design will not work as I expect it to. The main question is, what happens when computer A send a packet to IP Y, but receives a packet back in the same TCP stream from a computer at IP X? Will it continue to send packets to IP Y? Or will it switch over to IP X?
So it turns out this is possible, but only halfway so, and I will explain what I mean by this. I have three processes, one which is netcat, used to initiate an tcp request, a second process, the dsr-lb, which receives packets on a certain port, changes the destination ip to a backend server(passed in via command line arg), and forwards it off using raw sockets, and a third process which is a basic echo server. I got this working on a local setup. The local setup consists of netcat running on my desktop, and dsr-lb and echo servers running on two different linux VMs on the desktop as well. The path of the packets was like this:
nc -> dsr-lb -> echo -> nc
When I said it only half works, what I meant was that outgoing traffic has to always go through the dsr-lb, but returning traffic can go directly to the client. The client does not send further traffic directly to the backend server, but still goes through the dsr-lb. This makes sense since the client opened a socket to the dsr-lb ip, and internally still remembers this ip, regardless of where the packet came from.
The comment saying "if its from a different IP, it's not the same stream. tcp is connection-based" is incorrect. I read through the linux source code, specifically the receive tcp packet portion, and it turns out that linux uses source ip, source port, destination ip, and destination port to calculate a hash which is uses to find the socket that should receive the traffic. However, if no such socket matches the hash, it tries again using only the destination ip and destination port and that is how this "magic" works. I have no idea if this would work on a windows machine though.
One caveat to this answer is that I also spun up two remote VMs and tried the same experiment, and it did not work. I am guessing it worked while all the machines were on the same switch, but there might be a little more work to do to get it to work if it goes through different routers. I am still trying to figure this out, but from using tcpdump to analyze the traffic, for some reason the dsr-lb is forwarding to the wrong port on the echo server. I am not sure if something is corrupted, or if the checksum is wrong after changing the destination ip and some router along the way is dropping it or changing it somehow(I suspect this might be the case) but hopefully I can get it working over an actual network.
The theory should still hold though. The IP layer is basically a packet forwarding layer and routers should not care about the contents of the packets, they should just forward packets based on their routing tables, so changing the destination of the packet while leaving the source the same should result in the source receiving any answer. The fact that the linux kernel ultimately resolves packets to sockets just using destination ip and port means the only real roadblock to this working does not really exist.
Also, if anyone is wondering why bother doing this, it may be useful for a loadbalancer in front of websocket servers. Its not as great as a direct connection from client to websocket server, but it is better than a loadbalancer that handles both requests and responses, which makes it more scalable, and more able to run on less resources.
Introduction
I am currently trying to build up a networking layer for Unity from scratch. Currently I am testing the communication via UDP using Node.js for the server and the client. However I guess the language of the implementation will not matter for what I am asking for.
Current approach
The current approach using Node.js for the server and the client is pretty basic. I simply send a packet from a client to my server while the client and the server are not in the same local network. Both are behind a router and therefore also behind a NAT.
The server then sends back an answer to the IP and port received within the UDP packet that was sent from the client.
Problem
I am curious about the security on the client side regarding to ports being opened on the client machines and routers. So far I assumed that I don't need to do anything to secure the client from attackers or anything else that can do something with the ports that are used by my application. The following assumption shows why I think that I don't need to do anything to secure the clients.
Assumption
Server is setting up callbacks.
Server starts listening to a specific port which is also forwarded to the servers machine within the router.
Server now will call a callback when a UDP message was received. The server then will send a UDP message to the address and the port of the client obtained by the message received.
Client is setting up callbacks.
Client starts listening to port 0 which for Node.js's dgram means:
For UDP sockets, causes the dgram.Socket to listen for datagram messages on a named port and optional address. If port is not specified or is 0, the operating system will attempt to bind to a random port. - https://nodejs.org/api/dgram.html#dgram_socket_bind_port_address_callback
So the operating system now knows that packets sent to this port belong to my application.
Nobody can use this for something malicious.
Client, which knows the servers address and port, starts the process of sending a UDP message to the server.
Clients router receives the UDP message. NAT creates a random port (used on the public side) and maps it to the clients (local) address and port.
So the router now knows that packets sent to the public address and the newly generated port belong to the local address and port.
Nobody can use this for something malicious.
Clients router sends UDP message containing the public address and the NAT generated port to the server.
The worst thing that can happen is that a man-in-the-middle attacker can read the data the client is sending. Due to it is only gamedata like positions and so on that is sent this is not a big problem while developing the basics.
Nobody can use this for something malicious.
Server receives the message and calls the callback described in 3. So the server sends to the public address and the NAT generated port of the client.
The worst thing that can happen is that a man-in-the-middle attacker can read the data the server is sending. Due to it is only gamedata like positions and so on that is sent this is not a big problem while developing the basics.
Nobody can use this for something malicious.
Same as 7. with the servers router and the servers local address and port.
Same as 8. with the servers router.
Client receives the UDP message of the server and calls a callback which processes the message contents.
Due to the local port of the client is bound to my application only nobody can use this for something malicious due to I simply ignore the contents if they are not from the real server.
Question
So is my assumption correct and I really don't need to secure the client from any attacks that will harm the clients in any way?
I have one server and multiple clients. The server wants to run shell script on each device it wants to. Absolutely it's not possible via simple socket because we may have thousands of devices. Also server and devices should be always connected via socket. after a lot of search I found out that the solution might be NAT-T. But still I don't know how to use that or if there is another solution.
Please help me what should I do on clients and server.
If you don't know the clients address and port upfront, you need to connect to the server with the clients. 1000s of devices are no problem. You run in a socket limit around 65000 open ports (check ulimit). Build an object stream between client and server and execute the script based on the object the client receives. You could also set an interval on the clients and let them check with simple http(s) every n secs if there is something to do for them?
See for example here: Node Stream Docs
Or here: Node HTTP Docs
I have Node.js TCP server framework which acts as a central TCP server connecting to many TCP clients which are sensors gathering and sending data. This is essentially a machine-to-machine communication where the TCP client establishes a connection and starts sending data. The server has to authenticate and then process the data.
What I want to do? --->
Authenticate each client by identifying them and making sure they are in the users list in the database.
I can identify each incoming client by its ip and port. However, each client has a dynamic IP. This means that I cannot rely on it to compare against a list in my DB.
My goal here is to make sure that each connection is valid and is part of my user database. I was thinking along the lines of implementing an 'application layer' where both the sensor(client) and the TCP server know a string which they match. When the client setups a connection, it sends this string and this is used by the server to compare against a list in the database. This way the client emits a 'keyword' each time it establishes a connection.
If this is a viable method, how can I use the node.js 'NET' module to emit a keyword only when the connection is established? I don't see any such provision.
Also, is there a better way to identify clients in a M-M connections? Any pointers will be helpful.
Node-RSA Will do exactly this for you.
Else,
use the socket.write function in order to send data. Then if another Node app is listening on the other side you can utilize the socket data event to obtain what the sender has written. If you only want certain machines to connect to yours I'd indeed recommend using a secret key if they IP adresses aren't static. Though beware this is not completely safe as people could perform a bruteforce attack. It is possible to minimize the chance of someone actually guessing your secret key by you guessed it, utilizing a key which is a hashed string. The string could be anything, as long as both of you have it. It's even better to create and share the same function which alters the string every once in a while for a little bit more security.
Please note that this isn't as secure as the first option but it is definitely still something.
I have a C/S model that each client use 3 or more TCP connections to one Server, for example one connection for binary data, one for text messages and one for controlling.
However, I have no idea how to "group" these 3 connections as one client.
I've tried use IP to identical, but different client may have same IP address.
Though I'm using node.js, I think this is a common problem may appear in any languages/implements.
The only thing you can do is handle it at an application layer. Send some data that the clients have to return to you in some form for each connection. (Look at the SSL handshaking process. Maybe just use that!)