I am checking node-http-proxy and nodejs-proxy to build a DIY reverse proxy/load balancer in Node.js. After coding a small version, I setup 2 WEBrick servers for the same Rails app so I could load balance (round robin) between them. However each HTTP request is sent to one or another server which is very inefficient since the loading process of CSS and Javascript files from the home page is performed with more than 25 GET requests.
I tried to play a bit with socket events but I didn't get anywhere because by default it uses keep-alive connections (possibly this is why nginx just support http/1.0).
Ok, so I am wondering how can my proxy send a block of HTTP requests (for instance loading a webpage entirely, etc) to only one server so I could send the next block to another server.
You need to consider stickiness or session persistence. This will ensure future connections after the first connection inbound will get 'stuck' to the chosen server for the duration of the session or until the persistence connection times out.
Related
I'm developing an app with NestJs where I'm using throttle module for banning abusive requests.
One thing that I couldn't find a clear answer is that if it's going to block abusive requests (for example, more than 20 requests per minute) will it also block the frontend requests made by nodejs server?
I mean getServerSideProps will make a request in every render. If our website has more than 100 visitors per minute, what will be happened in this situation? Considering that
Frontend and backend projects both are on same server with same IP
They are hosted on different servers with different IP addresses
Your suspicion is valid because #nestjs/throttler does not differentiate between local and remote requests so yes your NextJs server will be blocked quickly.
I'd suggest you to use reverse proxies instead which are more mature and also does not check local requests.
I am working on a nodejs app with Socket.io and I did a test in a single process using PM 2 and it was no errors. Then I move to our production environment(We use Google Cloud Compute Instance).
I run 3 app processes and a iOS client connects to the server.
By the way the iOS client doesn't keep the socket connection. It doesn't send disconnect to the server. But it's disconnected and reconnect to the server. It happens continuously.
I am not sure why the server disconnects the client.
If you have any hint or answer for this, I would appreciate you.
That's probably because requests end up on a different machine rather than the one they originated from.
Straight from Socket.io Docs: Using Multiple Nodes:
If you plan to distribute the load of connections among different processes or machines, you have to make sure that requests associated with a particular session id connect to the process that originated them.
What you need to do:
Enable session affinity, a.k.a sticky sessions.
If you want to work with rooms/namespaces you also need to use a centralised memory store to keep track of namespace information, such as the Redis/Redis Adapter.
But I'd advise you to read the documentation piece I posted, things might have changed a bit since the last time I've implemented something like this.
By default, the socket.io client "tests" out the connection to its server with a couple http requests. If you have multiple server requests and those initial http requests don't go to the exact same server each time, then the socket.io connect will never get established properly and will not switch over to webSocket and it will keep attempting to use http polling.
There are two ways to fix this.
You can configure your clients to just assume the webSocket protocol will work. This will initiate the connection with one and only one http connection which will then be immediately upgraded to the webSocket protocol (with socket.io running on top of that). In socket.io, this is a transport option specified with the initial connection.
You can configure your server infrastructure to be sticky so that a request from a given client always goes back to the exact same server. There are lots of ways to do this depending upon your server architecture and how the load balancing is done between your servers.
If your servers are keeping any client state local to the server (and not in a shared database that all servers access), then you will need even a dropped connection and reconnect to go back to the same server and you will need sticky connections as your only solution. You can read more about sticky sessions on the socket.io website here.
Thanks for your replies.
I finally figured out the issue. The issue was caused by TTL of backend service in Google Cloud Load Balancer. The default TTL was 30 seconds and it made each socket connection tried to disconnect and reconnect.
So I updated the value to 3600s and then I could keep the connection.
I'm making an app using node.js' express framework which serves both html content over http and uses websockets for a chat feature. I'm wondering how I can accomplish both at the same time. My idea is to use a different port for websocket connections (so http requests would come to port 3000 and websockets would connect on port 3001) but I don't know if that's a good solution. I'm especially worried about deployment to something like heroku and if I can specify different ports for my app.
I'm wondering how I can accomplish both at the same time.
The webSocket protocol is specially designed so it can run on the same port as your regular web server requests. So, you don't need a separate port in order to have both a web server and chat running using webSockets.
This works because a webSocket connection is always initiated with an http request that sets a few special headers. The receiving web server can then detect those special headers and know that this incoming http request is actually a request to initiate a webSocket connection. With a particular response, the client and server then agree to "upgrade" the connection and switch to the webSocket protocol. From that point on, that particular TCP connection uses the webSocket protocol.
Meanwhile any incoming http request that does not have the special webSocket headers on it is treated by your web server as just a regular http request. In this way, the same server and the same port can be used for both webSocket connections and regular http requests. No second port is needed.
Another advantage of this scheme is that the client can avoid the cross-origin issues that it would run into if it was trying to use a different port than the web page it was loaded from.
I'm especially worried about deployment to something like heroku and
if I can specify different ports for my app.
If you were to actually use two ports, then you would need to create two separate servers, one listening on each port since a given server can only listen on one port. In node.js, the two servers could both be in the same node.js app (making it easier to share data between them) or you could put them in completely separate node.js processes (your choice).
And, if you used multiple ports, you'd also have to support CORS so that the browser would be allowed to connect to the separate port (to avoid same-origin restrictions).
I have a standard LAMP EC2 instance set-up running on Amazon's AWS. Having also installed Node.js, socket.io and Express to meet the demands of live updating, I am now at the stage of load balancing the application. That's all working, but my sockets aren't. This is how my set-up looks:-
--- EC2 >> Node.js + socket.io
/
Client >> ELB --
\
--- EC2 >> Node.js + socket.io
[RDS MySQL - EC2 instances communicate to this]
As you can see, each instance has an installation of Node and socket.io. However, occasionally Chrome debug will 400 the socket request returning the reason {"code":1,"message":"Session ID unknown"}, and I guess this is because it's communicating to the other instance.
Additionally, let's say I am on page A and the socket needs to emit to page B - because of the load balancer these two pages might well be on a different instance (they will both be open at the same time). Using something like Sticky Sessions, to my knowledge, wouldn't work in that scenario because both pages would be restricted to their respective instances.
How can I get around this issue? Will I need a whole dedicated instance just for Node? That seems somewhat overkill...
The issues come up when you consider both websocket traffic (layer 4 -ish) and HTTP traffic (layer 7) moving across a load balancer that can only inspect one layer at a time. For example, if you set the ELB to load balance on layer 7 (HTTP/HTTPS) then websockets will not work at all across the ELB. However, if you set the ELB to load balance on layer 4 (TCP) then any fallback HTTP polling requests could end up at any of the upstream servers.
You have two options here. You can figure out a way to effectively load balance both HTTP and websocket requests or find a way to deterministically map requests to upstream servers regardless of the protocol.
The first one is pretty involved and requires another load balancer. A good walkthrough can be found here. It's worth noting that when that post was written HAProxy didn't have native SSL support. Now that this is the case it might be possible to just remove the ELB entirely, if that's the route you want to go. If that's the case the second option might be better.
Otherwise you can use HAProxy on its own (or a paid version of Nginx) to implement a deterministic load balancing mechanism. In this case you would use IP hashing since socket.io does not provide a route-based mechanism to identify a particular server like sockjs. This would use the first 3 octets of the IP address to determine which upstream server gets each request so unless the user changes IP addresses between HTTP polls then this should work.
The solution would be for the two(or more) node.js installs to use a common session source.
Here is a previous question on using REDIS as a common session store for node.js How to share session between NodeJs and PHP using Redis?
and another
Node.js Express sessions using connect-redis with Unix Domain Sockets
I don't have the faintest clue on how a software or hardware load balancer works. I guess the hardware load balancer is basically a switch and based on some algorithm decides which node to switch to for a incoming request. On the software load balancer front, I guess the software picks up a node and uses a reverse proxy connection to it. In such a scenario, 2-way SSL wont work as the load balancer cannot have the client's private key.
Again, I don't how a software load balancer works but as my application would need a load balancer and as the application uses 2-way SSL connection, I wanted to know how does a software load balancer take care of a 2-way SSL connection.
No, SSL works with a load balancer. They typically work at the TCP level, so the clients connect to the LB IP address, but it NATs the connections on to the real servers. The connection persists to the same real server for its lifetime, but if the same client makes another one, it can (and typically would) go to a different server.
For HTTPS this works fine, except that if you have a web server which supports SSL session caching, then the SSL session cache will be lost if the client comes back to a different server. In practice this is not a big problem. Of course HTTP keep-alive sessions aren't affected because they are a single TCP connection so they stay on the same realserver.
Generally speaking, a software load balancer will note that there is a new incoming connection request, assess the workload on the machines available, and allocate the new request to the most appropriate machine. When there is a session-based service, that connection will last for the duration of the session; rebalancing would only occur if a server went down, and would probably establish new connections in a newly balanced configuration.
So, as Jon implied, the SSL session would be established with a server, and would continue with that server until the session terminates.
If you want to route connections more dynamically, then it may be that the SSL session has to be terminated (decrypted) in front of the software that dynamically sends requests to different servers.
All these are possible - they are not necessarily efficient or implemented.
A software load balancer will distribute sessions evenly across multiple servers.
So, if a user hits your load balancer, it will send him to a specific server and that server will negotiate the SSL. The user will continually talk to this server until his session expires. At that point, he will hit the load balancer again.