I'm adding third party logging to my nodejs app, the API has an ingestion endpoint I need to call with logging information.
The problem is that this request takes time, if I await all the logging requests it will add significant time to function executions.
My thinking was to do a 'fire and forget' approach however given NextJS will cancel all executing processes this may occasionally fail. I also found logging may arrive in a different order than sent (though this can be mediated with a timestamp).
What I'd love to do is fire off the log and ensure the request has been sent without waiting on the response from the server - is this possible?
I'm thinking to add a 1ms wait after each log since an initial http request with a small payload isn't going to take that long to send but wanted to know are their any best practices for handling these sorts of scenarios?
I have situation in my NodeJS API where I need to handle complex request which usually takes about few seconds to process. Problem is when I'm doing load tests on this API call I'm starting to receive back Timeouts (but requests still been processed in background). How and what is best option to handle this situation? Should I increase timeout or do something else? Where I need to do that, on NodeJS side or maybe on Nginx side because I use Nginx as reverse proxy?
I have been studying web sockets recently and plan to use them in my application even though the app is not realtime. I am mostly doing this because I want to try it out and further down the line it might open more possibilites for the app's functionality. Also I am not bothered about having an API for mobile at the moment but think it would still be possible to have some kind of api over web sockets if I needed it in the future.
However for in-production apps are there any real reasons why somebody would consider implementing websockets if there is no real-time element?
Are there any benefits over HTTP requests other than the real timeness of it?
HTTP requests include the full HTTP headers. Depending on the cookie load, this may reach a couple of KB per request. WebSocket protocol headers are minimal compared to that. If you have a lot of requests and care about bandwidth then going with WebSocket makes sense.
Additionally a HTTP connection is (traditionally) negotiated for each request, which means you have overhead on each request compared to WebSocket, which has persistent connections. Connection establishment takes time (hence the advantage in real-time applications), but it also uses resources on the server. Again, depending on your app's communication patterns, using WebSocket may make sense.
I have a specific/strange requirement where I need to do something like this:
app.put('/example/:entity', function(req, res, next) {
fs.writeFileSync(someFile);
});
Since its a sync operation, it will block the event loop and I am concerned that node.js/express will start dropping http requests.
For a short term solution:
Is there any queue/limit of incoming requests in node.js/express?
Is there any configuration that I can use to increase the queue size?
Would node.js/express bounce requests if the event loop is blocked?
Your service will still accept incoming connections while you have node's event loop blocked up but express won't be able to respond to anything. Incoming requests will get queued as part of the underlying TCP implementation.
Is there any queue/limit of incoming requests in node.js/express?
There's probably a virtual limit in terms of memory being used to queue incoming requests, but this is a detail of your underlying TCP stack, not node.js.
Is there any configuration that I can use to increase the queue size?
I think you can decerase the TCP connection cap in OS settings to prevent your server from getting easily overloaded. Increasing it is the wrong solution to your problem I think (see below).
Would node.js/express bounce requests if the event loop is blocked?
No, since its I/O under the hood, node.js/express are not bouncing the incoming requets you get while the event loop is blocked. Your requests won't get bounced until you hit some configured TCP connection limit. But you really don't want it to ever get this blocked up.
To be frank, blocking the event loop in node.js is almost always a bad idea. and there's always a way around it. In your case, if you need to write that file in series, try pushing events onto a backend queue that you can configure to handle one event at a time without blocking up the works. kue + redis is a good option for this.
We have a node.js server which implements a REST API as a proxy to a central server which has a slightly different, and unfortunately asymmetric REST API.
Our client, which runs in various browsers, asks the node server to get the tasks from the central server. The node server gets a list of all the task ids from the central one and returns them to the client. The client then makes two REST API calls per id through the proxy.
As far as I can tell, this stuff is all done asynchronously. In the console log, it looks like this when I start the client:
Requested GET URL under /api/v1/tasks/*: /api/v1/tasks/
This takes a couple seconds to get the list from the central server. As soon as it gets the response, the server barfs this out very quickly:
Requested GET URL under /api/v1/tasks/id/:id :/api/v1/tasks/id/438
Requested GET URL under /api/v1/workflow/id/:id :/api/v1/workflow/id/438
Requested GET URL under /api/v1/tasks/id/:id :/api/v1/tasks/id/439
Requested GET URL under /api/v1/workflow/id/:id :/api/v1/workflow/id/439
Requested GET URL under /api/v1/tasks/id/:id :/api/v1/tasks/id/441
Requested GET URL under /api/v1/workflow/id/:id :/api/v1/workflow/id/441
Then, each time a pair of these requests gets a result from the central server, another two lines is barfed out very quickly.
So it seems our node.js server is only willing to have six requests out at a time.
There are no TCP connection limits imposed by Node itself. (The whole point is that it's highly concurrent and can handle thousands of simultaneous connections.) Your OS may limit TCP connections.
It's more likely that you're either hitting some kind of limitation of your backend server, or you're hitting the builtin HTTP library's connection limit, but it's hard to say without more details about that server or your Node implementation.
Node's built-in HTTP library (and obviously any libraries built on top of it, which are most) maintains a connection pool (via the Agent class) so that it can utilize HTTP keep-alives. This helps increase performance when you're running many requests to the same server: rather than opening a TCP connection, making a HTTP request, getting a response, closing the TCP connection, and repeating; new requests can be issued on reused TCP connections.
In node 0.10 and earlier, the HTTP Agent will only open 5 simultaneous connections to a single host by default. You can change this easily: (assuming you've required the HTTP module as http)
http.globalAgent.maxSockets = 20; // or whatever
node 0.12 sets the default maxSockets to Infinity.
You may want to keep some kind of connection limit in place. You don't want to completely overwhelm your backend server with hundreds of HTTP requests under a second – performance will most likely be worse than if you just let the Agent's connection pool do its thing, throttling requests so as to not overload your server. Your best bet will be to run some experiments to see what the optimal number of concurrent requests is in your situation.
However, if you really don't want connection pooling, you can simply bypass the pool entirely – sent agent to false in the request options:
http.get({host:'localhost', port:80, path:'/', agent:false}, callback);
In this case, there will be absolutely no limit on concurrent HTTP requests.
It's the limit on number of concurrent connections in the browser:
How many concurrent AJAX (XmlHttpRequest) requests are allowed in popular browsers?
I have upvoted the other answers, as they helped me diagnose the problem. The clue was that node's socket limit was 5, and I was getting 6 at a time. 6 is the limit in Chrome, which is what I was using to test the server.
How are you getting data from the central server? "Node does not limit connections" is not entirely accurate when making HTTP requests with the http module. Client requests made in this way use the http.globalAgent instance of http.Agent, and each http.Agent has a setting called maxSockets which determines how many sockets the agent can have open to any given host; this defaults to 5.
So, if you're using http.request or http.get (or a library that relies on those methods) to get data from your central server, you might try changing the value of http.globalAgent.maxSockets (or modify that setting on whatever instance of http.Agent you're using).
See:
http.Agent documentation
agent.maxSockets documentation
http.globalAgent documentation
Options you can pass to http.request, including an agent parameter to specify your own agent
Node js can handle thousands of incoming requests - yes!
But when it comes down to ougoing requests every request has to deal with a dns lookup and dns lookup's, disk reads etc are handled by the libuv which is programmed in C++. The default value of threads for each node process is 4x threads.
If all 4x threads are busy with https requests ( dns lookup's ) other requests will be queued. That is why no matter how brilliant your code might be : you sometimes get 6 or sometimes less concurrent outgoing requests per second completed.
Learn about dns cache to reduce the amount of dns look up's and increase libuv size. If you use PM2 to manage your node processes they do have a well documentation on their side on environment variables and how to inject them. What you are looking for is the environment variable UV_THREADPOOL_SIZE = 4
You can set the value anywhere between 1 or max limit of 1024. But keep in mind libuv limit of 1024 is across all event loops.
I have seen the same problem in my server. It was only processing 4 requests.
As explained already from 0.12 maxsockets defaults to infinity. That easily overwhelms the sever. Limiting the requests to say 10 by
http.globalAgent.maxSockets = 20;
solved my problem.
Are you sure it just returns the results to the client? Node processes everything in one thread. So if you do some fancy response parsing or anything else which doesn't yield, then it would block all your requests.