I am using nodejs async module for concurrent connectios, Now my backend server only can handle 1000 connections at a time, I am using async.mapLimit to limit the connections, each and every job of async.mapLimit does multiple connections, when I am sending the same request which does async.mapLimit from multiple browser at the same time, then I am getting EMFILE error from Server side
([Error: connect EMFILE] code: 'EMFILE', errno: 'EMFILE', syscall: 'connect'),
My code somewhat looks like this:
async.mapLimit(jobList, 200, jobCallback, function(error, data) {
});
function jobCallback(job, callback) {
/* Make multiple connections to to backend server, this number is
dynamic, here also I use async.mapLimit */
}
Now I want to implement some wrapper function top of this mapLimit or anything, irrespective of number of parallel requests, I want to limit the concurrent connections, even irrespective of number of client calls also, it may be slower, but I do not bother.
How I can achieve this?
I am using restler library. I have tried to set
proto.globalAgent.maxSockets = 1000
to do concurrent 1000 connections at a time, but it seems it is not working.
Please advise.
-M-
You will have to control for throttling yourself, as that async instruction won't know if you have calls from other users adding to the 1000 limit.
In REST services, a service would typically send an http 429 response when such a limit is triggered, allowing your app to identify a bottleneck scenario and trigger a throttling mechanism.
A common way to do that is via exponential backoff
https://developers.google.com/api-client-library/java/google-http-java-client/backoff
I use the following line of code to manage the limit globally:
require('events').EventEmitter.prototype._maxListeners = 1000;
Thanks
Related
I'm currently load testing one of my API (Node.js + Express). This API makes a HTTP request to another server. Here's an example code:
var start = new Date()
axios.get('https://google.com')
.then(function (response) {
var end = (new Date() - start)/1000
console.info('Finished in %ds', end)
})
During the test, I find out that the more concurrent HTTP requests to the other server (in this example it's google.com), the slower the response becomes. I use Apache Jmeter for testing.
For example, if I do 1 request in one second:
Finished in 0.150s
But if I do 100 requests in one second:
Finished in 0.320s
...
Finished in 1.190s
Finished in 2.559s
Finished in 1.230s
Finished in 5.530s
At first I thought there must be a problem in the other server but that is not the case, even after I changed it to google.com (as per example), the same thing happened.
The more outbound http request that node.js has to make, the slower the response becomes. I have tried to improve my API by using node cluster, the workers help but I want to improve the response time even further.
Is there anything that I can do? or perhaps an explanation on why does this happen? I thought since my API makes asynchronous http requests, there should be no blocking, thus the response time should not be increased by such a significant amount.
Thanks.
I was facing a similar issue - in my instance I was awaiting each API call rather than allowing them to all occur asynchronously.
To do this you can push all of your async API calls into an array. For example, if you need to call a series of urls:
const requests = []
urls = ['http...a/get','http...b/get']
urls.map(item => {
request.push(axios.get(item))
})
Now that each of these calls are occurring asynchronously, be sure to wait for all of them to resolve before consuming the data.
const allAPIData = await Promise.all(requests)
Just be sure to handle your promise resolution in the event any of the API calls fail, perhaps with a helper function that nests axios.get(url). Otherwise any failed API promises could cause issues awaiting and resolving the Promise.all() statement.
I currently have a request which is made from an angular 4 app(which uses electron[which uses chromium]) to a bottleneck(nodejs/express) server. The server takes about 10 minutes to process the request.
The default timeout which I'm getting is 120 seconds.
I tried to use setting the timeout on the server using
App.use(timeout("1000s")
In the client side I have used
options = {
url,
method: GET
timeout : 600 * 1000}
let req = http.request(options, () => {})
req.end()
I have also tried to give the specific route timeout.
Each time the request hits 120 seconds the socket dies and I get a "socket timeout"
I have read many posts with the same questions but I didn't get any concrete answers. Is it possible to do a request with a long/no timeout using the tools above? Do I need to download a new library which handles long timeouts?
Any help would be greatly appriciated.
So after browsing through the internet I have discovered that there is no possible way to increase Chrome's timeout time.
My solution to this problem was to open the request and return a default answer(something like "started") then pinging the server to find out it's status.
There is another possible solution which will be to put a route in the client(I'm using electron and node modules in the client side so it is possible) and then let the server ping back to the client with the status of the query.
Writing this down so other people will have some possible patches. Will update if I'll find anything better.
I am using the request module to crawl a list of URLs and would like to
limit the number of open sockets to 2:
var req = request.defaults({
forever: true,
pool: {maxSockets: 1}
});
req(options, function(error, response, body) {
... code ...
done();
however, when looping over an array of URLs and issuing a new request to each - that does not seem to work.
is there a way to get the current number of open sockets to test it?
I believe that maxSockets maps to http.Agent.maxSockets, which limits the number of concurrent requests to the same origin (host:port).
This comment, from the developer of request, suggests the same:
actually, pooling controls the agent passed to core. each agent holds all hosts and throttles the maxSockets per host
In other words, you can't use it to limit the number of concurrent requests in general. For that, you need to use an external solution, for instance using limiter or async.queue.
I have a homework assignment to build an http server using only node native modules.
I am trying to protect the server from overloading, so each request is hashed and stores.
If a certain request reaches a high number, say 500, I call socket.destroy().
Every interval (one minute) I restart the hash-table. Problem is that when I do a socket that was previously dead is now working again. The only thing I do each interval is requests = {}, and nothing to do with the connections.
Any ideas why the connection is live again? Is there a better function to use than destroy()?
Thanks
Destroying the socket won't necessarily stop the client from retrying the request with a new socket.
You might instead try responding minimally with just a non-OK status code:
if (requests[path] >= 500) {
res.statusCode = 503;
res.end();
}
And, on the 503 status code:
The server is currently unable to handle the request due to a temporary overloading or maintenance of the server.
I'm writing proxy in Node.js + Express 2. Proxy should:
decrypt POST payload and issue HTTP request to server based on result;
encrypt reply from server and send it back to client.
Encryption-related part works fine. The problem I'm facing is timeouts. Proxy should process requests in less than 15 secs. And most of them are under 500ms, actually.
Problem appears when I increase number of parallel requests. Most requests are completed ok, but some are failed after 15 secs + couple of millis. ab -n5000 -c300 works fine, but with concurrency of 500 it fails for some requests with timeout.
I could only speculate, but it seems thant problem is an order of callbacks exectuion. Is it possible that requests that comes first are hanging until ETIMEDOUT because of node's focus in latest ones which are still being processed in time under 500ms.
P.S.: There is no problem with remote server. I'm using request for interactions with it.
upd
The way things works with some code:
function queryRemote(req, res) {
var options = {}; // built based on req object (URI, body, authorization, etc.)
request(options, function(err, httpResponse, body) {
return err ? send500(req, res)
: res.end(encrypt(body));
});
}
app.use(myBodyParser); // reads hex string in payload
// and calls next() on 'end' event
app.post('/', [checkHeaders, // check Content-Type and Authorization headers
authUser, // query DB and call next()
parseRequest], // decrypt payload, parse JSON, call next()
function(req, res) {
req.socket.setTimeout(TIMEOUT);
queryRemote(req, res);
});
My problem is following: when ab issuing, let's say, 20 POSTs to /, express route handler gets called like thousands of times. That's not always happening, sometimes 20 and only 20 requests are processed in timely fashion.
Of course, ab is not a problem. I'm 100% sure that only 20 requests sent by ab. But route handler gets called multiple times.
I can't find reasons for such behaviour, any advice?
Timeouts were caused by using http.globalAgent which by default can process up to 5 concurrent requests to one host:port (which isn't enough in my case).
Thouthands of requests (instead of tens) were sent by ab (Wireshark approved fact under OS X; I can not reproduce this under Ubuntu inside Parallels).
You can have a look at node-http-proxy module and how it handles the connections. Make sure you don't buffer any data and everything works by streaming. And you should try to see where is the time spent for those long requests. Try instrumenting parts of your code with conosle.time and console.timeEnd and see where is taking the most time. If the time is mostly spent in javascript you should try to profile it. Basically you can use v8 profiler, by adding --prof option to your node command. Which makes a v8.log and can be processed via a v8 tool found in node-source-dir/deps/v8/tools. It only works if you have installed d8 shell via scons(scons d8). You can have a look at this article to help you further to make this working.
You can also use node-webkit-agent which uses webkit developer tools to show the profiler result. You can also have a look at my fork with a bit of sugar.
If that didn't work, you can try profiling with dtrace(only works in illumos-based systems like SmartOS).