There is a node server hosted and from application we also make a request to some external api over http. This external service can process 10request/sec. Application is behind Nginx which has timeout 30 secs.
Now let say we put load of 10k request on nodejs app server. Since we do have dependency on external api which can process max 10*30 request in 30 secs. Only 300 request will be served and remaining will be terminated by Nginx. But still this 10k request got queued into https.globalAgent.requests queue and keep running. There is no way to specify sockettimeout or limit the size of the request queue. Further call to application will eventually be queued up for external service and later will be terminated by Nginx.
So questions are :
Is there any way we can set socketTimeOut?
Is there any way we can limit the size of the queue?.
Any workaround ?
Sample Code
var http = require('http');
var https = require('https');
var Q = require('q');
var file = require('fs');
var request = require('request');
http.globalAgent.maxSockets = 10;
https.globalAgent.maxSockets = 10;
https.globalAgent.keepAlive=true;
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
var st = new Date();
var rr = request(
{url:'https://amazon.com',timeout:100000,time:true},
function(err,resp,body){
var et = new Date();
// console.log( resp && resp.timings.socket,st-et,err);
console.log(https.globalAgent)
res.end('ok');
}
);
}).listen(9898);
I'll suggest you to use a rate limiting library like https://github.com/jhurliman/node-rate-limiter
In case of an exceded limit, you can answer immediately to the requests, with a 429 error code (https://httpstatuses.com/429) or a ko error message
or you can wait until you have an available request slot
Related
So i came across a problem.I am trying to send {id} to my rest API (node js) and in response, I get data on the socket.
Problem:
For first 5-6 time it works perfectly fine and display Id and send data back to socket.But after 6 time it does not get ID.
I tried this https://github.com/socketio/socket.io/issues/1145
and https://github.com/socketio/socket.io/issues/1145 but didn't solve the problem.
On re compiling the server it shows previous {ids} which i enter after 6 time.it like after 5-6 time it is storing id in some form of cache.
Here is my API route.
//this route only get {id} 5-6 times .After 5-6 times it does not display receing {id}.
const express = require("express");
var closeFlag = false;
const PORT = process.env.SERVER_PORT; //|| 3000;
const app = express();
var count = 1;
http = require('http');
http.globalAgent.maxSockets = 100;
http.Agent.maxSockets = 100;
const serverTCP = http.createServer(app)
// const tcpsock = require("socket.io")(serverTCP)
const tcpsock = require('socket.io')(serverTCP, {
cors: {
origin: '*',
}
, perMessageDeflate: false
});
app.post("/getchanneldata", (req, res) => {
console.log("count : "+count)
count++;// for debugging purpose
closeFlag = false;
var message = (req.body.val).toString()
console.log("message : "+message);
chanId = message;
client = dgram.createSocket({ type: 'udp4', reuseAddr: true });
client.on('listening', () => {
const address = client.address();
});
client.on('message', function (message1, remote) {
var arr = message1.toString().split(',');
}
});
client.send(message, 0, message.length, UDP_PORT, UDP_HOST, function (err, bytes) {
if (err) throw err;
console.log(message);
console.log('UDP client message sent to ' + UDP_HOST + ':' + UDP_PORT);
// message="";
});
client.on('disconnect', (msg) => {
client.Diconnected()
client.log(client.client)
})
}
);
There are multiple issues here.
In your app.post() handler, you don't send any response to the incoming http request. That means that when the browser (or any client) sends a POST to your server, the client sits there waiting for a response, but that response never comes.
Meanwhile, the browser has a limit for how many requests it will send simultaneously to the same host (I think Chrome's limit is coincidentally 6). Once you hit that limit, the browser queues the request and and waits for one of the previous connections to return its response before sending another one. Eventually (after a long time), those connections will time out, but that takes awhile.
So, the first thing to fix is to send a response in your app.post() handler. Even if you just do res.send("ok");. That will allow the 7th and 8th and so on requests to be immediately sent to your server. Every incoming http request should have a response sent back to it, even if you have nothing to send, just do a res.end(). Otherwise, the http connection is left hanging, consuming resources and waiting to eventually time out.
On a separate note, your app.post() handler contains this:
client = dgram.createSocket({ type: 'udp4', reuseAddr: true });
This has a couple issues. First, you never declare the variable client so it becomes an implicit global (which is really bad in a server). That means successive calls to the app.post() handler will overwrite that variable.
Second, it is not clear from the included code when, if ever, you close that udp4 socket. It does not appear that the server itself ever closes it.
Third, you're recreating the same UDP socket on every single POST to /getchanneldata. Is that really the right design? If your server receives 20 of these requests, it will open up 20 separate UDP connections.
My Perl script talks to a Node.js server and sends the commands that the Node.js server needs to execute. While some commands take less time, others take a lot longer. While the command is executing on the server, there is silence on the connection. After a while, I receive the error: 500: Server closed connection without sending any data back
During this error, the command is still executing on the server and the desired results are obtained (if you check the server logs). My problem is that I don't want the connection to reset as there are other follow on commands that need to run after these long running commands. Some commands might take 20 mins
Perl Side Code:
my $ua = LWP::UserAgent->new;
$ua->timeout(12000);
my $uri = URI->new('http://server');
my $json = JSON->new;
my $data_to_json = {DATA};
my $json_content = $json->encode($data_to_json);
# set custom HTTP request header fields
my $req = HTTP::Request->new(POST => $uri);
my $resp = $ua->request($req);
my $message = $resp->decoded_content;
NodeJS Code
var express = require('express');
var http = require('http');
var app = express();
app.use(express.json());
var port = process.env.PORT || 8080;
app.get('<API URL>', function (req, res) {
<get all the passed arguments>
<send output to the console>
});
app.post('<API URL>', function(req, res) {
.
.
.
req.connection.setTimeout(0);
var exec = require('child_process').exec;
var child = exec(command);
}
// start the server
const server = app.listen(port);
server.timeout = 12000;
console.log('Server started! Listening on port' + port);
I have tried to add the timeout for the server using server.timeout and req.connection.setTimeout(0);.
How do I make sure that the connection is not broken?
It's generally a bad idea to make a web worker carry out long running tasks. It ties up the worker and you end up having problems like this.
Instead, have the web worker add a job to some sort of job queue (Perl's Minion is really nice). The queue operates independently of the web server. Clients can poll the server to check on the status of a job and get the output or artifacts when it's complete.
Another advantage of a proper job queue is that you can restart jobs if they fail. The queue knows the job was there. As you've seen, a broken web connection means that it fails and you've probably lost track of those inputs.
We have a node application running behind Amazon Elastic Load Balancer (ELB), which randomly throws 502 errors when there are multiple concurrent requests and when each request takes time to process. Initially, we tried to increase the idle timeout of ELB to 5 minutes, but still we were getting 502 responses.
When we contacted amazon team, they said this was happening because the back-end is closing the connection with ELB after 5s.
ELB will send HTTP-502 back to your clients for the following reasons:
The load balancer received a TCP RST from the target when attempting to establish a connection.
The target closed the connection with a TCP RST or a TCP FIN while the load balancer had an outstanding request to the target.
The target response is malformed or contains HTTP headers that are not valid.
A new target group was used but no targets have passed an initial health check yet. A target must pass one health check to be considered healthy.
We tried to set our application's keep-alive/timeouts greater than ELB idle timeout (5 min), so the ELB can be responsible for opening and closing the connections. But still, we are facing 502 errors.
js:
var http = require( 'http' );
var express = require( 'express' );
var url = require('url');
var timeout = require('connect-timeout')
const app = express();
app.get( '/health', (req, res, next) => {
res.send( "healthy" );
});
app.get( '/api/test', (req, res, next) => {
var query = url.parse( req.url, true ).query;
var wait = query.wait ? parseInt(query.wait) : 1;
setTimeout(function() {
res.send( "Hello!" );
}, wait );
});
var server = http.createServer(app);
server.setTimeout(10*60*1000); // 10 * 60 seconds * 1000 msecs
server.listen(80, function () {
console.log('**** STARTING SERVER ****');
});
Try setting server.keepAliveTimeout to something other than the default 5s. See: https://nodejs.org/api/http.html#http_server_keepalivetimeout. Per AWS docs, you'd want this to be greater than the load balancer's idle timeout.
Note: this was added in Node v8.0.0
Also, if you're still on the Classic ELB, consider moving to the new Application Load Balancer as based on current experience this seems to have improved things for us a lot. You'll also save a few bucks if you have a lot of separate ELBs for each service. The downside could be that there's 1 point of failure for all your services. But in AWS we trust :)
In my case, I needed upgrade nodejs version:
https://github.com/nodejs/node/issues/27363
After that the problem was fixed.
Change your server.listen() to this:
const port = process.env.PORT || 3000;
const server = app.listen(port, function() {
console.log("Server running at http://127.0.0.1:" + port + "/");
});
You can read more about this here.
on a basic node http server like this:
var http = require('http');
var server = http.createServer(function(req, res){
console.log("number of concurr. connections: " + this.getConnections());
//var pendingConnections = ???
});
server.maxConnections = 500;
server.listen(4000);
if i send 500 requests at one time to the server, the number of concurr. connections is arround 350. with the hard limit set to 500 (net.server.backlog too), i want to know, how to access the number of pending connections (max. 150 in this example) when ever a new request starts.
so i think i have to access the underlying socket listening on port 4000 to get this info, but until now i was not able to get it.
EDIT
looking at node-http there is an event called connection, so i think the roundtrip of an request is as follows:
client connects to server socket --> 3-way-handshake, socket lasts in state CONNECTED (or ESTABLISHED ?!) then in node event connection is emitted.
the node http server accepts this pending connection an starts processing the request by emitting request
so the number of connections has to be at least as big as the number of requests, but with following example i could not confirm this:
var http = require('http');
var activeRequets = 0;
var activeConnections = 0;
var server = http.createServer(function(req, res){
activeRequests++;
res.send("foo");
});
server.on('connection', function (socket) {
socket.setKeepAlive(false);
activeConnections++;
});
setInterval(function(){
console.log("activeConns: " + activeConnections + " activeRequests: " + activeRequests);
activeRequests = 0;
activeConnections = 0;
}, 500);
server.maxConnections = 1024;
server.listen(4000, '127.0.0.1');
even if i stress the server with 1000 concurr connections and adding one delay in the response, activeRequests is mostly as high as activeConnections. even worse, the activeRequests are often higher then activeconnections, how could this be?
IIRC You can just count how many connections that have a status of SYN_RECV for that particular IP and port that you're listening on. Whether you use a child process to execute netstat and grep (or similar utilities) for that information, or write a binding to get this information using the *nix C API, is up to you.
I am trying to use node.js to setup a simple proxy server. The idea behind that is to get all web services calls made to one web service go through a node.js proxy in order to easily inspect and debug web service calls.
In order to do that, I am trying to use the following code to proxy the requests:
var
url = require('url'),
http = require('http'),
acceptor = http.createServer().listen(8008);
acceptor.on('request', function(request, response) {
console.log('request ' + request.url);
request.pause();
var options = url.parse(request.url);
options.headers = request.headers;
options.method = request.method;
options.agent = false;
var connector = http.request(options, function(serverResponse) {
serverResponse.pause();
response.writeHeader(serverResponse.statusCode, serverResponse.headers);
serverResponse.pipe(response);
serverResponse.resume();
});
request.pipe(connector);
request.resume();
});
But I can't figure out where to inspect / dump to file the response. With node-inspector, I was looking at the response object at line: serverResponse.pipe(response); but the body of the response is not yet available.
I found the following question node.js proxied request body but it is written in CoffeeScript.
The idea is write your own 'data' handler and don't use pipe().
You cannot eavesdrop on the data once you piped the stream.