Difference between NodeJS new Agent() and HTTP Keep-Alive header..? - node.js

I'm using axios to make HTTP requests.
axios offers the option to specify new http.Agent() in the request config when used in Node.
The http Agent options are:
const agentOptions = {
keepAlive: true, // Keep sockets around even when there are no outstanding requests, so they can be used for future requests without having to reestablish a TCP connection. Defaults to false
keepAliveMsecs: 1000, // When using the keepAlive option, specifies the initial delay for TCP Keep-Alive packets. Ignored when the keepAlive option is false or undefined. Defaults to 1000.
maxSockets: Infinity, // Maximum number of sockets to allow per host. Defaults to Infinity.
maxFreeSockets: 256 // Maximum number of sockets to leave open in a free state. Only relevant if keepAlive is set to true. Defaults to 256.
}
There is also the HTTP header with timeout and max options.
The options don't seem to line up (keepAliveMsecs, maxSockets and maxFreeSockets vs timeout and max), which suggests they are different.
What's the difference between Node's new Agent({ keepAlive: true }) and the HTTP header Keep-Alive: timeout=5, max=1000..?

This is based on what I've gathered myself.
The HTTP header Keep-Alive: timeout=5, max=1000 is just a header sent with HTTP requests. See it as a way to communicate between two hosts (client and server). The host says 'hey keep the connection alive please'. This is automatic for modern browsers and servers might implement it or not. The keepAlive: true of the agent is as the documentation says
Not to be confused with the keep-alive value of the Connection header.
What that means is that keepAlive: false != Connection: close. It doesn't really have anything to do with the header. The agent will take care of things at the TCP level with sockets and such on the HTTP client.
keepAlive boolean Keep sockets around even when there are no outstanding requests, so they can be used for future requests without having to reestablish a TCP connection
As soon as you use an agent for your HTTP client, the Connection: Keep-Alive will be used. Unless keepAlive is set to false and maxSockets to Infinity.
const options = {
port: 3000,
agent: new http.Agent({
keepAlive: false ,
maxSockets: Infinity,
})
};//----> Connection: close
What exactly is an agent?
An Agent is responsible for managing connection persistence and reuse for HTTP clients. It maintains a queue of pending requests for a given host and port, reusing a single socket connection for each until the queue is empty, at which time the socket is either destroyed or put into a pool where it is kept to be used again for requests to the same host and port. Whether it is destroyed or pooled depends on the keepAlive option.
Pooled connections have TCP Keep-Alive enabled for them, but servers may still close idle connections, in which case they will be removed from the pool and a new connection will be made when a new HTTP request is made for that host and port. Servers may also refuse to allow multiple requests over the same connection, in which case the connection will have to be remade for every request and cannot be pooled. The Agent will still make the requests to that server, but each one will occur over a new connection.
Regarding timeout and max, as far as I know, these are set (automatically?) when adding config for Apache
#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On
#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100
#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5
which gives
Connection:Keep-Alive
Keep-Alive:timeout=5, max=100
But these are irrelevant for NodeJS? I'll let more experimented people answer this. Anyway, the agent won't set these and won't modify Connection: Keep-Alive unless setting keepAlive to false and maxSockets to Infinity as said above.
However, for the agent config to have any meaning, Connection must be set to Keep-Alive.
Okay, now for a little experiment to see the agent at work!
I've set up a client for testing (since axios use http.agent for the agent anyway, I just use http).
const http = require('http');
const options = {
port: 3000,
agent: new http.Agent({
keepAlive: true,
maxSockets: 2,
}),
// headers: {
// 'Connection': 'close'
// }
};
var i = 0;
function request() {
console.log(`${++i} - making a request`);
const req = http.request(options, (res) => {
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.setEncoding('utf8');
res.on('data', (chunk) => {
console.log(`BODY: ${chunk}`);
});
res.on('end', () => {
console.log('No more data in response.');
});
});
req.on('error', (e) => {
console.error(`problem with request: ${e.message}`);
});
req.end();
}
setInterval(function(){ request(); }, 3000); // send a request every 3 seconds
And the server is an express application (I'll skip the details)
server.on('connection', function(socket) {
socket.id = shortid.generate();
//socket.setTimeout(500)
console.log("A new connection was made by a client." + ` SOCKET ${ socket.id }`);
socket.on('end', function() {
console.log(`SOCKET ${ socket.id } END: other end of the socket sends a FIN packet`);
});
socket.on('timeout', function() {
console.log(`SOCKET ${ socket.id } TIMEOUT`);
});
socket.on('error', function(error) {
console.log(`SOCKET ${ socket.id } ERROR: ` + JSON.stringify(error));
});
socket.on('close', function(had_error) {
console.log(`SOCKET ${ socket.id } CLOSED. IT WAS ERROR: ` + had_error);
});
});
To make you see that keepAlive: false != Connection: close, let set keepAlive to false and see what happens server-side.
agent: new http.Agent({
keepAlive: false,
maxSockets: 20
})
Server
Client
As you can see, I've not set maxSockets to Infinity so even though keepAlive in the agent was set to false, the Connection header was set to Keep-Alive. However, each time a request was sent to the server, the socket on the server was immediately closed after each request. Let's see what happens when we set keepAlive to true.
Server
Client
This time around, only one socket have been used. There was a persistent connection between the client and the server that persisted beyond a single request.
One thing I've learned, thanks to this great article is that on Firefox, you can have as many as 6 concurrent persistent connections at a time. And you can reproduce this with the agent by setting maxSockets to 6. For testing purposes, I'll set this to 2. And also, I won't return anything from the server so the connection will be left hanging.
agent: new http.Agent({
keepAlive: true,
maxSockets: 2,
}),
//res.send('response from the server');
Server
Client
The client keeps sending requests but only two have been received by the server. Yet after two minutes, see http_server_timeout
The number of milliseconds of inactivity before a socket is presumed to have timed out.
two new requests are accepted. Actually, the client has queued the subsequent requests and once the server freed the sockets, the client was able to send two new requests from the queue.
So, I hope this helps.

Related

Websocket (socket.io) vs HTTP(S) time performance

We are trying to achieve a faster response time for a server call, and we opted to move the HTTP functionality to Websockets through socket.io.
Strangely enough, even though the general opinion is that Websockets should be faster especially since there's no handshake and any other connection setup functionality, our results don't show this, and actually HTTP edges out better since consecutive requests get back faster.
The backend is in Node.js with compression enabled (app.use(compression())), while the socket.io server is initalized with the deflate extension, as:
const io = new Server(server, {
serveClient: false,
pingInterval: 5000,
pingTimeout: 10000,
allowEIO3: true,
cors: {
origin: config.cors.origins,
credentials: true,
},
perMessageDeflate: {
threshold: 2048, // defaults to 1024
zlibDeflateOptions: {
chunkSize: 8 * 1024, // defaults to 16 * 1024
},
zlibInflateOptions: {
windowBits: 14, // defaults to 15
memLevel: 7, // defaults to 8
},
clientNoContextTakeover: true, // defaults to negotiated value.
serverNoContextTakeover: true, // defaults to negotiated value.
serverMaxWindowBits: 10, // defaults to negotiated value.
concurrencyLimit: 20, // defaults to 10
},
});
The socket listens like this:
io.use(async (socket, next) => {
try {
const userId = await socketAuthentication(socket.request);
socket.on('search', async ({ data }) => {
const response = await app.locals.searchController.search(data, userId);
socket.emit('search', { ...response });
});
next();
} catch (err) {
next(err);
}
});
While express listens for the respective HTTP request normally, through a router which then calls the same controller function (await app.locals.searchController.search(data, userId)), so no difference there.
The server is in AWS, on an EC2 instance behind nginx, and with SSL.
Below are the results for the same request, done sequentially 32 times:
Are we missing anything? I thought that the deflate configuration could have something to do, since there is a general opinion of not having it enabled (socket.io from version 3 and up disables it).

Node HTTP library ignoring timeout argument during TCP handshake?

I have following code. It sends simple GET request to the target server.
However when there's no webserver listening on given IP address, node keeps hanging for around 2 minute+ - totally disrespecting timeout value - which should be few seconds.
I've launched tcpdump and i noticed that node keeps sending SYN packets constantly to target server during these 2 minutes.
Why is this happening on http library? I tested alternative library (node-request) and timeout is working correctly there. However for some reasons, i can't use it and need to stick to lower level libs.
http = require('http');
const options = {
hostname: '52.178.167.109',
port: 80,
path: '/upload',
method: 'GET',
timeout: 5000, //5s timeout
};
const req = http.request(options, (res) => {
console.log(`STATUS: ${res.statusCode}`);
req.end();
});
req.on('error', (e) => {
console.error(`problem with request: ${e.message}`);
});
// Write data to request body
req.end();
root#abc:~# time node test.js
problem with request: connect ETIMEDOUT 52.178.167.109:80
real **2m15.474s** <-------
user 0m0.616s
sys 0m0.084s
The timeout property is only applicable to socket inactivity. You want to use request.setTimeout to set a read timeout. The github ticket https://github.com/nodejs/node/issues/12005 clarifies this.

Does keep alive only matter with outbound requests?

If I have a client that may be making a request to an http Google Cloud Function multiple times in a relatively short amount of time how can I use keep-alive? Is having the client send the connection keep-alive header enough?
I saw this on the Google docs:
https://cloud.google.com/functions/docs/bestpractices/networking
const http = require('http');
const agent = new http.Agent({keepAlive: true});
/**
* HTTP Cloud Function that caches an HTTP agent to pool HTTP connections.
*
* #param {Object} req Cloud Function request context.
* #param {Object} res Cloud Function response context.
*/
exports.connectionPooling = (req, res) => {
req = http.request(
{
host: '',
port: 80,
path: '',
method: 'GET',
agent: agent,
},
resInner => {
let rawData = '';
resInner.setEncoding('utf8');
resInner.on('data', chunk => {
rawData += chunk;
});
resInner.on('end', () => {
res.status(200).send(`Data: ${rawData}`);
});
}
);
req.on('error', e => {
res.status(500).send(`Error: ${e.message}`);
});
req.end();
};
But, that would only apply to making outbound requests from the cloud function right?
There was also something about the global (instance-wide) scope here:
https://cloud.google.com/functions/docs/bestpractices/tips
Is there anything I need to do to reuse connections on requests sent from the end user?
When you define agent at the global scope in your function, that is only retained for as long as any given server instance where that is running. So, while your connections may keep alive in that one instance, it will not share any connections with other instances that are allocated when the load on your function increases. You don't have much direct control over when Cloud Functions will spin up a new instance to handle new load, or when it will deallocate that instance. You just have to accept that they will come and go over time, along with any HTTP connections that are kept alive by global scope objects.

How can I stop axios from retrying the request after 2 mins

I have developed a nodejs server and react client application. The client requests an information from server using axios. This information has to be retrieved from a database and takes approx 5 mins. But the client retries the request exactly after 2 mins. How do I ask the client to wait for 5 mins.
client code
axios.get(apiServer + 'teradata/'+table).then( (resp) => {
console.log(resp.data.data)
)
In server, I tried below. But nothing works.
server.on('connection', function(socket) {
//Teradata takes too long to respond. Set timeoute to 3 mins
socket.setTimeout(0);
})
server.on('connection', function(socket) {
//Teradata takes too long to respond. Set timeoute to 3 mins
socket.setTimeout(60 * 6 * 1000);
// 30 second timeout. Change this as you see fit.
})
Update:
On further debugging, I see that
1. the retry originates from browser. because, retry happens in firefox, but not in chrome.
2. the connection disconnects exactly at 2 mins. This leads to firefox to retry.
I used the following code in express js and saw that it indeed disconnects. But I don't know who disconnects it. Is it axios or express js?
req.on("close", function() {
console.log('request closed unexpectedly')
});
req.on("end", function() {
console.log('equest ended normally')
});
According to documentation you can create an instance of axios and configure timeout too.
const instance = axios.create({
baseURL: 'https://some-domain.com/api/',
timeout: 5000,
headers: {'X-Custom-Header': 'foobar'}
});

NodeJS: both request close and end events not getting fired on browser tab/window close

I am implementing a real time commenting tool using Redis PubSub and NodeJS. On the server, I watch for req close or end event to ensure I close/end the corresponding subscribers. On my OSX dev machine, I am able to detect the close event but on AWS elasticbeanstalk setup, the close event just doesn't get fired. I have tried listening to the other events as well(both for req and res) but those don't seem to be firing either!.
Both my dev and aws setups use Node v0.12.2, Express v4.5.1 nginx 1.6.2 on aws. The aws setup runs behind a load balancer.
Has anyone else ever faced anything similar to this issue?
var publisherClient = redis.createClient();
publisherClient.on("error",function(err){
console.log(err);
})
setInterval(function(){
publisherClient.publish("Ping", '{"Pong":'+"}");
},20000)
exports.amastream=function (req,res){
req.socket.setTimeout(40*1000);
var messageCount = 0;
var subscriberClient = redis.createClient();
subscriberClient.subscribe("Ping");
subscriberClient.on("error", function(err) {
console.log("Redis Error: " + err);
});
subscriberClient.on("message", function(channel, message) {
messageCount++;
res.write('id: ' + messageCount + '\n');
res.write("data: " +channel + '\n');
res.write("data: " +message + '\n\n');
res.flush();
});
req.on("end", function() {
console.log("closing connection due to end of request!");
subscriberClient.punsubscribe();
subscriberClient.quit();
});
req.on("close", function() {
console.log("closing connection!");
subscriberClient.punsubscribe();
subscriberClient.quit();
});
req.on("error", function() {
console.log("closing connection due to abrupt ending!");
subscriberClient.punsubscribe();
subscriberClient.quit();
});
req.on("timeout",function() {
console.log("closing connection due to timeout!");
subscriberClient.punsubscribe();
subscriberClient.quit();
})
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache',
'You-Are-Like-Us':'Contact us at tarkeshwar#fundamine.com'
});
res.write('id: ' + 0 + '\n');
res.write("data: " +"first message" + '\n\n');
res.flush();
}
I have the same issue. This behaviour is caused by the Elastic Load Balancing.
We should take a look to the Elastic Load Balancing Connection Timeout Management for better unsderstanind on how ELB works.
So, it has the client and the backend connection and idle timeout value.
For some reason ELB can not properly detect when your client connection is closed and close the associated backend connection. Acctually your backend connection stay opened. Thats why you don`t receive the close event.
Idle timeout at the backend side will not work since you are sending ping to client. Howhever, I wonder, where does the ELB send this data if the client connection does not exist?
There is a post on the AWS support forum about this issue. The topic starter tells that it takes approximately 55 mintues to close the connection to backend. From my investigation it could take much longer.
We also use the Server-Sent Events in our application and we discovered that it leads to the enourmous number of idle connections, which is holded by the ELB after the user closed or reloaded the page.
You could monitor the number of connections using the following code (I didn`t find a metric at AWS wich shows the number of active 'backend' connection to a particular server from ELB):
var server = require('http').createServer(app);
setInterval(function (){
server.getConnections(function (err, count){
console.log("Number of connections : " + count);
});
}, 60000);
Unfortunately we cant move to the web sockets because EBS does not support it without additional workarounds or using the HAProxy server.
As a workaround it is possible setup a timer wich will perform the req.end each N minutes. In this case client code should detect that server connection is closed and establish a new one.

Resources