Node server, socket, request and response timeouts - node.js

Problem
Node's default configuration timeouts requests after 2 minutes. I would like to change the request timeouts to:
1 minute for 'normal' requests
5 minutes for requests that serve static files (big assets in this case)
8 hours for uploads (couple of thousand pictures per request)
Research
Reading through Node's documentation, I've discovered that there are numerous ways of defining timeouts.
server.setTimeout
socket.setTimeout
request.setTimeout
response.setTimeout
I'm using Express which also provides middleware to define timeout's for (specific) routes. I've tried that, without success.
Question
I'm confused about how to properly configure the timeout limit globally and per route. Should I configure all of the above timeouts? How is setting the server's timeout different to setting the socket's or request's timeout?

As I saw on your other question concerning the usage of the timeout middleware, you are using it somehow differently.
See documentation of timeout-connect middleware.
Add your errorHandler-function as an EventListener to the request, as it is an EventEmitter and the middleware causes it to emit the timeout-event:
req.on("timeout", function (evt) {
if (req.timedout) {
if (!res.headersSent) {
res
.status(408)
.send({
success: true,
message: 'Timeout error'
});
}
}
});
This is called outside of the middleware stack, causing the function call to next(err) to be invalid. Also, you have to keep in mind, that if the timeout happens while the request is hanging server-side, you have to prevent your server code from further processing this request (because headers are already sent and its underlying connection will no longer be available).

Summary
nodejs timeout API are all inactivity timeout
expressjs/timeout package is response hard timeout
nodejs timeout API
server.timeout
inactivity/idle timeout
equal to socket timeout
default 2min
server.setTimeout
inactivity/idle timeout
equal to socket timeout
default 2min
have callback
socket.setTimeout
inactivity/idle timeout
callback responsible to end(), destroy() socket
default no timeout
response.setTimeout
socket.setTimeout front end
request.setTimeout
socket.setTimeout front end
expressjs/timeout package
response hard-timeout (vs inactivity)
have callback
Conclusion
max. time allowed for an action(request+response), express/timeout package is needed.
This is properly what you need, but the callback need to end the request/response. As the timeout only trigger the callback, it does not change the state or interfere with the connection. It is the callback job.
idle timeout, set nodejs api request/response timeout
I don't recommend touching these, as it is not necessary in most cases. Unless you want to allow a connection to idle(no traffic) over 2min.

There is already a Connect Middleware for Timeout support. You can try this middleware.
var timeout = express.timeout // express v3 and below
var timeout = require('connect-timeout'); //express v4
app.use(timeout(120000)); // should be changed with your desire time
app.use(haltOnTimedout);
function haltOnTimedout(req, res, next){
if (!req.timedout) next();
}

Related

AWS http api gateway + lambda (node/express) Internal Server Error

I get internal server error when I have a long running query.
Actually, I have to fetch historic data through an API, which sometime can take longer than 30 seconds. It depends on the query how complex it is. It can take 1 min also.
Not sure but guessing, API gateway timeout is set to 30 seconds (and I cann't increase it) and my query execution time is more then 30 seconds. So I get internal server error I believe.
HOW can I say above statement ?
because If I run the same query locally, I mean in node/express locally by running npm run start, it works fine even if takes1 mins, response will always come back.
But when I deploy node/express code to lambda function, it throws error if any query takes longer period to execute.
I have following setup of node/express
const express = require("express");
const serverless = require("serverless-http");
const app = express();
app.use(cors());
app.use((req, res, next) => {
res.setHeader('Connection', 'keep-alive'); // I added this line as suggested in some post but not helping
res.setHeader('Keep-Alive', 'timeout=30'); // I added this line as suggested in some post but not helping
res.setHeader("Access-Control-Allow-Headers", "X-Requested-With,content-type");
res.setHeader("Access-Control-Allow-Origin", "*");
res.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS, PUT, PATCH, DELETE");
res.setHeader("Access-Control-Allow-Credentials", true);
next();
});
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(`api-end-point/user`, userRoute);
....
if (process.env.NODE_ENV !== "lambda") {
PORT = process.env.PORT || 7000;
const server = app.listen(PORT, () => {
console.log(`node-express server running in ${process.env.NODE_ENV} mode on ${PORT}`);
});
server.timeout = 0;
}else {
module.exports.handler = serverless(app); // this is for lambda function
}
I deploy this code to AWS lambda function.
HTTP API gateway is configured with two routes /ANY, /{proxy+}
TIMEOUT
API gateway is set to default 30 seconds. [I can not increase this time as not allowed by AWS]
Lambda is set to 10 **mins**
CORS
I really have no idea how can I fix this problem ?
How can I increase API gateway timeout or How can I keep connection alive ?
You cannot increase the API Gateway timeout to greater than 30 seconds, as has already been mentioned.
The only solution I know of at this time is to run your Lambda asynchronously, but this cannot be done in an Http API. But if you're willing to change it to a REST API, then this can be done with a combination of turning on Lambda Proxy Integration in the REST API and invoking the Lambda asynchronously utilizing an invoke header X-Amz-Invocation-Type. This will allow your Lambda to run asynchronously (up to 15 minutes) with an API call.
https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html
Since timeout cannot be incresed, you might change using single HTTP request to
https://en.wikipedia.org/wiki/Post/Redirect/Get Pattern
Client POST query
Server response an url for the result
Client GET the url multiple times -- it will be 200 OK when the result ready
or WebSocket
The document says Idle Connection Timeout for WebSocket is up to 10 minutes
Using Lambda means subscribing to patterns from the Serverless catalog/philosophy. Which means using async whenever possible.
As far as I understand your Lambda needs receives a request, does another call to something (not specified) which takes 30~60s.
The API Gateway has a hardcoded timeout of 29s (hard limit).
To solve this problem the application would need to be re-architectured:
Trigger the Lambda asynchronously using X-Amz-Invocation-Type Event from the Frontend.
The Lambda calls the history API and stores the result in some storage (DynamoDB, S3, RDS, ...).
The frontend queries the backend from the frontend until the data is available (or use WebSockets)
This way the historic API call can take up to 15m and the calls can be cached in the storage to speed up further calls. If it needs more than 15m then I would ask the historic API to re-architecture.

Setting long timeout for http request via nodejs angular4 or express

I currently have a request which is made from an angular 4 app(which uses electron[which uses chromium]) to a bottleneck(nodejs/express) server. The server takes about 10 minutes to process the request.
The default timeout which I'm getting is 120 seconds.
I tried to use setting the timeout on the server using
App.use(timeout("1000s")
In the client side I have used
options = {
url,
method: GET
timeout : 600 * 1000}
let req = http.request(options, () => {})
req.end()
I have also tried to give the specific route timeout.
Each time the request hits 120 seconds the socket dies and I get a "socket timeout"
I have read many posts with the same questions but I didn't get any concrete answers. Is it possible to do a request with a long/no timeout using the tools above? Do I need to download a new library which handles long timeouts?
Any help would be greatly appriciated.
So after browsing through the internet I have discovered that there is no possible way to increase Chrome's timeout time.
My solution to this problem was to open the request and return a default answer(something like "started") then pinging the server to find out it's status.
There is another possible solution which will be to put a route in the client(I'm using electron and node modules in the client side so it is possible) and then let the server ping back to the client with the status of the query.
Writing this down so other people will have some possible patches. Will update if I'll find anything better.

what happens if neither res.send() nor res.end() is called in express.js?

I have a security issue that someone is trying to call random APIs that are not supported on our server but are frequently used for administrators API in general. and I set this code below to handle 404 to not respond to this attack
url-not-found-handler.js
'use strict';
module.exports = function () {
//4XX - URLs not found
return ((req, res, next) => {
});
};
what happens to client is that it waits until the server responds but I want to know if this will affect the performance of my express.js server also what happens behind the scene in the server without res.send() or res.end() ?
According to the documentation of res.end().
Ends the response process. This method actually comes from Node core,
specifically the response.end() method of http.ServerResponse.
And then response.end
This method signals to the server that all of the response headers and
body have been sent; that server should consider this message
complete. The method, response.end(), MUST be called on each response.
If you leave your request hanging, the httpserver will surely keep data about it. Which means that if you let hang many requests, your memory will grow and reduce your server performance.
About the client, he's going to have to wait until he got a request timeout.
The best to do having a bad request is to immediately reject the request, which is freeing the memory allowed for the request.
You cannot prevent bad requests (maybe have a firewall blocking requests from certains IP address?). Best you can do is to handle them as fast as possible.

Request ending early on ExpressJS

router.route('...')
.get(function(req, res, next) {
// This job() function doing some *long* async task and end of the task calling 3th param as callback
job(param1, param2, function(response) {
// printing response to console perfectly
console.log("callback fired", response);
res.send("response");
});
});
And I'm making my request with curl.
$ curl ... -m 300
curl: (52) Empty reply from server
cURL is waiting for a response for a few minutes and then I'm getting empty reply error. cURL is giving this error before nodejs printing callback fired message.
Same error if I make this request with a browser or with Postman.
I'm sure there are no res.send(), res.end() functions inside job() async function. I stuck, how can I track and found the error?
There are two possible timeouts that could be affecting you. Without seeing the actual network trace (to see what happens when the request ends), I can't tell which timeout might be causing your issue. But, you can just address both of them and it should handle your issue.
First, curl has a timeout value. It is unclear what its default setting is, but you can set whatever value you want with:
curl --max-time 900
where the value is in seconds.
Second, the nodejs http server has a timeout where if no response is sent to an open request, it will close the socket (this keeps dead sockets from building up over time). You can see the doc for server.setTimeout() here. The default for the http server object is 2 minutes (I don't know if Express changes that at all).
The general idea is this:
server.setTimeout(10 * 60 * 1000); // set response timeout to 10 minutes
where the server object would be your http server object (not the Express app object).

Is there a default timeout in Node.js for http.request?

In Node.js there is a default timeout for a server (for an incoming HTTP request) at 120000ms (2 minutes) (see HTTP's server.timeout documentation).
But if I want to do an HTTP request in Node.js (using http.request), looking at the documentation, I only find a function request.setTimeout() to set the timeout manually.
Anyone know if there is a default timeout for HTTP requests in Node.js? Or does Node.js try to send the HTTP request with no end?
You want to set the server.timeout property (it defaults to 120,000, as you've found).
Update: Node.js 13 has removed the default timeout:
https://nodejs.org/api/http.html#servertimeout
https://github.com/nodejs/node/pull/27558
I was also interested in this. By reading the code, Node.js uses Socket under the hood of http request (naturally). (The source link below is referring v8.8.0 at the point when I'm writing this)
https://github.com/nodejs/node/blob/master/lib/_http_outgoing.js
And Socket does not have the timeout by default by this document
https://nodejs.org/dist/latest-v6.x/docs/api/net.html#net_socket_settimeout_timeout_callback
And the source tells the same.
https://github.com/nodejs/node/blob/master/lib/net.js
No. There is no default timeout.
Node.js uses Socket under the hood of http request and socket does not have the timeout by default.
Use the {timeout: XX} parameter of http.request to configure a proper request timeout.

Resources