I am trying to limit the amount of async calls I can make to the server to 1 call per second. I looked at async eachOfLimit, but not sure if this is the right solution.
I'm looking for a library that already exists, that would wrap around the request module and actually throttle requests to 1 per second. I just don't know how to handle this kind of situation, and I'm hoping for some kind of standard.
If you're using express you can use express-rate-limit package from npm. Usage below -
const rateLimit = require("express-rate-limit");
app.enable("trust proxy"); // only if you're behind a reverse proxy (Heroku, Bluemix, AWS ELB, Nginx, etc)
const limiter = rateLimit({
windowMs: 60 * 1000, // 1 minute
max: 1 // limit each IP to 1 requests per windowMs
});
// apply to all requests
app.use(limiter);
Related
I get internal server error when I have a long running query.
Actually, I have to fetch historic data through an API, which sometime can take longer than 30 seconds. It depends on the query how complex it is. It can take 1 min also.
Not sure but guessing, API gateway timeout is set to 30 seconds (and I cann't increase it) and my query execution time is more then 30 seconds. So I get internal server error I believe.
HOW can I say above statement ?
because If I run the same query locally, I mean in node/express locally by running npm run start, it works fine even if takes1 mins, response will always come back.
But when I deploy node/express code to lambda function, it throws error if any query takes longer period to execute.
I have following setup of node/express
const express = require("express");
const serverless = require("serverless-http");
const app = express();
app.use(cors());
app.use((req, res, next) => {
res.setHeader('Connection', 'keep-alive'); // I added this line as suggested in some post but not helping
res.setHeader('Keep-Alive', 'timeout=30'); // I added this line as suggested in some post but not helping
res.setHeader("Access-Control-Allow-Headers", "X-Requested-With,content-type");
res.setHeader("Access-Control-Allow-Origin", "*");
res.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS, PUT, PATCH, DELETE");
res.setHeader("Access-Control-Allow-Credentials", true);
next();
});
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(`api-end-point/user`, userRoute);
....
if (process.env.NODE_ENV !== "lambda") {
PORT = process.env.PORT || 7000;
const server = app.listen(PORT, () => {
console.log(`node-express server running in ${process.env.NODE_ENV} mode on ${PORT}`);
});
server.timeout = 0;
}else {
module.exports.handler = serverless(app); // this is for lambda function
}
I deploy this code to AWS lambda function.
HTTP API gateway is configured with two routes /ANY, /{proxy+}
TIMEOUT
API gateway is set to default 30 seconds. [I can not increase this time as not allowed by AWS]
Lambda is set to 10 **mins**
CORS
I really have no idea how can I fix this problem ?
How can I increase API gateway timeout or How can I keep connection alive ?
You cannot increase the API Gateway timeout to greater than 30 seconds, as has already been mentioned.
The only solution I know of at this time is to run your Lambda asynchronously, but this cannot be done in an Http API. But if you're willing to change it to a REST API, then this can be done with a combination of turning on Lambda Proxy Integration in the REST API and invoking the Lambda asynchronously utilizing an invoke header X-Amz-Invocation-Type. This will allow your Lambda to run asynchronously (up to 15 minutes) with an API call.
https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html
Since timeout cannot be incresed, you might change using single HTTP request to
https://en.wikipedia.org/wiki/Post/Redirect/Get Pattern
Client POST query
Server response an url for the result
Client GET the url multiple times -- it will be 200 OK when the result ready
or WebSocket
The document says Idle Connection Timeout for WebSocket is up to 10 minutes
Using Lambda means subscribing to patterns from the Serverless catalog/philosophy. Which means using async whenever possible.
As far as I understand your Lambda needs receives a request, does another call to something (not specified) which takes 30~60s.
The API Gateway has a hardcoded timeout of 29s (hard limit).
To solve this problem the application would need to be re-architectured:
Trigger the Lambda asynchronously using X-Amz-Invocation-Type Event from the Frontend.
The Lambda calls the history API and stores the result in some storage (DynamoDB, S3, RDS, ...).
The frontend queries the backend from the frontend until the data is available (or use WebSockets)
This way the historic API call can take up to 15m and the calls can be cached in the storage to speed up further calls. If it needs more than 15m then I would ask the historic API to re-architecture.
I set up a local express server with:
const express = require('express');
const app = express();
app.get('/test', (request, response) => {
response.sendStatus(200);
});
const port = 3000;
app.listen(port, () => {});
Then I ran a script with:
const axios = require('axios');
async function main() {
console.time('time');
const requests = Array(5000).fill().map(() => axios.get('http://localhost:3000/test'));
await Promise.all(requests);
console.timeEnd('time');
}
main();
And my question is why this script takes 3 seconds on my machine?
I'd expect it to take a few milliseconds just like with any other for loop of 5000 iterations.
Because I'm running the server locally and calling it via localhost, I expect no latency, therefore, the waiting time for the promises should be almost 0.
Can anyone explain to me what's going on?
Also, how can I do many requests at the same time faster?
EDIT
Looking here https://stressgrid.com/blog/webserver_benchmark/ I'd expect my single process node server to be able to handle at least 20k requests concurrently without any delay.
So I'm guessing there is some configuration missing on my machine. Maybe some flag when starting the node server?
3 things:
That benchmark is not properly setup.
Express is the slowest of all NodeJS web frameworks.
Your machine might be misconfigured.
You can find better benchmarks and a comparison of different frameworks here: https://www.fastify.io/benchmarks/
Their github repo explains all the setup they've done, so you can compare your machine against theirs too.
1. Benchmarking
To put it plainly, the benchmark you set up is not valid. It doesn't reproduce any real world scenario, and is not optimized for the synthetic scenario it creates.
Just to exemplify, since on Node everything is single threaded, you'd have better performance running requests serially so that connections can be reused (would also need to change your request framework to one that can reuse connections). HTTP 1 doesn't reuse connections if you issue requests in parallel, AND your client isn't setup to reuse connections anyways.
Let's take a look at what results look like after fixing that. On my machine, the benchmark you posted doesn't even run--node crashes if you try to open that many connections simultaneously on the same port. This version has about the same theoretical performance as your benchmark, and it runs:
const axios = require("axios");
async function main() {
console.info(process.hrtime.bigint() / 1000000n + "ms");
for (let i = 0; i < 5000; ++i) {
await axios.get("http://localhost:3000/test");
}
console.info(process.hrtime.bigint() / 1000000n + "ms");
}
main();
That takes around 3 seconds on my machine (about the same time as yours). Now let's reuse connections:
const axios = require("axios");
const http = require("http");
async function main() {
const httpAgent = new http.Agent({ keepAlive: true });
console.info(process.hrtime.bigint() / 1000000n + "ms");
for (let i = 0; i < 5000; ++i) {
await axios.get("http://localhost:3000/test", { httpAgent });
}
console.info(process.hrtime.bigint() / 1000000n + "ms");
}
main();
This takes 800ms.
There's a lot of other details like this that your benchmark misses. I can't summarize all of them. You can compare your benchmark to Fastify's (linked above) to see how each difference impacts your measurement.
2. Frameworks
Express has its popularity for being simple, but it is not a fast framework. Take a look at more modern ones such as Koa or Fastify. Note that your app likely will do much more than just serve an empty page, so performance of your web framework is likely not important. That said, I don't think anyone should be using express in 2021 if they have a choice anyways, since their developer experience is also outdated (eg there's no support for awaiting a request within a middleware).
3. Local Machine
It could also just be that your computer is slow, etc. That's another reason to start by rerunning a standardized benchmark instead of creating your own.
Define slow to begin with. You have Array(5000).fill() which we can interpreted as reserved me 5000 slots in memory for me in other word you do a for loop of 5000 then you do 5000 request so that means 10,000 looping. Do the same 10,000 looping on java and compare then tell me if JavaScript is slow.
Also I don’t know if you have, but axios has quite a few internal validations
I'm using express-rate-limit npm package, I deployed my backend on AWS (t2 micro ec2 instance), while limiter is on, requests are blocked from ALL users who try to interact with my API, it works for a couple of minutes and stops for about 10 minutes.
when I comment out the limiter part everything is working fine,I think too many requests should be blocked for only one user who tries to hammer the server with requests but what happens is ALL users get blocked, all users are treated like only 1 user, that's my conclusion.
If that's the case what should I do? I need my rate limiter on, and if there is any other explanation what would it be?
By default, express-rate-limit has a keyGenerator of req.ip. When I log this on my server it is '::ffff:127.0.0.1' which is obviously going to be the same for every request, thus limiting for all IP addresses once it's limited for one.
My solution was to use request-ip to get the correct IP address like so:
const rateLimit = require('express-rate-limit');
const requestIp = require('request-ip');
const app = express();
app.use(requestIp.mw());
app.use(rateLimit({
windowMs: 60 * 1000, // 1 minute
max: 30, // limit each IP to 30 requests per windowMs
keyGenerator: (req, res) => {
return req.clientIp // IP address from requestIp.mw(), as opposed to req.ip
}
}));
keyGenerator: function (req: any) {
return req.headers["x-forwarded-for"] || req.connection.remoteAddress;
}
It blocks based on iP
The express-rate-limit package blocks requests based on IP Address and that's because it provides a very basic configuration for rate-limiting that would be suitable for most applications. If you block based on user, someone can easily configure a bot to hit your APIs until the limit is reached on one user account and make a new account automatically to start hitting your server again. Blocking based on IP avoids such risks as one IP means one Device no matter how many users request from that IP. In most cases, one device is most likely to be used by one person so this solution works pretty well.
I am trying to solve the issue of Slow Post Vulnerability on my application.
Issue: https://blog.qualys.com/securitylabs/2011/07/07/identifying-slow-http-attack-vulnerabilities-on-web-applications
To limit the number of connections from a user, I have used express-rate-limit so that the application does not go unavailable.
const rateLimit = require('express-rate-limit')
const limiter = rateLimit({ windowMs: 60 * 1000, // 1 minute max: 100 // limit each IP to 100 requests per windowMs })
app.use(limiter)
But If I try to test my application with slowtesttool and run a test with 2 connections (with rate 1 connection per sec and follow up data every 10sec), I see the connections never get closed.
I have set timeout to the connection as below, but it doesn't seem to work!
app.use((req, res, next) => {
req.connection.setTimeout(30000, () => {
req.socket.end()
})
next()
})
Is there a way I can limit the rate of accepting data, i.e. specifying the max time I can wait for every next chunk of body?
One solution could be to use the capacities of your front webserver (I assume that you will expose your app behind a server such as nginx, apapche, caddy, ...).
Nginx and caddy have this built-it, others probably too.
I'm writing an api using nodejs and express and my app is hosted by openshift free plan.
I want to protect my routes from brute force. For example if an IP sends more than 5 requests /sec then block it for 5 minutes. :)
There's nothing stopping you from implementing this in Node.js/express directly, but this sort of thing is typically (and almost certainly more easily) handled by using something like nginx or Apache httpd to handle traffic to your app.
This has the added benefit of allowing you to run the app entirely as an unprivileged user because nginx (or whatever) will be binding to ports 80 and 443 (which requires administrative/superuser/whatever privileges) rather than your app. Plus you can easily get a bunch of other desirable features, like caching for static contents.
nginx has a module specifically for this:
The ngx_http_limit_req_module module (0.7.21) is used to limit the request processing rate per a defined key, in particular, the processing rate of requests coming from a single IP address.
There are several packages on NPM that are dedicated to this, if you are using the Express framework:
express-rate-limiter
express-limiter
express-brute
These can be used for limiting by ip, but also by other information (e.g. by username for failed login attempts).
It is better to limit rates on reverse-proxy, load balancer or any other entry point to your node.js app.
However, it doesn't fit requirements sometimes.
rate-limiter-flexible package has block option you need
const { RateLimiterMemory } = require('rate-limiter-flexible');
const opts = {
points: 5, // 5 points
duration: 1, // Per second
blockDuration: 300, // block for 5 minutes if more than points consumed
};
const rateLimiter = new RateLimiterMemory(opts);
const rateLimiterMiddleware = (req, res, next) => {
// Consume 1 point for each request
rateLimiter.consume(req.connection.remoteAddress)
.then(() => {
next();
})
.catch((rejRes) => {
res.status(429).send('Too Many Requests');
});
};
app.use(rateLimiterMiddleware);
You can configure rate-limiter-flexible for any exact route. See official express docs about using middlwares
There are also options for Cluster or distributed apps and many others useful