How to block the api repeated request with in certain time limit - node.js

I am using Rest API with node js.
My requirement is that if I get repeated requests within a certain time interval, I need to block it for 15 minutes (because token expired in 15 mints).
My condition allow only 15 requests in 10 seconds from the same client.
once you receive the 16th request in the 11th second from the same client then client is need to be black listed and gets a Forbidden status from then on – until their Token expires.
Token expiry time limit is 15 mints from the time of token creation, this will be maintained in node session.
How to achieve this please help me any one I am using below method
var rateLimit = require('express-rate-limit');
var limiter = rateLimit({/* config */});
app.use('/users', limiter);
I am using default config values here. If I send repeated request then its says 429-Too Many Requests. After a few seconds its working no error response. Here I need to block the request from same place for 15 minutes, after 15 minutes token will be expired. I then want to continue the process then, creating a new token.

You need to decide 2 things:
A time limit
How many requests you want to allow in that time limit
If you want to limit requests to a maximum of 15 every 10 seconds, you would configure express-rate-limit like so:
var rateLimit = require('express-rate-limit');
var limiter = rateLimit({
windowMs: 10 * 1000, // 10 seconds
max: 15,
});
app.use('/users', limiter);

Related

Calling an external api request every hour for a loggedIn user in node js

when a user logins I want to call an external api(linkedIn api) every hour for the next 10 hours. How will I achieve this in node js? Every logged In user has different hours access e.g if the loggedIn user has (1 to 5) hours then the linkedIn api will be called every hour for the next 5 hours.
You can use the built in setInterval() function to execute code at a given rate (in milliseconds)
setInterval(() => {
// Your code here
}, 3600000) // one hour is 3,600,000 milliseconds

Slow Post Vulnerability (R U Dead Yet) - Express.js - data rate limit Solution?

I am trying to solve the issue of Slow Post Vulnerability on my application.
Issue: https://blog.qualys.com/securitylabs/2011/07/07/identifying-slow-http-attack-vulnerabilities-on-web-applications
To limit the number of connections from a user, I have used express-rate-limit so that the application does not go unavailable.
const rateLimit = require('express-rate-limit')
const limiter = rateLimit({ windowMs: 60 * 1000, // 1 minute max: 100 // limit each IP to 100 requests per windowMs })
app.use(limiter)
But If I try to test my application with slowtesttool and run a test with 2 connections (with rate 1 connection per sec and follow up data every 10sec), I see the connections never get closed.
I have set timeout to the connection as below, but it doesn't seem to work!
app.use((req, res, next) => {
req.connection.setTimeout(30000, () => {
req.socket.end()
})
next()
})
Is there a way I can limit the rate of accepting data, i.e. specifying the max time I can wait for every next chunk of body?
One solution could be to use the capacities of your front webserver (I assume that you will expose your app behind a server such as nginx, apapche, caddy, ...).
Nginx and caddy have this built-it, others probably too.

Delay koa request 30 minutes but send 200 header immediately

I'm looking to delay all requests to an endpoint 30 minutes from when it was originally received. I'm working with Shopify and am responding to an order creation webhook, passing along data to another 3rd party service.
This is what happens: an order is placed, and then a webhook fires via Shopify, which contacts my koa endpoint with the associated order data.
I want to receive the webhook body (which includes order data) immediately, but then wait 30 minutes, then send along that information to another service. But I want to send a 200 header back to Shopify immediately, regardless of whether or not the code executes properly after the 30 minute delay (if it doesn't, I'll be informed via an error email anyway and can handle it manually, it wouldn't be the end of the world).
I've tried setTimeout methods, but the issue seems to be this: I can't send a 200 header back to Shopify until after the 30 minutes is up (or I shouldn't anyway, according to Koa docs?).
Meaning, if I do something like
context.response.status = 200; // tried to send response code immediately due to 30 minute timeout.
await handleRequest(context.request.body); // code waits 30 mins via settimeout
the 200 header isn't being sent right away
By the time 30 minutes has passed, Shopify has long given up on receiving a 200 header and has automatically put the request in a retry queue, which means the webhook will fire again before the 30 minutes is up, so I'll be sending duplicate requests to the service I'm trying to wait 30 minutes for.
I've considered cron, but it's not that I want these to be processed every 30 minutes, I want each individual request to wait 30 minutes to send to a third party service after it was received.
I'm really trying to avoid a database.
I'm fine with using setTimeout, if there is a way to send a 200 header back to Shopify before the timeout and subsequent service call after 30 minutes.
Status code goes with the first part of the HTTP message, so you need to start sending, or end the HTTP response.
In this case, you don't need to block the response on the 3rd party call. Move the logic you want to run in 30 minutes into its own function and don't await on the timer. You don't need it in your response logic.
Some pseudo code:
function talkTo3rdPartyDelayed(context, delay) {
setTimeout(()=>{
// talk to 3rdparty
}, delay);
return; // continue what you were doing before, this is non-blocking
}
function listenForShopify(req, res, next) {
talkTo3rdPartyDelayed({req.body}, 30*60*1000);
res.statusCode = 200;
next(); // or res.end() which sends the status code
}
As an aside, in a robust system, you'd store that 3rd party request you want delayed for 30 minutes in a queue system, with a visibility timer set to 30 minutes ahead. That way, if the process dies, you don't lose that work you intended to do. Having to manually handle any issues that come up will get tiresome.

How to limit the amount of async calls in node js?

I am trying to limit the amount of async calls I can make to the server to 1 call per second. I looked at async eachOfLimit, but not sure if this is the right solution.
I'm looking for a library that already exists, that would wrap around the request module and actually throttle requests to 1 per second. I just don't know how to handle this kind of situation, and I'm hoping for some kind of standard.
If you're using express you can use express-rate-limit package from npm. Usage below -
const rateLimit = require("express-rate-limit");
app.enable("trust proxy"); // only if you're behind a reverse proxy (Heroku, Bluemix, AWS ELB, Nginx, etc)
const limiter = rateLimit({
windowMs: 60 * 1000, // 1 minute
max: 1 // limit each IP to 1 requests per windowMs
});
// apply to all requests
app.use(limiter);

My API request is slow - where to start investigation on what is causing delays?

I have app hosted on heroku + heroku postgres instance.
My REST api fetches some data from DATABASE and returns data back to client.
I am using Sequelize.js as ORM and restify.js as server.
My API function looks like this:
app.get('/test', function(req,res,next){
t1 = new Date()
db.getSomeData.done(function(err,result){
t2 = new Date()
console.log((t2.getTime()-t1.getTime())+'ms')
res.json({}) // it is not mistake, I return empty json to make sure that amount of data transfered back to client is not causing dely
next()
})
})
For tests I always fetch exactly same data from DB. console.log says that DB query takes 400-500ms. However when I am testing ajax requests (in chrome) and I can see that average request time is 500-800ms, but from time to time (it happens around 2-3 times per 10 tries) response is received after 3-5 seconds. This is strange because db query time is still normal (400ms for example) and I am not sending any data back to client. API is used only by me so there is no load on heroku server. I though that it might be caused by my internet connection so I made second test with following handler:
app.get('/test', function(req,res,next){
res.json({}) // it is not mistake, I return empty json to make sure that amount of data transfered back to client is not causing dely
next()
})
And for 100 times ther eis no delay - all requeststs are completed in 73-89 ms.
Do you know what might be a problem? It seems that request sometimes takes more time just because db query was done, no matter how much data is sent back. (despite that db query itself does not take time longer than normal)...
That is super-strange because I tried something like this:
app.get('/test', function(req,res,next){
setTimeout(function(){
res.json({}) // it is not mistake, I return empty json to make sure that amount of data transfered back to client is not causing dely
next()
},500)
})
And problem still occurs: 3-4 time per 10 tries response is received after 1.5 - 3.5 seconds where average is 800-900ms.
Also Response-Time header set by restify is correct despite the delay. For example:
Chrome says: 3.5s
Response-Time: 600s
It means that lag is caused by Heroku?
Resolved. It was a Heroku platform issue.
https://status.heroku.com/incidents/649
Investigating
Our automated systems have detected potential platform errors.
We are investigating.
Posted Jul 12, 2014 23:38 UTC

Resources