I'm using NestJS (with Express Server) for a project and trying to optimize the performance on some of the endpoints, using New Relic I noticed that a big chunk of the response time of all endpoints is spent in an anonymous middleware, reaching 89% at some points.
Is there a way to find out which middleware is this?
I've made an answer to this in another question, but I figured I should go into a bit more depth here about what's going on under the hood of Nest.
A route handler in Nest is technically a middleware in express. I.e.
#Controller('test')
export class TestController {
#Get()
testGet() {
return 'do the thing';
}
}
Get's transformed under the hood (through some really awesome code) to
app.get('test', (req, res, next) => {
res.send('do the thing');
})
Now when we add on a filter
#Controller('test')
export class TestController {
#Get()
#UseFilter(SomeExceptionFilter)
testGet() {
return 'do the thing';
}
}
More magic happens and we get this
app.get('test', (req, res, next) => {
let returnVal = '';
try {
returnVal = controller.testGet();
} catch (err) {
returnVal = someExceptionFilterInstnace.catch(err, customArgumentHostThatNestMaintains);
}
res.send(returnVal);
})
(Disclaimer: this isn't a perfect 1:1 representation of what actually happens, as there's a lot going on, but this should get the idea across that it's all middleware in the end)
This follows too as you add in services, they're just method calls inside of the express/fastify middleware, and so they don't get broken out in a nice way in newrelic.
Unfortunately, I don't know of a good way to help with this in newrelic or how to add better annotations. Looking at the timing of the middleware, you've got 218 ms, which isn't too bad. Most likely a database operation somewhere, so I'd check your pipes, guards, and interceptors, along with your service, and see if you have any possibly large queries. If nothing else, you can do some profiling locally and get a bit more raw information about the time being taken
Related
I want to change an entity name from Person to Individual. I want to keep the old /person endpoint (for temporary backward compatibility) and add a new /individual endpoint.
What would be the easiest way to do it in Node.js using Nest?
I can copy the code but I'm hoping for a better solution that won't require duplication
The #Controller() decorator accepts an array of prefix, thus you can use like this:
import { Controller, Get } from '#nestjs/common';
#Controller(['person', 'individual'])
export class IndividualController {
#Get()
findAll(): { /* ... */ }
}
to me this is the easiest way.
source
In NestJS, We can have multiple routes for the whole controller or for a single route. This is supported for all HTTP methods (POST, GET, PATCH etc., )
#Controller(['route-1', 'route-2'])
export class IndividualController {
#Get(['/sub-route-1','/sub-route-2'])
public async getSomething(...){...}
All HTTP methods support either a single string route or an array of string routes. We could use this technique to deprecate a bad route and start introducing a better route without breaking the consumers immediately.
if you mean expressjs instead of jestjs (which is a testing framework), my approach would be the following:
simply exclude your controller code into a function and pass it to your routes.
// your controller code
const doSomethingWithPersonEntity = (req, res, next) => {
res.status(200).json(persons);
}
router.get("/person", doSomethingWithPersonEntity);
router.get("/individual", doSomethingWithPersonEntity);
I'm walking through the Javascript demos of pg-promise-demo and I have a question about the route /api/users/:name.
Running this locally works, the user is entered into the database, but is there a reason this wouldn't be a POST? Is there some sort of advantage to creating a user in the database using GET?
// index.js
// --------
app.get('/api/users/:name', async (req, res) => {
try {
const data = (req) => {
return db.task('add-user', async (t) => {
const user = await t.users.findByName(req.params.name);
return user || t.users.add(req.params.name);
});
};
} catch (err) {
// do something with error
}
});
For brevity I'll omit the code for t.users.findByName(name) and t.users.add(name) but they use QueryFile to execute a SQL command.
EDIT: Update link to pg-promise-demo.
The reason is explained right at the top of that file:
IMPORTANT:
Do not re-use the HTTP-service part of the code from here!
It is an over-simplified HTTP service with just GET handlers, because:
This demo is to be tested by typing URL-s manually in the browser;
The focus here is on a proper database layer only, not an HTTP service.
I think it is pretty clear that you are not supposed to follow the HTTP implementation of the demo, rather its database layer only. The demo's purpose is to teach you how to organize a database layer in a large application, and not how to develop HTTP services.
I am implementing a simple Access Control system in Node, and I am wondering what can be the best approach for what I am doing.
I am using Node ACL and it is not clear to me how to block on a per-resource basis.
Let's take the following example:
USER ->* PROJECT ->* ENTRY. Users can have multiple projects which contains many entries. Users can be ADMIN or USER.
I created an endpoint /entry/{ID} where user can access an entry detail. The endpoint is accessible to everyone, ADMINs can see all entries, but for User I need to do something similar:
app.get('/entry/{id}', (req, res) => {
if (user.admin) {
// Return eveything
}
else {
if (entry.project == user.project) {
// return it
}
else {
// Unathorized
}
}
})
Is there a better approach/pattern to implement this checks on ownership on a resource?
It's a very broad question so I'll try to give you a couple hints as my answer, but
Is there an ACL pattern in javascript?
There's a number of solutions but I wouldn't call any of those a pattern. I'll be very subjective now, but the ways of passport.js and similar modules are non-transparent to say the least - and it's not really an ACL...
Someone may say - hey, it's node.js, there must be module to do that and make your node_modules heavier but searching for a good acl module in npm, I only found some outdated ones and tightly bound with express. Since your question wasn't which is the best npm module for acl I gave up looking for such at page 3, which doesn't mean there ain't something ready so you may want to look more closely.
I think your implementation could be considered acceptable, with some minor corrections or hints as I mentioned:
Separate your request logic from access control logic
In your code everything happens in one callback - that's definitely very efficient, but also very hard to support in longer term. You see, it'll end up in the same code in lots of those if's above in all the callbacks. It's very simple to separate the logic - simply implement the same path in two callbacks (they'll be run in the order they were defined), so:
app.all('/entry/{id}', (req, res, next) => {
const {user, entry} = extractFromRequest(req);
if (user.admin || entry.project === user.project) {
next();
} else {
res.status(403).send("Forbidden");
}
});
app.get('/entry/{id}', (req, res) => {
// simply respond here
})
This way the first callback checks if the user has access and this won't affect the logic of the response. The usage of the next() is specific to express-like frameworks which I assumed you use looking at your code - when you call it the next handler will be executed, otherwise no other handlers will be run.
See Express.js app.all documentation for an acl example.
Use a service wide acl
It's much more secure to keep a basic ACL in a single place and not to define it per path unless necessary. This way you won't omit one path and won't leave a security hole somewhere in middle of request. For this we need to split the ACL into parts:
URL access check (if path is public/open for all users)
User and session validity check (user is logged in, session is not expired)
Admin/user check (so permission level)
Otherwise we don't allow anything.
app.all('*', (req, res, next) => {
if (path.isPublic) next(); // public paths can be unlogged
else if (user.valid && user.expires > Date.now()) next(); // session and user must be valid
else if (user.admin) next(); // admin can go anywhere
else if (path.isOpen && user.valid) next(); // paths for logged in users may also pass
else throw new Error("Forbidden");
});
This check is not very restrictive but we won't need to repeat ourselves. Also notice the throw Error at the bottom - we'll handle this in an error handler:
app.use(function (err, req, res, next) {
if (err.message === "Forbidden") res.status(403).send("Forbidden");
else res.status(500).send("Something broke");
})
Any handler with 4 arguments will be considered an error handler by Express.js.
On the specific path level if there's any need for ACL's, simply throw an error to the handler:
app.all('/entry/{id}', (req, res, next) => {
if (!user.admin && user.project !== entry.project) throw new Error("Forbidden");
// then respond...
});
Which reminds me of another hint...
Don't use user.admin
Ok, fine, use it if you like. I don't. The first attempt to hack your code will be by trying to set admin on any object that has properties. It's a common name in a common security check so it's like leaving your WiFI AP login at factory defaults.
I'd recommend using roles and permissions. A role contains a set of permissions, a user has some roles (or one role which is simpler but gives you less options). Roles may be also assigned to project.
It's easily a whole article about this so here's some further reading on Role-based ACL.
Use standard HTTP responses
Some of this mentioned above, but it's a good practice to simply use one of standard 4xx HTTP code status as response - this will be meaningful for the client. In essence reply 401 when the user is not logged in (or session expired), 403 when there's no sufficient priviledge, 429 when use limits are exceeded. more codes and what to do when the request is a teapot in Wikipedia.
As to implementation itself I do like to create a simple AuthError class and use it to throw errors from the app.
class AuthError extends Error {
constructor(status, message = "Access denied") {
super(message);
this.status = status;
}
}
It's really easy to both handle and throw such an error in the code, like this:
app.all('*', (req, res, next) => {
// check if all good, but be more talkative otherwise
if (!path.isOpen && !user.valid) throw new AuthError(401, "Unauthenticated");
throw new AuthError(403);
});
function checkRoles(user, entry) {
// do some checks or...
throw new AuthError(403, "Insufficient Priviledges");
}
app.get('/entry/{id}', (req, res) => {
checkRoles(user, entry); // throws AuthError
// or respond...
})
And in your error handler you send your status/message as caught from your code:
app.use(function (err, req, res, next) {
if (err instanceof AuthError) res.send(err.status).send(err.message);
else res.status(500).send('Something broke!')
})
Don't reply immediately
Finally - this is more of a security feature and a safety feature at the same time. Every time you respond with an error message, why not sleep a couple seconds? This will hurt you in terms of memory, but it will hurt just a little and it'll hurt a possible attacker a lot because they wait for the outcome longer. Moreover it's super simple to implement in just one place:
app.use(function (err, req, res, next) {
// some errors from the app can be handled here - you can respond immediately if
// you think it's better.
if (err instanceof AppError) return res.send(err.status).send(err.message);
setTimeout(() => {
if (err instanceof AuthError) res.send(err.status).send(err.message);
else res.status(500).send('Something broke!')
}, 3000);
})
Phew... I don't think this list is exhaustive, but in my view it's a sensible start.
I wonder what way should I organize my routing in expressJS :
Params parsing in Controller
router.get('/users/:id', UserController.get);
class UserController {
get(res, req) {
var id = res.params.id;
UserModel.get(id, function(user) {
res.send(user);
}
}
}
Params parsing in Route
router.get('/users/:id', function(req, res) {
var id = req.params.id;
UserController.get(id, function(user) {
res.json(user);
}
});
class UserController {
get(id, fn) {
UserModel.get(id, fn);
}
}
I find the second version Params parsing in Route easier for
unit test
In case of change in the URL params or request body
but most of the example I found use the first version, why ?
If you consider a much larger, messier real world application, with route names that no longer match controller names etc., it might be beneficial to place the full routing table (all of the router.xxx calls) in one place, such as a routes.js. For a given url, this makes it much simpler for a new developer to figure out which code handles which url.
If you included all of the parameter parsing in your routes.js, it would become really messy and you'd likely lose some of the benefit of having collected all that into one file in the first place.
That said, there's no reason why you cant have the best of both worlds by separating the routing, the parameter parsing/response formatting, and the controller logic each into their own modules.
I have two situations to get data from DB
To show normal data
http://exampleapp.com/task/{{taskId}}
To edit data via posting
http://exampleapp.com/task/{{taskId}}/?state={{app.state}}
Both url have the same http://exampleapp.com/task/{{taskId}} just a little bit different with last phrase ?state={{app.state}}
I use Express routing as followed:
app.get('/task/:taskId/(?state=:status(pending|cancel|confirmed|deleted))?', routes.task.show);
But I dont know why it does not work ?
For example error: Cannot GET /task/51d2c53f329b8e0000000001 when going to h**p://exampleapp.com/task/51d2c53f329b8e0000000001
Query strings cannot be defined in routes. You access query string parameters from req.query.
app.get('/task/:taskId', function(req, res) {
if (req.query.state == 'pending') { ... }
});
However, if you're modifying a task, this is not the appropriate way to do it. GET requests SHOULD be idempotent: the request SHOULD NOT modify state. That's what POST requests are for.
app.get('/task/:taskId', function(req, res) {
// show task info based on `req.params.taskId`
});
app.post('/task/:taskId', function(req, res) {
// set task `req.params.taskId` to state `req.body.state`
});
You could either have a <form> that posts to the task, or make an ajax request:
$.post('/task/1', { state: 'pending' }, function() { ... });
According to the Express API, you cannot mix RegExp routes with string routes.
You should do something like this (I'm assuming taskId is an integer):
app.get(/^\/task/([0-9]+)/(?state=:status(pending|cancel|confirmed|deleted))?, routes.task.show);
However, I don't see why you cannot only check if req.query.state is defined in your route. It's probably less error prone and easier:
app.get("/task/:taskId", function( req, res, next ) {
if (req.query.state) {
// Do things
}
next();
});
Your problem is that query strings are not considered in routing. You will either have to redesign your urls (ie, include the state into the url itself, instead of the query string) or check the query string in your route handler function.