From within a middleware module, I'm trying to count the number of page requests made from the parent app. Think of it as a generic request monitor that knows nothing about the pages the parent serves. My initial stab at it simply listened for requests and incremented a counter with each response generated, only to find that each page request generated n number of additional responses from all of the included requests (favicon.ico, script files, etc). I expected that, but hoped there was a way to filter out the secondary files.
Is there a way to differentiate between them so that I can ignore the included files in my count?
So far, I've used the request and express-req-metrics middlewares to look at the response properties, but haven't yet seen any property that was helpful.
If you're using middleware to serve static assets, then one solution is to
just reorder your middleware so that static asset requests never make it
to your counter middleware (if they are handled by upstream middleware):
var counter = 0;
app.use(serveStatic('public'));
app.use(function(req, res, next) {
counter++;
next();
})
// ... routes defined down here ...
However, one downside is that, if you don't have a favicon.ico file for
example, the serveStatic middleware will not handle the request and your counter
middleware will count those requests.
Another solution is to write your counter middleware so that it inspects
the request path first to ensure that the path doesn't end in
'.ico', '.js', '.jpg', etc. Here I just use a basic regular expression:
var counter = 0;
app.use(function(req, res, next) {
if (! /(.ico|.js|.css|.jpg|.png)$/i.test(req.path)) {
counter++;
}
next();
})
// ... routes defined down here ...
Related
I would like to require pages in my Node.js server based on the requested URI.
However I concern that this could be a severe security issue since user can inject some malicous chars into the url, something like ../../ and reach to my root server point and reveal all of the code.
So just like throwing a bottle of water to a big fire, I have eliminated the option to send . to the request.
This is not a silverbullet, probably :)
Maybe is there some standard/best practice/guide or keypoints about URI sanitizing in REST API based on Node.js?
Edit - here the code uses the require
// app.js
app.use(require('./services/router')(app));
// router.js middleware
function router(app) {
return function(req, res, next) {
try {
// checking for . in the url
if (req.url.indexOf(".")!=-1) cast.badRequest();
// req.url.split('/')[2] should be customers, users or anything else
require('../../resources/' + req.url.split('/')[2] + '/' + req.url.split('/')[2] + '-router')(app);
next();
} catch(err) { cast.notFound(); }
}
}
module.exports = router;
// rides-router.js (this could be users-router.js or customers-router.js)
module.exports = function(app) {
// GET ride - select a ride
app.get("/v1/rides/:id", dep.verifyToken(), require('./api/v1-get-ride'));
// POST ride - insert a new ride
app.post("/v1/rides", dep.verifyToken(), require('./api/v1-set-ride'));
app.use((req, res, next) => {
cast.notFound();
});
}
You asked how to do it safer. My recommendation is that you put all the resources in an array and run all the app.use() statements with one loop that pulls the resource names from the array at server startup.
I don't like running synchronous require() during a request and I don't like loading code based on user specified characters. Both are avoided with my recommendation.
// add routes for all resources
const resourceList = ['rides', 'products', ...];
for (let r of resourceList) {
app.use(`/${r}`, require(`./resources/${r}/${r}-router`));
}
This seems like less code and 100% safe and no running of synchronous require() during a request.
Advantages:
Fully whitelisted.
No user input involved in selecting code to run.
No synchronous require() during request processing.
All routes installed at server initialization time.
Any errors in route loading (like a missing route file) occur at server startup, not during a user request.
I'm novice in Express and a little bit confused about how it handles middlewares? So basically I have two middlewares which looks like:
app.use(require('_/app/middlewares/errors/404'))
app.use(require('_/app/middlewares/errors/500'))
404
var log = require('_/log')
module.exports = function (req, res, next) {
log.warn('page not found', req.url)
res.status(404).render('errors/404')
}
500
var log = require('_/log')
module.exports = function (er, req, res, next) {
log.error(er.message)
res.locals.error = er
res.status(500).render('errors/500')
}
So now I want to add my custom middleware app.use(require('_/app/middleware/shareLocals')) which looks like:
module.exports = function (req, res, next) {
res.locals.base_url = req.protocol + '://' + req.get('host');
next();
}
The main problem is that now when I try to use base_url I get 404 error...
So how Express understands what middleware do? That is between my middleware and 404 are no visual differences:
it receives same params
it doesn’t have any if's in it, just throws 404 error
Appears the feeling the middlewares in Express are made for errors (when excepts err as first param) and for 404 (when there is no first err)...
P.S.
Is there any difference defining middlewares before or after routes?
P.S. Is there any difference defining middlewares before or after routes?
Yes.
The order in which you register your middlewares (and routes) have a lot to say.
Image express as a giant list. Starting at the first element in the list, you have the first middleware OR route you have defined, next is the second, etc.
When express gets a request, it appears to be matching your route/name of route/middleware, and if it's a hit, it executes the middleware/route and potentially waits for a "next()" call.
So if you have a route "/test" it will only be executed if you have a request matching "/test". routes with different names obviously wont get triggered. middlewares can also have names: app.use("/test", middlewareA). This will also only trigger if "/test" is requested. The way you do it, all requests (within the routes namespace) will be triggered app.use(middlewareA). It's like a wildcard.
Now, to the implications of things being ordered:
Your 404 middleware should only be used AFTER all routes have been defined. that way, when the list reached the 404 middleware, no routes have actually been found.
returning/sending result/not calling next() at the end of a middleware will all potentially create problems in your flow. I wont go into details about this, but be aware of it.
I am guessing your own middleware is added after the 404 middleware. That is probably the problem. If not, you should surrender more of your code so we can take a better look. But remember, order is everything :)
I have multiple routes, split into different files (my app consists of different "modules", which I maintain in separate folders. For each folder, there is an index.js file in which I manage the routes per module, and I require these in the app.js file).
For every route, I will require to check the auth, and pass the loggedIn status to the header of every page:
//Default variables for the ejs template
var options = {
loggedIn: true
};
res.render("home/home", options);
If the logged in status is true, then the user's name will be displayed. If not, the login / signup labels are displayed.
What is the best way to centralise this, so that I don't need to require the auth script in every of these index.js (route) files?
I need to be able to pass the auth status to the view via the options object (see example).
In your auth, module, use a middleware function. That function can check and store res.locals.loggedIn which will be available for any view that will eventually be rendered. Just make sure the app.use call executes prior to your other routes and it will work properly.
app.use(function auth(req, res, next) {
res.locals.loggedIn = true; // compute proper value here
next();
});
From what I understand you need to do this for every request.One common thing is adding this as middleware so that all the request gets this .
For Example :
var http = require('http');
var connect = require('connect');
var app = connect();
app.use(function(req, res) {
res.end('Hello!');
});
http.createServer(app).listen(3000)
Now for every request , Hello is printed . You could extract this as a module and reuse it across projects. Check here for more details
I'm creating a CMS in node.js and Express. I allow users to create their own subsections in the site. A subsection can be a blog, a page or a forum. These sub-sections can be installed one level deep in the site url path, so for instance:
domain.com/custom-path-blog/
I would have to support the following url structure with express routes:
domain.com/custom-path-blog/ -> blog index
domain.com/custom-path-blog/page/5 -> list posts on page 5
domain.com/custom-path-blog/guides/ -> list posts that belong to guides category
domain.com/custom-path-blog/guides/this-is-a-post -> shows a post
I would also have to support other sub-sections with different url structures. I have to make a call to a database to check out what the first level in the url actually is before I can dispatch it to the appropriate route.
Since this is a saaas website I dont want to dynamically register the routes on my node process as I could end up having thousands of users with possibly millions of routes. This is not doable. I have to go to the database for that first chunk of information.
Once I know a sub section is a blog or a forum or a e-commerce store how do I send the url past that "custom-path-blog" to be processed by the appropriate express routing mechanism?
I'm starting to think this might be too complicated to do with express routes and I will have to do it by hand.
Thanks!
If you have already have 3 separated apps (page, blog, forum), and you want to launch it in 1 node process you can do this:
app.use('/page', pageApp);
app.use('/blog', blogApp);
app.use('/forum', forumApp);
express will strip out the first component of url for you.
In your case, the first component is customize by user, so you need to write a middleware for it:
function appSelector(req, res, next) {
var firstComponent = getFirtCompoent(req.url.pathname) // return page or blog ...
var userID = req.user.id;
detectAppForCurrentUser(firstCompoent, userID, function (type) {
if(type === 'page') {
removeFirstComponent(req);
return pageApp(req, res, next);
}
if(type === 'blog') {
removeFirstComponent(req);
return blogApp(req, res, next);
}
next(); // if not found continue with other routes
}
}
app.use(appSelector);
// TODO other routes here
there are many way to solve problem, but is it important rule: app.use, app.get are called on initialization phase only
In my ExpressJS app, several of my urls handlers have the following logic:
See if the user has permission to access a resource
If so, continue
Else, redirect to the main handler.
Is there a way to insert a pre-handler for certain url handlers, via ConnectJS or ExpressJS?
I know I can do it globally, for all handlers, (which I do to insert missing headers as a result from IE's broken XDR).
But, can I do this for a subset of handlers?
I do something like this:
lib/auth.js
exports.checkPerm = function(req, res, next){
//do some permission checks
if ( authorized ) {
next();
} else {
res.render('/401');
return;
}
};
app.js
var auth = require('./lib/auth');
...
app.get('/item/:itemid', auth.checkPerm, routes.item.get);
You can stack middleware before your final route handler like the above line has. It has to have same function signature and call next();
If I understand this question correctly, you know about:
// This is too general
app.use(myAuthMiddleware());
And you are aware that you can add it manually to certain url-handlers:
app.get('/user/profile/edit', myAuthMiddleware(), function(req,res){
/* handle stuff */ });
// but doing this on all your routes is too much work.
What you might not know about express' mounting feature:
// Matches everything under /static/** Cool.
app.use('/static', express.static(__dirname + '/public'));
Or app.all():
// requireAuthentication can call next() and let a more specific
// route handle the non-auth "meat" of the request when it's done.
app.all('/api/*', requireAuthentication);