Best explained by an example. Say I have a directory /images, where I have images a.png, b.png, and c.png.
Then I have a directory /foo/images, which has an image b.png, which is different than the b.png in /images.
I want it so if a request comes in for http://mydomain.com/foo/images/a.png, it will serve the image /images/a.png. But if a request comes in for http://mydomain.com/foo/images/b.png, it will get the version of b.png in /foo/images. That is, it first checks foo/images/ and if there is not file by that name, it falls back on /images.
I could do this using res.sendfile(), but I'd prefer use built-in functionality if it exists, or someone's optimized module, while not losing the benefits (caching, etc) that might be provided by the middleware.
This would intercept requests to /foo/images/ and redirect them if the file doesn't exist, still using static middleware and caching appropriately
var imageProxy = require('./imageProxy.js');
// intercept requests before static is called and change the url
app.use( imageProxy );
// this will still get cached
app.use( express.static(__dirname + '/public') );
And inside imageProxy.js:
var url = require('url');
var fs = require('fs');
var ROOT = process.execPath + '/public';
exports = function(req, res, next) {
var parts = url.parse(req.url);
// find all urls beginnig with /foo/images/
var m = parts.pathname.match(/^(\/foo(\/images\/.*))/);
if( m ) {
// see if the override file exists
fs.exists(ROOT+m[1], function (exists) {
if( !exists ) { req.url = ROOT+m[2]; }
// pass on the results to the static middleware
next();
});
}
});
If you wanted to access the original URL for some reason, it's still available at req.originalUrl
Related
I'm trying to write a very simple Express app, which is supposed to serve multiple static folders.
I have a root folder "stories" which contains multiple folders (story-1, story2, etc...). Each story folder contains static assets (scripts, CSS stylesheets, subpages...).
My users can unlock each of those stories, so each story folder must be protected. (If anyone tries to access http://backend/stories/story-1, it should give a 401 Forbidden).
My initial thought was to generate a one-time JWT upfront (like a signed url; not a bearer), add it to query params like http://backend/stories/story-1?jwt=the-jwt-token, then do some backend logic to test if the user has access to this content before serving it.
I tried fiddling with a basic express configuration + a custom authorization middleware :
Project structure :
...
-- /src
-- /stories ⬅️ custom public folder
-- /story-1 ⬅️ public but protected
- index.html
- /subpages
-page2.html
-page3.html
- /styles
- /scripts
-- /story-2 ⬅️ public but protected
- index.html
- /subpages
-page2.html
-page3.html
- /styles
- /scripts
-- /story-3 ⬅️ public but protected
- index.html
- /subpages
-page2.html
-page3.html
- /styles
- /scripts
etc...
index.js :
const express = require("express");
const { authorized } = require("./middlewares/authorized");
const app = express();
const port = 3000;
app.use("/stories/:story", authorized);
app.use("/stories", express.static(__dirname + "/stories"));
app.listen(port, () => {
console.log(`Example app listening on port ${port}`);
});
authorized.js :
exports.authorized = (req, res, next) => {
const jwt = req.query.jwt;
if (!jwt) return res.sendStatus(401);
// todo : custom logic to test if the user has access to this content, if yes we do next(), if no we return a 401.
return next();
};
This simple example works partially, when I try to go to http://localhost:3000/stories/first-story (without JWT), I get a 401 (that's ok).
But when I add the jwt :
http://localhost:3000/stories/first-story/?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
The middleware runs for every assets that are linked in the index.html, but those assets urls don't have the JWT query params, which leads to a 401.
I guess it's totally normal because that's how middlewares are intended to work. My guess is that i'm configuring express router wrong :
app.use("/stories/:story", authorized);
app.use("/stories", express.static(__dirname + "/stories"));
I would like to run the middleware only once, when any of the /:story subfolders inside /stories are asked to be served.
You write:
I would like to run the middleware only once, when any of the /:story subfolders inside /stories are asked to be served.
But if every .html (sub-)page is served by a separate HTTP request, each of these requests must be protected, assuming that the HTML contains material that is worthy of protection. (The styles and scripts may not need this extra protection.)
Therefore it is OK that the authorized middleware runs for each such request. And if the JWT was in a cookie (as suggested by Kaneki21), it would be present automatically in each request.
I would separate out the access control logic from the identity logic. You can use your jwt to verify that the user is who they say the are, and then use your existing knowledge of who that user is to grant them access.
I put a simple example using cookie-backed sessions below, note that you can add sequential middleware a, b, and c all in one function via app.use('/foobar',a,b,c).
// other setup
...
const session = require('express-session'),
fs = require('fs'),
{ Router } = require('express');
const secret = (() => {
let secretFile = '/path/to/my/secret.txt';
try {
// try reading a saved secret
return fs.readFileSync(secretFile,'utf8');
}
catch(err) {
// otherwise generate secret and save it
let random = require('crypto').randomBytes(128).toString('base64');
fs.writeFileSync(secretFile,random);
return random;
}
})();
// Add the session middleware to the app
app.use(session(
{ secret,
name: 'stories-and-whatnot',
cookie: { sameSite: true } }
));
// Create a router for stories and add it to the app
let storyRouter = Router();
app.use('/stories', storyRouter);
// add identity middleware to storyRouter
storyRouter.use( authorized);
let storyMax = 10;
for(let i=0; i<storyMax; i++) {
// set up the individual story routers
storyRouter.use(
`/story-${i}`,
function(req,res,next) {
if(!req.session.storyAccess || !req.session.storyAccess[i]) {
// If the user's session doesn't show it has access, reject with 401
res.status(401).end(`You do not have access to story ${i}`);
}
else {
// Otherwise let them proceed to the static router
next();
}
},
express.static(require('path').join(__dirname,`stories/story-${i}`)
);
}
...
// And somewhere else you have something like this
app.get('/access-granted', authorized, function(req,res,next) {
let { id } = req.query;
if(!req.session.storyAccess)
req.session.storyAccess = {};
req.session.storyAccess[id] = true;
res.end(`Access granted to story ${id}`);
});
You might consider, not using a middleware at all for the serving of content. But rather, to setup the user's set of "approved" paths.
That way a user, failing authentication, would have no valid paths, other then perhaps a preset collection of "base" paths.
This way, later after your authentication middleware the "routing" can be constrained to just that users set of "granted" paths.
Essentially model the access using sessions, which are established on first request, and then updated and maintained as things progress.
One solution is that you check if the user has the right to view the page on the client side. You'll need some JavaScript on the client side to do this.
You can store the token in LocalStorage after login. Then, at the beginning of the protected HTML file, you include your JS code to retrieve the token, and send a request to the server to check if the user is authenticated or not. Then, based on the response of the server you show the content or hide it.
To be honest, I rarely see the JWT in the URL. People talk about it here, here, here... You should revise your current approach carefully.
I'm trying to create a NodeJS Express API (route) which has the following characteristics:
It has a base path, in my case it is /web/views. This part is a static value and doesn't change for as long as the server is up.
I can do this as follows:
const BASE = '/web/views'; // defined externally/elsewhere
app.get(BASE, function handleRequest(req, res) {
// handle API request...
}
Next, I expect to be provided with a resource. Given the name of this resource, I locate a file and send it to the client.
I can do this as follows:
app.get(BASE + '/:resource', function handleRequest(req, res) {
var resource = req.params.resource;
// handle API request...
}
So on the client, I invoke it this way:
GET /web/views/header
All of this works so far... but my problem is that my 'resource' can actually be a path in itself, such as:
GET /web/views/menu/dashboard
or a longer path, such as:
GET /web/views/some/long/path/to/my/xyz
I was using the following REGEX mapping:
const DEFAULT_REGEX = '/(\*/)?:resource';
or more precisely:
app.get(BASE + DEFAULT_REGEX, function handleRequest(req, res) {
var resource = req.params.resource;
// handle API request...
}
This works with an arbitrary length path between my BASE value and the :resource identifier, but the problem is that my resource variable only has
the xyz portion of the path and not the full path (ie: /some/long/path/to/my/xyz).
I could simply cheat and strip the leading BASE from the req.url, but I though there would be a REGEX rule for it.
If anyone knows how to do such advanced REGEX routing, I'd appreciate it.
Thanks!
Sure, so I think the easiest way is to simply not worry about using Regex, but instead just use a wildcard. You lose the cool params name, but otherwise it works as you're looking for. For example:
const express = require('express');
const app = express();
const port = 3000;
const BASE = '/web/views'
app.get(`${BASE}/*`, (req, res) => {
res.send(req.url);
});
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
If you hit http://localhost:3000/web/views/path/to/my/resource, in my example the response content will be /web/views/path/to/my/resource, so from there it's some simple string manipulation to pull the bit you want:
let resource = req.url.split('/web/views')[1];
// resource will equal /path/to/my/resource if the above URL is used
Of course you could get fancier with your string parsing to check for errors and such, but you get the idea.
You could even setup a middleware to get that resource piece for other handlers to work from:
app.use(`${BASE}/*`, (req, res, next) => {
const resource = req.url.split(BASE)[1];
req.resource = resource;
next();
});
Then all subsequent routes will have access to req.resource.
In our application we store our reports in a user defined folders. User can add their own folders during runtime. Iam showing the history of those files in a web page. on clicking the file name i should show the file from the folder. How can i show the files from a non public directory.Since its given during runtime i havent added them as static dir to the express server.
One idea we tried was to use node-static-server and create a file server with the folder and serve the file. for each file we create this. it works fine but i get an error saying "port already in use". is there any better idea to do this? is this the right approach?
You can do this in NodeJS using a express.static:
const FS = require('fs')
const express = require('express')
const bp = require('body-parser')
const app = express()
function fileTest(req, res, next){
if (/\.|\/|\\/.test(req.params.file))
return res.sendStatus(400)
return next();
}
app.get(
'/static/:file',
fileTest,
function(req, res, next){
req.url = req.url.replace('/static','')
next()
},
express.static(
'./static',
{
fallthrough: false
}
)
)
app.post(
'/static/:file',
fileTest,
bp.text(),
function (req, res) {
FS.writeFile(
'./static/'+req.params.file,
req.body,
function (err) {
if(err)
return res.sendStatus(500)
return res.sendStatus(200)
}
)
}
)
app.listen(
1337
)
This is a simple example showing a server that will:
[POST]
Take a text body and load it into memory( pitfall: large bodies in memory )
Based on the URL, save it as a file in the static folder
[GET]
Search for a file
If found return file
The good news is that you can make the file and then request the file without restarting the server. Bad news is that this is a VERY SLOW server( comparatively to other options ).
As with all examples no good practices were followed, so be sure to adapt it to your needs.
Things to think about as you adopt it:
How do I allow people to save files to other folders?
How do I disallow people from saving files to other folders I don't want them to?
PROPER AUTHORIZATION
I use node.js in a simple way to serve a static web.
...
app.use(express.static('./build'));
http.createServer(app).listen(port, ipaddress);
...
This serves the files 1:1 (with index.html as default resource), e.g.
//server/a.html -> ./build/a.html
//server/bbb/x.html -> ./build/bbb/x.html
//server/ccc/ -> ./build/index.html
But now, I need to be able to remove 'one level' of the request url, but it shall serve still the same web, e.g.
//server/aaaa/a.html -> ./build/a.html
//server/bbbb/a.html -> ./build/a.html
//server/xxxx/bbb/x.html -> ./build/bbb/x.html
//server/yyy/ccc/ -> ./build/ccc/index.html
So I need a wildcard matching in the request url. I tried this:
app.use('/\*', express.static('./build'));
http.createServer(app).listen(port, ipaddress);
But with no luck. No more page is accessible. What is wrong?
[Edited to show that the server should serve index.html as default resource]
Depending on your application, you might put express.static() on separate Router instances that are mounted on your app. For example:
var routerA = new express.Router();
// You could also reuse the same static file handler since they
// are all using the same root path
routerA.use(express.static('./build'));
// and other `routerA` route handlers ...
var routerB = new express.Router();
routerB.use(express.static('./build'));
// and other `routerB` route handlers ...
// etc.
However if you don't have your application broken up like this already, you could also specify multiple routes like:
app.use(['aaaa', 'bbbb', 'xxxx'], express.static('./build'));
Or if nothing else, you could just use a custom middleware, calling the static file handler manually (although this is kind of a hack, as it was what separate, mounted Routers were designed to help solve):
var staticHandler = express.static('./build');
app.use(function(req, res, next) {
var m = /^\/[^/]+(\/.+)$/.exec(req.url);
if (m) {
// Temporarily override the `req.url` so that the path
// concatenation will happen correctly
var oldUrl = req.url;
req.url = m[1];
staticHandler(req, res, function(err) {
// Reverting the to the original `req.url` allows
// route handlers to match the request if a file
// was not found
req.url = oldUrl;
next(err);
});
} else
next();
});
app.get('/aaa/foo', function(req, res) {
res.end('hello from /aaa/foo!');
});
My final solution is:
// serve all files from ./web directory regardless of first element in url
app.get('/:leveltoremove/*', function(req, res) {
var path = req.params[0] ? req.params[0] : 'index.html';
res.sendfile(path, {root: './web'});
});
http.createServer(app).listen(port, ipaddress);
Usually you render a Jade page in a route like this:
app.get('/page', function(req, res, next){
res.render('page.jade');
});
But I want to serve all Jade pages (automatically rendered), just like how one would serve static HTML
app.use(express.static('public'))
Is there a way to do something similar for Jade?
"static" means sending existing files unchanged directly from disk to the browser. Jade can be served this way but that is pretty unusual. Usually you want to render jade to HTML on the server which by definition is not "static", it's dynamic. You do it like this:
app.get('/home', function (req, res) {
res.render('home'); // will render home.jade and send the HTML
});
If you want to serve the jade itself for rendering in the browser, just reference it directly in the url when loading it into the browser like:
$.get('/index.jade', function (jade) {
//...
});
https://github.com/runk/connect-jade-static
Usage
Assume the following structure of your project:
/views
/partials
/file.jade
Let's make jade files from /views/partials web accessable:
var jadeStatic = require('connect-jade-static');
app = express();
app.configure(function() {
app.use(jadeStatic({
baseDir: path.join(__dirname, '/views/partials'),
baseUrl: '/partials',
jade: { pretty: true }
}));
});
Now, if you start your web server and request /views/partials/file.html in browser you
should be able see the compiled jade template.
Connect-jade-static is good, but not the perfect solution for me.
To begin with, here are the reasons why I needed jade:
My app is a single page app, there are no HTMLs generated from templates at runtime. Yet, I am using jade to generate HTML files because:
Mixins: lots of repeated / similar code in my HTML is shortened by the use of mixins
Dropdowns: I know, lots of people use ng-repeat to fill the options in a select box. This is a waste of CPU when the list is static, e.g., list of countries. The right thing to do is have the select options filled in within the HTML or partial. But then, a long list of options makes the HTML / jade hard to read. Also, very likely, the list of countries is already available elsewhere, and it doesn’t make sense to duplicate this list.
So, I decided to generate most of my HTML partials using jade at build time. But, this became a pain during development, because of the need to re-build HTMLs when the jade file changes. Yes, I could have used connect-jade-static, but I really don’t want to generate the HTMLs at run time — they are indeed static files.
So, this is what I did:
Added a 'use' before the usual use of express.static
Within this, I check for the timestamps of jade and the corresponding html file
If the jade file is newer, regenerate the html file
Call next() after the regeneration, or immediately, if regeneration is not required.
next() will fall-through to express.static, where the generated HTML will be served
Wrap the ‘use’ around a “if !production” condition, and in the build scripts, generate all the HTML files required.
This way, I can also use all the goodies express.static (like custom headers) provides and still use jade to generate these.
Some code snippets:
var express = require('express');
var fs = require('fs')
var jade = require('jade');
var urlutil = require('url');
var pathutil = require('path');
var countries = require('./countries.js');
var staticDir = 'static'; // really static files like .css and .js
var staticGenDir = 'static.gen'; // generated static files, like .html
var staticSrcDir = 'static.src'; // source for generated static files, .jade
if (process.argv[2] != 'prod') {
app.use(‘/static', function(req, res, next) {
var u = urlutil.parse(req.url);
if (pathutil.extname(u.pathname) == '.html') {
var basename = u.pathname.split('.')[0];
var htmlFile = staticGenDir + basename + '.html';
var jadeFile = staticSrcDir + basename + '.jade';
var hstat = fs.existsSync(htmlFile) ? fs.statSync(htmlFile) : null;
var jstat = fs.existsSync(jadeFile) ? fs.statSync(jadeFile) : null;
if ( jstat && (!hstat || (jstat.mtime.getTime() > hstat.mtime.getTime())) ) {
var out = jade.renderFile(jadeFile, {pretty: true, countries: countries});
fs.writeFile(htmlFile, out, function() {
next();
});
} else {
next();
}
} else {
next();
}
});
}
app.use('/static', express.static(staticDir)); // serve files from really static if exists
app.use('/static', express.static(staticGenDir)); // if not, look in generated static dir
In reality, I have a js file containing not just countries, but various other lists shared between node, javascript and jade.
Hope this helps someone looking for an alternative.