I'm running a few node.js servers on the same framework right now. Somewhere along the lines of code of the framework I have this little snippet:
//...
if (subdomain.staticFolder) {
let relativePath = subdomain.staticFolder + path.sep + path.normalize(url);
if (fs.existsSync(relativePath)) {
let fileStats = fs.statSync(relativePath);
if (fileStats.isFile()) {
//...
This snippet is located inside the handler that manages request traffic. In a nutshell, what it's doing is looking inside a static/ folder for the domain and seeing if any file matches the request.
My concern is that a malicious user may come along and attempt to gain access to the web server's files, exploiting this lookup method of static files. For example:
https://www.example.com/../index.js
https://www.example.com/../serverStructure.json
Is this possible, or am I just worrisome? And, if it is possible, what are some potential solutions? Would moving static files to an S3 bucket be viable? Even if it isn't possible, is this still bad backend practice anyway?
Related
I'm working on an app that instead of a database uses file system in the server's root directory. It's basically a note application that allows me to save notes. Each note is a serialized object of Note class represented by following structure \Data\Notes\MyUsername\Title.txt
When I'm testing this on localhost through IIS Express everything works fine and I can easily go step by step there.
However, once I publish the app to Azure, the folder structure is still there (made a test Controller that uses Directory.GetFiles() and .GetDirectories() to simulate folder browsing so I'm sure that the files are there) but the file simply doesn't get loaded.
Loading script that's being called:
public T Load<T>(string filePath) where T : new()
{
StreamReader reader = null;
try
{
reader = new StreamReader(filePath);
var RawDB = reader.ReadToEnd();
return JsonConvert.DeserializeObject<T>(RawDB);
}
catch
{
return default(T);
}
finally
{
if (reader != null)
reader.Dispose();
}
}
Since I can't normally debug the app on Azure I tried to dump as much info as I can through ViewData and even there, everything looks okay and the paths match, but the deserialized object is still null, and this is only when trying to open an existing note WITHOUT creating a new one first (more on that later)
Additionally, like I said, those new notes get saved in the folder structure, and there's a Note sidebar on the left that allows users to switch between notes. The note browser is nothing more but a list that's collected with a .GetFiles() of that folder.
On Azure, this works normally and if I were to delete one manually it'd be removed from the sidebar as well.
Now here's the kicker. On localhost, adding a note adds it to the sidebar and I can switch between them normally.
Adding a note on Azure makes all Views only display that new note regardless of which note I open and the new note does NOT get stored in the structure (I don't know where it ended up at all!) even though the path is defined at that point normally and it should save just like it does on localhost.
var model = new ViewNoteModel()
{
Note = Load<Note>($#"{NotePath}\{Title}.txt"), //Works on localhost, fails on Azure on many levels. Title is a URL param.
MyNotes = GetMyNotes() //works fine, reads right directory on local and Azure
};
To summarize:
Everything works fine on localhost, Important part doesn't work on Azure.
If new note is not created but an existing note is opened, Correct note gets loaded (based on URL Param) on Localhost, it breaks on Azure and loads default Note object (not null, just the default constructor data since it's required by JsonConvert)
If a new note is created, you'll see it on Localhost and you'll be able to open all other notes regardless, you will see only the new note on Azure regardless of note picked.
It's really strange and I have no idea what could cause this? I thought it had something to do with Azure requests being handled differently so maybe controller pushes the View before the model is initialized completely but that doesn't make sense since there's nothing async here.
However the fact that it loads a note that doesn't exist on the server it's even more apsurd and I have no explanation for that.
Additionally this issue is not linked with a session. I logged in through my phone and it showed the fake note there as well right away.
P.S. Before you say anything about storage, please note this. Our university grants us a very limited Azure subscription. Simple lowest tier App service and 5DTU SQL server and 99% of the rest is locked out of our subscription. This is why I'm storing stuff on the server, not because I believe it's the smart thing to do.
I would like to use the swagger-ui dist 'as-is'...well almost as-is.
Pulled down the latest release from github (2.0.24) and stuck it in a folder in my app. I then server it out statically with express:
app.use('/swagger', express.static('./node_modules/swagger-ui/dist'));
That works as expected when I go to:
https://mydomain.com/swagger
However I want to populate the url field to my swagger json dynamically. IE I may deploy to different domains:
https://mydomain.com/api-docs
https://otherdomain.com/api-docs
And when I visit:
https://mydomain.com/swagger
https://otherdomain.com/swagger
I would like to dynamically set the url.
Is that possible?
Assuming the /api-docs (or swagger.json) are always on the same path, and only the domain changes, you can set the url parameter of the SwaggerUi object to "/path/to/api-docs" or "/path/to/swagger.json"instead of a full URL. That would make the UI load that path as relative to the domain the UI is hosted on.
For reference, I'm leaving the original answer as well, as it may prove useful in some cases.
You can use the url parameter to set the URL the UI should load.
That is, if you're hosting it under https://mydomain.com/swagger you can use https://mydomain.com/swagger?url=https://mydomain.com/api-docs and https://mydomain.com/swagger?https://otherdomain.com/api-docs to point at the different locations.
However, as far as I know, this functionality is only available at the current alpha version (which should be stable enough) and not with 2.0.24 that you use (though it's worth testing).
Another method would be to use the swagger-ui middleware located in the swagger-tool.
let swaggerUi = require('../node_modules/swagger-tools/middleware/swagger-ui');
app.use(swaggerUi(config.swagger));
The variable config.swagger contains the swagger.yaml or swagger.json. I have in my setting
let config = {
appRoot: __dirname,
swagger: require('./api/swagger/swagger.js')
};
Note: I am using the require('swagger-express-mw') module
You could try with this on index.html file of the swagger-ui... It works for me.
if (url && url.length > 1) {
url = decodeURIComponent(url[1]);
} else {
url = window.location.origin + "/path/to/swagger.json";
}
I am working on a development platform, I have code similar to the following:
app.use('/public', express.static( config.directory.public ));
The issue is that there are many (100s) of projects each with its own directory structure. The project will be selected via the URL:
http://localhost/dev/accounts
Where accounts is a project with its own directory tree and static public directory.
I do not want to run a separate copy of node for each project. Once a project has been selected via the URL then express needs to be reconfigured to serve files for that request.
However, that approach is probably not feasible because we may be working on many projects at the same time. So every request for static files would have to be processed according to the project URL. It seems to negate the benefit of static directories.
I think what I am after is a way to put variables into the directory path
http://localhost/dev/accounts
Would set a variable called prj = "accounts" and then somehow set express so that the root directory is "c:\projects\" + prj + "\public".
If I simply issue a new app.use(..) statement for every request I imagine bad things will happen.
Maybe I am better off just manually reading the file contents for each static request and sending the contents back.
Is there another way to approach this problem?
I'm not sure if I understood your question correctly, but express serves static files in file directories automatically for you. If you have a bunch of projects in some 'path/to/public' folder, you just need to do something like
app.use('/', express.static( __dirname + '/public' ));
That way, you just need to type some url like
http://localhost/project1
or
http://localhost/project2
Is there a service that creates basically a one-time download of a file, preferably something I can use from NodeJS?
I've done some research on FilePicker, and haven't found anything about regenerating the link it gives you for a file. There may be a way to do this with NodeJS, but I'm using Meteor at the same time so many Node things probably will conflict.
You could build it with meteor. Using meteor-router with meteorite & use server side routing to deliver the files.
You need a collection to keep track of downloaded files:
Server JS
var downloads = new Meteor.Collection("downloads");
//create a link
downloads.insert({url:"/mydownload.zip",downloaded:false})
Meteor.Router.add('/file/:id', 'GET', function(id) {
download = downloads.findOne(id);
if( download) {
if(dowload.downloaded) {
this.response.send("You've already downloaded me")
}
else
{
//I guess you could just redirect or stream the file for an extra layer of surety
this.response.redirect(download.url);
}
}
});
On the client you can use /files/{{_id}} with _id of the file from downloads the person has as the link
My recommendation would also be to add custom server-side logic to count # of uploads (or just flag a file as downloaded/not downloaded) and respond accordingly. The closest you could do with Filepicker.io would be using the security policies to restrict downloading the file to a specific time interval.
in addition to using the router package
in Meteor.startup you can add
var require = __meteor_bootstrap__.require;
fs = require( 'fs' );
the fs variable should be declared on the server only. the fs package is used by Meteor and does not need to be added separately.
once you have done this, you can create files with Meteor.uuid() as their name which makes them unique and very difficult to guess. It is also possible to delete the file after a certain amount of time by using Meteor.setTimeout
the question is: where do the files to be downloaded come from?
Solution using Heroku Cloud and NodeJS Meteor Hooks
Heroku in particular is actually great for temporary file download links: they offer a "temporary scratchpad" filesystem that is reset every time the program restarts, and each running Node server cannot see the files other instances have created.
Each dyno gets its own ephemeral filesystem, with a fresh copy of the
most recently deployed code. During the dyno’s lifetime its running
processes can use the filesystem as a temporary scratchpad, but no
files that are written are visible to processes in any other dyno and
any files written will be discarded the moment the dyno is stopped or
restarted.
Taken from the Heroku documentation: https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem
Thus, any files written to the "filesystem" will be temporary.
This allows for a very easy solution to this problem: you can simply use NodeJS filesystem manipulation to create temporary files on the server, serve them once (or for a limited time), and then remove them so they cannot be downloaded again.
This in combination with something like $.download() will make a seamless experience which in turn prevents unauthorized downloads.
What is the procedure for securing static assets (javascript and css) behind the firewall?
I have an admin section which uses javascript heavily. I don't really want to expose the code to the public.
I currently compile all my javascript using assetic to files in /web/admin/js/xyz.js
Is there a simple way to do this that I'm overlooking?
You could use a controller to serve the static file and secure that controller. Something like:
/**
* Serves static javascript file.
* We have configured /secure to be secured by some firewall
*
* #Route("/secure/xyz.js", name="static_xyz")
*/
public function staticXyzAction()
{
$headers = array(
'Content-Type' => 'text/javascript',
);
return new Response(file_get_contents($this->get('kernel')
->getRootDir().'../web/admin/js/xyz.js'), 200, $headers);
}
This is just an example with the data you provided. Obviously in your final code the file being served should be located in some directory which is not directly accesible by the web server.
The obvious downside to this approach is performance. PHP is much slower for serving an static file than your web server but depending on your load this may not be an issue.
Why do you want to "hide" these admin js files? The js should not perform critical auth or check rights, but just converse with your Sf2 Apis / Controllers which do that, and should not be critical if read. This is a conception matter.
If you are afraid that a lambda user / hacker sees these js files, you could set a very complicated random js output in Assetic. The Symfony .htaccess allows user to access static files only if they know their exact url, they cannot list your repository where you store your builded assets, the firewall catch that.
And last security mesure, use yui-minifier with Assetic to minify and obfusacate your builded js files.