We had to implement an image uploader for a node.js project. As framework we are using express.js We did it like described here: http://howtonode.org/really-simple-file-uploads
But we are not sure how to secure this image uploader. What we did so far is:
checking the file size
checking extension and header
rename the file
file is only accessible over a special route and is not in the root folder
Is this enough? We don't feel very comfortable with the following line:
// CHECKING FOR FILESIZE, EXTENSION, HEADERS
fs.readFile(req.files.displayImage.path, function (err, data) {
...
...
...
// RENAMING FILE
// SAVE FILE
...
...
...
}
Is it save to read the image this way? We are afraid, there could be malicious code in req.files.displayImage.path. Do we need to add more checks or are our checks sufficient? What attack vectors do we offer an attacker if we use the code as described?
Thank you for your advices
Tschoartschi
If you are concerned for opening malicious images on client side as posted in your comments. Try opening third party scripts and untrusted files inside a sandboxed iframe this will protect your users.
Related
Update: This question used to ask about Google Cloud Storage, but I have since realized the issue actually is reproducable merely trying to save the download to local disk. Thus, I am rephrasing the question to be entirely about file downloads in Typescript and to no longer mention Google Cloud Storage.
When attempting to download and save a file in Typescript with WebRequests (though I experienced the same issue with requests and request-promises), all the code seems to execute correctly, but the resultant file is corrupted and cannot be viewed. For example, if I download an image, the file is not viewable in any applications.
// Seems to work correctly
const download = await WebRequest.get(imageUrl);
// `Buffer.from()` also takes an `encoding` parameter, but it's unclear how to determine the encoding of a download
const imageBuffer = Buffer.from(download.content);
// I *think* this line is straightforward
const imageByteArray = new Uint8Array(imageBuffer);
// Saves a corrupted file
const file = fs.writeFileSync("/path/to/file.png", imageByteArray);
I suspect the issue lies within the Buffer.from call not correctly interpreting the downloaded content, but I'm not sure how to do it right. Any help would be greatly appreciated.
Thanks so much!
From what I saw in the examples for web-request, download.content is just a string. If you want to upload a string to Cloud Storage using the node SDK, you can use File.save, passing that string directly.
Alternatively, you could use one the solutions seen here.
I hope I'm saying this correctly. What I'm trying to do is write to a json file using fs.writeFile.
I can get it to work using the command line but what I want to do is call a function maybe a button click to update the json file.
I figure I would need some type of call to the node server which is local port 8080. I was researching and seen somebody mention using .post but still can't wrap my head around how to write the logic.
$(".button").on("click", function(event) {
fs.writeFile("./updateme.json", "{test: 1}", function(err) {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
});
Using jQuery along with fs? Wow that could be great! Unfortunately that is not as simple as that!
Let me introduce you to server-side VS client-side JavaScript. Well actually there are a lot of resources on the net about that - just google it, or check the answers to this other StackOverflow question. Basically JavaScript can run either on a browser (Chrome, Mozilla...) or as a program (usually a server written in NodeJS), and while the language is (almost) the same, both platforms don't have the same features.
The script that you're showing should run in a browser, because it's using jQuery and interacting with buttons and stuff (aka the DOM). Can you imagine what a mess it would be if that script could interact with the file system? Any page you'll visit will be able to crawl around in your holiday pictures and other personal stuff you keep on your computer. Bad idea! That is why some libraries like fs are not available in the browser.
Similarly, some libraries like jQuery are not available (or simply useless) in the server, because there is no HTML and user interaction, only headless programs running.
So, what can I do to write a JSON file after a user clicks on a button?
You can set up:
A NodeJS server that will write a JSON file
Make jQuery call this server with the data to be written after the user clicks on a button
If you want further guidelines on this, tell me in the comments! I'll be ready to edit my question so as to include instructions on setting up such an environment.
Is there a service that creates basically a one-time download of a file, preferably something I can use from NodeJS?
I've done some research on FilePicker, and haven't found anything about regenerating the link it gives you for a file. There may be a way to do this with NodeJS, but I'm using Meteor at the same time so many Node things probably will conflict.
You could build it with meteor. Using meteor-router with meteorite & use server side routing to deliver the files.
You need a collection to keep track of downloaded files:
Server JS
var downloads = new Meteor.Collection("downloads");
//create a link
downloads.insert({url:"/mydownload.zip",downloaded:false})
Meteor.Router.add('/file/:id', 'GET', function(id) {
download = downloads.findOne(id);
if( download) {
if(dowload.downloaded) {
this.response.send("You've already downloaded me")
}
else
{
//I guess you could just redirect or stream the file for an extra layer of surety
this.response.redirect(download.url);
}
}
});
On the client you can use /files/{{_id}} with _id of the file from downloads the person has as the link
My recommendation would also be to add custom server-side logic to count # of uploads (or just flag a file as downloaded/not downloaded) and respond accordingly. The closest you could do with Filepicker.io would be using the security policies to restrict downloading the file to a specific time interval.
in addition to using the router package
in Meteor.startup you can add
var require = __meteor_bootstrap__.require;
fs = require( 'fs' );
the fs variable should be declared on the server only. the fs package is used by Meteor and does not need to be added separately.
once you have done this, you can create files with Meteor.uuid() as their name which makes them unique and very difficult to guess. It is also possible to delete the file after a certain amount of time by using Meteor.setTimeout
the question is: where do the files to be downloaded come from?
Solution using Heroku Cloud and NodeJS Meteor Hooks
Heroku in particular is actually great for temporary file download links: they offer a "temporary scratchpad" filesystem that is reset every time the program restarts, and each running Node server cannot see the files other instances have created.
Each dyno gets its own ephemeral filesystem, with a fresh copy of the
most recently deployed code. During the dyno’s lifetime its running
processes can use the filesystem as a temporary scratchpad, but no
files that are written are visible to processes in any other dyno and
any files written will be discarded the moment the dyno is stopped or
restarted.
Taken from the Heroku documentation: https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem
Thus, any files written to the "filesystem" will be temporary.
This allows for a very easy solution to this problem: you can simply use NodeJS filesystem manipulation to create temporary files on the server, serve them once (or for a limited time), and then remove them so they cannot be downloaded again.
This in combination with something like $.download() will make a seamless experience which in turn prevents unauthorized downloads.
A web game I play on that allows user uploaded content has been having a lot of issues with people using the navigateToURL function to send players to random websites. I was curious if there was a way to disable this function using Actionscript 2 or 3. I have seen a way to do it using the HTML embed but I do not have administrative access to the website.
After doing some more research, I have come up with a solid answer:
You should use a combination of PHP and an executable called swfdump on the server side to validate the user uploaded content.
swfdump is an exe file located in the bin folder of the Flex SDK. You can run it from PHP using exec.
It will read the bytecode of the swf and produce a report. From that you can easily locate which files contain navigateToURL() and reject the files.
I tested a file of my own using swfdump -abc -out myfilereport.swfx myfile.swf
and in that output I found this:
findpropstrict flash.net:navigateToURL
findpropstrict flash.net:URLRequest
pushstring "http://www.plasticsturgeon.com"
constructprop flash.net:URLRequest (1)
callproperty flash.net:navigateToURL (1)
The url I was using was "http://www.plasticsturgeon.com". But it would be far easuer to just eliminate any swf that includes flash.net.navigateToURL. Once you identify tha is present you can generate an error notice to your end user.
So using this method you can find and reject any swf that is using navigate to URL. You could even create a batch to run and invalidate any existing assert with this problem.
More information about using bytecode:
http://code.google.com/p/redtamarin/wiki/ABC
And about decompiling ASbytecode:
http://dougmccune.com/flex/FOTB_Decompiling_Doug_McCune.pdf
In relation with my early question of how to add manifest cache in node.js, my question now is related with how to cache the HTML generated by node.js. As we didn't have a physical file like in php (index.php) we cannot cache such kind of files.
How we can cache a "non existing" page? Just adding in cache:
CACHE MANIFEST
CACHE:
# plain files to cache
/javascripts/client.js
/stylesheets/style.css
/stylesheets/style.styl
# generated files like /
/
/content
Any idea of how to solve this problem?
Thanks!
Solution:
Add router to return the cache.manifest file with the correct mime-type:
app.get("/offline.manifest", function(req, res){
res.header("Content-Type", "text/cache-manifest");
res.end("CACHE MANIFEST");
});
Found at stackoverflow
The cache manifest list URLs that should be cached. The client accessing those urls has no knowledge whether these are static html files on top of Apache or dynamic content generated by node.js or anything else.
You are basically instructing the client:
Read my list of urls
Go through each url
Download the response and store it someplace safe
Check back on my cache.manifest if it has changed and then proceed to step 1
So as long as your data generated by node.js is reachable via a URL there is no problem in defining it as a line in the cache manifest.
And if you are worried "how will I know which urls there are" you can always generate the cache.manifest file programmatically from node.js itself -- but remember to serve the correct content-type text/cache-manifest