I am uploading a one megabyte file that is just 1mb of ones. It appears that any post page that does not handle file uploads will cause the server to save a temporary instance of that file that has an unknown time of existance. This is an issue that can be experienced in any post request. How can I fix this so that no one could use this as a vector of attack?
POST page http://cryptsy.tv/speedtest
speedtest: function(req,res){
res.end("Done");
}
https://github.com/justin7674/PeerJS-SwarmConnection/tree/master/Server
When using express-fileupload, any file upload to any post page (regardless of whether or not the post page is intended to handle file uploads) the temporary file will upload without permissions and the created file will stay in the server directory with no expiry time known/provided. This could allow for a server to be overfilled and memory maxed by simply sending a file to any post page available. I fixed it by passing this to every page utilizing POST or PUT.
,handlefiles: function(req,res,next){
const files = req.files || {};
function cleanupFiles () {
res.removeListener('finish', cleanupFiles);
res.removeListener('close', cleanupFiles);
const files = req.files || {};
for(var fn in files){
fs.unlinkSync(files[fn].path);
}
}
res.on('finish', cleanupFiles);
res.on('close', cleanupFiles);
next();
},
Related
I have created file storage system for my application. The problem I'm facing is that I'm unable to download the file from the server.
This is my code in the node js (express) controller
exports.downloadFiles = async (req, res, next) => {
const file = await File.findById(req.params.fileId);
return res.download(`uploads/${file.fileName}`, file.originalName);
}
I'm getting files correctly and on the frontend this is what I get.
But I should be prompted to save the file to my computer.
And here is my code that sends request to the backend
const requestData = { filePassword: password };
await axios.post(`${url-to-api}/file/${fileId}`, requestData);
I have tried looking into some answers (like this one), express documentation but can't find any solution.
Please note that these files are not in a static directory.
Please help me with this.
They way Multer works on top of express is wired!, why Multer should precede the controller in the chain of Middlewares, which which by design causes the server to upload stuff before the DB operation is even checked?
For instance if there was a post operation to articles, and it contains a bunch of fields one of them is a file.
articleModel{title:String,image:String};
router.post('/', multer, articleController.createArticle);
now at the time the request hits, first thing in the chain is to upload the file in the request, but what if an error happened at the execution of the record to the DB like validation or even duplicates, what if I am going to update the article title only? the old files will be uploaded again?
How to make multer upload the files in the response of the http operation callback?
You can indeed make all kind of stuff before Multer actually process the image:
var upload = multer({
dest: 'uploads/',
fileFilter: function (req, file, cb) {
// only images are allowed
var filetypes = /jpeg|jpg|png/;
var mimetype = filetypes.test(file.mimetype);
var extname = filetypes.test(path.extname(file.originalname).toLowerCase());
if (mimetype && extname) {
return cb(null, true);
}
cb("Error");
}
}).single('localImg');
app.post('/api/file', checkBody, auth, uploadFile, controller.aController);
Take this code for example, you can make all kind of middleware actions BEFORE multer process your file, but multer is a library to process multipart/form-data, not files only, people use multipart for sending files mainly but you can send all kind of data too and it will append them to the body (req.body)
Your question is: "Why multer should upload files before any operation?"
You can execute multer when ever you want, but multer will process the request and get your data into the body. Unless you don't need the body data first hand, you need multer to be in the first middleware.
Your other question is: "what if I am going to update the article title only? the old files will be uploaded again?"
No, it will be uploaded once, if there is any problem with the database, any error or reject, you can always use the filesystem (fs) to remove the file from your server, if you already upload it to a third party system, you can delete it.
Hope it helps
I'm working with NodeJS and using HAPI to create API for upload and download file. When uploading, I read the file information (filename, mime type and file content) and store it in database. File content is stored as base64 encoded string.
What I want to do is to create API, so when client hits it will be forced to download a file that is constructed based on the stored information using the code below
server.route({
method: 'GET',
path:'/file',
handler: function (request, reply) {
var fileData = // get file content;
var mime = // get file mime-type;
var fileBuffer = new Buffer(fileData, 'base64');
reply(fileBuffer)
.header('Content-disposition', 'attachment; filename=' + fileName)
.header('Content-type', mime).header('Content-length', fileBuffer.length).encoding('binary');
}
})
But this code looks like still not work, if I hit the API it will be loading process forever and no file downloaded. Anybody can give me suggestion on how to do it correctly?
UPDATE
the code is correct and works perfectly. the problem I had before is caused by another reason, incorrect encoding/decoding mechanism.
Check out the inert plugin for hapi which handles files, the repo is here
I'm working on a project using Google Cloud Storage to allow users to upload media files into a predefined bucket using Node.js. I've been testing with small .jpg files. I also used gsutil to set bucket permissions to public.
At first, all files generated links that downloaded the file. Upon investigation of the docs, I learned that I could explicitly set the Content-Type of each file after upload using the gsutil CLI. When I used this procedure to set the filetype to 'image/jpeg', the link behavior changed to display the image in the browser. But this only worked if the link had not been previously clicked prior to updating the metadata with gsutil. I thought that this might be due to browser caching, but the behavior was duplicated in an incognito browser.
Using gsutil to set the mime type would be impractical at any rate, so I modified the code in my node server POST function to set the metadata at upload time using an npm module called mime. Here is the code:
app.post('/api/assets', multer.single('qqfile'), function (req, res, next) {
console.log(req.file);
if (!req.file) {
return ('400 - No file uploaded.');
}
// Create a new blob in the bucket and upload the file data.
var blob = bucket.file(req.file.originalname);
var blobStream = blob.createWriteStream();
var metadata = {
contentType: mime.lookup(req.file.originalname)
};
blobStream.on('error', function (err) {
return next(err);
});
blobStream.on('finish', function () {
blob.setMetadata(metadata, function(err, response){
console.log(response);
// The public URL can be used to directly access the file via HTTP.
var publicUrl = format(
'https://storage.googleapis.com/%s/%s',
bucket.name, blob.name);
res.status(200).send(
{
'success': true,
'publicUrl': publicUrl,
'mediaLink': response.mediaLink
});
});
});
blobStream.end(req.file.buffer);
});
This seems to work, from the standpoint that it does actually set the Content-Type on upload, and that is correctly reflected in the response object as well as the Cloud Storage console. The issue is that some of the links returned as publicUrl cause a file download, and others cause a browser load of the image. Ideally I would like to have both options available, but I am unable to see any difference in the stored files or their metadata.
What am I missing here?
Google Cloud Storage makes no assumptions about the content-type of uploaded objects. If you don't specify, GCS will simply assign a type of "application/octet-stream".
The command-line tool gsutil, however, is smarter, and will attach the right Content-Type to files being uploaded in most cases, JPEGs included.
Now, there are two reasons why your browser is likely to download images rather than display them. First, if the Content-Type is set to "application/octet-stream", most browsers will download the results as a file rather than display them. This was likely happening in your case.
The second reason is if the server responds with a 'Content-Disposition: attachment' header. This doesn't generally happen when you fetch GCS objects from the host "storage.googleapis.com" as you are doing above, but it can if you, for instance, explicitly specified a contentDisposition for the object that you've uploaded.
For this reason I suspect that some of your objects don't have an "image/jpeg" content type. You could go through and set them all with gsutil like so: gsutil -m setmeta 'Content-Type:image/jpeg' gs://myBucketName/**
I'm trying to generate a zip file in memory on the server and sending it to the client as a download file.
Basically, there's a html page where the client types what file he wants.
The server receives what the client typed (via socket.io) and does a search on a mongodb. At this point, the server returns a link like this one
<a href='#' onclick='socket.emit("generateFile", id, path); return false;'>link</a>
Where id is the id of the entry in the database and path is the location of one of the files that will be in the zip.
After that, that server is supposed the create a json containing the entry that has that id and zip it along with some local files available on the server. My question is: how do I send this generated zip to the client? Remember: I want to send it without having to save it in the server's hdd. I tried using Express after the zip is created but the code doesn't reach this piece:
app.get('/', function(res, req)
{
res.set('Content-Type', 'application/zip')
res.send(generatedZipFile)
})
How to proceed?
This is from express-static-zip implementation
var contentType = mime.lookup(name);
if (contentType != 'application/octet-stream') {
res.set('Content-type', contentType);
var charSet = mime.charsets.lookup(contentType);
if (charSet) {
entryData = entryData.toString(charSet);
}
}
res.status(200).send(entryData);
You might want to set status code and mime format.