I have built a Node/Express server behind Nginx & a javascript webapp to upload files. The webapp uses multer for uploads. I have adjusted Nginx to accept up to 10GB files.
When I attempt to send a 4GB file, Chrome craps itself with a "stalled" message in the Timing tab. There is no message about the error at /var/logs/nginx/error.log
Is there some practical limit to file upload using multer (multer-s3)? What strategy can I use for uploading large files like this? I have read about 'chunking' in answers from 2012, but surely this is a thoroughly solved problem.
Thank you
Related
I am trying to upload file using nodejs & multer.
I am using plesk server and it work probably on my local host but when I trying upload file on server (plesk) I got the 413(Request Entity Too Large).
upload file how size? you can enlarge upload size
However, it is not recommended to be too large
I have tried the PUT request by XMLHttpRequest. There is a browser-side limitation that doesn't allow me to upload files larger than 2GB. Then I have tried the POST request from an HTML form that doesn't require Javascript side preprocessing. It has 5GB upload size limitation in a single operation.
AWS recommended multipart upload in larger upload scenarios. That requires files to chunk down then upload into pieces. How to do it right from the browser, when the file size is greater than 10GB.
You can use chucks in combination with signed URLs. See this link for more details https://github.com/prestonlimlianjie/aws-s3-multipart-presigned-upload.
I am using Angular2 and nodeJS (loopback) for my application. I want to upload a large file to my backend server as I have not tried much code for file uploading.
So can someone help me with some efficient way to make a large file upload fast?
Also as per my understanding Angular works on the client side so file uploading will be done by nodeJs, right?
Another thing will breaking the large file into chunks and then uploading will help?
If yes then how will those multiple chunks are can be merged at backend node (loopback).
I'm about to start a project that will involve the user uploading audioMP3 files less than 16MB, which is less than the maximum file size for mongoDB. I have been searching but with no luck on how to implement this in the server side to recieve the mp3 file, save it to the DB and retrieve in the client. Any one done something similar or got any idea how to implement this specially the node express side.
Yes Thank you, what I finally did was to load the files in AWS and store their url MongoDB, later requested for them in my application and load them in the browser
This question already has answers here:
How to upload a file from the browser to Amazon S3 with node.js, Express, and knox? [closed]
(6 answers)
Closed 9 years ago.
Hello :) I'm looking at the feasibility of having my node application stream HTTP POST file uploads directly through to an Amazon S3 Bucket.
I'm looking at using Formidable and Knox for this.
The part I'm unsure about is that S3 requires that you know the total number of bytes in the file before transmission.
Would I be right in thinking that the only way to accomplish this then would be to use the HTML5 File API (And possibly an ActiveX control on Internet Explorer) to check the file size on the client and send this as part of the HTTP POST request ?
With the recent CORS support, you can easily send files directly to s3, without your server having to handle anything.
I recently wrote a short tutorial, with rails, but again the server is just used to compute some keys, so adapting it to express shouldn't be hard at all.
But with such a solution, you'll need to use the jQuery File Upload plugin, and you probably won't need knox
https://pjambet.github.io/blog/direct-upload-to-s3
Hope it'll help you.
Maybe this can help - I made this to use the JQuery File Upload plugin upload directly to S3. I did not need to check the size of the file.
https://gist.github.com/3995819