Im having problems uploading files to a node.js application using restify.
This is my upload code https://gist.github.com/maumercado/7ab5cbbfd27c6b825044
Apparently the events are not being triggered, but I dont really know the reason, also the files are being created, but once I see the size info it says 0 bytes.
also this is the server.js file https://gist.github.com/maumercado/ecf948b4b8fc7d39e69e
Im using a post request in order to upload the file and node 0.10.7.
What can be wrong with the code??
Thank you
It looks like restify.bodyParser() works the same as express.bodyParser(), in that it will handle the upload for you (and in the process consume the body data passed with the request, so there's nothing left to read once your handler is called – hence, no events either).
The uploaded data is written to a temporary file, req.files.selfie.path in your case.
As for what you're intending to do (upload progress), I don't think it's possible, unless you implement the functionality of bodyParser() itself. But I'm not overly familiar with the inner workings of Restify to be 100% sure about that.
If you're working with a browser as a client, you could implement upload progress there instead.
Related
I am using Node JS as web API server, for the front end, I am using Angular 4, Android and IOS. I have successfully implemented file upload using post request. I am exposing only one port outside. I need to implement file upload with a progress bar. I have only worked with POST and GET requests in past.I am not getting how to implement it on the server and send the progress responses to the front end. Additionally is there any other method that can be used.
It would be of great help if someone could help me with this process
For file upload you can use DropzoneJS. Since you are using Angular for front end use Dropzone for Angular.
I was looking into it and heard socket.io can be used to achieve the same. i need to user sharp package on the streamed data
For the server side, I use the built in express routers using multer. Multer parses multipart form data and uses callbacks that play very nicely with express routers.
Then for the UI you need an element that can upload multipart form data. If your using HTML 5 then you should be able to use just about any library that is out there. I personally use Vaadin uploads. It's a nice drop in element that does all the work for you. It even has a nice upload bar and status built in.
I have a need to create a pdf or html document within a Node.js express API which then sends that document over HTTP to an API managing our CMS.
So functionally I would like to create the document and POST it as part of a multipart-form upload POST request to an external service.
I see how to do this if after I create the file, I then turn around and write it disk. After that point I can do a read stream of the file from that path to format the POST request with the file.
However I'm wondering how I can perform this action without writing the file to disk and then reading it into a read stream. It seems I should be able to accomplish this without that IO.
Anybody able to point me to a good example or library that does something along these lines?
You can extend Writable and/or Readable streams. By the first look this library do what you need, with the same way - extending built-in streams.
I need to upload a local file to s3 and save its link in the database. Right now I am converting the image to base64 and sending it to my rails server, which saves it on s3 and returns a url. I send this URL in the next HTTP request. Now, how about I save it via express get a link and then use it for the request. What be the better approach? Using middleware or backend server?
For file upload, i suggest you to use multer middleware, because native multipart implementation is a little bit tricky. For interaction with amazon s3 middleware is used.
To send file somewhere else you could use pipes:
fs.createReadStream(rqPath).pipe(res);
In above example, file is read from local system and piped to response.
All mentioned modules could be find at NPM
If you're still trying to figure this out, I was struggling with the same issue, decided to POST binary data (converted from base64) directly in body without dealing with multipart forms, and whipped up the base64-image-upload package to make this easy.
I have an image upload view on my client (ember.js) that send the resized image to nodejs rest api;
it works well but it is easy for someone expert to force upload of a non-resized image;
I would like to keep the resize process on the client because this allows users to select heavy-weight images, that are resized locally and uploaded only after that, when they are lightweight;
If someone else uses something like this, I'm interested on how it is possible to make this as safe as possible;
As a rule of thumb when developing web applications is never ever trust any data coming from the client side, always try to do a check in your server side!
Use authentication, this ensures that user only allow to upload data to their own account and not fiddling others files.
Add a special message passing between your server and client, a simple example would be
i. send a post API request first (that contains the image information and targeted compressed size) to your server indicating that your client is starting to compress the picture
ii. when uploading, add a metadata to include the complete compressed image, and check the uploaded image with your server if it is within the accepted threshold, else discard it
You could enhance the security of the message passing to be more complicated!
This would be my simple security, anyone else got better solution? :)
Approaches here also work for file uploads. You can use a combination of checking:
content-length header and/or (i.e. req.headers['content-length'] > x)
reading stream size as it's being read by server. (i.e req.on('data'))
If the stream data exceeds a certain size you can respond accordingly. Check out something like Multer for file uploads, specifically the limits section. Best approach would probably the second option.
How do you send files on node.js/express.
I am using Rackspace Cloudfiles and wanna send images/videos to their remote storage but I am not sure that it's as simple as reading the file (fs.readFileSync()) and send the data in the request body, or is it?
What should the headers be.
What if it's a very large file on a couple of GBs?
Is it possible to use superagent (http://visionmedia.github.com/superagent) for this or is there a better library for sending files?
Please give me some information about this.
Thanks!
app.get('/img/bg.png', function(req, res) {
res.sendFile('public/img/background.png')
})
https://expressjs.com/en/api.html#res.sendFile
use "res.sendFile". "res.sendfile" is deprecated.
For large files, you will want to use node.js's concept of piping IO streams together. You want to open the local file for reading, start the HTTP request to rackspace, and then pipe the data events from the file read process to the HTTP request process.
Here's an article on how to do this.
Superagent is fine for small files, but because the superagent API presumes your entire request body is loaded into memory before starting the request, it's not the best approach for large file transfers.
Normally you won't need to worry specifically about the request headers as node's HTTP request library will send the appropriate headers for you. Just make sure you use whatever HTTP method your API requires (probably POST), and it looks like for rackspace you will need to add the X-Auth-Token extra header with your API token as well.
I am using Rackspace Cloudfiles and wanna send images/videos to their remote storage but I am not sure that it's as simple as reading the file (fs.readFileSync()) and send the data in the request body, or is it?
You should never use fs.readFileSync in general. When you use it, or any other method called somethingSync, you block the entire server for the duration of that call. The only acceptable time to make synchronous calls in a node.js program is during startup.
What should the headers be.
See RackSpace Cloud Files API.
Is it possible to use superagent (http://visionmedia.github.com/superagent) for this or is there a better library for sending files?
While I don't have any experience with superagent, I'm sure it will work fine. Just make sure you read the API documentation and make your requests according to their specification.