How do you send files on node.js/express.
I am using Rackspace Cloudfiles and wanna send images/videos to their remote storage but I am not sure that it's as simple as reading the file (fs.readFileSync()) and send the data in the request body, or is it?
What should the headers be.
What if it's a very large file on a couple of GBs?
Is it possible to use superagent (http://visionmedia.github.com/superagent) for this or is there a better library for sending files?
Please give me some information about this.
Thanks!
app.get('/img/bg.png', function(req, res) {
res.sendFile('public/img/background.png')
})
https://expressjs.com/en/api.html#res.sendFile
use "res.sendFile". "res.sendfile" is deprecated.
For large files, you will want to use node.js's concept of piping IO streams together. You want to open the local file for reading, start the HTTP request to rackspace, and then pipe the data events from the file read process to the HTTP request process.
Here's an article on how to do this.
Superagent is fine for small files, but because the superagent API presumes your entire request body is loaded into memory before starting the request, it's not the best approach for large file transfers.
Normally you won't need to worry specifically about the request headers as node's HTTP request library will send the appropriate headers for you. Just make sure you use whatever HTTP method your API requires (probably POST), and it looks like for rackspace you will need to add the X-Auth-Token extra header with your API token as well.
I am using Rackspace Cloudfiles and wanna send images/videos to their remote storage but I am not sure that it's as simple as reading the file (fs.readFileSync()) and send the data in the request body, or is it?
You should never use fs.readFileSync in general. When you use it, or any other method called somethingSync, you block the entire server for the duration of that call. The only acceptable time to make synchronous calls in a node.js program is during startup.
What should the headers be.
See RackSpace Cloud Files API.
Is it possible to use superagent (http://visionmedia.github.com/superagent) for this or is there a better library for sending files?
While I don't have any experience with superagent, I'm sure it will work fine. Just make sure you read the API documentation and make your requests according to their specification.
Related
Using Twitter as an example: Twitter has an endpoint for uploading file data. https://developer.twitter.com/en/docs/media/upload-media/api-reference/post-media-upload-append
Can anyone provide an example of a real HTTP message containing, for example, image file data, showing how it is supposed to be structured? I'm fairly sure Twitter's documentation is nonsense, as their "example request" is the following:
POST https://upload.twitter.com/1.1/media/upload.json?command=APPEND&media_id=123&segment_index=2&media_data=123
Is the media_data really supposed to go in the URL? What if you have raw binary media data? Would it go in the body? How is the REST service to know how the data is encoded?
You're looking at the chunked uploader - it's intended for sending large files, breaking them into chunks, so a network failure doesn't mean you have to re-upload a 100 MB .mp4. It is, as a result, fairly complicated. (Side note: The file data goes in the request body, not the URL as a GET parameter... as indicated by "Requests should be multipart/form-data POST format.")
There's a far less complicated unchunked uploader that'll be easier to work with if you're just uploading a regular old image.
All of this gets a lot easier if you use one of Twitter's recommended libraries for your language.
to upload a file, you need to send it in a form, in node.js server you save accept the incoming file using formidable.
You can also use express-fileupload or multer
I'm currently using a framework in Node.js ( the botbuilder module from Microsoft Bot Framework) which uses the request[2] module to make HTTP requests.
I'm encountering a problem : this framework seems to send a malformed JSON to Microsoft's servers, but I fail to see why and what is this JSON message made of.
So I'm looking for a way to log those messages, to take a peek at this malformed JSON, as I don't have access to the request object (unless I heavily alter the framework code, which is not something one shall do)
So far, I'm able to log the response body (by adding request to the NODE_DEBUG environment variable), but not the original request body. I did try a tcpdump on our server but since it's all HTTPS there's nothing I can use there.
Any idea / tool that might help ?
Thanks for your time.
Use Node.js middleware to log all your requests. For example, you could use the module request-debug.
Another popular request logging middleware worth knowing about is Morgan, from the Express.js server team.
I need to download a file by POSTing to a REST server.
I was first using http when it was get. But now i need to use POST, and node's http post is way too complicated, I dont want to build a low level request, so I dont want to use it.
I am now using request. https://www.npmjs.com/package/request
Now, my server either sends {isUpdateAvailable:false} or it sends a tar file.
So i need to save the file, or show 'Already up to date'. How do I pipe to a filestream by checking the response header?
I need to set the pipe along with the request code, so I'm not able to separate the code properly. I need to fs.createWriteStream only if its necessary. Will it be possible?
So, how do i do it?
I'm writing a node SDK and one of the endpoints allows for a file_upload parameter. I'm currently using the standard https library to make my calls, and I wonder if I should continue using it or move to something the "requests" library given that I need to do file uploads.
Here is an article I was reading through to build multi-part file upload functionality into the https module, but the article doesn't say the best way to combine the multi-part file form data and additional parameters say "test_mode=true" or something like that.
how to upload a file from node.js
Wondering if I should move over to requests complete or if this approach seems good then how can I add the above multi-part form functionality but extend it to allow for additional parameters in the body as well as a file binary.
The request module uses the form-data module for sending multipart/form-data requests. You could also use form-data by itself if you don't want to use request.
I'm using an external service to create images. I'd like my users to be able to hit my API and ask for the image. Then my Express server would retrieve it from the external service, then serve it to the user. Sort of like a proxy I suppose, but not exactly.
Is there an easy way to do this, preferably one that doesn't involve downloading the image to the hard drive, then reading it back in and serving it?
Using the request library, I was able to come up with this:
var request = require("request");
exports.relayImage = function(req, res){
request(req.params.url).pipe(res);
}
That seems to work. If there is a more efficient way to do this (meaning on server resources, not in terms of lines of code), speak up!
What you are doing is exactly what you should be doing, and is the most efficient method. Using pipe, the data is sent as it comes in, requiring no additional resources than are needed to buffer and transmit.
Also be mindful of content type and other response headers that you may want to relay. Finally, realize that you've effectively built an open proxy where anyone can request anything they want through your servers. This is a bit dangerous, so be sure to lock it down in your final application.
You should be able to use the http module to make a request to the external image service with a callback that returns the image as the response. It won't write to disk unless you explicitly tell it to.