I'm trying to build an http module that suppose to work with an express server.
while reading the http module api, I see that it doesn't save the body inside the request object.
So my questions are:
If I want to build an express server which works with the official http module, how should I get the body?
I consider to implement the http module in the following way: listening to the socket, and if I get content-length header, listetning to the rest of the socket stream till I get all the body, save it as a memeber of the http request, and only then send the request object to the express server handler.
What are the pros and cons of my suggestion above vs letting the express server to "listen" to the body of the request via request.on('data',callback(data))
I mean , why shouldn't I keep the body inside the 'request' object the same way I keep the headers?
It's hard to answer your question without knowing exactly what you want to do. But I can give you some detail about how the request body is handled by Node/Express, and hopefully you can take things from there.
When handling a request (either directly via Node's request handler, or through Express's request handlers), the body of the request won't automatically be received: you have to open an HTTP stream to receive it.
The type of the body content should be determined by the Content-Type request header. The two most common body types are application/x-www-form-urlencoded and multipart/form-data. It's possible, however, to use any content type you want, which is usually more common for APIs (for example, using application/json is becoming more common for REST APIs).
application/x-www-form-urlencoded is pretty straightforward; name=value pairs are URL encoded (using JavaScript's built-in encodeURIComponent, for example), then combined with an ampersand (&). They're usually UTF-8 encoded, but that can also be specified in Content-Type.
multipart/form-data is more complicated, and can also typically be quite large, as vkurchatkin's answer points out (meaning you may not want to bring it into memory).
Express makes available some middleware to automatically handle the various types of body parsing. Usually, people simply use bodyParser, though you have to be careful with that middleware. It's really just a convenience middleware that combines json, urlencoded, and multipart. However, multipart has been deprecated. Express is still bundling Connect 2.12, which still includes multipart. When Express updates its dependency, though, the situation will change.
As I write this, bodyParser, json, urlencoded, and multipart have all been removed from Connect. Everything but multipart has been moved into the module body-parser (https://github.com/expressjs/body-parser). If you need multipart support, I recommend Busboy (https://npmjs.org/package/busboy), which is very robust. At some point, Express will update it's dependency on Connect, and will most likely add a dependency to body-parser since it has been removed from Connect.
So, since bodyParser bundles deprecated code (multipart), I recommend explicitly linking in only json and urlencoded (and you could even omit json if you're not accepting any JSON-encoded bodies):
app.use(express.json());
app.use(express.urlencoded());
If you're writing middleware, you probably don't want to automatically link in json and urlencoded (much less Busboy); that would break the modular nature of Express. However, you should specify in your documentation that your middleware requires the req.body object to be available (and fail gracefully if it isn't): you can go on to say that json, urlencoded, and Busboy all provide the req.body object, depending on what kind of content types you need to accept.
If you dig into the source code for urlencoded or json, you will find that they rely on another Node module, raw-body, which simply opens the request stream and retrieves the body content. If you really need to know the details of retrieving the body from a request, you will find everything you need in the source code for that module (https://github.com/stream-utils/raw-body/blob/master/index.js).
I know that's a lot of detail, but they're important details!
You can do that, it fairly simple. bodyParser middleware does that, for example (https://github.com/expressjs/body-parser/blob/master/index.js#L27). The thing is, request body can be really large (file upload, for example), so you generally don't want to put that in memory. Rather you can stream it to disk or s3 or whatnot.
Related
I know that body parser is used to extract data, but how this works. I'm sending JSON.stringify(data) from my angular 2 view but in my node js + express rest API I don't need to use JSON.parse(req.body) to use the data, please explain to me how this works exactly.
The body-parser module runs JSON.parse on the request body automatically when using the json mode (docs here), as long as the Content-Type header of the incoming request matches the mode used (application/json in this example).
See the source here for how body-parser actually does it.
Is it possible to use the request library in node to include a body for a get request? https://github.com/request/request#requestoptions-callback
It looks like the body option only works for POST/PUT/PATCH methods according to documentation. I was wondering if there was a known workaround for this. I know this is not conventional but the api that I will be hitting does accept a get request with a body and putting the data in query string is not an option because the url becomes too long. (I do not have the ability to implement api changes)
Turns out Node's request library does accept body in the get request although it doesn't mention it in the documentation. Just passing in options.body = {}, with options.json = true, worked great.
I Am using the MEAN stack to develop an application following some video tutorials
in the video he use the body-parser node.js module in the server side I want to understand why :
does it convert application/json HTTP requests to a JSON object? so node.js can use JSON format to work on the Request ?
by pure express you have only request headers and body as set of strings
but you want to work with set of parameters and values for doing backend jobs
https://github.com/expressjs/body-parser
it's allow you to parse request body into Object with {param: 'value'} structure, and deal with arrays and other complex structures inside body data
the name of library pretty self-explaining -> Parsing body data
You can do it by yourself from scratch, but then you write another body-parser
at the github docs you'll see all supported content-types
I'm developing a RESTful API for a mobile client application with the combination of Node.js, Express.js and Mongodb.
Now I'm trying to handle the upload of the user profile image and I've found a module called "multer" (that is the one ecommended by express.js team itself) that allow the express.app to handle multipart/form-data requests.
Now I need to test the app and, moreover, the upload function but I'm not able to simulate a http-form request (via postman chrome plugin).
Multer returns this error:
[Error: Multipart: Boundary not found]
In fact, comparing an http-form request (which works) with a custom http request, the second one has not the Boundary header property.
What Boundary property is?
If you are using Postman, you can try removing the Header: "Content-type": "multipart/form-data". I removed it and now it works.
Boundary in a multipart form indicates some delimiter string separating text and binary data. You can do this in postman but it sounds like you aren't sending both file and text so postman maybe defaults to a regular form. do you see something like:
If you click preview in postman you can see the boundary in the Content-type header and in the body.
solutions:
1) don't specify the content-type at client
2) use the naming convention(imageUpload) in upload.single('imageUpload') same as field name
I am new to node.js and express and have been experimenting with them for a while. Now I am confused with the design of the express framework related to parsing the request body.
From the official guide of express:
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(app.router);
app.use(logErrors);
app.use(clientErrorHandler);
app.use(errorHandler);
After setting up all the middleware, then we add the route that we want to handle:
app.post('/test', function(req, res){
//do something with req.body
});
The problem with this approach is that all request body will be parsed first before the route validity is checked. It seems very inefficient to parse the body of invalid requests. And even more, if we enable the upload processing:
app.use(express.bodyParser({uploadDir: '/temp_dir'}));
Any client can bombard the server by uploading any files (by sending request to ANY route/path!!), all which will be processed and kept in the '/temp_dir'. I can't believe that this default method is being widely promoted!
We can of course use the bodyParser function when defining the route:
app.post('/test1', bodyParser, routeHandler1);
app.post('/test2', bodyParser, routeHandler2);
or even perhaps parse the body in each function that handle the route. However, this is tedious to do.
Is there any better way to use express.bodyParser for all valid (defined) routes only, and to use the file upload handling capability only on selected routes, without having a lot of code repetitions?
Your second method is fine. Remember you can also pass arrays of middleware functions to app.post, app.get and friends. So you can define an array called uploadMiddleware with your things that handle POST bodies, uploads, etc, and use that.
app.post('/test1', uploadMiddleware, routeHandler1);
The examples are for beginners. Beginner code to help you get the damn thing working on day 1 and production code that is efficient and secure are often very different. You make a certainly valid point about not accepting uploads to arbitrary paths. As to parsing all request bodies being 'very inefficient', that depends on the ratio of invalid/attack POST requests to legitimate requests that are sent to your application. The average background radiation of attack probe requests is probably not enough to worry about until your site starts to get popular.
Also here's a blog post with further details of the security considerations of bodyParser.