Creating a Nodejs server that send response with multipart form data - node.js

There are lots of posts about how to handle request with multipart form data . But my use case is that I have a client that expects multipart form data response from server, and I need to write a simple nodejs server in order to test my client.
To write the simple server, I have the following:
var express = require('express');
var bodyParser = require('body-parser');
var FormData = require('form-data');
var app = express();
app.use(bodyParser.urlencoded({extended: true}));
app.use(bodyParser.json());
app.get('/describe', function(req, res) {
var form = new FormData();
form.append('part1', 'part 1 data');
form.append('part2', 'part 2 data');
res.setHeader('Content-Type', 'multipart/form-data');
res.send(form);
});
app.listen(3030, "0.0.0.0");
console.log('Listening on port 3030...');
Now when my client request localhost:3030/describe, the response header shows the following without the boundary value
Content-Type: multipart/form-data; charset=utf-8
And the content is downloaded as file instead of in the response body.
{"_overheadLength":208,"_valueLength":22,"_valuesToMeasure":[],"writable":false,"readable":true,"dataSize":0,"maxDataSize":2097152,"pauseStreams":true,"_released":false,"_streams":["----------------------------315683163006570790405079\r\nContent-Disposition: form-data; name=\"part1\"\r\n\r\n","part 1 data",null,"----------------------------315683163006570790405079\r\nContent-Disposition: form-data; name=\"part2\"\r\n\r\n","part 2 data",null],"_currentStream":null,"_boundary":"--------------------------315683163006570790405079"}
So my questions:
1. how do we make the boundary appears in the response header?
2. how do we make the form data response content show up in the response body instead as download file?

To send your form data you'll want to pipe it (see documentation), like this:
form.pipe(res);
To add the boundary into the header, you can do something like this:
res.setHeader('Content-Type', 'multipart/form-data; boundary='+form.getBoundary());
Now, about the "save as" box: The browser uses content-type to figure out what to do with the file. So if you want it to be displayed in the browser window, a good choice would be text/plain (or possibly text/html if that doesn't work). Then you could put your multipart/form-data into something like x-content-type.
(I'm assuming eventually you'll be using XHR or fetch, and at that point you can switch the content-type back. the text/plain thing is just a temporary solution to get the data to show up in a standard web browser for testing.)
Putting it all together:
app.get('/describe', function(req, res) {
var form = new FormData();
form.append('part1', 'part 1 data');
form.append('part2', 'part 2 data');
res.setHeader('x-Content-Type', 'multipart/form-data; boundary='+form._boundary);
res.setHeader('Content-Type', 'text/plain');
form.pipe(res);
});

In my case, it was POST request that was being called by a webhook and I used multer did the magic too. Here is the code.
const express = require('express');
const bodyParser = require('body-parser');
var multer = require('multer');
var upload = multer();
const app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.post('/', upload.none(), function (req, res) {
console.log("Data",req.body)
res.status(200).send('OK');
});
app.listen(port);

Related

req.body returns an empty object eventhough data is passed through form

this is my index.js code and it returns an empty object even though data is passed on from the front-end
const express = require("express");
const cors = require("cors");
const app = express();
app.use(cors());
app.use(express.json());
app.post("/api/register", (req, res) => {
console.log(req.body);
res.json({ status: "ok" });
});
app.listen(8000, () => {
console.log("listening on port 8000 . . . ");
});
For the specific case you're talking about, you usually need oa body parser to be able to access the form input fields. The minimum example that I advice you to build above it is the following:
// parse requests of content-type - application/json
app.use(express.json());
// parse requests of content-type - application/x-www-form-urlencoded
app.use(express.urlencoded({ extended: true }));
Some other hints
Make sure that the request is being submitted with the header Content-Type: application/x-www-form-urlencoded or Content-Type: application/json
Check if there any CORS problems
Here's More reference for you
The main reason this does not work is
some how the data passed in body is in text format while req.body is
expecting json data
make sure to double chek the 'Content-Type':'application/json' is set on the request headers
if you are using multipart/form-data or require a file upload from frontend you will need a multer as middleware for your post/patch requests, otherwise you can set your frontend to send application/json
[edit]
this line looks missing from your index.js
app.use(express.urlencoded({extended: true}))

Body-parser fails to/do not parse urlencoded parameters from GET request

I'm creating a web platform with a Nodejs server. I'm trying to retrieve urlencoded data sent from my front but can't manage to.
How I send the GET request :
xhr.open("GET", address + "?limit=1&offset=1",true);
xhr.setRequestHeader('Authorization', 'Bearer ' + token);
xhr.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
xhr.send(null);
xhr.addEventListener("readystatechange", processRequest, false);
On the server side :
const bodyParser = require('body-parser');
var urlencodedParser = bodyParser.urlencoded({ extended: true });
app.get('/guid_list', urlencodedParser, function (req, res) {
console.log(req.body.limit);
console.log(req.body.offset);
var headerjwt = HeaderGetJWT(req);
...
}
I have no problem retrieving the jwt token I'm sending, but always get undefined for urlencoded parameters.
I was wondering if I should use multipart content type instead, since I'm sending both a token and urlencoded data ? And maybe "multer" module in that case, since body-Parser does not support that content type.
I would suggest accessing your parameters in Node.js as follows (since they are being passed as query parameters):
app.get('/guid_list', parser, function (req, res) {
console.log("req.query.limit:", req.query.limit);
console.log("req.query.offset:", req.query.offset);
});
or just log all parameters:
app.get('/guid_list', parser, function (req, res) {
console.log("req.query:", req.query);
});

Postman send strange response for raw JSON post (node js)

I'm trying to do a POST request using raw json.
In the Body tab I have "raw" selected with this body:
{
"name": "book"
}
On the Node js side I'm doing res.send(JSON.stringify(req.body))
router.post('/', (req, res, next) => {
res.send(JSON.stringify(req.body));
}
And in POSTMAN response I receive:
{"{\n\"name\": \"book\"\n}":""}
When expected something like
{"name":"book"}
Have no idea - where could be a reason for it?
You'll need to use the Express JSON body parser, install using
npm install body-parser;
Then:
const bodyParser = require('body-parser');
app.use(bodyParser.json());
Once you do this, the JSON data will be parsed correctly and when you send it back it will render correctly.
Also make sure you have your Content-Type header set to "application/json" in your Postman request (go to "Headers" and add a new "Content-Type" header with value "application/json")
Here's a simple express app that will echo any JSON POST:
const express = require("express");
const port = 3000;
const app = express();
const bodyParser = require('body-parser')
app.use(bodyParser.json());
app.post('/', (req, res, next) => {
console.log("Body: ", req.body);
res.send(JSON.stringify(req.body));
})
app.listen(port);
console.log(`Serving at http://localhost:${port}`);
If you're on Express v4.16.0 onwards, try to add this line before app.listen():
app.use(express.json());
This is a built-in middleware function in Express. It parses incoming requests with JSON payloads and is based on body-parser.
Looks to me like its not a fault of Postman, but your NodeJS service is applying JSON.stringify twice?
Can you log the response type from the server to console to check whether its already json content or not?
try with hard coded json response and then with dynamic variable
res.json({"name":"book"});

How can I read the data received in application/x-www-form-urlencoded format on Node server?

I'm receiving data on a webhook URL as a POST request. Note that the content type of this request is application/x-www-form-urlencoded.
It's a server-to-server request. And On my Node server, I simply tried to read the received data by using req.body.parameters but resulting values are "undefined"?
So how can I read the data request data? Do I need to parse the data? Do I need to install any npm module? Can you write a code snippet explaining the case?
If you are using Express.js as Node.js web application framework, then use ExpressJS body-parser.
The sample code will be like this.
var bodyParser = require('body-parser');
app.use(bodyParser.json()); // support json encoded bodies
app.use(bodyParser.urlencoded({ extended: true })); // support encoded bodies
// With body-parser configured, now create our route. We can grab POST
// parameters using req.body.variable_name
// POST http://localhost:8080/api/books
// parameters sent with
app.post('/api/books', function(req, res) {
var book_id = req.body.id;
var bookName = req.body.token;
//Send the response back
res.send(book_id + ' ' + bookName);
});
You must tell express to handle urlencoded data, using an specific middleware.
const express = require('express');
const app = express();
app.use(express.urlencoded({
extended: true
}))
And on your route, you can get the params from the request body:
const myFunc = (req,res) => {
res.json(req.body);
}
The accepted answer uses express and the body-parser middleware for express. But if you just want to parse the payload of an application/x-www-form-urlencoded ContentType sent to your Node http server, then you could accomplish this without the extra bloat of Express.
The key thing you mentioned is the http method is POST. Consequently, with application/x-www-form-urlencoded, the params will not be encoded in the query string. Rather, the payload will be sent in the request body, using the same format as the query string:
param=value&param2=value2
In order to get the payload in the request body, we can use StringDecoder, which decodes buffer objects into strings in a manner that preserves the encoded multi-byte UTF8 characters. So we can use the on method to bind the 'data' and 'end' event to the request object, adding the characters in our buffer:
const StringDecoder = require('string_decoder').StringDecoder;
const http = require('http');
const httpServer = http.createServer((req, res) => {
const decoder = new StringDecoder('utf-8');
let buffer = '';
req.on('data', (chunk) => {
buffer += decoder.write(chunk);
});
req.on('end', () => {
buffer += decoder.end();
res.writeHead(200, 'OK', { 'Content-Type': 'text/plain'});
res.write('the response:\n\n');
res.write(buffer + '\n\n');
res.end('End of message to browser');
});
};
httpServer.listen(3000, () => console.log('Listening on port 3000') );
Express 4.16+ has implemented their own version of body-parser so you do not need to add the dependency to your project.
app.use(express.urlencoded()); //Parse URL-encoded bodies
Non-deprecated alternative to body-parser in Express.js
If you are creating a NodeJS server without a framework like Express or Restify, then you can use the NodeJS native querystring parser. The content type application/www-form-urlencoded format is the same as the querystring format, so we can reuse that built-in functionality.
Also, if you're not using a framework then you'll need to actually remember to read your body. The request will have the method, URL, and headers but not the body until you tell it to read that data. You can read up about that here: https://nodejs.org/dist/latest/docs/api/http.html
If you are using restify, it would be similar:
var server = restify.createServer()
server.listen(port, () => console.log(`${server.name} listening ${server.url}`))
server.use(restify.plugins.bodyParser()) // can parse Content-type: 'application/x-www-form-urlencoded'
server.post('/your_url', your_handler_func)

Node JS: Where is my POST data?

I have no lack with sending POST request to node js server. I have a simple request and a simple server.
My server code is:
var http = require('http');
var bodyParser = require('body-parser');
http.createServer(function (req, res) {
console.log(req.body);
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/')
my client request code is:
var val = JSON.stringify({ city:"SomeCity", name:"MyNameIsHere" });
alert(val);
$.ajax({
url: 'http://127.0.0.1:1337',
type: 'POST',
data: { value: val},
success: function(result) {
alert('the request was successfully sent to the server');}
});
So I suppose to get SomeCity and MyNameIsHere strings in the request body at the node js server, but the req.body field is undefined. Have to say that I open my test.html with request code locally with URL like this:
file:///D:/Projects/test.html
May be Im blind and overseen something trivial, but I have no idea what:)
Have to say that I open my test.html with request code locally with URL like this:
file:///D:/Projects/test.html
You're trying to post cross-domain, which you cannot do in this case. Serve your HTML over HTTP so that you can make a POST. If you use your browser's development tools, you will see that the request will never hit your Node.js server (except for a possible pre-flight request for CORS).
Another problem is that you're not actually using body-parser. If you want the post data, you will have to read from req like a stream.
You are including "body-parser" in var bodyParser = require('body-parser');, but you never actually use it. It won't magically parse the request for you. The default request object does not have a body property... see the documentation.
If you want to use the body-parser module, you should probably use express.js and read the documentation on how to connect it as middleware.
If you want to get the body using the built-in http server, you need to consume the request object first using something like this:
if (req.method == 'POST') {
var body = '';
req.on('data', function(data) {
body += data;
if (body.length > 1000000) {
req.connection.destroy();
}
});
req.on('end', function () {
console.log(req.body);
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
});
}
Adding express library and bodyparser as middleware did the trick. However I could use the code from neelsg answer but working with built-in http server and handling Post data by my own is too burdensome.
So piece of my working code is here:
var express = require('express');
var http = require('http');
var url = require('url');
var bodyParser = require('body-parser');
var app = express();
app.use(express.bodyParser(
{
keepExtensions: true,
limit: 30*1024*1024 // lets handle big data
}
));
app.use(bodyParser.urlencoded());
Bodyparser by default can handle only 100Kb data in the post request, so I had to increase limit using its config options.

Resources