Sending BLOB to nodejs API results in empty body on server - node.js

I need to send BLOB to the server in order to make an image on same.
I am using axios on reactJs client and sending data by using this code.
/**
* Returns PDF document.
*
*/
getPDF = (blob) =>
{
let formatData = new FormData();
formatData.append('data', blob);
return axios({
method: 'post',
url: 'http://172.18.0.2:8001/export/pdf',
headers: { 'content-type': 'multipart/form-data' },
data: {
blob: formatData
}
}).then(response => {
return {
status: response.status,
data: response.data
}
})
}
I tried to console.log this blob value on client and there is regular data.
But on server request body is empty.
/**
* Exports data to PDF format route.
*/
app.post('/export/pdf', function (request, response) {
console.log(request.body.blob);
response.send('ok');
});
If I remove headers still empty body when sending blob, but if I remove blob and send some string, a server receives data.
But when the blob is sent server has an empty body.

NodeJS natively does not handle multipart/form-data so you have to use external module eg :- multer
Code Example(Not Tested):
var upload = multer({ dest: __dirname + '/public/uploads/' });
var type = upload.single('upl');
/**
* Exports data to PDF format route.
*/
app.post('/export/pdf', type, function (request, response) {
// Get the blob file data
console.log(request.file);
response.send('ok');
});
you can read about multer here
I hope this will work for you.

Are you using body-parser?
body-parser doesn't handle multipart bodies, which is what FormData is submitted as.
Instead, use a module like multer
let multer = require('multer');
let upload = multer();
app.post('/export/pdf', upload.fields([]), (req, res) => {
let formData = req.body;
console.log('Data', formData);
res.status(200).send('ok');
});

I had 2 problems that I had to solve for this. 1 firebase functions has a bug that doesn't allow multer. 2 you may be getting a blob back from response.blob() and that doesn't seem to produce a properly formatted blob for firebase functions either.

Related

Send req.file to another EndPoint using multer result is undefined

I'm trying to send the filesI received using multer to another endpoint but on the other endpoint I get undefined req.file
- Here I have created a form and added the data I received to it, then I'm sending it to the other endpoint using axios
const body = new FormData();
body.append('file', Readable.from(req.files[i].buffer)),{
filename: req.files[i].originalname,
}
body.append('mimetype' , req.files[i].mimetype);
const response = await axios.post("http://localhost:8080/api/image/images/create", body, {
headers: {
"Content-Type": "multipart/form-data",
},
})
- Here I'm using multer in the other endpoint, however in the controller the file I get is undefined, I only receive the body of the request
const multer = require('multer')
const upload = multer({ dest: 'uploads/' })
imageRouter.post('/images/create', upload.single("file"), imageController.postImage);
Try not including headers, and also try appending file buffer without converting it to stream. Also, mimetype can be included in the file information object.
Try this:
const body = new FormData();
body.append('file', req.files[i].buffer, {
filename: req.files[i].originalname,
contentType: req.files[i].mimetype
})
const response = await axios.post("http://localhost:8080/api/image/images/create", body);

Reading .csv data after uploading via a POST request to an AWS Lambda Node.js function

I have a scenario where I need to upload a .csv file from a react frontend, using axios, to an AWS Lambda function written in Node.js, through AWS API Gateway. From there I will process the lines of the csv file and save them to a database.
So far I've managed to get the data into the lambda function using the code below:
React Snippet
...
export const upload = async (file: any) => {
await axios({
method: "post",
url: url,
data: file,
headers: { "Content-Type": "multipart/form-data" },
})
.then(function (response) {
//handle success
console.log(response);
return response;
})
.catch(function (response) {
//handle error
console.log(response);
return response;
});
const upload = async () => {
setIsLoadng(true);
if (selectedFile) {
debugger;
const formData = new FormData();
formData.append("File", selectedFile);
await upload(formData);
setIsLoadng(false);
} else {
setIsLoadng(false);
alert("Please choose a file to upload.");
}
};
...
but it arrives encoded in base64. At this point this is how I extract the data from the lambda function:
AWS Lambda Function
exports.handler = async (event) => {
try {
console.log(`Event: ${JSON.stringify(event)}`);
// Decoding the base64 encoded body from AWS API Gateway
let buff = Buffer.from(event.body, "base64");
console.log(`Buffer data decoded: ${buff}`);
console.log(`Type: ${typeof buff}`);
const response = {
statusCode: 200,
body: JSON.stringify("Some data here..."),
};
return response;
} catch (err) {
console.log(err);
const response = {
statusCode: 200,
body: JSON.stringify(err),
};
return response;
}
};
This is all fine and well, however, after decoding it I cant seem to parse it easily to JSON format for processing because the data is surrounded by a webkit form boundray:
'------WebKitFormBoundary0YYQOz5kaQyRTGk4
Content-Disposition: form-data; name="File"; filename="file001.csv"
Content-Type: application/vnd.ms-excel
...
the actual .csv data
...
------WebKitFormBoundary0YYQOz5kaQyRTGk4--
'
I think that axios has formatted my data and sent the request correctly based on content-type: multipart/form-data. I also believe that AWS API Gateway encodes data like this into base64 which is what I receive in the lambda function. However, once I decode the data I get this string that includes all the header information and the "------WebKitFormBoundary0YYQOz5kaQyRTGk4" top and bottom bits.
Is there a better way to handle the .csv data so that I can have a nicely parsed JSON object in my lambda function to then process. So far I've been trying to hack together a string "find and concat" solution but this feels so wrong.
Any help or guidance is sincerely appreciated!
B
So I finally managed to parse the data received from AWS API Gateway and needed a body parsing library to get the .csv data I needed as Tomalak said.
Here's what I did in the end:
Set up AWS API Gateway with Proxy Integration which have me the .csv data in a base64 encoded format in the lambda function.
Used the node "Buffer" to decode the data from base64
To parse the data I used a library called "multipart"
Then I converted the data part of the converted FormData (I only had one file so I called the file at index 0) to JSON using a library called "csvtojson"
From there I had a nicely formatted JSON object to work with. I didnt realise you could pull the Boundry information directly from the header which you will need to parse the data with the "parse-multipart" library.
Lambda Code Snippet
const multipart = require("parse-multipart");
const csv = require("csvtojson");
exports.handler = async event => {
console.log(`Event: ${JSON.stringify(event)}`);
const body = Buffer.from(event["body"].toString(), "base64"); // AWS case
const boundary = multipart.getBoundary(event.headers["Content-Type"]);
const buff = multipart.Parse(body, boundary);
const csvDataString = buff[0].data.toString("utf8");
const csvData = await csv().fromString(csvDataString);
console.log(csvData);
}

Node, Multer & Axios : Sending a media to server

I am struggling with a simple media (mp3/mp4) upload to a server using axios.
I have an angular application that creates a formData and send this formData to node server via :
return this.http.post(this.apiURL + '/uploadFile', formData);
My server method looks like this :
app.post('/api/uploadFile', upload.single('file'), (req, res) => {
inputFile = req.file;
let fd = new FormData();
fd.append('file',inputFile.buffer, inputFile.originalname);
axios.post(uploadFileURL , fd, { headers: { 'Content-Type': 'multipart/form-data' } })
.then((response) => {
console.log(response);
})
.catch((error) => {
console.error(error)
})
})
The inputFile contains the original files. The error I get now is that the request is not a multipart request...
I tried as well to define the formData differently :
formData = {
file: {
value: inputFile.buffer,
options: {
filename: inputFile.originalname,
contentType: inputFile.mimetype
}
}
};
Which brought me to a different error : 'Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: the request was rejected because no multipart boundary was found'
Am I doing something wrong ?
I am wondering if this could be link to the fact that I use const bodyParser = require('body-parser'); for some of my other requests.
Any help would be appreciated!
Thanks
EDIT :
Here is my need and what I've done so far :
I have a web application that allow users to upload media files.
I have to send those files to a server, but I can not use the browser to send the request directly.
I created a nodejs application to realize the proxy task of getting the files from the browser and sending it to my remote server.

Azure Blob Storage - Node.JS - createReadStream() as file in POST Request Form

I've been trying to asynchronously send a Blob image to a REST Api using the request module and Azure Storage module. I don't want to download the Blob to a local file and then create a Readable stream from the local file because it's not performant. This is what I have attempted, but it is throwing the error "Unexpected end of MIME multipart stream. MIME multipart message is not complete." From the request docs, sending a file in the form data requires you pass it a Readable Stream. It seems the Readable Stream from the Azure Storage client isn't compatible with the request module's format. Any ideas how to get this to work?
const request = require('request');
const storage = require('azure-storage');
const blobService = storage.createBlobService(process.env.AzureWebJobsStorage);
let stream = blobService.createReadStream(
containerName,
blobName,
function(err, res) {
});
let formData = {
rootMessageId: messageId,
file: stream
};
request.post({
url:'https://host-name/Api/comment',
headers: {'Authorization': `Token ${authToken}`},
formData: formData
}, (err, res, body) => {
console.log(res)
}
});
I tried to use your code to upload an image blob to my owner local url http://localhost/upload, then I found there is missing some properties in the file property of your formData.
Here is my code works.
const request = require('request');
const storage = require('azure-storage');
var accountName = '<your storage account name>';
var accountKey = '<your storage account name>';
var blobService = storage.createBlobService(accountName, accountKey);
let stream = blobService.createReadStream(containerName, blobName, function(err, res){
formdata.file.options.contentType = res.contentSettings.contentType;
console.log(formdata);
});
var formdata = {
rootMessageId: messageId,
file: { // missing some properties
value: stream,
options: {
filename: function(blobName) {
var elems = blobName.split('/');
return elems[elems.length-1];
}(blobName),
knownLength: stream // a required property of `file` is `knownLength` which will cause server error if be missed.
},
}
}
request.post({
url: 'https://host-name/Api/comment', // I used my url `http://localhost/upload` at here
headers: {'Authorization': `Token ${authToken}`}, // I used a empty {} as header at here
formData: formdata
}, (err, res, body) => {
console.log(res)
}
});
Thinking for the code above, it must pipe a download stream to an upload stream and all data also need to flow through your webapp machine. Per my experience, I think you can generate a SAS url of a blob to post to your REST API and then download the blob via your REST server if you can change the code of your REST application server.

React, node js, mongoDB - How to make post, put, delete with FormData?

How to create a post service with formdata?
I sent formdata by Axios.
However, the value of 'req.body.title' on the node-express server is empty.
So now I am sending fetch in the following format.
But I need to upload the file to the server, so I want to send it using formData.
let bodys = 'title=a1&contents=b'
fetch("http://localhost:5000/test", {
method : 'post',
headers : {
'Content-type' : 'application/x-www-form-urlencoded; charset=UTF-8'
},
body: bodys
})
.then(function(response){
console.log('Request Succeded ', response);
})
.catch(function (error){
console.log('Failed ', error)
})
I wrote new data using append with new FormData(),
I checked that FormData contains a value on React.
However, the node-express server did not enter the body.
please let me know what to do...
Try sending FormData object instead of raw string as your request body.
const bodys = new FormData();
bodys.append('title', 'a1');
bodys.append('contents', 'b');
This form data will be available in request.body in express.js server.
Edit: to parse the FormData in express.js, you need a middleware like multer
const upload = require('multer');
app.use('/', upload.any(), yourRouteHandler);
You are sending just a string so
You can access your body like below
let bodys = 'title=a1&contents=b'
console.log(req.body); //will print title and contents as you are sending
If you want to access title and contents separately then you have to send data as an object
const bodys = {“title”: “a1”, “contents”: “b”}
console.log(“title”, req.body.title); //will print a1
console.log(“contents”, req.body.contents); //will print b
Chec this thread for more details https://github.com/github/fetch/issues/263

Resources