I need to send BLOB to the server in order to make an image on same.
I am using axios on reactJs client and sending data by using this code.
/**
* Returns PDF document.
*
*/
getPDF = (blob) =>
{
let formatData = new FormData();
formatData.append('data', blob);
return axios({
method: 'post',
url: 'http://172.18.0.2:8001/export/pdf',
headers: { 'content-type': 'multipart/form-data' },
data: {
blob: formatData
}
}).then(response => {
return {
status: response.status,
data: response.data
}
})
}
I tried to console.log this blob value on client and there is regular data.
But on server request body is empty.
/**
* Exports data to PDF format route.
*/
app.post('/export/pdf', function (request, response) {
console.log(request.body.blob);
response.send('ok');
});
If I remove headers still empty body when sending blob, but if I remove blob and send some string, a server receives data.
But when the blob is sent server has an empty body.
NodeJS natively does not handle multipart/form-data so you have to use external module eg :- multer
Code Example(Not Tested):
var upload = multer({ dest: __dirname + '/public/uploads/' });
var type = upload.single('upl');
/**
* Exports data to PDF format route.
*/
app.post('/export/pdf', type, function (request, response) {
// Get the blob file data
console.log(request.file);
response.send('ok');
});
you can read about multer here
I hope this will work for you.
Are you using body-parser?
body-parser doesn't handle multipart bodies, which is what FormData is submitted as.
Instead, use a module like multer
let multer = require('multer');
let upload = multer();
app.post('/export/pdf', upload.fields([]), (req, res) => {
let formData = req.body;
console.log('Data', formData);
res.status(200).send('ok');
});
I had 2 problems that I had to solve for this. 1 firebase functions has a bug that doesn't allow multer. 2 you may be getting a blob back from response.blob() and that doesn't seem to produce a properly formatted blob for firebase functions either.
Related
I'm trying to send the filesI received using multer to another endpoint but on the other endpoint I get undefined req.file
- Here I have created a form and added the data I received to it, then I'm sending it to the other endpoint using axios
const body = new FormData();
body.append('file', Readable.from(req.files[i].buffer)),{
filename: req.files[i].originalname,
}
body.append('mimetype' , req.files[i].mimetype);
const response = await axios.post("http://localhost:8080/api/image/images/create", body, {
headers: {
"Content-Type": "multipart/form-data",
},
})
- Here I'm using multer in the other endpoint, however in the controller the file I get is undefined, I only receive the body of the request
const multer = require('multer')
const upload = multer({ dest: 'uploads/' })
imageRouter.post('/images/create', upload.single("file"), imageController.postImage);
Try not including headers, and also try appending file buffer without converting it to stream. Also, mimetype can be included in the file information object.
Try this:
const body = new FormData();
body.append('file', req.files[i].buffer, {
filename: req.files[i].originalname,
contentType: req.files[i].mimetype
})
const response = await axios.post("http://localhost:8080/api/image/images/create", body);
I have a scenario where I need to upload a .csv file from a react frontend, using axios, to an AWS Lambda function written in Node.js, through AWS API Gateway. From there I will process the lines of the csv file and save them to a database.
So far I've managed to get the data into the lambda function using the code below:
React Snippet
...
export const upload = async (file: any) => {
await axios({
method: "post",
url: url,
data: file,
headers: { "Content-Type": "multipart/form-data" },
})
.then(function (response) {
//handle success
console.log(response);
return response;
})
.catch(function (response) {
//handle error
console.log(response);
return response;
});
const upload = async () => {
setIsLoadng(true);
if (selectedFile) {
debugger;
const formData = new FormData();
formData.append("File", selectedFile);
await upload(formData);
setIsLoadng(false);
} else {
setIsLoadng(false);
alert("Please choose a file to upload.");
}
};
...
but it arrives encoded in base64. At this point this is how I extract the data from the lambda function:
AWS Lambda Function
exports.handler = async (event) => {
try {
console.log(`Event: ${JSON.stringify(event)}`);
// Decoding the base64 encoded body from AWS API Gateway
let buff = Buffer.from(event.body, "base64");
console.log(`Buffer data decoded: ${buff}`);
console.log(`Type: ${typeof buff}`);
const response = {
statusCode: 200,
body: JSON.stringify("Some data here..."),
};
return response;
} catch (err) {
console.log(err);
const response = {
statusCode: 200,
body: JSON.stringify(err),
};
return response;
}
};
This is all fine and well, however, after decoding it I cant seem to parse it easily to JSON format for processing because the data is surrounded by a webkit form boundray:
'------WebKitFormBoundary0YYQOz5kaQyRTGk4
Content-Disposition: form-data; name="File"; filename="file001.csv"
Content-Type: application/vnd.ms-excel
...
the actual .csv data
...
------WebKitFormBoundary0YYQOz5kaQyRTGk4--
'
I think that axios has formatted my data and sent the request correctly based on content-type: multipart/form-data. I also believe that AWS API Gateway encodes data like this into base64 which is what I receive in the lambda function. However, once I decode the data I get this string that includes all the header information and the "------WebKitFormBoundary0YYQOz5kaQyRTGk4" top and bottom bits.
Is there a better way to handle the .csv data so that I can have a nicely parsed JSON object in my lambda function to then process. So far I've been trying to hack together a string "find and concat" solution but this feels so wrong.
Any help or guidance is sincerely appreciated!
B
So I finally managed to parse the data received from AWS API Gateway and needed a body parsing library to get the .csv data I needed as Tomalak said.
Here's what I did in the end:
Set up AWS API Gateway with Proxy Integration which have me the .csv data in a base64 encoded format in the lambda function.
Used the node "Buffer" to decode the data from base64
To parse the data I used a library called "multipart"
Then I converted the data part of the converted FormData (I only had one file so I called the file at index 0) to JSON using a library called "csvtojson"
From there I had a nicely formatted JSON object to work with. I didnt realise you could pull the Boundry information directly from the header which you will need to parse the data with the "parse-multipart" library.
Lambda Code Snippet
const multipart = require("parse-multipart");
const csv = require("csvtojson");
exports.handler = async event => {
console.log(`Event: ${JSON.stringify(event)}`);
const body = Buffer.from(event["body"].toString(), "base64"); // AWS case
const boundary = multipart.getBoundary(event.headers["Content-Type"]);
const buff = multipart.Parse(body, boundary);
const csvDataString = buff[0].data.toString("utf8");
const csvData = await csv().fromString(csvDataString);
console.log(csvData);
}
I am struggling with a simple media (mp3/mp4) upload to a server using axios.
I have an angular application that creates a formData and send this formData to node server via :
return this.http.post(this.apiURL + '/uploadFile', formData);
My server method looks like this :
app.post('/api/uploadFile', upload.single('file'), (req, res) => {
inputFile = req.file;
let fd = new FormData();
fd.append('file',inputFile.buffer, inputFile.originalname);
axios.post(uploadFileURL , fd, { headers: { 'Content-Type': 'multipart/form-data' } })
.then((response) => {
console.log(response);
})
.catch((error) => {
console.error(error)
})
})
The inputFile contains the original files. The error I get now is that the request is not a multipart request...
I tried as well to define the formData differently :
formData = {
file: {
value: inputFile.buffer,
options: {
filename: inputFile.originalname,
contentType: inputFile.mimetype
}
}
};
Which brought me to a different error : 'Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: the request was rejected because no multipart boundary was found'
Am I doing something wrong ?
I am wondering if this could be link to the fact that I use const bodyParser = require('body-parser'); for some of my other requests.
Any help would be appreciated!
Thanks
EDIT :
Here is my need and what I've done so far :
I have a web application that allow users to upload media files.
I have to send those files to a server, but I can not use the browser to send the request directly.
I created a nodejs application to realize the proxy task of getting the files from the browser and sending it to my remote server.
I've been trying to asynchronously send a Blob image to a REST Api using the request module and Azure Storage module. I don't want to download the Blob to a local file and then create a Readable stream from the local file because it's not performant. This is what I have attempted, but it is throwing the error "Unexpected end of MIME multipart stream. MIME multipart message is not complete." From the request docs, sending a file in the form data requires you pass it a Readable Stream. It seems the Readable Stream from the Azure Storage client isn't compatible with the request module's format. Any ideas how to get this to work?
const request = require('request');
const storage = require('azure-storage');
const blobService = storage.createBlobService(process.env.AzureWebJobsStorage);
let stream = blobService.createReadStream(
containerName,
blobName,
function(err, res) {
});
let formData = {
rootMessageId: messageId,
file: stream
};
request.post({
url:'https://host-name/Api/comment',
headers: {'Authorization': `Token ${authToken}`},
formData: formData
}, (err, res, body) => {
console.log(res)
}
});
I tried to use your code to upload an image blob to my owner local url http://localhost/upload, then I found there is missing some properties in the file property of your formData.
Here is my code works.
const request = require('request');
const storage = require('azure-storage');
var accountName = '<your storage account name>';
var accountKey = '<your storage account name>';
var blobService = storage.createBlobService(accountName, accountKey);
let stream = blobService.createReadStream(containerName, blobName, function(err, res){
formdata.file.options.contentType = res.contentSettings.contentType;
console.log(formdata);
});
var formdata = {
rootMessageId: messageId,
file: { // missing some properties
value: stream,
options: {
filename: function(blobName) {
var elems = blobName.split('/');
return elems[elems.length-1];
}(blobName),
knownLength: stream // a required property of `file` is `knownLength` which will cause server error if be missed.
},
}
}
request.post({
url: 'https://host-name/Api/comment', // I used my url `http://localhost/upload` at here
headers: {'Authorization': `Token ${authToken}`}, // I used a empty {} as header at here
formData: formdata
}, (err, res, body) => {
console.log(res)
}
});
Thinking for the code above, it must pipe a download stream to an upload stream and all data also need to flow through your webapp machine. Per my experience, I think you can generate a SAS url of a blob to post to your REST API and then download the blob via your REST server if you can change the code of your REST application server.
How to create a post service with formdata?
I sent formdata by Axios.
However, the value of 'req.body.title' on the node-express server is empty.
So now I am sending fetch in the following format.
But I need to upload the file to the server, so I want to send it using formData.
let bodys = 'title=a1&contents=b'
fetch("http://localhost:5000/test", {
method : 'post',
headers : {
'Content-type' : 'application/x-www-form-urlencoded; charset=UTF-8'
},
body: bodys
})
.then(function(response){
console.log('Request Succeded ', response);
})
.catch(function (error){
console.log('Failed ', error)
})
I wrote new data using append with new FormData(),
I checked that FormData contains a value on React.
However, the node-express server did not enter the body.
please let me know what to do...
Try sending FormData object instead of raw string as your request body.
const bodys = new FormData();
bodys.append('title', 'a1');
bodys.append('contents', 'b');
This form data will be available in request.body in express.js server.
Edit: to parse the FormData in express.js, you need a middleware like multer
const upload = require('multer');
app.use('/', upload.any(), yourRouteHandler);
You are sending just a string so
You can access your body like below
let bodys = 'title=a1&contents=b'
console.log(req.body); //will print title and contents as you are sending
If you want to access title and contents separately then you have to send data as an object
const bodys = {“title”: “a1”, “contents”: “b”}
console.log(“title”, req.body.title); //will print a1
console.log(“contents”, req.body.contents); //will print b
Chec this thread for more details https://github.com/github/fetch/issues/263