I have a Node.js (16.13.1) REST API using Express and one of my endpoints receives one or more uploaded files. The client (web app) uses FormData into which the files are appended. Once they're submitted to my API, the code there uses multer to grab the files from the request object.
Now I'm having trouble trying to send those same files to another API. multer attaches the files to req.files and each file object in that array has several properties one of which is buffer. I tried using the stream package's Duplex object to convert this buffer to a stream so that I could append the file to another FormData object, but when the server the second API is running on receives the request, I get an error from the web server saying that "a potentially dangerous request.form value was detected from the client.".
Any suggestions?
I am working on a nest project I was also facing this issue did some research and found that we need to create a Readable from the Buffer of that file and it's working for me.
// Controller
#UseInterceptors(FileInterceptor('file'))
async uploadFile(#UploadedFile() file: Express.Multer.File) {
return this.apiservice.upload(file);
}
// Service
uploadFile(file: Express.Multer.File) {
const readstream = Readable.from(file.buffer)
console.log(readstream)
const form = new FormData();
form.append('file', file, { filename: extra.filename });
const url = `api_endpoint`;
const config: AxiosRequestConfig = {
headers: {
'Content-Type': 'multipart/form-data'
},
};
return axios.post(url, form, config);
}
Related
I am in a Node.JS application. (Node version 14 if that matters)
Also I am using node-fetch (v2) to send my request.
I am trying to send a Send Fax Request to Documo. The documentation can be found here: https://docs.documo.com/#da8dc725-8327-470c-83e4-34b40205cfa2.
As part of the request I want to send a pdf file that I have stored in S3. I have the following code to accomplish that:
const s3Object = await s3Client.send(
new GetObjectCommand({
Bucket: bucket,
Key: key,
})
const formData = new FormData();
formData.append("faxNumber", faxNumber);
formData.append("coverPage", "true");
formData.append("recipientName", recipientName);
formData.append("senderName", senderName);
formData.append("subject", subject);
formData.append("notes", note);
formData.append("webhookId", webhookId);
formData.append("attachments", s3Object.Body);
// ERROR: File extension 'pdf%22&x-id=getobject' is not allowed. Allowed formats: doc,docx,fodt,gif,htm,html,jpeg,jpg,odp,ods,odt,pdf,png,ppt,pptx,rtf,tif,tiff,txt,xls,xlsx,csv
const response = await fetch(`${BASE_DOCUMO_URL}/v1/faxes`, {
method: "POST",
headers: {
Authorization: DEMO_API_KEY,
ContentType: "multipart/form-data",
},
body: formData,
});
I am getting the following error from Documo:
"File extension 'pdf%22&x-id=getobject' is not allowed. Allowed formats: doc,docx,fodt,gif,htm,html,jpeg,jpg,odp,ods,odt,pdf,png,ppt,pptx,rtf,tif,tiff,txt,xls,xlsx,csv"
From what I understand it looks like the GetObjectCommand from S3 is appending an x-id in the file stream which the Documo client is not happy with. Ideally I don't want to recreate this file in memory and I just want to take the result from S3 to send through my POST request. (Although I could be convinced to just do that if there is no better option or I don't need to worry about holding files in memory).
What are my options. I've tried playing around with ResponseContentDisposition in GetObjectCommand to no avail
I use Postman to POST a binary (Excel) file to my Node.js (Typescript) Azure function, which PUTs the body of the request on to Microsoft Graph API as a new file upload - so it acts as a proxy for an upload to OneDrive using the Graph API. I use axios rather than the Graph client.
The file is created at the expected location in OneDrive but the file data is not identical to the original - and when opened in Excel, it sees it as corrupted.
So sending it via Postman:
this is the (simplified) Typescript code receiving it and passing it onto Graph API:
const httpTrigger: AzureFunction = async function (context: Context, req: HttpRequest): Promise<void> {
const fileBytes = req.body;
const config:AxiosRequestConfig = {
headers: {
'Authorization': `Bearer ${bearerToken}`,
'Content-Type': 'text/plain'
}
}
const url:string = `https://graph.microsoft.com/v1.0/me/drive/root:/Testing/TDA/Test.xlsx:/content`;
const uploadResult:AxiosResponse<any, any> = await axios.put(url, fileBytes, config);
}
The file size is larger (22KB vs the original 18KB) and the encoding looks to be different as opening the original and its new copy in OneDrive in Notepad++ shows there are differences (one example highlighted in green):
What am I missing here? Is there a way I can ensure the format/encoding of the file uploaded and saved into OneDrive through Microsoft Graph API ends up being identical (in binary terms) to the original file sent through Postman?
Cheers - Nicolaas
so I need to download a file from S3 bucket and then, with either its buffer or readStream, append it to a FormData on node. I've tried different libraries such as formdata-node, isomorphic-form-data and form-data. Only formdata-node printed a FormData type on the log, while still having no values attached to it, and the others crashed as soon as I tried to instantiate a FormData.
const form = new FormData();
form.set("UsuarioInclusaoDR", body.UsuarioInclusaoDR);
form.set(
"TipoDocumentoDigitalizadoDR",
body.TipoDocumentoDigitalizadoDR
);
form.set(
"DescDocumentoDigitalizado",
body.DescDocumentoDigitalizado
);
form.set("TipoDocumento", fileType);
form.set("NomeDocumento", file.originalname);
form.set("Documento", {
name: file.originalname,
[Symbol.toStringTag]: "File",
stream() {
return s3Object;
},
});
I've tried removing the "Documento" form.set and it also didn't work. Any clues on what's impeding my FormData to get values set, or appended (i've tried both?)
Also, I'm really struggling trying to get a valid File from my S3 Buffer, is there any way to do that?
I'm using Axios npm package in my Node.js application which is a built-in microservices architecture. API-gateway is using Axios to call other services.
According to axios documentation, for POST method, request format is like: axios#post(url[, data[, config]]) . means only one argument data.
Can anyone tell me how I can send req.body & req.files data separately to Axios?
According to answer in Axios community to familiar question this could be achieved with the npm package form-data, which simulates WebAPI FormData on the node side.
This would allow you to send files and form fields to the API and receive it in form of multipart form data.
const axios = require('axios');
const FormData = require('form-data');
const form = new FormData();
// Second argument can take Buffer or Stream (lazily read during the request) too.
// Third argument is filename if you want to simulate a file upload. Otherwise omit.
form.append('field', 'a,b,c', 'blah.csv');
axios.post('http://example.org/endpoint', form, {
headers: form.getHeaders(),
})
.then(result => {
// Handle result…
console.log(result.data);
});
Bundle your data up as an object, and pass that object through to the axios request as a single payload.
On the other side of the API, you can access the object, and do as you wish.
var payload = { property1: 'values', property2: 'values2' }
now you pass payload around
I am trying to add attachments to existing invoices in xero.
I am using xero-node sdk (https://github.com/XeroAPI/xero-node#readme) for this integration and they provide a method for adding attachment as follows:
this.xero.accountingApi.createInvoiceAttachmentByFileName(tenantId, invoiceid, filenameInvoice,includeOnline,readStream )
The issue here is it requires an fs.ReadStream object for readStream.
The file I am trying to upload is present in cloud and I cannot download it and store it in file system before sending to Xero. I want to send the file present in azure cloud directly to xero. I have the url of file so I can get the content as a variable by making http request but there is no option to send this content to Xero.
There is an API available for this as well (here https://developer.xero.com/documentation/api/attachments) apart from the sdk. But I am not sure how I can send the file that I have to this API in body as it expects RAW data. Are there any specific headers or encodings required to call this API with file content in body? Because this is also not working for me if I just pass the body of the response I got from azure file url, as body to this Xero Attachment API. It tries for a long time and gives timeout error.
yes you are correct. There are additional headers/manipulation you need to do to upload files.
Please checkout the sample app - we've got it queued up to show exactly how to upload files: https://github.com/XeroAPI/xero-node-oauth2-app/blob/master/src/app.ts#L1188
Something like the following should get you sorted:
import * as fs from "fs";
const path = require("path");
const mime = require("mime-types");
const totalInvoices = await xero.accountingApi.getInvoices('your-tenantId-uuid', undefined, undefined, undefined, undefined, undefined, undefined, ['PAID']);
// Attachments need to be uploaded to associated objects https://developer.xero.com/documentation/api/attachments
// CREATE ATTACHMENT
const filename = "xero-dev.png";
const pathToUpload = path.resolve(__dirname, "../path-to-your.png");
const readStream = fs.createReadStream(pathToUpload);
const contentType = mime.lookup(filename);
const fileAttached = await xero.accountingApi.createInvoiceAttachmentByFileName(req.session.activeTenant.tenantId, totalInvoices.body.invoices[0].invoiceID, filename, true, readStream, {
headers: {
"Content-Type": contentType,
},
});
I ended up adding the link to file in History and Notes section of the invoice. Even though this is not the best solution, It serves the purpose of showing invoices to the customer.
Thanks to #SerKnight for your answer.