NodeJS, Axios - post file from local server to another server - node.js

I have an API endpoint that lets the client post their csv to our server then post it to someone else server. I have done our server part which save uploaded file to our server, but I can't get the other part done. I keep getting error { message: 'File not found', code: 400 } which may mean the file never reach the server. I'm using axios as an agent, does anyone know how to get this done? Thanks.
// file = uploaded file
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
Update
As axios doc states as below and the API I'm trying to call requires a file
// data is the data to be sent as the request body
// Only applicable for request methods 'PUT', 'POST', and 'PATCH'
// When no transformRequest is set, must be of one of the following types:
// - string, plain object, ArrayBuffer, ArrayBufferView, URLSearchParams
// - Browser only: FormData, File, Blob
// - Node only: Stream, Buffer
Is there any way to make axios send a file as a whole? Thanks.

The 2 oldest answers did not work for me. This, however, did the trick:
const FormData = require('form-data'); // npm install --save form-data
const form = new FormData();
form.append('file', fs.createReadStream(file.path));
const request_config = {
headers: {
'Authorization': `Bearer ${access_token}`,
...form.getHeaders()
}
};
return axios.post(url, form, request_config);
form.getHeaders() returns an Object with the content-type as well as the boundary.
For example:
{ "content-type": "multipart/form-data; boundary=-------------------0123456789" }

I'm thinking the createReadStream is your issue because its async. try this.
Since createReadStream extends the event emitter, we can "listen" for when it finishes/ends.
var newFile = fs.createReadStream(file.path);
// personally I'd function out the inner body here and just call
// to the function and pass in the newFile
newFile.on('end', function() {
const form_data = new FormData();
form_data.append("file", newFile, "filename.ext");
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
});

This is what you really need:
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios
.post(url, form_data, request_config);

In my case, fs.createReadStream(file.path) did not work.
I had to use buffer instead.
const form = new FormData();
form.append('file', fs.readFileSync(filePath), fileName);
const config = {
headers: {
Authorization: `Bearer ${auth.access_token}`,
...form.getHeaders(),
},
};
axios.post(api, form.getBuffer(), config);

I have made an interceptor you can connect to axios to handle this case in node: axios-form-data. Any feedback would be welcome.
npm i axios-form-data
example:
import axiosFormData from 'axios-form-data';
import axios from 'axios';
// connect axiosFormData interceptor to axios
axios.interceptors.request.use(axiosFormData);
// send request with a file in it, it automatically becomes form-data
const response = await axios.request({
method: 'POST',
url: 'http://httpbin.org/post',
data: {
nonfile: 'Non-file value',
// if there is at least one streamable value, the interceptor wraps the data into FormData
file: createReadStream('somefile'),
},
});
// response should show "files" with file content, "form" with other values
// and multipart/form-data with random boundary as request header
console.log(response.data);

I had a same issue, I had a "pdf-creator-service" for generate PDF document from html.
I use mustache template engine for create HTML document - https://www.npmjs.com/package/mustache
Mustache.render function returns html as a string what do I need to do to pass it to the pdf-generator-service ? So lets see my suggestion bellow
//...
async function getPdfDoc(props: {foo: string, bar: string}): Promise<Buffer> {
const temlateFile = readFileSync(joinPath(process.cwd(), 'file.html'))
mustache.render(temlateFile, props)
const readableStream = this.getReadableStreamFromString(htmlString)
const formData = new FormData() // from 'form-data'
formData.append('file', options.file, { filename: options.fileName })
const formHeaders = formData.getHeaders()
return await axios.send<Buffer>(
{
method: 'POST',
url: 'https://pdf-generator-service-url/pdf',
data: formData,
headers: {
...formHeaders,
},
responseType: 'arraybuffer', // ! important
},
)
}
getReadableStreamFromString(str: string): Readable {
const bufferHtmlString = Buffer.from(str)
const readableStream = new Readable() // from 'stream'
readableStream._read = () => null // workaround error
readableStream.push(bufferHtmlString)
readableStream.push(null) // mark end of stream
return readableStream
}

For anyone who wants to upload files from their local filesystem (actually from anywhere with the right streams architecture) with axios and doesn't want to use any external packages (like form-data).
Just create a readable stream and plug it right into axios request function like so:
await axios.put(
url,
fs.createReadStream(path_to_file)
)
Axios accepts data argument of type Stream in node context.
Works fine for me at least in Node v.16.13.1 and with axios v.0.27.2

Related

Axios : send image with FormData on Prestashop API

I'm trying to upload on image on the Prestashop API with form-data/axios.
For that, i just need to send a post request with the images joined in an "image" parameter.
I did this simple node.js script (mine is much more complicated but i simplified for here):
const FormData = require("form-data");
const fs = require("fs");
const axios = require("axios");
// Read the image file into a buffer
const imageBuffer = fs.readFileSync("image.jpg");
// Create a FormData object
const form = new FormData();
// Append the image buffer to the form data
form.append("image", imageBuffer);
// Make an HTTP POST request to the PrestaShop API
axios({
method: "POST",
url: "https://XXX/api/images/products/15924",
data: form, // set the request body using the data field
params: {
ws_key: "XXX",
},
headers: {
"Content-Type": "multipart/form-data; boundary=" + form.getBoundary(),
},
})
.then((response) => {
console.log(response);
})
.catch((error) => {
console.error(error.response.data);
});
But i get this answer:
<message><![CDATA[Please set an "image" parameter with image data for value]]></message>
I tried a LOT of things. With fs.createReadStream instead of formData, with http instead of axios, with a buffer or a file, etc ... and i alway end with this error.
I would be glad if someone has an idea :-)
Thx !
I finally succeed!
Two changed were needed:
Adding the filename in the append (thanks Dmitriy Mozgovoy for pointing that out in the comment)
Adding the size in the headers with "Content-Length": formData.getLengthSync()
Without these 2 missing elements, it seems that PHP received an empty $_FILES variable.
Final version:
const FormData = require("form-data");
const fs = require("fs");
const axios = require("axios");
// Read the image file into a buffer
const imageBuffer = fs.readFileSync("image.jpg");
// Create a FormData object
const form = new FormData();
// Append the image buffer to the form data
form.append("image", imageBuffer, "image.jpg");
// Make an HTTP POST request to the PrestaShop API
axios
.post("https://example.com/api/images/products/15924", form, {
params: {
ws_key: "EXAMPLE",
},
headers: {
...form.getHeaders(),
"Content-Length": form.getLengthSync(),
},
})
.then((response) => {
console.log(response);
})
.catch((error) => {
console.error(error.response.data);
});

Cannot upload document via Telegram bot API

I'm using NestJS, there is axios module inside it. I read that Telegram accepts only multipart/form-data for files, so I use form-data package. I have only 400 errors, that don't represent anything.
I don't want to use packages like telegraf (it's awesome, I know) for this simple usecase.
const config: AxiosRequestConfig = {
headers: { 'Content-Type': 'multipart/form-data' },
};
const file = await readFile(path);
const body = new FormData();
body.append('chat_id', this.chatId.toString());
body.append('document', file, { filename: 'document.pdf' });
try {
const res = await firstValueFrom(this.http.post(`${this.tgURL}/sendDocument`, body, config));
} catch (e) {
console.log(e.message);
}
Telegram needs boundary in header.
So:
const config: AxiosRequestConfig = {
headers: { 'Content-Type': `multipart/form-data; boundary=${body.getBoundary()}` },
};

Grabbing an image from a URL, then passing it to an API as a file in multipart form data

So I have a URL that contains an image, and I want to pass that image as part of multipart form data to an API (to be specific, if it matters, the ClickUp API). I'm doing all of this inside of a Figma plugin, which is a browser environment.
The url looks something like https://s3-alpha-sig.figma.com....
The request works perfectly for a local image that I add manually, such as in Postman. Here is the code for a successful Postman request to this endpoint:
var axios = require('axios');
var FormData = require('form-data');
var fs = require('fs');
var data = new FormData();
data.append('attachment', fs.createReadStream('7UI7S5-pw/fdb54856-9c05-479f-b726-016ef252d9f5.png'));
data.append('filename', 'example.png');
var config = {
method: 'post',
url: 'https://api.clickup.com/api/v2/task/2phh5bf/attachment',
headers: {
'Authorization': '(my auth token)',
...data.getHeaders()
},
data : data
};
axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
However, I don't have access to local files and need to upload from a URL, so here's what I've done so far:
var data = new FormData();
data.append('attachment', open(imgURL));
data.append('filename', 'screenshot.png');
fetch(`(the URL)`, {
"method": "POST",
"muteHttpExceptions": true,
"headers": {
'Authorization': '(my auth token)',
...data.headers
},
data: data
}).then(response => {
console.log(response)
})
How should I be converting the URL into something I can input as Form Data? Thanks so much in advance!

Autodesk Forge - Reality Capture Issue - Specified Photoscene ID doesn't exist in the database

i'm trying to upload my files as form-data, after i've created a scene. But I receive always the error "Specified Photoscene ID doesn't exist in the database" (which were created directly before).
My upload function:
// Upload Files
async function uploadFiles(access_Token, photoSceneId, files) {
try {
const params = new URLSearchParams({
'photosceneid': photoSceneId,
'type': 'image',
'file': files
})
const headers = Object.assign({
Authorization: 'Bearer ' + access_Token,
'Content-Type': 'multipart/form-data' },
files.getHeaders()
)
let resp = await axios({
method: 'POST',
url: 'https://developer.api.autodesk.com/photo-to-3d/v1/file',
headers: headers,
data: params
})
let data = resp.data;
return data;
} catch (e) {
console.log(e);
}
};
I've also tried a few varaints, e.g. adding the photosceneId to the form data (form.append(..), but doesn't works either.
Any helpful suggestion are appreciated. Thx in advance.
There might be two problems here.
First, I am not sure of it, as I don't have experience of URLSearchParams as a "packer" for POST requests. This might be the reason why you get "Specified Photoscene ID doesn't exist in the database" error - perhaps the way the data are serialized using URLSearchParams is not compatible.
The second problem, I am sure of it, is regarding the way you submit the files.
According to documentation, you have to pass the files one by one, like
"file[0]=http://www.autodesk.com/_MG_9026.jpg" \
"file[1]=http://www.autodesk.com/_MG_9027.jpg"
and not just passing an array to the "file" field.
Having this said, try this approach:
var axios = require('axios');
var FormData = require('form-data');
var fs = require('fs');
var data = new FormData();
var TOKEN = 'some TOKEN';
const photoSceneID = 'some_photoscene_id';
data.append('photosceneid', photoSceneID);
data.append('type', 'image');
data.append('file[0]', fs.createReadStream('/C:/TEMP/Example/DSC_5427.JPG'));
data.append('file[1]', fs.createReadStream('/C:/TEMP/Example/DSC_5428.JPG'));
data.append('file[2]', fs.createReadStream('... and so on ...'));
var config = {
method: 'post',
url: 'https://developer.api.autodesk.com/photo-to-3d/v1/file',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'Bearer ' + TOKEN,
},
data : data
};
axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});
Also, I always recommend instead of jumping right into the code, to check first the workflow using apps like Postman or Insomnia and then, after you validated the workflow (created the photoscene, all images were properly uploaded and so on), you can translate this into the code.
At the end of this blogpost you will find the link to alrady created Postman collection, but I highly recommend building your own collection, as part of the learning step.
This is the solution that worked for me. Please note that the upload should be limited by a maximum of 20 files per call.
// Upload Files
async function uploadFiles(access_Token, photoSceneId) {
try {
let dataPath = path.join(__dirname, '../../data')
let files = fs.readdirSync(dataPath)
var data = new FormData();
data.append('photosceneid', photoSceneId)
data.append('type', 'image')
for(let i=0; i < files.length; i++) {
let filePath = path.join(dataPath, files[i])
let fileName = 'file[' + i + ']'
data.append(fileName, fs.createReadStream(filePath))
}
const headers = Object.assign({
Authorization: 'Bearer ' + access_Token,
'Content-Type': 'multipart/form-data;boundary=' + data.getBoundary()},
data.getHeaders()
)
let resp = await axios({
method: 'POST',
url: 'https://developer.api.autodesk.com/photo-to-3d/v1/file',
headers: headers,
maxContentLength: Infinity,
maxBodyLength: Infinity,
data: data
})
let dataResp = resp.data;
return dataResp;
} catch (e) {
console.log(e);
}
};

Get image from Axios and send as Form Data to Wordpress API in a Cloud Function

What I'm trying to accomplish is using a Firebase Cloud Function (Node.js) to:
First download an image from an url (f.eg. from unsplash.com) using an axios.get() request
Secondly take that image and upload it to a Wordpress site using the Wordpress Rest API
The problem seems (to me) to be that the formData doesnt actually append any data, but the axios.get() request actually does indeed retrieve a buffered image it seems. Maybe its something wrong I'm doing with the Node.js library form-data or maybe I get the image in the wrong encoding? This is my best (but unsuccessfull) attempt:
async function uploadMediaToWordpress() {
var FormData = require("form-data");
var formData = new FormData();
var response = await axios.get(
"https://images.unsplash.com/photo-1610303785445-41db41838e3e?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=634&q=80"
{ responseType: "arraybuffer" }
);
formData.append("file", response.data);
try {
var uploadedMedia = await axios.post("https://wordpresssite.com/wp-json/wp/v2/media",
formData, {
headers: {
"Content-Disposition": 'form-data; filename="example.jpeg"',
"Content-Type": "image/jpeg",
Authorization: "Bearer <jwt_token>",
},
});
} catch (error) {
console.log(error);
throw new functions.https.HttpsError("failed-precondition", "WP media upload failed");
}
return uploadedMedia.data;
}
I have previously successfully uploaded an image to Wordpress with Javascript in a browser like this:
async function uploadMediaToWordpress() {
let formData = new FormData();
const response = await fetch("https://images.unsplash.com/photo-1610303785445-41db41838e3e?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=634&q=80");
const blob = await response.blob();
const file = new File([blob], "image.jpeg", { type: blob.type });
formData.append("file", file);
var uploadedMedia = await axios.post("https://wordpresssite.com/wp-json/wp/v2/media",
formData, {
headers: {
"Content-Disposition": 'form-data; filename="example.jpeg"',
"Content-Type": "image/jpeg",
Authorization: "Bearer <jwt_token>",
},
});
return uploadedMedia.data;
},
I have tried the last couple of days to get this to work but cannot for the life of me seem to get it right. Any pointer in the right direction would be greatly appreciated!
The "regular" JavaScript code (used in a browser) works because the image is sent as a file (see the new File in your code), but your Node.js code is not really doing that, e.g. the Content-Type value is wrong which should be multipart/form-data; boundary=----...... Nonetheless, instead of trying (hard) with the arraybuffer response, I suggest you to use stream just as in the axios documentation and form-data documentation.
So in your case, you'd want to:
Set stream as the responseType:
axios.get(
'https://images.unsplash.com/photo-1610303785445-41db41838e3e?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=634&q=80',
{ responseType: 'stream' }
)
Use formData.getHeaders() in the headers of your file upload request (to the /wp/v2/media endpoint):
axios.post( 'https://wordpresssite.com/wp-json/wp/v2/media', formData, {
headers: {
...formData.getHeaders(),
Authorization: 'Bearer ...'
},
} )
And because the remote image from Unsplash.com does not use a static name (e.g. image-name.jpg), then you'll need to set the name when you call formData.append():
formData.append( 'file', response.data, 'your-custom-image-name.jpeg' );
I hope that helps, which worked fine for me (using the node command for Node.js version 14.15.4, the latest release as of writing).

Resources