I am using node js for testing apis. In 1 API i need to upload a file. I am not able to attach file in the payload. I am able to do it in postman but not do it with nodejs. I am using request-promise for managing requests. How can i go about it?
Thanks.
Slight modification in the #rustam's answer,
return request({
method: 'POST',
uri: 'http://localhost:8333',
formData ={
'file': fileStream,
}
});
If you just need to specify the file in payload, then please look at this invented example:
const fs = require('fs');
const request = require('request-promise')
function sendFile() {
const readStream = fs.createReadStream('./test.txt');
return request({
method: 'POST',
uri: 'http://localhost:8333',
body: readStream
});
}
async function main() {
const response = await sendFile();
console.log({ response });
}
main();
In this example, I hard-coded the file name and url, but for you this may be function parameters.
Related
I tried
const myData = uri; // this looks like file:///data/expo/...
const myDataResponse = await API.SendFile({
myData: myData
});
But file is not being recieved in the backend. Is there any other way to do this?
IN the backend I get
console.log(req.body)
// I get a string same as uri of file
console.log(req.file)
// undefined
Backend is setup perfectly. I am getting successful response when using postman.
Any help is appreciated. I am a begginer.
You should use FormData.
const data = new FormData();
// first argument is the key name received on you API
// second argument is your file path
data.append('file', filePath)
// then your request should look like this :
await fetch(url, {
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
body: data,
})
I have this payload format in my hapi route that accepts only multipart/form-data data and the output is set as a stream:
payload: {
maxBytes: 1024,
output: 'stream',
parse: true,
allow: 'multipart/form-data',
}
I want to test that route and my payload is this:
const FD = new FormData();
FD.append('field1', 'value');
FD.append('field2', 'value');
The hapi inject method looks like this:
const res = await server.inject({
method,
url,
headers: {
...
'Content-Type': 'multipart/form-data; boundary=--SEPARATION--',
},
payload: FD,
});
I am getting
Invalid multipart payload format
I tried to set a stream using a Steam object too but it doesn't work. Also I tried to send a File object.
At this point I just want to send something to the route that will not result in an error. It's not necessary to be a FormData. Anything that will be accepted by the route is fine as long as I can add some custom fields to test it further.
Thanks in advance.
As far as I know, FormData doesn’t exist in node.js, so I assume you’re using the form-data module?
In that case, you need to ask form-data for the buffer and headers separately, like this:
const FormData = require('form-data');
const FD = new FormData();
FD.append('field1', 'value');
FD.append('field2', 'value');
const response = await server.inject({
method,
url,
headers: {
<your other headers>
...FD.getHeaders(),
},
payload: FD.getBuffer(),
});
If you want to test sending files, you also need to provide the filename property to append(…) for the binary data to be decoded correctly by the server:
const fileContent = <any Buffer>;
FD.append('form_file_property', fileContent, {filename: 'a testfile'});
I have an API endpoint that lets the client post their csv to our server then post it to someone else server. I have done our server part which save uploaded file to our server, but I can't get the other part done. I keep getting error { message: 'File not found', code: 400 } which may mean the file never reach the server. I'm using axios as an agent, does anyone know how to get this done? Thanks.
// file = uploaded file
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
Update
As axios doc states as below and the API I'm trying to call requires a file
// data is the data to be sent as the request body
// Only applicable for request methods 'PUT', 'POST', and 'PATCH'
// When no transformRequest is set, must be of one of the following types:
// - string, plain object, ArrayBuffer, ArrayBufferView, URLSearchParams
// - Browser only: FormData, File, Blob
// - Node only: Stream, Buffer
Is there any way to make axios send a file as a whole? Thanks.
The 2 oldest answers did not work for me. This, however, did the trick:
const FormData = require('form-data'); // npm install --save form-data
const form = new FormData();
form.append('file', fs.createReadStream(file.path));
const request_config = {
headers: {
'Authorization': `Bearer ${access_token}`,
...form.getHeaders()
}
};
return axios.post(url, form, request_config);
form.getHeaders() returns an Object with the content-type as well as the boundary.
For example:
{ "content-type": "multipart/form-data; boundary=-------------------0123456789" }
I'm thinking the createReadStream is your issue because its async. try this.
Since createReadStream extends the event emitter, we can "listen" for when it finishes/ends.
var newFile = fs.createReadStream(file.path);
// personally I'd function out the inner body here and just call
// to the function and pass in the newFile
newFile.on('end', function() {
const form_data = new FormData();
form_data.append("file", newFile, "filename.ext");
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
});
This is what you really need:
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios
.post(url, form_data, request_config);
In my case, fs.createReadStream(file.path) did not work.
I had to use buffer instead.
const form = new FormData();
form.append('file', fs.readFileSync(filePath), fileName);
const config = {
headers: {
Authorization: `Bearer ${auth.access_token}`,
...form.getHeaders(),
},
};
axios.post(api, form.getBuffer(), config);
I have made an interceptor you can connect to axios to handle this case in node: axios-form-data. Any feedback would be welcome.
npm i axios-form-data
example:
import axiosFormData from 'axios-form-data';
import axios from 'axios';
// connect axiosFormData interceptor to axios
axios.interceptors.request.use(axiosFormData);
// send request with a file in it, it automatically becomes form-data
const response = await axios.request({
method: 'POST',
url: 'http://httpbin.org/post',
data: {
nonfile: 'Non-file value',
// if there is at least one streamable value, the interceptor wraps the data into FormData
file: createReadStream('somefile'),
},
});
// response should show "files" with file content, "form" with other values
// and multipart/form-data with random boundary as request header
console.log(response.data);
I had a same issue, I had a "pdf-creator-service" for generate PDF document from html.
I use mustache template engine for create HTML document - https://www.npmjs.com/package/mustache
Mustache.render function returns html as a string what do I need to do to pass it to the pdf-generator-service ? So lets see my suggestion bellow
//...
async function getPdfDoc(props: {foo: string, bar: string}): Promise<Buffer> {
const temlateFile = readFileSync(joinPath(process.cwd(), 'file.html'))
mustache.render(temlateFile, props)
const readableStream = this.getReadableStreamFromString(htmlString)
const formData = new FormData() // from 'form-data'
formData.append('file', options.file, { filename: options.fileName })
const formHeaders = formData.getHeaders()
return await axios.send<Buffer>(
{
method: 'POST',
url: 'https://pdf-generator-service-url/pdf',
data: formData,
headers: {
...formHeaders,
},
responseType: 'arraybuffer', // ! important
},
)
}
getReadableStreamFromString(str: string): Readable {
const bufferHtmlString = Buffer.from(str)
const readableStream = new Readable() // from 'stream'
readableStream._read = () => null // workaround error
readableStream.push(bufferHtmlString)
readableStream.push(null) // mark end of stream
return readableStream
}
For anyone who wants to upload files from their local filesystem (actually from anywhere with the right streams architecture) with axios and doesn't want to use any external packages (like form-data).
Just create a readable stream and plug it right into axios request function like so:
await axios.put(
url,
fs.createReadStream(path_to_file)
)
Axios accepts data argument of type Stream in node context.
Works fine for me at least in Node v.16.13.1 and with axios v.0.27.2
I need to add an file attachment to a soap request from node.js application.
I am able to send request with node-soap library, and now I need to add a file to the request.
I did it with a java client or with soapUI, but I have to do it in node.js, maybe it's possible to do that overriding default request object ?
I didn't find a solution with node-soap, I had to build manually my soap request :
var soapHeader = '<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header/><soapenv:Body><ws:method>';
var soapFooter = '</ws:method></soapenv:Body></soapenv:Envelope>';
function sendSoapRequestWithAttachments(soap,files){
var soapRequest = jsonToXml.buildObject(mail);
var finalSoapRequest = soapHeader + soapRequest + soapFooter;
var multipartMail = [];
// Add soap request
multipartMail.push({
'Content-Type': 'text/xml; charset=utf-8',
body: finalSoapRequest
});
// Add attachments
if (files) {
files.forEach(function (file) {
multipartMail.push({
'Content-Id': '<' + file.uuid + '>',
'Content-Type': 'application/octet-stream',
'Content-Transfer-Encoding': 'binary',
body: fs.createReadStream(file.path)
});
});
}
var options = {
uri: URL,
method: 'POST',
multipart: multipartMail
};
request.post(options, function (error, response) {
...
}
}
I found a way to send attachment using node-soap using base64 encoding
here is an example
import soap from 'soap'
import fs from 'fs'
async function main(){
const filepath = '/path/to/attachment/file.extension'
const client = await soap.createClientAsync(/* ... */)
const result = await client.AnApiMethodAsync({
expectedKeyForTheFile: await fs.readFileAsync(filepath, 'base64')
})
}
main()
I'm switching one of my projects from request over to something a bit more light-weight (such as got, axios, or fetch). Everything is going smoothly, however, I'm having an issue when attempting to upload a file stream (PUT and POST). It works fine with the request package, but any of the other three return a 500 from the server.
I know that a 500 generally means an issue on the server's end, but it is consistent only with the HTTP packages that I'm testing out. When I revert my code to use request, it works fine.
Here is my current Request code:
Request.put(`http://endpoint.com`, {
headers: {
Authorization: `Bearer ${account.token.access_token}`
},
formData: {
content: fs.createReadStream(localPath)
}
}, (err, response, body) => {
if (err) {
return callback(err);
}
return callback(null, body);
});
And here is one of the attempts using another package (in this case, got):
got.put(`http://endpoint.com`, {
headers: {
'Content-Type': 'multipart/form-data',
Authorization: `Bearer ${account.token.access_token}`,
},
body: {
content: fs.createReadStream(localPath)
}
})
.then(response => {
return callback(null, response.body);
})
.catch(err => {
return callback(err);
});
Per the got documentation, I've also tried using the form-data package in conjunction with it according to its example and I still get the same issue.
The only difference between these 2 I can gather is with got I do have to manually specify the Content-Type header otherwise the endpoint does give me a proper error on that. Otherwise, I'm not sure how the 2 packages are constructing the body with the stream, but as I said, fetch and axios are also producing the exact same error as got.
If you want any of the snippets using fetch or axios I'd be happy to post them as well.
I know this question was asked a while ago, but I too am missing the simple pipe support from the request package
const request = require('request');
request
.get("https://res.cloudinary.com/demo/image/upload/sample.jpg")
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
and had to do some experimenting to find similar features from current libraries.
Unfortunately, I haven't worked with "got" but I hope the following 2 examples help someone else that are interested in working with the Native http/https libraries or the popular axios library
HTTP/HTTPS
Supports piping!
const http = require('http');
const https = require('https');
console.log("[i] Test pass-through: http/https");
// Note: http/https must match URL protocol
https.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
(imageStream) => {
console.log(" [i] Received stream");
imageStream.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
);
}
);
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
)
Axios
Note the usage of imageStream.data and that it's being attached to data in the Axios config.
const axios = require('axios');
(async function selfInvokingFunction() {
console.log("[i] Test pass-through: axios");
const imageStream = await axios.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
{
responseType: "stream", // Important to ensure axios provides stream
}
);
console.log(" [i] Received stream");
const upload = await axios({
method: "post",
url: "http://127.0.0.1:8000/api/upload/stream/",
data: imageStream.data,
headers: {
"Content-Type": imageStream.headers["content-type"],
},
});
console.log("Upload response", upload.data);
})();
Looks like this was a headers issue. If I use the headers directly from FormData (i.e., headers: form.getHeaders()) and just add in my additional headers afterwards (Authorization), then this ends up working just fine.
For me just works when I added other parameters on FormData.
before
const form = new FormData();
form.append('file', fileStream);
after
const form = new FormData();
form.append('file', fileStream, 'my-whatever-file-name.mp4');
So that way I can send stream from my backend to another backend in node, waiting a file in multipart/form-data called 'file'