Convert multipart/form-data cURL to Fetch node js - node.js

With the following cURL request I am able to successfully interact with a pulsar api to update the configurations of a function.
Working cURL Request
curl --request PUT -H "Content-Type: multipart/form-data" -F functionConfig='{"parallelism":2};type=application/json' http://pulsar-test:8080/admin/v3/functions/test/ingest/test-entity-FeedTransformer
But I'm having trouble converting to an equivalent fetch request in Node.
Note: I'm using formdata-node to import FormData.
import { FormData } from 'formdata-node'
Non Working Fetch (Bad Request)
const body = new FormData()
body.append('functionConfig', '{"parallelism":2};type=application/json')
return fetch(
'http://pulsar-test:8080/admin/v3/functions/test/ingest/test-entity-FeedTransformer',
{
body,
headers: {
Authorization: 'Bearer abcd'
},
method: 'PUT'
}
)
Apparently the form data is not being set in a way the api server can recognize.
{"reason":"Function config is not provided"}
What am I missing here?

the problem is most likely that multipart request is not recognized/not well formed, or something like that, so try using another library, here's an example with Form-Data:
import { FormData } from 'form-data';
const body = new FormData();
body.append('functionConfig', '{"parallelism":2};type=application/json')
return fetch(
'http://pulsar-test:8080/admin/v3/functions/test/ingest/test-entity-FeedTransformer', {
body,
headers: {
Authorization: 'Bearer abcd'
},
method: 'PUT'
}
)
alternatively, you could run your curl command with child-process
see: How to use curl with exec nodejs

Related

How to upload an image to Cloudflare Images from Node using fetch?

I'm trying to upload an image from my computer to Cloudflare Images using Node and the fetch API. They provided me with a curl command that works fine.
curl -X POST -F file=#./<file_name> -H "Authorization: Bearer <api_token>" https://api.cloudflare.com/client/v4/accounts/<domain_id>/images/v1
When I tried to convert it to fetch, Cloudflare keeps sending me a 400 in their response.
const cloudflarePostBody = new FormData();
cloudflarePostBody.append("file", fs.createReadStream("testing.jpeg"));
const cloudflareResponse = await fetch("https://api.cloudflare.com/client/v4/accounts/<my_id>/images/v1", {
method: "POST",
headers: {
Authorization: `Bearer ${cloudflareApiToken}`,
"Content-Type": "multipart/form-data"
},
body: cloudflarePostBody,
});
My guess is that I'm doing something wrong with how I'm reading the file with createReadStream, but a lot of the examples I've looked up showed exactly that. Does anyone have any advice? Thanks in advance.
Try fs.readFile() instead of fs.createReadStream():
fs.readFile("testing.jpeg", (err, data) => {
if (err) throw err;
cloudflarePostBody.append("file", new Blob([data]), "testing.jpeg");
});

node curl error "Received HTTP code 503 from proxy after CONNECT"

I get a JWT token after authenticating on an API with a POST req.
With that token, I can make a successful curl call from my command prompt:
curl --request GET https://corpURL/customer/wlsAccountManagement/v1/billingAccount/23435657 --header "env: it04" --header "Authorization: Bearer tokenstring..."
But when I try to exec this in node, it fails with Received HTTP code 503 from proxy after CONNECT.
I even tried a basic curl in the command prompt and it works:
curl -v https://corpURL/customer/wlsAccountManagement/v1/billingAccount/23435657
It just tells me I am unauthorized {"message":"Unauthorized"}, which is correct.
In node, I can't even get the {"message":"Unauthorized"}, it still gives me Received HTTP code 503 from proxy after CONNECT.
The reason I'm using curl is because I can see more info. Using axios it gives me a "socket hang up" error.
I have been trying to get this to work and searching online for a solution for over a day. Does anyone have any idea what's going on here? Why does it work in the command prompt but not in node? Thanks!
For anyone who can't get an exec with curl to work in node, or using the packages axios or request, use the https package in node like so:
const options = {
hostname: 'hostDomain', // no https protocol
port: 443,
path: '/path/to/get',
method: 'GET',
headers: {
Authorization: `Bearer tokenString`,
'Content-Type': 'application/json',
},
};
const req = https.request(options, response => {
response.on('data', d => {
process.stdout.write(d);
});
});
req.on('error', e => {
console.error(e);
});
req.end();
This was the only way I was able to get the response so far, in node. I may tweak the structure as it doesn't seem that clean, but it works! You may also need to set the env var for your cert file NODE_EXTRA_CA_CERTS=cert.pem as I already had. Peace.
EDIT
I found a solution with axios, you need to create instances with a baseURL like the https request has it:
const axiosInstance = axios.create({
baseURL: 'https://corpURL',
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
env: 'it04',
},
});
axiosInstance
.get('/get/path')
.then(response => console.log('response', response.data))
.catch(err => console.log('err', err));

URI encode query parameter in request

I want to replicate the below curl request in node using request module.
curl -X POST \
'https://control.msg91.com/api/sendotp.php?authkey=fdfdfdfdfdfd&mobile=%203456786444&message=Your%20OTPis%20456897&otp=456897' \
-H 'cache-control: no-cache'
How ever when I tried the below code where I have used qs object to send the query parameters seems the + in mobile number is not uri encoded as above. How can I achieve this. Is qs is not the right parameter ? I have also tried using form .
My node code
var request = require('request');
request.post(
{
url:'https://control.msg91.com/api/sendotp.php',
headers: {
'content-type': 'application/json'
},
qs : {
authkey:'fdfdfdfdfdfd',
message:'Your OTP is 456789',
mobile :'+3456786444',
otp: '456789'
}}, function(err,httpResponse,body){
console.log(err);
});

How to POST with multipart/form-data header and FormData using fetch

This is a CURL example which works fine:
curl -X POST \
<url> \
-H 'authorization: Bearer <token>' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
-F file=#algorithm.jpg \
-F userId=<userId>
I'm trying to reproduce this request using isomorphic-fetch.
I've tried the following code:
const formData = new FormData();
formData.append('file', file);
formData.append('userId', userId);
return fetch(`<url>`, {
method: 'POST',
headers: {
'Content-Length': file.length
'Authorization: Bearer <authorization token>',
'Content-Type': 'multipart/form-data'
},
body: formData
})`
I use fs.readFileSync in order to generate the file passed to FormData.
The previous example returns a 401 HTTP status code (unauthorized) with an error message saying that the userId embedded in the token (sent via header) does not match the userId passed from formData.
So my suspicion is that the FormData that arrives to the REST API is not adequately formed.
The problem may be related with the Content-Length header, but I didn't find a better way to calculate it (if I don't use the Content-Length header I get a 411 HTTP status code Content-Length header missing).
Could be the case that this is failing because of an incorrect value in the Content-Length header?
Any other suggestions on why this is failing or how to better debug it?
If further info is needed to clarify this problem, please just ask.
UPDATE
I've tried the form-data module in order to get the right Content-Length value using the method formData.getLengthSync()
However the problem remains the same (401 error HTTP status code response).
Just remove the Content-Length and Content-Type headers from your code as these headers will be set automatically by the browser.
If you open up your network inspector, run this code snippet, and submit the form you should see that the Content-Length is set correctly:
const foo = document.getElementById('foo')
foo.addEventListener('submit', (e) => {
e.preventDefault()
const formData = new FormData(foo)
formData.append('userId', 123)
fetch('//example.com', {
method: 'POST',
body: formData
})
})
<form id="foo">
<input id="file" type="file" name="file"/><br><br>
<button type="submit">Submit</button>
</form>
I hit my head against a similar wall, specifically using isomorphic-fetch on node to POST a multipart form. The key for me was finding .getHeaders(). Note that NPM description for form-data suggests that it'll "just work" without this, but it doesn't seem to, at least not in node (I think browsers inject header stuff?).
// image is a Buffer containing a PNG image
// auth is the authorization token
const form_data = new FormData();
form_data.append("image", png, {
filename: `image.png`,
contentType: 'application/octet-stream',
mimeType: 'application/octet-stream'
});
const headers = Object.assign({
'Accept': 'application/json',
'Authorization': auth,
}, form_data.getHeaders());
try {
const image_res = await fetch(url, {
method: 'POST',
headers: headers,
body: form_data
});
if (!image_res.ok) {
const out = await image_res.json();
console.dir(out);
return;
}
}
catch (e) {
console.error(`Chart image generation exception: ${e}`);
}

How do I duplicate this CURL request within nodejs?

I am trying to duplicate the following CURL request within nodejs using request-promise (eventually, I need to use promises so I would prefer this method):
curl -H "Authorization: Token token=[API Key]" -H "Accept: application/vnd.moonclerk+json;version=1" https://api.moonclerk.com/customers
The following code snippet shows my attempt:
var rp = require('request-promise');
var querystring = require('querystring');
//this is what I think is causing me trouble
var bodyHeaders = querystring.stringify({
"Authorization": "Token token=[token taken out in code snippet]",
"Accept": "application/vnd.moonclerk+json;version=1"
});
var options = {
uri: 'https://api.moonclerk.com/customers',
method: 'GET',
body: bodyHeaders
};
var cb = function () {
return function (response) {
console.log("response: ", response);//this should spit out the JSON text I'm looking for
}
}
rp(options).then(cb())
But I am getting Possibly unhandled StatusCodeError: 401 - HTTP Token: Access denied. in the nodejs console as a response. What's the issue here?
PS -- Note that my uri is an HTTPS (i.e., 'https://api.moonclerk.com/customers'); is this what's causing the problem?
You can't pass HTTP headers in the request body, server can't recognize them there. See https://github.com/request/request#custom-http-headers for correct usage of the request library.
var options = {
// ...
headers: {
"Authorization": "Token token=[token taken out in code snippet]",
"Accept": "application/vnd.moonclerk+json;version=1"
}
});

Resources