Been spending the past few hours figuring out why Axios is doing this.
I tried to do this in the request library in nodeJS and it works fine, but Axios isn't.
Essentially what i'm doing is sending a request of XML data:
var options = {
'method': 'POST',
'url': 'https://rvices.svc',
'headers': {
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData};
with XMLData looking simliar to :
let xmlData = `<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:com="http://p" xmlns:pws="http://"
etc etc (it has mostly private data in it so I stripped most of it out).
when I try to use axios to get this
const testData = await axios(options)
i get returned a whole lot of code that looks like this:
'���_\x1B�\f�10�\x17�Ч�{O�\x00��5h�S�������\x0F���s�(+Ғ�\x0F�����m�\x15\x01\x13��6b��\x06%\x00��\x15p8<;��W�4����\x0B���\x01���e�\x7FvZ�{���Ï������\x06��-�z��\x01}�!�r�A�\x13��\x11O�w6ũ���{�\x03����;{����\x01\x7Fy��KoՎ���\x1Bߚe��W��mЇ�qD�a��[�7Ӄ���#��F<\x1C/mF�{\x03�h��#�\x16�\x11\x1F\x1F�L9\x0FM\x8A\x0E�\x
17�h���\x03�4�7�f=bj*8�p�\x13_�\x17�5���_�Ӑ�|M>����\r��F�8q�iE�#��\x0E?�v�������O�xq3���x�Q�튱\x1F?G&HG9��6���V\x1B⫯Ev\x01rc\x13\x10�\'�7��`�Ii��x�~LM6�#˒74##�����f�*\x7F\x16(5|\x1CWl��\x07\t\x1F��z�\x15\x00\x1B��4�\x13���LCTG�\x1FI�����\fec�h\x02�~��i`�:Ғ�\x0F���y\b#�]V��g��Ӈ�\x14|���6~\x19~c`�/�O���M\x01��k\x
10�\'+���\x07S\r?|��T�A�\x0FӒ�\x0F��ܷ\'.s�!>�tbX\x05�\fs\x18�\r�"`���\x10lV٠\x05#ܲ�\x02\x0E\x07h���\n' +
'���[�7}�>54 r�����ʦ\x15�\x17��\x0E:
that is the right amount of characters (100k +) but jumbled
compared to doing this with request which returns the xml back I expect ala:
</b:ProjectTaskTypeDetail></b:PwsProjectTaskTypeElement><b:PwsProjectTaskTypeElement><b:ProjectTaskTypeDetail><b:ExternalSystemIdentifier i:nil="true"/><b:ProjectTaskTypeId i:nil="true"/><b:ProjectTaskTypeUid>5776</b:ProjectTaskTypeUid><b:ProjectTaskTypeName>Faon</b:Proj
ectTaskTypeName>
one thing I noticed is axios is breaking my request up into multiple lines like this:
'<com:PwsProjectRef><com:ProjectCode>201268</com:ProjectCode></com:PwsProjectRef>\n' +
'\n' +
'<com:PwsProjectRef><com:ProjectCode>210115-01</com:ProjectCode></com:PwsProjectRef>\n' +
'\n' +
even though there's no \n's in my request or breaks like that.
So i'm wondering if anyone has ran into this before and knows how to solve it?
Request is working but request (from what I can tell?) doesn't work with asynch code (i'm probably wrong about this)
Sorry for the vagueness!
You should be using the responseType config option to set the expected response which reflects the Accept HTTP header and not the Content-Type one:
const options = {
method: 'POST',
url: 'https://rvices.svc',
headers: {
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData,
responseType: 'document',
responseEncoding: 'utf8'
};
const testData = await axios(options);
The response is being returned in a compressed binary format, but
Axios does not understand the compression format that is being returned from the server. Try forcing the response to a specific compression algorithm like 'deflate' which axios understands. 'gzip' may also work.
The axios 'decompress' option tells axios to automatically decompress the binary data.
var options = {
'method': 'POST',
'url': 'https://rvices.svc',
'headers': {
'Accept-Encoding': 'deflate'
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData,
decompress: true
};
<!-- end snippet -->
In v1.2.1, it is fixed this error.
try with this code
Save as get-data.js file
const axios = require("axios");
const getData = async () => {
try {
const resp = await axios.get('your xml URL',
{
headers: {
'Accept-Encoding': 'application/xml',
}
}
);
console.log(resp.data);
} catch (err) {
// Handle Error Here
console.error(err);
}
};
getData()
npm install axios
node get-data.js
Related
I am calling an API that returns a gzip file in it's body. The problem I am having is I don't know how to extract specific data from that csv file as I need to use it further in my code. This is my code:
const response = await axios.request({
method: 'GET',
url: api,
decompress: true,
responseType: 'arraybuffer',
headers: {
Authorization: `Bearer ${token}`
}
}).then(resp => {
zlib.gunzip(resp.data,(err,output) => {
console.log(output);
})
});
The console log in the zlib callback displays the file as I want it but i don't know how to assign that value to response once the gunzip is finished.
Do you want the variable response to be set to output? If so you can just return output using an intermediary variable. I just called it out, but of course you can rename it to whatever seems fitting. Of course, you also need to handle the error cases, which I did not do below, but that should be a relatively easy task.
const response = await axios.request({
method: 'GET',
url: api,
decompress: true,
responseType: 'arraybuffer',
headers: {
Authorization: `Bearer ${token}`
}
}).then(resp => {
let out;
zlib.gunzip(resp.data,(err,output) => {
out = output;
})
return out;
});
Above the const response is set to the variable output in the gunzip func.
I'm having some problems to do a simple POST request with UrlFetchApp on Google Scripts.
This code works fine on NodeJS with node-fetch lib.
const fetch = require('node-fetch');
const URL = "https://login.XXXXXXXXXXXXX.com.br/api/login"
fetch(URL, {
"body": "{'my_json_data': 'login data'}",
"method": "POST",
}).then(res => res.text())
.then(body => console.log(JSON.parse(body)));
The same request on a Google Scripts project using UrlFetchApp give me a 403 Forbidden HTTP error.
var url = 'https://login.XXXXXXXXX.com.br/api/login';
var data = {
'email':'EMAIL',
'password':'PASS'
}
var options = {
method: 'POST',
payload: JSON.stringify(data)
}
var response = UrlFetchApp.fetch(url, options);
Logger.log(response)
What am I missing here?
Edit: Already tried with payload.
I believe your goal as follows.
You want to convert the following Node.js script to Google Apps Script.
const fetch = require('node-fetch');
const URL = "https://login.XXXXXXXXXXXXX.com.br/api/login"
fetch(URL, {
"body": "{'my_json_data': 'login data'}",
"method": "POST",
}).then(res => res.text())
.then(body => console.log(JSON.parse(body)));
Modification points:
In your Node.js script, the data of "{'my_json_data': 'login data'}" is sent to data as text/plain. So in this case, when UrlFetchApp is used, the content type is required to be set. Because when the content type is not set, the data is sent as form-data.
When above points are reflected to your script, it becomes as follows.
Modified script:
From:
var options = {
method: 'POST',
payload: JSON.stringify(data)
}
To:
var options = {
method: 'POST',
payload: JSON.stringify(data),
contentType: "text/plain"
}
If contentType: "text/plain" occurs an error, please modify it to contentType: "text/plain;charset=UTF-8".
Although I'm not sure about the specification of your API, I thought that contentType: "application/json" might be able to be also used. But this is my guess.
Reference:
Class UrlFetchApp
I need to make an XML POST (I know don't ask it's government...) and I can't get it to work with node request promise native.
I have tried turning my string of XML into a buffer, String(), .toString etc. The POST works if I turn json:true so I don't think it's a networking issue ( when passed a xml string with json true it sends a json like { 'variablename': 'stringed XML that I want to send as the body' } )
Here is what I'm working with. I've been banging my head for a while here any help appreciated.
Would ideally be promise/async.
Maybe I should look to find a XMLHttp request npm module?
var request_promise_native_options = {
method: 'POST',
uri: 'http://requestbin.fullcontact.com/198flbs1',
headers: {
'User-Agent': 'Request-Promise',
'Content-Type': 'text/xml'
//'Content-Length': Buffer.byteLength(my_xml_string) //I've tried multiple ways to use this
},
body: {
my_xml_string //also tried many ways here Buffer, String() etc
},
json: false // automatically stringifys body to json if true
};
request_promise(request_promise_native_options)
.then(function (response) {
console.log("success");
})
.catch(function (err) {
console.log(err);
})
Kudos to #kevin-b to help me see the obvious. Simply remove {}
var request_promise_native_options = {
method: 'POST',
uri: 'http://requestbin.fullcontact.com/198flbs1',
headers: {
'User-Agent': 'Request-Promise',
'Content-Type': 'text/xml'
'Content-Length': Buffer.byteLength(my_xml_string)
},
body: my_xml_string,
json: false // automatically stringifys body to json if true
};
I'm trying to make a POST request using node/HTTPS, but I keep getting this error:
BODY: {"message":"MalformedJsonException:
com.google.gson.stream.MalformedJsonException: Use
JsonReader.setLenient(true) to accept malformed JSON at line 1 column
11 path $","mdcKey":""}
Here's the code that initiates the request:
const https = require('https');
const querystring = require('querystring');
const POSTData = querystring.stringify({
'addressTo': 'myemail#address.com',
'subject': 'testing your email template',
'templateName': 'genericTemplate',
'bodyText': 'this is a test'
});
console.log(POSTData);
const HTTPOptions = {
hostname: 'url.com',
port: 00000,
path: '/path',
method: 'POST',
headers: {
'Content-Type': 'application/json'
}
};
const HTTPSrequest = https.request(HTTPOptions, (response) => {
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
response.setEncoding('utf8');
response.on('data', (chunk) => {
console.log(`BODY: ${chunk}`);
});
response.on('end', () => {
console.log('No more data in response.');
});
});
HTTPSrequest.on('error', (error) => {
console.log(`problem with request: ${error}`);
});
// write data to request body
HTTPSrequest.write(POSTData);
HTTPSrequest.end();
I'm assuming it's the POSTData that is the "malformed JSON", here's what it looks like stringified:
addressTo=name%40companyname.com&subject=testing%20your%20email%20template&templateName=genericTemplate&bodyText=this%20is%20a%20test
I'm not sure what I'm doing that is causing the malformed JSON. What am I missing?
You're sending an application/x-www-form-urlencoded payload but telling the server it's application/json. The server is complaining because of this mismatch.
The simple solution should be to just change querystring.stringify to JSON.stringify.
You may also need to specify a Content-Length header since some servers may not support chunked requests (which is the default in node). If you do add this, make sure you use Buffer.byteLength(POSTData) as the header value instead of just POSTData.length since the former works for multi-byte characters and the latter returns the number of characters (not bytes).
Help more with explication from #mscdex
Your options object after querystring.stringify has returned. Otherwise you won't know the length of the stringified body data.
It is important to note that you need two extra headers in your options object in order to make a successful post request. These are :
'Content-Type': 'application/json',
'Content-Length': POSTData.length
Check example:
const HTTPOptions = {
hostname: 'url.com',
port: 00000,
path: '/path',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': POSTData.length
}
};
You can see this examples too: 1st post.
JsonReader.setLenient(true) to accept malformed JSON at line 1 column 11 path $","mdcKey":""}
I'm switching one of my projects from request over to something a bit more light-weight (such as got, axios, or fetch). Everything is going smoothly, however, I'm having an issue when attempting to upload a file stream (PUT and POST). It works fine with the request package, but any of the other three return a 500 from the server.
I know that a 500 generally means an issue on the server's end, but it is consistent only with the HTTP packages that I'm testing out. When I revert my code to use request, it works fine.
Here is my current Request code:
Request.put(`http://endpoint.com`, {
headers: {
Authorization: `Bearer ${account.token.access_token}`
},
formData: {
content: fs.createReadStream(localPath)
}
}, (err, response, body) => {
if (err) {
return callback(err);
}
return callback(null, body);
});
And here is one of the attempts using another package (in this case, got):
got.put(`http://endpoint.com`, {
headers: {
'Content-Type': 'multipart/form-data',
Authorization: `Bearer ${account.token.access_token}`,
},
body: {
content: fs.createReadStream(localPath)
}
})
.then(response => {
return callback(null, response.body);
})
.catch(err => {
return callback(err);
});
Per the got documentation, I've also tried using the form-data package in conjunction with it according to its example and I still get the same issue.
The only difference between these 2 I can gather is with got I do have to manually specify the Content-Type header otherwise the endpoint does give me a proper error on that. Otherwise, I'm not sure how the 2 packages are constructing the body with the stream, but as I said, fetch and axios are also producing the exact same error as got.
If you want any of the snippets using fetch or axios I'd be happy to post them as well.
I know this question was asked a while ago, but I too am missing the simple pipe support from the request package
const request = require('request');
request
.get("https://res.cloudinary.com/demo/image/upload/sample.jpg")
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
and had to do some experimenting to find similar features from current libraries.
Unfortunately, I haven't worked with "got" but I hope the following 2 examples help someone else that are interested in working with the Native http/https libraries or the popular axios library
HTTP/HTTPS
Supports piping!
const http = require('http');
const https = require('https');
console.log("[i] Test pass-through: http/https");
// Note: http/https must match URL protocol
https.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
(imageStream) => {
console.log(" [i] Received stream");
imageStream.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
);
}
);
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
)
Axios
Note the usage of imageStream.data and that it's being attached to data in the Axios config.
const axios = require('axios');
(async function selfInvokingFunction() {
console.log("[i] Test pass-through: axios");
const imageStream = await axios.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
{
responseType: "stream", // Important to ensure axios provides stream
}
);
console.log(" [i] Received stream");
const upload = await axios({
method: "post",
url: "http://127.0.0.1:8000/api/upload/stream/",
data: imageStream.data,
headers: {
"Content-Type": imageStream.headers["content-type"],
},
});
console.log("Upload response", upload.data);
})();
Looks like this was a headers issue. If I use the headers directly from FormData (i.e., headers: form.getHeaders()) and just add in my additional headers afterwards (Authorization), then this ends up working just fine.
For me just works when I added other parameters on FormData.
before
const form = new FormData();
form.append('file', fileStream);
after
const form = new FormData();
form.append('file', fileStream, 'my-whatever-file-name.mp4');
So that way I can send stream from my backend to another backend in node, waiting a file in multipart/form-data called 'file'