I am currently trying to get a XML from an external source using request / request-promise.
I did the following:
const request = require('request-promise');
const options = {
method: 'GET',
uri: '',
headers: {
'Content-Type': 'application/xml; charset=utf-8',
},
};
request(options).then((response) => {
console.log(response);
});
The external XML contains characters like ü,ö,ä. They are all missing and are replaced by a question mark in a box.
I thought setting the header would be enough and my google research didn't bring up any other solutions for this scenario.
Am I missing something obvious?
Thanks for every hint!!
Related
Been spending the past few hours figuring out why Axios is doing this.
I tried to do this in the request library in nodeJS and it works fine, but Axios isn't.
Essentially what i'm doing is sending a request of XML data:
var options = {
'method': 'POST',
'url': 'https://rvices.svc',
'headers': {
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData};
with XMLData looking simliar to :
let xmlData = `<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:com="http://p" xmlns:pws="http://"
etc etc (it has mostly private data in it so I stripped most of it out).
when I try to use axios to get this
const testData = await axios(options)
i get returned a whole lot of code that looks like this:
'���_\x1B�\f�10�\x17�Ч�{O�\x00��5h�S�������\x0F���s�(+Ғ�\x0F�����m�\x15\x01\x13��6b��\x06%\x00��\x15p8<;��W�4����\x0B���\x01���e�\x7FvZ�{���Ï������\x06��-�z��\x01}�!�r�A�\x13��\x11O�w6ũ���{�\x03����;{����\x01\x7Fy��KoՎ���\x1Bߚe��W��mЇ�qD�a��[�7Ӄ���#��F<\x1C/mF�{\x03�h��#�\x16�\x11\x1F\x1F�L9\x0FM\x8A\x0E�\x
17�h���\x03�4�7�f=bj*8�p�\x13_�\x17�5���_�Ӑ�|M>����\r��F�8q�iE�#��\x0E?�v�������O�xq3���x�Q�튱\x1F?G&HG9��6���V\x1B⫯Ev\x01rc\x13\x10�\'�7��`�Ii��x�~LM6�#˒74##�����f�*\x7F\x16(5|\x1CWl��\x07\t\x1F��z�\x15\x00\x1B��4�\x13���LCTG�\x1FI�����\fec�h\x02�~��i`�:Ғ�\x0F���y\b#�]V��g��Ӈ�\x14|���6~\x19~c`�/�O���M\x01��k\x
10�\'+���\x07S\r?|��T�A�\x0FӒ�\x0F��ܷ\'.s�!>�tbX\x05�\fs\x18�\r�"`���\x10lV٠\x05#ܲ�\x02\x0E\x07h���\n' +
'���[�7}�>54 r�����ʦ\x15�\x17��\x0E:
that is the right amount of characters (100k +) but jumbled
compared to doing this with request which returns the xml back I expect ala:
</b:ProjectTaskTypeDetail></b:PwsProjectTaskTypeElement><b:PwsProjectTaskTypeElement><b:ProjectTaskTypeDetail><b:ExternalSystemIdentifier i:nil="true"/><b:ProjectTaskTypeId i:nil="true"/><b:ProjectTaskTypeUid>5776</b:ProjectTaskTypeUid><b:ProjectTaskTypeName>Faon</b:Proj
ectTaskTypeName>
one thing I noticed is axios is breaking my request up into multiple lines like this:
'<com:PwsProjectRef><com:ProjectCode>201268</com:ProjectCode></com:PwsProjectRef>\n' +
'\n' +
'<com:PwsProjectRef><com:ProjectCode>210115-01</com:ProjectCode></com:PwsProjectRef>\n' +
'\n' +
even though there's no \n's in my request or breaks like that.
So i'm wondering if anyone has ran into this before and knows how to solve it?
Request is working but request (from what I can tell?) doesn't work with asynch code (i'm probably wrong about this)
Sorry for the vagueness!
You should be using the responseType config option to set the expected response which reflects the Accept HTTP header and not the Content-Type one:
const options = {
method: 'POST',
url: 'https://rvices.svc',
headers: {
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData,
responseType: 'document',
responseEncoding: 'utf8'
};
const testData = await axios(options);
The response is being returned in a compressed binary format, but
Axios does not understand the compression format that is being returned from the server. Try forcing the response to a specific compression algorithm like 'deflate' which axios understands. 'gzip' may also work.
The axios 'decompress' option tells axios to automatically decompress the binary data.
var options = {
'method': 'POST',
'url': 'https://rvices.svc',
'headers': {
'Accept-Encoding': 'deflate'
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData,
decompress: true
};
<!-- end snippet -->
In v1.2.1, it is fixed this error.
try with this code
Save as get-data.js file
const axios = require("axios");
const getData = async () => {
try {
const resp = await axios.get('your xml URL',
{
headers: {
'Accept-Encoding': 'application/xml',
}
}
);
console.log(resp.data);
} catch (err) {
// Handle Error Here
console.error(err);
}
};
getData()
npm install axios
node get-data.js
I'm having some problems to do a simple POST request with UrlFetchApp on Google Scripts.
This code works fine on NodeJS with node-fetch lib.
const fetch = require('node-fetch');
const URL = "https://login.XXXXXXXXXXXXX.com.br/api/login"
fetch(URL, {
"body": "{'my_json_data': 'login data'}",
"method": "POST",
}).then(res => res.text())
.then(body => console.log(JSON.parse(body)));
The same request on a Google Scripts project using UrlFetchApp give me a 403 Forbidden HTTP error.
var url = 'https://login.XXXXXXXXX.com.br/api/login';
var data = {
'email':'EMAIL',
'password':'PASS'
}
var options = {
method: 'POST',
payload: JSON.stringify(data)
}
var response = UrlFetchApp.fetch(url, options);
Logger.log(response)
What am I missing here?
Edit: Already tried with payload.
I believe your goal as follows.
You want to convert the following Node.js script to Google Apps Script.
const fetch = require('node-fetch');
const URL = "https://login.XXXXXXXXXXXXX.com.br/api/login"
fetch(URL, {
"body": "{'my_json_data': 'login data'}",
"method": "POST",
}).then(res => res.text())
.then(body => console.log(JSON.parse(body)));
Modification points:
In your Node.js script, the data of "{'my_json_data': 'login data'}" is sent to data as text/plain. So in this case, when UrlFetchApp is used, the content type is required to be set. Because when the content type is not set, the data is sent as form-data.
When above points are reflected to your script, it becomes as follows.
Modified script:
From:
var options = {
method: 'POST',
payload: JSON.stringify(data)
}
To:
var options = {
method: 'POST',
payload: JSON.stringify(data),
contentType: "text/plain"
}
If contentType: "text/plain" occurs an error, please modify it to contentType: "text/plain;charset=UTF-8".
Although I'm not sure about the specification of your API, I thought that contentType: "application/json" might be able to be also used. But this is my guess.
Reference:
Class UrlFetchApp
I need to make a http call within my node server .The optional parameter is:
'name='
This means that url (relative path) should look like:
/v1/clans?name=**exampleValue**
So far the options for my http request looks like:
app.get('/v1/:clans?=:name', (req, res) => {
console.log(req.path)
const options = {
host: 'api.clashofclans.com',
path: req.path,
method: 'GET',
headers: {
'Content-Type': 'application/json',
Authorization: 'Bearer *token*'
}
};
const x = https.get(options, (request) => {...});
But that doesnt work out. Does someone know how to include the optional parameters in my path property?
You don't. That's not a parameter you're thinking of. That's a query parameter and your path should look like this:
'/v1/clans
You retrieve the query parameter using req.query.<parameter> in your case req.query.name
The optional url parameter you're thinking of would be like this /v1/clans/:name and would be accessible using req.params.name.
I am somewhat new to Nodejs. I am working on a pair of Microservices, and I need one to post data to the other. I am using the request-promise-native library.
My code is to make the call is like this:
const options = {
method: 'POST',
uri: url,
formData: {
command: command,
version: version,
session_id: sid,
aicc_data: data
},
headers: {
'content-type' : 'application/x-www-form-urlencoded'
}
}
rp(options)
However, when I inspect the request as it come in to the other server, the header I have specified does not appear.
headers: { 'content-type': 'multipart/form-data; boundary=--------------------------395968157759002211606136',
host: 'localhost:9000',
'content-length': '513',
connection: 'close' }
What am I doing wrong?
options includes a formData object which enforces multipart/form-data.
You should add the form object instead when you want to use application/x-www-form-urlencoded.
I'm trying to send bath request to GMail API using request for nodejs. The problem is even trough I generate the same query as in documentation, gmail always respond with 400 BadRequest
Those are my headers in form of json
{ 'Content-Type': 'multipart/mixed; boundary=c2db2692-8877-4159-bf02-03e6c5d3ffbf',
authorization: 'Bearer MY_TOKEN',
'content-length': 231 }
Example of body content
--c2db2692-8877-4159-bf02-03e6c5d3ffbf
Content-Type: application/http
Content-ID: <item1:12930812#barnyard.example.com>
GET /gmail/v1/users/sneg0k32#gmail.com/messages/14c6c0c43e9bb16b
--c2db2692-8877-4159-bf02-03e6c5d3ffbf--
Any suggestions why this happens? Request generates extra few spaces and make headers in lowercase, however I'm not sure if this is the problem.
Example of nodejs code
request({
method: 'POST',
uri: 'https://www.googleapis.com/batch',
auth: {
bearer: key
},
headers: {
'Content-Type': 'multipart/mixed'
},
multipart: _.map(queries, function(query) {
return {
'Content-Type': 'application/http',
'Content-ID': "<item1:12930812#barnyard.example.com>",
body: 'GET ' + query
}
})
}, function(error, response, body) {
console.log(response.request.headers)
_.map(response.request.body, function(chunk) {
console.log(chunk.toString())
})
console.log(body)
})
UPD: This is clearly an issue with how I make my node call, send same request via HTTPTester and everything worked fine
UPD2: Decided to use bachelor, however I haven't figured what was causing this issue with plain request
Quite hard to troubleshoot such a problem. I would more than likely say it's down to the spaces. I had a similar issue that was because the request body was seen as invalid due to it's structure.
Uppercase or lowercase headers shouldn't cause an issue.
If it helps I've created a node module to creating batch requests: https://www.npmjs.com/package/batchelor. Feel free to use it.