Node HTTPS Request – MalformedJsonException - node.js

I'm trying to make a POST request using node/HTTPS, but I keep getting this error:
BODY: {"message":"MalformedJsonException:
com.google.gson.stream.MalformedJsonException: Use
JsonReader.setLenient(true) to accept malformed JSON at line 1 column
11 path $","mdcKey":""}
Here's the code that initiates the request:
const https = require('https');
const querystring = require('querystring');
const POSTData = querystring.stringify({
'addressTo': 'myemail#address.com',
'subject': 'testing your email template',
'templateName': 'genericTemplate',
'bodyText': 'this is a test'
});
console.log(POSTData);
const HTTPOptions = {
hostname: 'url.com',
port: 00000,
path: '/path',
method: 'POST',
headers: {
'Content-Type': 'application/json'
}
};
const HTTPSrequest = https.request(HTTPOptions, (response) => {
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
response.setEncoding('utf8');
response.on('data', (chunk) => {
console.log(`BODY: ${chunk}`);
});
response.on('end', () => {
console.log('No more data in response.');
});
});
HTTPSrequest.on('error', (error) => {
console.log(`problem with request: ${error}`);
});
// write data to request body
HTTPSrequest.write(POSTData);
HTTPSrequest.end();
I'm assuming it's the POSTData that is the "malformed JSON", here's what it looks like stringified:
addressTo=name%40companyname.com&subject=testing%20your%20email%20template&templateName=genericTemplate&bodyText=this%20is%20a%20test
I'm not sure what I'm doing that is causing the malformed JSON. What am I missing?

You're sending an application/x-www-form-urlencoded payload but telling the server it's application/json. The server is complaining because of this mismatch.
The simple solution should be to just change querystring.stringify to JSON.stringify.
You may also need to specify a Content-Length header since some servers may not support chunked requests (which is the default in node). If you do add this, make sure you use Buffer.byteLength(POSTData) as the header value instead of just POSTData.length since the former works for multi-byte characters and the latter returns the number of characters (not bytes).

Help more with explication from #mscdex
Your options object after querystring.stringify has returned. Otherwise you won't know the length of the stringified body data.
It is important to note that you need two extra headers in your options object in order to make a successful post request. These are :
'Content-Type': 'application/json',
'Content-Length': POSTData.length
Check example:
const HTTPOptions = {
hostname: 'url.com',
port: 00000,
path: '/path',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': POSTData.length
}
};
You can see this examples too: 1st post.

JsonReader.setLenient(true) to accept malformed JSON at line 1 column 11 path $","mdcKey":""}

Related

Axios returning binary data instead of XML

Been spending the past few hours figuring out why Axios is doing this.
I tried to do this in the request library in nodeJS and it works fine, but Axios isn't.
Essentially what i'm doing is sending a request of XML data:
var options = {
'method': 'POST',
'url': 'https://rvices.svc',
'headers': {
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData};
with XMLData looking simliar to :
let xmlData = `<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:com="http://p" xmlns:pws="http://"
etc etc (it has mostly private data in it so I stripped most of it out).
when I try to use axios to get this
const testData = await axios(options)
i get returned a whole lot of code that looks like this:
'���_\x1B�\f�10�\x17�Ч�{O�\x00��5h�S�������\x0F���s�(+Ғ�\x0F�����m�\x15\x01\x13��6b��\x06%\x00��\x15p8<;��W�4����\x0B���\x01���e�\x7FvZ�{���Ï������\x06��-�z��\x01}�!�r�A�\x13��\x11O�w6ũ���{�\x03����;{����\x01\x7Fy��KoՎ���\x1Bߚe��W��mЇ�qD�a��[�7Ӄ���#��F<\x1C/mF�{\x03�h��#�\x16�\x11\x1F\x1F�L9\x0FM\x8A\x0E�\x
17�h���\x03�4�7�f=bj*8�p�\x13_�\x17�5���_�Ӑ�|M>����\r��F�8q�iE�#��\x0E?�v�������O�xq3���x�Q�튱\x1F?G&HG9��6���V\x1B⫯Ev\x01rc\x13\x10�\'�7��`�Ii��x�~LM6�#˒74##�����f�*\x7F\x16(5|\x1CWl��\x07\t\x1F��z�\x15\x00\x1B��4�\x13���LCTG�\x1FI�����\fec�h\x02�~��i`�:Ғ�\x0F���y\b#�]V��g��Ӈ�\x14|���6~\x19~c`�/�O���M\x01��k\x
10�\'+���\x07S\r?|��T�A�\x0FӒ�\x0F��ܷ\'.s�!>�tbX\x05�\fs\x18�\r�"`���\x10lV٠\x05#ܲ�\x02\x0E\x07h���\n' +
'���[�7}�>54 r�����ʦ\x15�\x17��\x0E:
that is the right amount of characters (100k +) but jumbled
compared to doing this with request which returns the xml back I expect ala:
</b:ProjectTaskTypeDetail></b:PwsProjectTaskTypeElement><b:PwsProjectTaskTypeElement><b:ProjectTaskTypeDetail><b:ExternalSystemIdentifier i:nil="true"/><b:ProjectTaskTypeId i:nil="true"/><b:ProjectTaskTypeUid>5776</b:ProjectTaskTypeUid><b:ProjectTaskTypeName>Faon</b:Proj
ectTaskTypeName>
one thing I noticed is axios is breaking my request up into multiple lines like this:
'<com:PwsProjectRef><com:ProjectCode>201268</com:ProjectCode></com:PwsProjectRef>\n' +
'\n' +
'<com:PwsProjectRef><com:ProjectCode>210115-01</com:ProjectCode></com:PwsProjectRef>\n' +
'\n' +
even though there's no \n's in my request or breaks like that.
So i'm wondering if anyone has ran into this before and knows how to solve it?
Request is working but request (from what I can tell?) doesn't work with asynch code (i'm probably wrong about this)
Sorry for the vagueness!
You should be using the responseType config option to set the expected response which reflects the Accept HTTP header and not the Content-Type one:
const options = {
method: 'POST',
url: 'https://rvices.svc',
headers: {
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData,
responseType: 'document',
responseEncoding: 'utf8'
};
const testData = await axios(options);
The response is being returned in a compressed binary format, but
Axios does not understand the compression format that is being returned from the server. Try forcing the response to a specific compression algorithm like 'deflate' which axios understands. 'gzip' may also work.
The axios 'decompress' option tells axios to automatically decompress the binary data.
var options = {
'method': 'POST',
'url': 'https://rvices.svc',
'headers': {
'Accept-Encoding': 'deflate'
'Content-Type': 'text/xml; charset=utf-8',
'SOAPAction': 'http://etProject'
},
data: xmlData,
decompress: true
};
<!-- end snippet -->
In v1.2.1, it is fixed this error.
try with this code
Save as get-data.js file
const axios = require("axios");
const getData = async () => {
try {
const resp = await axios.get('your xml URL',
{
headers: {
'Accept-Encoding': 'application/xml',
}
}
);
console.log(resp.data);
} catch (err) {
// Handle Error Here
console.error(err);
}
};
getData()
npm install axios
node get-data.js

POST request to DeepL api through node.js not working

Can anyone spot any problems that may explain why the api client is giving me the forbidden error? I know the credentials are correct, as GET requests w the same info in the url work find.
Thank you in advance
app.get('/translate', (req, res) => {
var textToTranslate = "Hello friend"
const targetLanguage = "ES"
var link = `https://api-free.deepl.com/v2/translate`
var options =
{
method: 'POST',
headers: {
"Host": 'api-free.deepl.com',
"Content-Length": 54,
"Content-Type": 'application/x-www-form-urlencoded',
"User-Agent": "YourApp",
"Accept": "*/*",
},
body: JSON.stringify({
'auth_key': deeplAccessCode,
'text': textToTranslate,
'target_lang': targetLanguage
}),
}
return fetch(link, options)
.then((response) => {
console.log(response)
return response.json(); //Transform http body to json
})
.then((json)=> {
res.send(json) //return json to browser
})
.catch(e => {
console.log(e)
return res.sendStatus(400);
});
})
It's probably failing because you're setting your Content-Type of your body to be application/x-www-form-urlencoded (which is correct as per the DeepL API specification) but then you provide a JSON body (which would require content type to be application/json).
You need to provide a URL-encoded body instead, like the part you can also append to the URL after the ?. See also this answer on SO.

Core https library vs. npm 'request' library

I am running into a very strange issue when trying to use the built-in node https library.
Request Headers:
let requestDetails = {
hostname: 'api.domain.com',
method: 'POST',
path: '/endpointIWant/goHere
headers: {
'Client-ID': clientId,
'Content-Type': 'application/json',
Authorization: bearerToken
},
};
Request body:
let body = JSON.stringify({
"content_type": "application/json",
"message" : message
});
This is my standard call using the default https library of node:
let req = https.request(requestDetails, function (res){
let responseBody = undefined;
res.on('body', function(res) {
responseBody = '';
});
res.on('data', function(chunk) {
responseBody += chunk;
});
res.on('end', function() {
console.log(responseBody);
});
});
req.write(body);
req.on('error', function(e) {
console.log(e);
});
req.end();
Now whenever I send this request to the relevant server I get a:
Your browser sent a request that this server could not understand.
Reference #7.24507368.1554749705.3185b29b
However when I use the popular 'request' library on NPM it works fine and I get the response I expect.
This leads be to believe there is something different in maybe the 'encoding' or 'chunking' of the requests between these two libraries, but I cannot figure out what.
Does anyone have experience with the Node https library and understand any gotcha's there?
I prefer to use built-in libraries as much as possible to keep my package size low.
When using native http or https modules, you need to use the querystring module to stringify your body.
const querystring = require('querystring');
let body = querystring.stringify({
"content_type": "application/json",
"message" : message
});
//also include the content length of your body as a header
let requestDetails = {
hostname: 'api.domain.com',
method: 'POST',
path: '/endpointIWant/goHere
headers: {
'Client-ID': clientId,
'Content-Type': 'application/json',
'Content-Length' : body.length
Authorization: bearerToken
},
};
'request' is built on top of the native modules and does this internally when you pass it a json body

"Body" of a post request always in unicode

If I make a POST request to a specific server using a Chrome plugin, I can see the "response body" comes back just fine in a JSOn format.
However when I try to do this with either "request" or "https.request", the "body" comes back as unicode which I can't seem to decipher. Anyone know how I can the body to come back as regular JSON, or how I could decipher this unicode? I tried a few stackoverflow solutions to decipher the unicode but no luck.
raw body:
��RPP�M-.NLOUR�RP�I,�K�P�,V��/QHL.�,KU��J�O�*─
�⎽�����wt�
⎽U��┬�>H ---
If I do this: JSON.stringify(body, null, 4)
"\┤001°�\␉\┤0000\┤0000\┤0000\┤0000\┤0000\┤0000\┤0000��RPP�M-.NLOUR�RP�I,�K�P�,V��/QHL.�,KU�\┤0001�J�O�*─\°�⎽����\┤000°�┬├\┤000␊�\°⎽U�\┤0005\┤0000�┬�>H\┤0000\┤0000\┤0000"
And here's the two code snippets I use to try and make POST requests:
request({
'url': 'https://api.nike.com/launch/entries/v2',
'method': 'POST',
'json': entriesPayload,
'headers': {
'authorization': authId,
"Accept": "application/json, text/plain, */*",
'Content-Type': "application/json;charset=utf-8"
},
},
...
and:
var options = {
hostname: 'api.nike.com',
port: 443,
path: '/launch/entries/v2',
method: 'POST',
json: entriesPayload,
headers: {
'authorization': authId,
"Accept": "application/json, text/plain, */*",
'Content-Type': 'application/json;charset=utf-8'
}
};
var req = https.request(options, (res) => {
res.on('data', (d) => {
process.stdout.write(d);
});
});
req.on('error', (e) => {
console.error(e);
});
req.write(postData);
req.end();
Also I get some real weird stuff happening after I make the request, in my IDE's logs. This is what shows up - it like translates everything into some different symbols (the last 3 lines are how I restarted my nodejs server).
"⎻⎺⎼├": 443,
"␤⎺⎽├┼▒└␊": "▒⎻␋.┼␋┐␊.␌⎺└",
"␤▒⎽␤": ┼┤┌┌,
"⎽␊▒⎼␌␤": ┼┤┌┌,
"─┤␊⎼≤": ┼┤┌┌,
"⎻▒├␤┼▒└␊": "/┌▒┤┼␌␤/␊┼├⎼␋␊⎽/┴2",
"⎻▒├␤": "/┌▒┤┼␌␤/␊┼├⎼␋␊⎽/┴2",
"␤⎼␊°": "␤├├⎻⎽://▒⎻␋.┼␋┐␊.␌⎺└/┌▒┤┼␌␤/␊┼├⎼␋␊⎽/┴2"
£,
"└␊├␤⎺␍": "POST",
"␤␊▒␍␊⎼⎽": π
"▒┤├␤⎺⎼␋≥▒├␋⎺┼": "B␊▒⎼␊⎼ ␊≤J␤␉G␌␋O␋JSU≥I1N␋I⎽I└├⎻ZCI6I┘␌2YWI1NT␤┐LWM┬ZTM├NGV␤Y␋05MT┌┘LTJ┐Y┘A3Y┘F┘N2N␤MHN⎻Z≤J9.␊≤J0␌┼V≥␍CI6MTA┬LCJ⎻YXQ␋O┘E1NDM2N≥␌┬ODA⎽I└V4␌CI6MTU0M≥Y4MDY4MC┬␋▒XN≥I┘⎺␋␉2F1␍G±≤YWN┘I␋┬␋▒┼R⎻I┘⎺␋ZGV␋MTZ┘M2Q├YT±│OS00OTE3LWF␤MD┐├NWV␤N≥R└M┘U┬MWM4I␋┬␋␉GF0I┘⎺│NTQ≥N┘␌3MD±┬LCJ␤␍WQ␋O␋J┘␉20┤␉└┌⎼ZS5┐▒W␍⎻␍GF⎽I␋┬␋␌3V␋I┘⎺␋Y29├L└5⎻▒2U┤Y29├␉WV≤Y2U┤␌25⎼␌┼M┤␍2V␋I␋┬␋␌2J0I┘⎺␋␉└┌⎼ZT⎻␤␌HA␋LCJ≥Y3A␋O┌⎽␋Y29├␉WV≤Y2U␋XS┬␋␌HJ┤I┘⎺␋M└I≥M≥J┐Y└U├MWE1Y≤00ZWJ┐LT┐0Z└Y├MGU┬N≥V┘OGV┘NGQ│I␋┬␋␌HJ0I┘⎺␋␉└┌⎼ZT⎻┬␉HV≥I┼0.I┴▒┘└U2W≤␉__┬AD└J±⎻Z≥␉␋-VVUV┘H⎽⎼EI├┼T┌␍1I9°┬OSL▒┬┤8≥0Z3┐C␍␊G⎽␊O≤│≥␌␉3RB┌GKXV┤SDRDH±IYZ␉H1X⎼5␉Q-┼R≥└┬I␤│␉M0─I⎺R┤␋␍␤▒I␋├NC␍⎺G±6⎺XQ␋°R┴W5␍⎺SZ┼⎻4YR2TN6U␍9Q≥⎻61NS⎺⎻F␌2V13NJ└0P7│K5-09⎻▒1│6P␍M≤┼⎽IWF─␤II⎼G≤K┘HO⎻BV└┌┤␋A≥°┼AF┤K1GC┌ZGD⎺TC␋8⎻JY_⎺HI-E8D±M┐O4KSN⎻H97KLHO-┴Z│2┬YLJ2°␉0⎼F┐D≤≤└A␍K⎼9┴┬┤9┼│XF␍⎼≥⎽┘22≤39KD⎻-⎽⎻R┐I⎺MD2▒└␋┼L1CA│8─-␉L│DY└┬GQ02C├I0─┘±",
"A␌␌␊⎻├": "▒⎻⎻┌␋␌▒├␋⎺┼/┘⎽⎺┼, ├␊│├/⎻┌▒␋┼, */*",
"C⎺┼├␊┼├-T≤⎻␊": "▒⎻⎻┌␋␌▒├␋⎺┼/┘⎽⎺┼;␌␤▒⎼⎽␊├=┤├°-8",
"␌⎺┼├␊┼├-┌␊┼±├␤": 575
£
£
£
[┼⎺␍␊└⎺┼] ⎼␊⎽├▒⎼├␋┼± ␍┤␊ ├⎺ ␌␤▒┼±␊⎽...
[┼⎺␍␊└⎺┼] ⎽├▒⎼├␋┼± ◆┼⎺␍␊ ▒⎻⎻.┘⎽◆
E│⎻⎼␊⎽⎽ ⎽├▒⎼├␊␍ ⎺┼ ⎻⎺⎼├ 3000
The response body is not unicode encoded, but GZIP encoded (compressed data). We can check if a response is compressed with the Content-Encoding header:
var encoding = res.headers['content-encoding'];
https doesn't unzip the response body automatically like a browser does, and so you get all those strange characters instead of a JSON string. However, we can use the built-in zlib library and decdode the response to a string.
const https = require('https');
const zlib = require('zlib');
var options = {
hostname: 'api.nike.com',
port: 443,
path: '/launch/entries/v2',
method: 'POST',
headers: {
'Authorization': 'my-token',
'Accept': 'application/json, text/plain, */*',
'Content-Type': 'application/json;charset=utf-8'
}
};
var data = JSON.stringify({});
var req = https.request(options, (res) => {
var encoding = res.headers['content-encoding'];
console.log('code:', res.statusCode);
console.log('encoding:', encoding);
deflate = zlib.createGunzip();
res.pipe(deflate);
deflate.on('data', (chunk) => {
console.log('data: ' + chunk.toString());
});
});
req.on('error', (e) => {
console.error(e);
});
req.write(data);
req.end();
The chrome plugin that you're using to POST is probably setting the content-type correctly, while your regular request from your server isn't.
Can you change your request so it includes
request({
'url': externalUrl,
'method': 'POST',
'json': entriesPayload,
'Content-Type': 'application/x-www-form-urlencoded'
});

Using Bulit-in Http Module instead of Request Module on Node.js

I want to send form data to web-server by using Node.js, So I use "request" module that is very famous in nodeland. And It's cool, There is no problem, But because of some reason (write-stream encoding non-supported), I have to change it to built-in module, "http".
I think beneath codes are same to post some data to web-server, When I using "request" module, There is no problem so can get 200 response, success to sending data.
But in "http" module, I got a 302 response that redirects to another page. and failed post data. I don't know what is problem with, maybe it is something URL trouble, http use 'host and path' on the other hand, request use 'url' . I don't know how can I solve this, I stucked 2 days, please let me know If you have some hints..
Thanks.
By Using "Request" Module
function postFormByRequestModule() {
request({
url: 'http://finance.naver.com/item/board_act.nhn',
headers: { 'Content-Type': 'text/plain' },
method: 'POST',
form: {
code:'000215',
mode: 'write',
title: 'This is Title',
body:'This is body'
}
}, function (error, response, body) {
if (error) {
console.log(error);
} else {
console.log(response.statusCode, response.body);
}
});
}
By Using "Http" Module
var postData = querystring.stringify({
code:'000215',
mode: 'write',
title: 'This is Title',
body:'This is body'
});
var options = {
host: 'finance.naver.com',
path: '/item/board_act.nhn',
method: 'POST',
headers: { 'Content-Type': 'text/plain', }
};
var req = http.request(options, function(res) {
console.log('STATUS: ' + res.statusCode);
res.setEncoding('utf8');
res.on('data', function (chunk) {
console.log('BODY: ' + chunk);
});
res.on('end', function() {
console.log('No more data in response.')
})
});
req.on('error', function(e) {
console.log('problem with request: ' + e.message);
});
function postFormByBuiltInHttpModule() {
req.write(postData);
req.end();
}
The built-in http client does not automatically follow forwards, whereas the request module does (and has many other "high level" features). So if you want to continue using the built-in client, you will need to manually check res.headers.location and retry the request at that url.

Resources