Porting legacy ColdFusion code to Node.js - cfhttp/request.js - node.js

I am primary a Coldfusion developer, for over 10+ years but its time to make a change. I am working through porting some old Colfusion 9 code to node js and I am struggling getting connected to the 3rd party API to access data for our company.
This is the current Coldfusion code that is connecting to the external service:
<cfsavecontent variable="thiscontent">
<post>
<username>username#domain.com</username>
<password>Pa$$w0rd</password>
</post>
</cfsavecontent>
<cfhttp url="https://API.ENDPOINT" method="post" result="httpResponse" >
<cfhttpparam type="FormField" name="xml" value="#Trim(thiscontent)#" />
</cfhttp>
This code works find, and returns the expected XML object from the service. However, this what is interesting is that if I remove the 'method="post"' perameter, I get the same error I do when trying to connect with node, more on this in a second.
For node, I am using express.js to interact with the endpoint. Here is the code I am using:
reqOpts = {
url: 'http://API.ENDPOINT',
method: 'post',
headers: {
'Content-Type': 'application/xml'
},
body: '<post><username>user#domain.com</username><password>Pa44w0rd</password></post>'
}
var getNew = request(reqOpts, function(err, resp, body){
console.log(body)
}) ;
This then return the following error:
<?xml version="1.0"?>
<response><status>FAILURE</status><message>No XML string passed</message></response>
Remember when I said that removing the post parameter from cfhttp causes the same error? I cant seem to get this to work at all in node.
I have tried using request().form, request().auth etc with no success, always the same NO XML STRING PASSED error.
I would be very grateful for any assistance.

In your ColdFusion code you used a FormField named xml.
Do the same in Node.js instead of putting the XML straight in the request body:
reqOpts = {
url: 'http://API.ENDPOINT',
method: 'post',
headers: {
'Content-Type': 'application/xml'
},
form: {
xml: '<post><username>user#domain.com</username><password>Pa44w0rd</password></post>'
}
}
var getNew = request(reqOpts, function(err, resp, body) {
console.log(body)
}) ;

Related

Getting JSON Data into image format

I have this GET request I built to call Google APIS dot com to get an image of a house at a given address. It's all working fine. In Postman it displays the image from the body of the request. All good!
I converted the code to NodeJS REQUEST. Put that code into my project. It all works though the data returned is all �����JFIF������ like this in the BODY returned.
Can you point me to some resources or can you tell me in NODEJS how I get that into a Image type variable. I want to then display it using JSON code into Messenger Bot. I have the JSON code to send a IMAGE type back to Messenger - I just need to get the results of the GET above into a format in NODEJS that will work - like a PNG or JPG format.
This is the code I used from Postman CODE:
var options = { method: 'GET',
url: 'https://maps.googleapis.com/maps/api/streetview',
qs:
{ size: '450x450',
location: 'N108W15303%20Bel%20Aire%20Ln%2053022',
fov: '90',
heading: '235',
pitch: '10',
key: 'xxx' },
headers:
{ 'Postman-Token': 'xxx',
'Cache-Control': 'no-cache',
Accept: 'application/json' } };
requestGoogle(options, function (error, response, body) {
if (error) throw new Error(error);
console.log(body);
});
The BODY var is is all ����JFIF��C
It displays great in the Postman App. so you are somehow converting it to display it - what I am looking for.
Any help would be appreciated - or direct me to a resource that can help that would be great.

Sending URL encoded string in POST request using node.js

I am having difficulty sending a url encoded string to the Stormpath /oauth/token API endpoint. The string is meant to look like this:
grant_type=password&username=<username>&password=<password>
Using Postman I was successful in hitting the endpoint and retrieving the data I want by providing a string similar to the one above in the request body by selecting the raw / text option. But when I generate the code snippet it looks like this:
var request = require("request");
var options = { method: 'POST',
url: 'https://<My DNS label>.apps.stormpath.io/oauth/token',
headers:
{ 'postman-token': '<token>',
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
host: '<My DNS label>.apps.stormpath.io',
accept: 'application/json' },
form: false };
request(options, function (error, response, body) {
if (error) throw new Error(error);
console.log(body);
});
Where did that string go? I would like some help in understanding the disconnect between knowing I sent a url encoded string to the API endpoint using Postman and not seeing it in the code generated by Postman. Because now I don't know how to reproduce a successful call to the endpoint in my actual app.
To me it seems like I should simply provide a body to the request, but the response comes out to be {"error":"invalid_request","message":"invalid_request"}. I have also tried appending the url encoded string to the url but that returns a 404 error.
I'm just now getting back into using an API and am not very experienced doing so.
The form data needs to be posted as an object, here is an example:
request.post('http://service.com/upload', {form:{key:'value'}})
Taken from this documentation:
https://github.com/request/request#forms
Hope this helps!

Parameters too long using Node.js & HTTPS for Email Audit API

I'm using Email Audit API to monitor emails of my company. Unfortunately, I can't upload my public key as asked here following the instructions here https://developers.google.com/admin-sdk/email-audit/auth.
In that example, I will use the public key (base64 encoded) given on the google page :
LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tDQpWZXJzaW9uOiBHbn
VQRyB2MS40LjEwIChHTlUvTGludXgpDQoNCm1RRU5CRXJXYUQ0QkNBQ3QybmdmczYv
K1FPR1lieE5iYzNnTG5YSHRxcDdOVFRYTlc0U0pvKy9BMW9VWm9HeEENClF4NnpGWG
hRLzhNWFc2Nis4U1RTMVlxTkpPQVJGdGpiSUtQd2pyZGN1a2RQellWS0dacmUwUmF4
Q25NeUNWKzYNCkY0WU5RRDFVZWdIVHUyd0NHUjF1aVlPZkx4VWE3L2RvNnMzMVdSVE
g4dmJ0aVBZOS82b2JFSXhEakR6S0lxWU8NCnJ2UkRXcUFMQllrbE9rSjNIYmdmeWw0
MkVzbkxpQWhTK2RNczJQQ0RpMlgwWkpDUFo4ZVRqTHNkQXRxVlpKK1INCldDMUozVU
R1RmZtY3BzRFlSdFVMOXc2WU10bGFwQys5bW1KM0FCRUJBQUcwVjBSaGMyaGxjaUJV
WlhOMElDaFUNCmRHVnlNa0JrWVhOb1pYSXRhSGxrTFhSbGMzUXVZMjl0UG9rQk9BUV
RBUUlBSWdVQ1N0Wm9QZ0liRFFZTENRZ0gNCmsxOVFja1Rwd0Jkc2tFWXVtRnZtV3Zl
NVVYMlNWVjdmek9DMG5adGdGeHRaR2xKaEdtanNBM3J4RlRsYitJcmENCldaYXlYQ1
dZaUN6ZDdtOXo1L0t5R0QyR0ZUSy85NG1kbTI1TjZHWGgvYjM1cElGWlhCSS9yWmpy
WXJoWVJCRnUNCkd0ekdGSXc5QUFuRnlVekVVVVZmUFdVdEJlNXlITVc1NEM2MG5Iaz
V4WUlhNnFGaGlMcDRQWXFaQ3JZWDFpSXMNCmZSUk9GQT09DQo9U1RIcg0KLS0tLS1F
TkQgUEdQIFBVQkxJQyBLRVkgQkxPQ0stLS0tLQ==
Here is my code :
var params =
"<atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>"+
"<apps:property name='publicKey' value='"+key+"'/>"+
"</atom:entry>";
// An object of options to indicate where to post to
var post_options = {
host: 'apps-apis.google.com',
path: '/a/feeds/compliance/audit/publickey/' + encodeURIComponent(domain),
method: 'POST',
headers: {
'Authorization': 'OAuth ' + token,
'Content-Type': 'application/atom+xml;charset=utf-8',
}
};
// Set up the request
var post_req = https.request(post_options, function(res) {
res.setEncoding('utf8');
res.on('data', function (chunk) {
console.log("REPONSE");
console.log("-------");
console.log(chunk);
});
});
// post the data
post_req.write(params);
post_req.end();
And as the result come as multiples i guess my params are too long :
<?xml version="1.0" encoding="UTF-8"?>
<AppsForYourDomainErrors>
<error errorCode="1411" invalidInput="LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tDQpWZXJzaW9uOiBHbn VQRyB2MS40LjEwIChHTlUvTGludXgpDQoNCm1RRU5CRXJXYUQ0QkNBQ3QybmdmczYv K1FPR1lieE5iYzNnTG5YSHRxcDdOVFRYTlc0U0pvKy9BMW9VWm9HeEENClF4NnpGWG hRLzhNWFc2Nis4U1RTMVlxTkpPQVJGdGpiSUtQd2pyZGN1a2RQellWS0dacmUwUmF4 Q25NeUNWKzYNCkY0WU5RRDFVZWdIVHUyd0NHUjF1aVlPZkx4VWE3L2RvNnMzMVdSVE g4dmJ0aVBZOS82b2JFSXhEakR6S0lxWU8NCnJ2UkRXcUFMQllrbE9rSjNIYmdmeWw0 MkVzbkxpQWhTK2RNczJQQ0RpMlgwWkpDUFo4ZVRqTHNkQXRxVlpKK1INCldDMUozVU R1RmZtY3BzRFlSdFVMOXc2WU10bGFwQys5bW1KM0FCRUJBQUcwVjBSaGMyaGxjaUJV WlhOMElDaFUNCmRHVnlNa0JrWVhOb1pYSXRhSGxrTFhSbGMzUXVZMjl0UG9rQk9BUV RBUUlBSWdVQ1N0Wm9QZ0liRFFZTENRZ0gNCmsxOVFja1Rwd0Jkc2tFWXVtRnZtV3Zl NVVYMlNWVjdmek9DMG5adGdGeHRaR2xKaEdtanNBM3J4RlRsYitJcmENCldaYXlYQ1 dZaUN6ZDdtOXo1L0t5R0QyR0ZUSy85NG1kbTI1TjZHWGgvYjM1cElGWlhCSS9yWmpy WXJoWVJCRnUNCkd0ekdGSXc5QUFuRnlVekVVVVZmUFdVdEJlNXlITVc1NEM2MG5Iaz V4WUlhNnFGaGlMcDRQWXFa
REPONSE
-------
Q3JZWDFpSXMNCmZSUk9GQT09DQo9U1RIcg0KLS0tLS1F TkQgUEdQIFBVQkxJQyBLRVkgQkxPQ0stLS0tLQ==" reason="EncryptionPublicKeyInvalidFormat" />
</AppsForYourDomainErrors>
As it seems the response is a divided several parts, i guess the problem come from response.write() .. So i tried adding in the headers :
'Content-Length': params.length
And still the same problem :
REPONSE
-------
<?xml version="1.0" encoding="UTF-8"?>
<AppsForYourDomainErrors>
<error errorCode="1411" invalidInput="LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tDQpWZXJzaW9uOiBHbn VQRyB2MS40LjEwIChHTlUvTGludXgpDQoNCm1RRU5CRXJXYUQ0QkNBQ3QybmdmczYv K1FPR1lieE5iYzNnTG5YSHRxcDdOVFRYTlc0U0pvKy9BMW9VWm9HeEENClF4NnpGWG hRLzhNWFc2Nis4U1RTMVlxTkpPQVJGdGpiSUtQd2pyZGN1a2RQellWS0dacmUwUmF4 Q25NeUNWKzYNCkY0WU5RRDFVZWdIVHUyd0NHUjF1aVlPZkx4VWE3L2RvNnMzMVdSVE g4dmJ0aVBZOS82b2JFSXhEakR6S0lxWU8NCnJ2UkRXcUFMQllrbE9rSjNIYmdmeWw0 MkVzbkxpQWhTK2RNczJQQ0RpMlgwWkpDUFo4ZVRqTHNkQXRxVlpKK1INCldDMUozVU R1RmZtY3BzRFlSdFVMOXc2WU10bGFwQys5bW1KM0FCRUJBQUcwVjBSaGMyaGxjaUJV WlhOMElDaFUNCmRHVnlNa0JrWVhOb1pYSXRhSGxrTFhSbGMzUXVZMjl0UG9rQk9BUV RBUUlBSWdVQ1N0Wm9QZ0liRFFZTENRZ0gNCmsxOVFja1Rwd0Jkc2tFWXVtRnZtV3Zl NVVYMlNWVjdmek9DMG5adGdGeHRaR2xKaEdtanNBM3J4RlRsYitJcmENCldaYXlYQ1 dZaUN6ZDdtOXo1L0t5R0QyR0ZUSy85NG1kbTI1TjZHWGgvYjM1cElGWlhCSS9yWmpy WXJoWVJCRnUNCkd0ekdGSXc5QUFuRnlVekVVVVZmUFdVdEJlNXlITVc1NEM2MG5Iaz V4WUlhNnFGaGlMcDRQWXF
REPONSE
-------
aQ3JZWDFpSXMNCmZSUk9GQT09DQo9U1RIcg0KLS0tLS1F TkQgUEdQIFBVQkxJQyBLRVkgQkxPQ0stLS0tLQ==" reason="EncryptionPublicKeyInvalidFormat" />
</AppsForYourDomainErrors>
I'm running out of ideas, any help would be welcome :)
I found a solution using http://unirest.io/nodejs.html.
Now it's working perfectly. Here is the code for those who might want it :
function uploadPublicKey(domain, key, token) {
var params =
"<atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>"+
"<apps:property name='publicKey' value='"+key+"'/>"+
"</atom:entry>";
unirest
.post('https://apps-apis.google.com/a/feeds/compliance/audit/publickey/' + encodeURIComponent(domain))
.headers({
'Authorization': 'OAuth ' + token,
'Content-Type': 'application/atom+xml;charset=utf-8',
'Content-Length': params.length}
)
.send(params)
.end(function (response) {
console.log(response.body);
});
}
You have spaces in your base64-encoded string (this is shown in the response). Remove the whitespace (e.g. key.replace(/\s/g, '')) and you should be ok.
Based from the documentation,
An attempt to upload an invalid key will return with error code 1411 (EncryptionPublicKeyInvalidFormat).
Also noted that, invalid OpenPGP keys are generally the most common cause of errors. For this, please try to follow the summary of the step-by-step instructions to generate a public key for the Email Audit API. Last step mentioned about double checking that you copy all the lines, including the header and not adding any extra line.
Lastly, in uploading the public key, start by creating an XML entry with the base64 encoded public key as shown in the example below:
<atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>
<apps:property name="publicKey" value="the base64 Encoded Key"/>
</atom:entry>
Send an HTTP POST request to the 'publickey' feed URI in your Google Apps domain:
POST https://apps-apis.google.com/a/feeds/compliance/audit/publickey/{domain name}
If successful, the server returns a 201 CREATED status code found in the Google Data API HTTP status codes documentation.
Please try going through the given documentations for more information including details on authorization protocols.

Problems making http-post requests with German umlauts (ä ö ü) or special chars (ß) with Node.js

TLDR; solved, see comments below :)
I am really getting nuts with this: I want to post JSON to the Redmine API to send time entries imported via CSV files. Everything works well, until I try to send strings with German umlauts (ä ü ö) or special chars (ß).
I am using the request package with the following code:
var options = {
url: self.getHost() + path,
headers: {
'Content-Type': 'application/json; charset=utf-8'
},
method: method
};
options.headers[self.getApiKeyHeader()] = self.getApiKey();
if (method !== HttpMethods.Get) {
params.time_entry.comments = 'äöüß';
options.body = JSON.stringify(params);
options.headers['Content-Length'] = options.body.length;
}
And send it like this:
request(options, function (error, response) {
if (response.statusCode === 201) {
console.log('Success');
}
else {
console.log('Error: ' + response.statusCode);
}
});
I always get an HTTP500 Error with server logs saying "Invalid byte sequence in UTF-8". I get the same error when I try to post the JSON via Postman. As my coworkers have no problems with Ruby and PHP script, I guess I got something terribly wrong in my script. Also tried the following alternatives for setting the options.body content:
options.body = new Buffer(JSON.stringify(params), encoding='utf8');
options.body = new Buffer(JSON.stringify(params, 'ascii').toString('utf8');
With both also not working.
I am using https://www.npmjs.com/package/request for the requests.
What am I doing wrong?

Getting 401 uploading file into a table with a service account

I am using nodejs and the REST API to interact with bigquery. I am using the google-oauth-jwt module for JWT signing.
I granted a service account write permission. So far I can list projects, list datasets, create a table and delete a table. But when it comes to upload a file via multipart POST, I ran into two problems:
gzipped json file doesn't work, I get an error saying "end boundary missing"
when I use uncompressed json file, I get a 401 unauthorized error
I don't think this is related to my machine's time being out of sync since other REST api calls worked as expected.
var url = 'https://www.googleapis.com/upload/bigquery/v2/projects/' + projectId + '/jobs';
var request = googleOauthJWT.requestWithJWT();
var jobResource = {
jobReference: {
projectId: projectId,
jobId: jobId
},
configuration: {
load: {
sourceFormat: 'NEWLINE_DELIMITED_JSON',
destinationTable: {
projectId: projectId,
datasetId: datasetId,
tableId: tableId
},
createDisposition: '',
writeDisposition: ''
}
}
};
request(
{
url: url,
method: 'POST',
jwt: jwtParams,
headers: {
'Content-Type': 'multipart/related'
},
qs: {
uploadType: 'multipart'
},
multipart: [
{
'Content-Type':'application/json; charset=UTF-8',
body: JSON.stringify(jobResource)
},
{
'Content-Type':'application/octet-stream',
body: fileBuffer.toString()
}
]
},
function(err, response, body) {
console.log(JSON.parse(body).selfLink);
}
);
Can anyone shine some light on this?
P.S. the documentation on bigquery REST api is not up to date on many things, wish the google guys can keep it updated
Update 1:
Here is the full HTTP request:
POST /upload/bigquery/v2/projects/239525534299/jobs?uploadType=multipart HTTP/1.1
content-type: multipart/related; boundary=71e00bd1-1c17-4892-8784-2facc6998699
authorization: Bearer ya29.AHES6ZRYyfSUpQz7xt-xwEgUfelmCvwi0RL3ztHDwC4vnBI
host: www.googleapis.com
content-length: 876
Connection: keep-alive
--71e00bd1-1c17-4892-8784-2facc6998699
Content-Type: application/json
{"jobReference":{"projectId":"239525534299","jobId":"test-upload-2013-08-07_2300"},"configuration":{"load":{"sourceFormat":"NEWLINE_DELIMITED_JSON","destinationTable":{"projectId":"239525534299","datasetId":"performance","tableId":"test_table"},"createDisposition":"CREATE_NEVER","writeDisposition":"WRITE_APPEND"}}}
--71e00bd1-1c17-4892-8784-2facc6998699
Content-Type: application/octet-stream
{"practiceId":2,"fanCount":5,"mvp":"Hello"}
{"practiceId":3,"fanCount":33,"mvp":"Hello"}
{"practiceId":4,"fanCount":71,"mvp":"Hello"}
{"practiceId":5,"fanCount":93,"mvp":"Hello"}
{"practiceId":6,"fanCount":92,"mvp":"Hello"}
{"practiceId":7,"fanCount":74,"mvp":"Hello"}
{"practiceId":8,"fanCount":100,"mvp":"Hello"}
{"practiceId":9,"fanCount":27,"mvp":"Hello"}
--71e00bd1-1c17-4892-8784-2facc6998699--
You are most likely sending duplicate content-type headers to the Google API.
I don't have the capability to effortlessly make a request to Google BigQuery to test, but I'd start with removing the headers property of your options object to request().
Remove this:
headers: {
'Content-Type': 'multipart/related'
},
The Node.js request module automatically detects that you have passed in a multipart array, and it adds the appropriate content-type header. If you provide your own content-type header, you most likely end up with a "duplicate" one, which does not contain the multipart boundary.
If you modify your code slightly to print out the actual headers sent:
var req = request({...}, function(..) {...});
console.log(req.headers);
You should see something like this for your original code above (I'm using the Node REPL):
> req.headers
{ 'Content-Type': 'multipart/related',
'content-type': 'multipart/related; boundary=af5ed508-5655-48e4-b43c-ae5be91b5ae9',
'content-length': 271 }
And the following if you remove the explicit headers option:
> req.headers
{ 'content-type': 'multipart/related; boundary=49d2371f-1baf-4526-b140-0d4d3f80bb75',
'content-length': 271 }
Some servers don't deal well with multiple headers having the same name. Hopefully this solves the end boundary missing error from the API!
I figured this out myself. This is one of those silly mistakes that would have you stuck for the whole day and at the end when you found the solution you would really knock on your own head.
I got the 401 by typing the selfLink URL in the browser. Of course it's not authorized.

Resources