In Express I have a route that makes an http call to Amazon S3 to see if a specific image exists or not (it's basically a proxy call to bypass my inability to make a cross-domain call directly to Amazon from the browser).
The problem is that it works successively for a while and then suddenly just stops and is only fixed by my restarting my express app.
Just in case (though I believe the code is totally generic) here's the code:
Router.get('/testS3Image',
function(request, response){
try{
var imagePath = request.param('imagePath');
var https = require('https');
var options = {
method:'GET',
host: 's3.amazonaws.com',
path: '/' + imagePath,
port: 443
};
console.log("MAKING S3 call with options " + JSON.stringify(options));
var req = https.request(options,
function(res) {
try{
if(!res || !res.headers || !res.headers["content-length"])
throw ("image not found on S3");
console.log("returned from S3 call with success");
response.json({success:true,headers:res.headers});
}catch(e){
console.log("returned from S3 call with failure");
response.json({success:false, error:e});
}
}
);
req.end();
}catch(e){
}
});
after the https.request() call is made, nothing happens. It simply goes dark. I'm new enough to Node to not know how I can follow along to see what's happening beneath the hood. We're running this in nginx.
Since it works perfectly for a long time and then suddenly breaks with no obvious rhyme or reason I am suspecting some sort of http call limitation, but I am unclear on what I can do differently.
You're not reading the response data from S3; I wonder if this is causing the issue.
Related
I have this following code snippet which works with Google, but I noticed that trying to reach websites like Amazon which force HTTPS will throw an error 301 (permanently moved). I think the problem may be that I’m using the http package, but the HTTPS package confuses me. If anyone could help me out, that would be stupendous.
var vars = {
host: “www.google.com”,
port: 80,
path: “/index.html”
}
http.get(vars, function(res) {
console.log(res.statusCode);
res.setEncoding(“utf8”);
res.on(“data”, function(data) {
console.log(data);
}
})
You can just use https.get(). But, for https, you have to use a different port (443). I prefer to just pass in the URL and let the library handle the default port for me:
const https = require('https');
https.get("https://www.google.com/index.html", function(res) {
console.log(res.statusCode);
res.setEncoding('utf8');
res.on('data', function(data) {
console.log(data);
});
}).on('error', function(err) {
console.log(err);
});
This may return the data in multiple data events so if you want all the data, you'd have to manually combine all the data.
Personally, I prefer to use a higher level library that is promise-based and makes lots of things simpler:
const got = require('got');
got("https://www.google.com/index.html").then(result => {
console.log(result);
}).catch(err => {
console.log(err);
});
Among many other features, the got() library will automatically collect the whole response for you, uses promises, will follow redirects, will automatically parse JSON results, will check the status and provide an error for 4xx and 5xx statuses, supports lots of authentication means, etc... It's just easier to use than the plain http/https libraries.
I have a system that is running on Node 8.11.1 on AWS. There is a function that writes logs to another server. This function takes a request object that it logs.
My problem arises during the actual POST attempt, giving me the following error:
Error: write EPROTO 139746875082624:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:
I cannot see anything wrong with my code.
Why is the error occurring, and what can I do to fix it?
Here is the code inside of the function:
const https = require('https');
try
{
const postData = "New log finished " + JSON.stringify(request, null, 2);
const options =
{
hostname: LOG_DOMAIN,
port: LOG_PORT,
path: '/',
method: 'POST'
};
const req = https.request(options);
req.on('error', (e) =>
{
console.error("ERROR writing logs: " + e);
});
req.write(postData);
req.end();
}
catch (e)
{
console.log(e);
}
LOG_DOMAIN and LOG_PORT are variables passed to the function.
After further research, it appears that Aravind Voggu (see comments) was in the right vein: the error comes from attempts to use HTTPS for a server that only allows HTTP.
The OpenSSL dies when attempting to secure a connection to something that is unsecured.
The only change required to make my code work correctly was to remove the "s" from "https" in the locations used.
const http = require("http");
This issue can be encountered if server side ssl certificate gets expired
UPDATE: I had a mistake on my http request endpoint. I had not set the appropriate authentication options so that fixed a lot of errors possibly this specific one.
My question is similar to one here:
Node.js Lambda function returns "The response is invalid" back to Alexa Service Simulator from REST call
However the solution to that question does not solve my problem. So I make an http request call to an xsjs service in Hana cloud. I am getting the 'response is invalid' error message. I can't see why. Here is my function:
// Create a web request and handle the response.
function httpGet(query, callback) {
console.log("/n QUERY: "+ query);
var host = 'datacloudyd070518trial.hanatrial.ondemand.com';
var path = '/LocationInformation/getLocationInfo.xsjs?location=';
var hostname = 'https://' + host + path + query;
var auth = 'user1:D1anafer';
var req = http.request({'hostname': hostname,
'auth': auth
}, (res) => {
var body = '';
res.on('data', (d) => {
body += JSON.stringify(d);
});
res.on('end', function () {
callback(body);
});
});
req.end();
req.on('error', (e) => {
console.error(e);
});
}
And the function that calls it:
'getNewsIntent': function () {
//self = this;
httpGet(location, function (response) {
// Parse the response into a JSON object ready to be formatted.
//var output = JSON.parse(response);
//output = output['change'];
var output = response;
var cardTitle = location;
var cardContent = output;
alexa.emit(':tellWithCard', output, cardTitle, cardContent);
});
},
Thank You
-Diana
Inside your AWS account go to your Lambda function and click on the monitoring tab, where you should see "View Logs in Cloudwatch" in the right hand corner. If you click that link and you should see the errors that are being produced.
You can also use console.log() to log any information being returned from your REST api, which will be logged in cloudwatch and can help you see where your errors are.
This is just a guess from the top of my head. To really help some detailed error message would be required like mentioned about.
But just a guess: Your http.request() is using the http module (https://nodejs.org/api/http.html) and your are accessing the a https resource. If so there is a https (https://nodejs.org/api/https.html) module or use something like axios https://www.npmjs.com/package/axios or requestjs (https://github.com/request/request) this will handle both.
Like I said just a blind guess without detailed error message and seeing your require statements but I am happy to dive deeper if you happen to have details.
HTH
Your callback from the Lambda has to return a valid status code and body. Like this:
let payload = {
statusCode: 400,
body: JSON.stringify('body'),
headers: {"Access-Control-Allow-Origin": "*"}
};
callback(null, payload);
On top of that, to call this from client side code, you have to pass the CORS header back.
I'm trying to mock request and response objects for my node/express handlers. I've tried a few mocking libraries and have run into issues with API compatibility, which has made them too unreliable for testing purposes.
What I would like to do is create the raw request and response objects myself and direct the output somewhere other than a live connection.
Here's what I have so far:
env.mockReq = function(o){
o = o || {};
o.hostname = 'www.tenor.co';
o.protocol = 'https';
o.path = o.url;
o.createConnection = function(){
console.log('mockReq createConnection');
};
var req = new http.ClientRequest(o);
req.url = o.url;
req.method = o.method;
req.headers = o.headers || {};
return req;
};
env.mockRes = function(o){
var res = new http.ServerResponse({
createConnection: function(){
console.log('mockRes createConnection');
}
});
return res;
};
Here's some test code:
var req = env.mockReq({method: 'GET', url: '/'});
var res = env.mockRes();
res.on('end', function(arguments){
expect(this.statusCode).toBe(200);
expect(this._getData().substr(-7)).toEqual('</html>');
scope.done();
done();
});
// my express app
app.handle(req, res);
My handler is piping a stream data source to the response:
stream.pipe(response);
It works fine when I load the requests in a browser, but my test times out because the response end event never gets fired. I should note that I have logging statements in my handler that's under test and it completes right to the end.
To complicate matters, I'm using nock to mock out some API requests. I had to add the following to prevent an error:
// Prevents "Error: Protocol "https" not supported. Expected "http:""
nock('http://www.example.com')
.persist()
.get('/non-existant-path')
.reply(function(uri, requestBody) {
console.log('nock path:', this.req.path);
return ''
});
That nock callback never actually gets called though. But without this code I get that error, even if I don't use https. The live version of my site redirects all traffic to https, so maybe a live connection is being made, but then why is my handler executing?
I have a problem with the following code that calls a remote service with a POST request and receives JSON data:
[..]
var options = {
host: REMOTE_HOST,
port: REMOTE_HOST_POST,
path: REMOTE_PATH,
method: 'POST'
};
var req = http.request(options, function(res) {
res.setEncoding('utf8');
res.on('data', function (data) {
// handles OK
});
});
req.on('error', function(e) {
// handles KO
});
req.write('user=myname&password=mypassword');
req.end();
[..]
It works ok on my OSX machine, but when I try it on a co-worker Windows computer, the data sent to the server is wrong and I don't know what could be causing the problem. The remote application doesn't receive user=myname&password=mypassword as expected but:
35\r\nuser=myname&password=mypassword
What could be the cause of this? thanks
EDIT
Seems like the problem is not Node.js Further investigation evidences that also my OSX box gives the same error when calling the dev server, while it works ok in production. Seems like a Rails problem. I'