I'm trying to get a webpage via node https.request(). Doing so results in an error getting logged by my code. Using the node request module has the same result:
problem with request: 140398870042432:error:140773F2:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message:s23_clnt.c:658:
The following indicates the wrong SSL version is being used, but I cannot find a way to change the version: curl error: "sslv3 alert unexpected message". Using curl from my terminal returns a response as does hitting the URL in my browser (it is a login page). My code is below.
var request = require('request')
request.get("https://icmserver.wit.ie/balance", function(err, res, body) {
if (err) {
return console.log(err)
}
return body;
});
Does anyone have any idea what might be happening here?
Try to use options = { secureProtocol: 'SSLv3_method' } in the request you are making.
We hit the same problem. By default, request uses the https.globalAgent. So we added the code near the top of our script.
var https = require('https');
https.globalAgent.options.secureProtocol = 'SSLv3_method';
All of a sudden everything worked.
In case website uses ECDH curve, for me the issue resolved only by adding this option:
request({ url, agentOptions: {
ecdhCurve: 'P-521:P-384:P-256',
},(err,res,body) => {
JFYI, May be this will help someone.
Related
I'm trying to fetch a page from IMDB, but for some odd reason it gives me error 400 when I'm using request-promise
But the same query works fine if I'm using CURL:
curl "https://www.imdb.com/title/tt6306064/mediaviewer/rm3146075904"
My nodejs code:
async function getMoviePosterImage(mediaViewerUrl) {
const options = {
uri: mediaViewerUrl
};
try {
const mediaViewerHtml = await request.get(options);
return mediaViewerHtml;
} catch (error) {
console.error(error.statusCode);
}
}
await getMoviePosterImage(
"https://www.imdb.com/title/tt6306064/mediaviewer/rm3146075904"
);
Things I have tried so far:
Setting a user agent
Setting keep-alive
Having cookie jar enabled
On my end, I just tried locally and the same error happened for me. Not a definitive answer, but I have a feeling that IMDB protects against web scrapers.
I am using request npm module in my app, to make to create a http client, as this.
var request = require('request');
And each time, I make a request to some server, I pass the options as below:
var options = {
url: "whateverurl...",
body: { some json data for POST ... }
}
request(options, cb(e, r, body) {
// handle response here...
})
This was working fine, until I started testing with high load, and I started getting errors indicating no address available (EADDRNOTAVAIL). It looks like I am running out of ephemeral ports, as there is no pooling or keep-alive enabled.
After that, I changed it to this:
var options = {
url: "whateverurl...",
body: { some json data for POST ... },
forever: true
}
request(options, cb(e, r, body) {
// handle response here...
})
(Note the option (forever:true)
I tried looking up request module's documentation about how to set keep-alive. According to the documentation and this stackoverflow thread, I am supposed to add {forever:true} to my options.
It didn't seem to work for me, because when I checked the tcpdump, the sever was still closing the connection. So, my question is:
Am I doing something wrong here?
Should I not be setting a global option to request module, while I am "require"ing it, instead of telling it to use {forever:true}, each time I make a http request? This is confusing to me.
Thanks for so many fast response.
I used NodeJS(v4.3.2) and ExpressJs(v4.x) to build up website.
I used a lot AJAX and all AJAX url point to one static IP(AWS Server itself).
Because I would deploy to several servers, I don't want to change AJAX url separately.
My idea is when I run "node bin/www" command line, Can I change it to "node bin/www 50.50.50.50(my AWS address)" and I can set all AJAX url to the right IP?
Is it possible or other alternative solustion?
Thanks
Your issue is related to CORS : basically, you cannot access a domain http://www.example1.com from http://www.example2.com via Ajax if http://www.example1.com does not explicitly allows it in the response.
This is a "security" feature on most modern browsers. You won't encounter this problem using command line such as curl or chrome extension Postman.
To fix this, make sure the domain requesting the data (http://www.example2.com) is allowed in the Access-Control-Allow-Origin header in your server's response, as well as the http verb (GET, POST, PUT... or * for every http methods).
It all comes down to add the two following headers to the http://www.example1.com server's response :
Access-Control-Allow-Origin: http://www.example2.com
Access-Control-Allow-Methods: *
Edit
Following #Paulpro's comment, you have to rebase all your urls so that they reach your server's IP instead of your localhost server.
I fix this problem.
First, in bin/www
append these code to retrieve URL for command line and store into json file.
function setUpURL(){
var myURL = process.argv[2];
console.log('myURL: ', myURL);
var outputFilename = 'public/myURL.json';
var myData = {
port:myURL
}
fs.writeFile(outputFilename, JSON.stringify(myData, null, 4), function(err) {
if(err) {
console.log(err);
} else {
console.log("JSON saved to " + outputFilename);
}
});
};
Then in each JS containing ajax add these code in the head of JS file (it need to load before ajax)
var myURL;
$.getJSON("myURL.json", function(json) {
myURL = json.port.toString();
console.log(myURL);
});
new ajax format
$.ajax({
type: "POST",
url: myURL + "/forgetPwd",
data: sendData,
success: function(data){
window.location.replace(myURL);
},
error: function(data){
}
});
Finally, you can run server
node bin/www your_aws_ip
It works for me now. Hope these will help you guys.
By the way, you should be careful about the path of myURL.json.
Objective:
Te get the response.headers when I get the status code as 302 from redirects uri response
Issue:
I did a bit of googling about an issue which i'm facing at my end about the Redirect URL returning 500 rather than the expected 302. I found the npm request() can be used for redirects. However I'm not able to get the response headers as it always returns 500. But when I used the API end points manually with Postman (with interceptors turned ON) along with body and headers, I was able to get 302 redirects. This is essentially used for Automation testing for API
request(
url: 'https://blah/blah',
client_id:'something.com.au'
redirect_uri:'http://something' // used for mobile app and so it is OK to be have a valid url
response_type :'code'
scope : 'ALLAPI',
Username : 'cxxx'
Password : 'ccxcxc'
headers : {
'Content-Type': 'application/x-www-form-urlencoded',
'followRedirect' : true
}, function (error, response) {
console.log('The response val is', response.statusCode);
});
Question:
Not sure if npm request can do the justice or Am I using request in-correctly or should I have to use axios/follow-redirects etc.. something like that. Please advise. If anyone can provide proper directions on this, it'll be really helpful
Thanks,
Brad
I met the same problem, needle works for me.
var needle = require('needle'),
url = 'https://www.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=GBE03831USEN&';;
needle.get(url, {follow_max: 5}, function(error, res) {
console.log(res.statusCode);
});
#all. I've resolved using form property but with using req.post() method which returned the statuscode as 302 with response headers. Thanks all once again.
I've used the method
Parse.User.become("session-token-here").then(function (user) {
// The current user is now set to user.
}, function (error) {
// The token could not be validated.
});
This method will call back to Parse to validate the session token and fetch the associated user, then set the current user on the client like is explained in this website
http://blog.parse.com/announcements/bring-your-own-login/
This method was working perfectly but I recently update the last version of npm parse 1.5.0 and now I got the following error:
Error: It is not secure to become a user on a node.js server environment.
at Function.Parse.User.Parse.Object.extend.become (/home/...
Anybody has a solution for this problem?
Thanks in advance
Ran into the same issue. While bypassing the Javascript SDK may work, it appears that you can use Parse.User.enableUnsafeCurrentUser() help bypass this error within the SDK.
In the Parse 1.5.0 Javascript SDK section of their change log they provided the following update:
Removed the concept of the current user when running in node.js Current users can be enabled in node with Parse.User.enableUnsafeCurrentUser() Many requests now support passing an explicit session token as an option
There may be some unintended security issues with this method. In reading through the source code this will also allow you to also use Parse.User.current and other like features as well. Using .become is probably still the safest option while managing your own session information.
I've been having a lot of issues with Parse.User.become(), both between the error you mentioned, and it returning a 101 invalid user session. My workaround was to bypass the Javascript SDK and make a Parse REST API request.
var request = require("request"),
q = require("q"),
deferred = q.defer(),
options = {
url: "https://api.parse.com/1/users/me",
method: "GET",
json: true,
headers: {
"X-Parse-Session-Token": token,
"X-Parse-Application-Id": "PARSE_APPLICATION_ID",
"X-Parse-REST-API-Key": "PARSE_RESET_API_KEY"
}
};
request(options, function (error, response, body) {
if (!error && response.statusCode == 200) {
return deferred.resolve(user);
} else {
return deferred.reject(error);
}
});
return deferred.promise;