Error: getaddrinfo ENOTFOUND parishackers.org parishackers.org:80 - node.js

Error: getaddrinfo ENOTFOUND parishackers.org parishackers.org:80 error happens.
I wrote the codes,
var Crawler = require("node-webcrawler");
var url = require('url');
var c = new Crawler({
maxConnections : 10,
// This will be called for each crawled page
callback : function (error, result, $) {
// $ is Cheerio by default
//a lean implementation of core jQuery designed specifically for the server
if(error){
console.log(error);
}else{
console.log($("title").text());
}
}
});
// Queue just one URL, with default callback
c.queue('http://www.amazon.com');
// Queue a list of URLs
c.queue(['http://www.google.com/','http://www.yahoo.com']);
// Queue URLs with custom callbacks & parameters
c.queue([{
uri: 'http://parishackers.org/',
jQuery: false,
// The global callback won't be called
callback: function (error, result) {
if(error){
console.log(error);
}else{
console.log('Grabbed', result.body.length, 'bytes');
}
}
}]);
// Queue some HTML code directly without grabbing (mostly for tests)
c.queue([{
html: '<p>This is a <strong>test</strong></p>'
}]);
but when I run the code,
Google
Yahoo
Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more
{ Error: getaddrinfo ENOTFOUND parishackers.org parishackers.org:80
at errnoException (dns.js:50:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:92:26)
code: 'ENOTFOUND',
errno: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: 'parishackers.org',
host: 'parishackers.org',
port: 80 }
error happens. I think program scrape data part of it, but I do not know why program can do it completely. I installed library like npm install node-webcrawler. I read another site, so I think this error happens because of wrong link, right? How should I fix this? What is wrong in my code?

Error (getaddrinfo ENOTFOUND parishackers.org parishackers.org:80) is throwed because of invalid domain http://parishackers.org. Use valid url links, your node webcrawler will works like a charm. Modified the snippet for your reference
var Crawler = require("node-webcrawler");
var url = require('url');
var c = new Crawler({
maxConnections : 10,
// This will be called for each crawled page
callback : function (error, result, $) {
// $ is Cheerio by default
//a lean implementation of core jQuery designed specifically for the server
if(error){
console.log(error);
}else{
console.log($("title").text());
}
}
});
// Queue just one URL, with default callback
c.queue('http://www.amazon.com');
// Queue a list of URLs
c.queue(['http://www.google.com/','http://www.yahoo.com']);
// Queue URLs with custom callbacks & parameters
c.queue([{
uri: 'http://www.amazon.com',
jQuery: false,
// The global callback won't be called
callback: function (error, result) {
if(error){
console.log(error);
}else{
console.log('Grabbed', result.body.length, 'bytes');
}
}
}]);
// Queue some HTML code directly without grabbing (mostly for tests)
c.queue([{
html: '<p>This is a <strong>test</strong></p>'
}]);

Related

Getting the Error: getaddrinfo ENOTFOUND api.twitter.com api.twitter.com:443 using Node js for twitter posting

const Twitter = require('twitter');
const client = new Twitter({
consumer_key: 'XXXX',
consumer_secret: 'XXXX',
access_token_key: 'XXXX',
access_token_secret: 'XXXX'
});
client.post('statuses/update', {status: 'Posting via the Node is awesome!'}, function(
error,
tweet,
response
) {
if (error) throw error;
// console.log(tweet); // Tweet body.
// console.log(response); // Raw response object.
});
Please Find Below the error:
if (error) throw error;
^
Error: getaddrinfo ENOTFOUND api.twitter.com api.twitter.com:443
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)
Note : I was trying to hit the twitter api using the firewall application proxy. Actually by default api.twitter.com was blocked by firewall but I requested from env team to add the api.twitter.com in proxy application Link list and they added. Still I am getting the below error any idea how i can check this what is happening and how to solve this.

getting Error: getaddrinfo ENOTFOUND while performing rest api call in node.js using http.request

i have created api in node.js which consume set of api hosted at http://dev.abc.co.in:20081
not every time but randomly sometimes it throws the error
Error: getaddrinfo ENOTFOUND dev.abc.co.in
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:60:26) {
errno: 'ENOTFOUND',
code: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: 'dev.abc.co.in'
}
to call those api i have used request node module because i started getting this error i switched to fetch-node npm module and finally replace the code with internal node module http but getting same error
here is the code i have written using http.request
try{
const options = {
hostname: "dev.abc.co.in",
port : 20081,
path: "/api/entity/workorder",
method: Config.method
};
if(Config.headers){
options.headers = Config.headers
}
const req = http.request(options, (res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
callback(res, data);
});
req.socket.destroy();
}).on("error", (err) => {
console.log("===Error: ", err);
callback(null, err);
});
if(Config.method!="GET" && Config.body){
Config.headers["Content-Length"] = Config.body.length;
req.write(Config.body);
}
req.end();
}catch(e){
console.log("Exception=====",e);
}
as shown in error message issue related to DNS so i try to resolve this DNS using
node -pe 'require("dns").lookup("dev-vsg.dovertech.co.in",function(){console.dir(arguments)})
but still not resolved.
1) Omit 'http://' from the beginning of your demain and all slashes from the end or any path after the actual domain.
2) Try to resolve your hostname:
const dns = require('dns');
dns.resolve("testdomain.com", 'ANY', (err, records) => {
if (err) {
console.log("Error: ", err);
} else {
console.log(records);
}
});
If dns records has been returned, then you will know it's a node js problem and after that we can investigate further. If not, then it's a domain configuration issue.

How do you run RServe on AWS Lambda with NodeJS?

While this question is quite open-ended, I'm generally trying to follow this excellent post here: https://aws.amazon.com/blogs/compute/analyzing-genomics-data-at-scale-using-r-aws-lambda-and-amazon-api-gateway/ which describes setting up R to run with python. I, on the other hand, am trying to get R to work with NodeJs.
I've packaged up my dependencies, deployed to Lambda, and can run simple Node scripts. However, I am having difficulty connecting to RServe from Node using the npm package Rio (https://www.npmjs.com/package/rio). RServe, on both my localhost and on Heroku, will accept the default connection of 127.0.0.1 and port 6331. No luck with AWS Lambda.
'use strict';
var rio = require('rio');
var Promise = require('bluebird');
var exec = require('child_process').exec;
var whenReady = new Promise(function(resolve){
// require libraries and bootup RServe
exec('Rscript init.R', function(error, stdout, stderr) {
(function check() {
// Attempt to connect to RServe through Rio using my 'up' test function
rio.e({
entrypoint: 'up',
callback: function (err) {
console.log(err);
if (err) return setTimeout(check, 100);
// If no connection error, rserve is running
console.log("Rserve up");
resolve();
}
});
})();
});
});
exports.handler = function(event, context, callback) {
whenReady.then(function () {
// Call hello world
rio.e({
entrypoint: 'hello',
data: {name:'Will'},
callback: function(err, result){
console.log("Error", err);
callback(null, result);
}
});
});
};
This ends with connection refused errors
2017-03-01T22:58:33.210Z 96f69baf-fed2-11e6-9164-e91b9773d645 {
[Error: connect ECONNREFUSED 127.0.0.1:6311] code: 'ECONNREFUSED',
errno: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1',
port: 6311 }
Any ideas on how to fix this one? I'm hoping we don't need to get complicated: https://aws.amazon.com/blogs/aws/new-access-resources-in-a-vpc-from-your-lambda-functions/
Thank you in advance!
** Update **
init.R does the following
// Require some libraries
...
require('jsonlite');
up <- function () {
toJSON(TRUE)
}
run.Rserve()
** Last Update **
Gave up and went to the python example as posted in the first link.
Will

Get HTML page by URL

Here is my code:
utilitesRouter.route('/url')
.post(function(request, response) {
console.log(request.body.uri);
var urlOpts = { host: request.body.uri, path: '/', port: '80', method: 'GET' };
var re = /(<\s*title[^>]*>(.+?)<\s*\/\s*title)>/gi;
http.get(urlOpts, function (response) {
response.on('data', function (chunk) {
var str=chunk.toString();
console.log(str);
var match = re.exec(str);
if (match && match[2]) {
console.log(match[2]);
}
});
});
response.json({ url: request.body.uri });
});
If I use POST request with this JSON {"uri":"google.ru" } I get:
302 Moved
google.ru
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
here.
</BODY></HTML>
If I use POST requiet with JSON {"uri":"http://google.ru" } I get the error message:
events.js:85
throw er; // Unhandled 'error' event
^
Error: getaddrinfo ENOTFOUND http://google.ru
at errnoException (dns.js:44:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26)
I can open http://google.ru in my browser.
How can I get the HTML using node.js ?
You may want to request to do that. It just pretty easy.
var request = require("request");
router.get('/proxy', function(req, res, next){
request.get( req.body.uri, function(error, response, body){
if( error )
return next(error);
res.send(body);
});
});
request also support streaming and other cool features too.
You get the error because in your urlOpts the attribute host has to be a domain name, like google.ru or www.google.ru. As you are putting a URL into it, it can't be resolved to an IP via DNS, that's why you get the error at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26).
If you want to use http.get() like the way you do, you would always have to extract the domain part out of your passed uri, i.e. getting google.ru out of http://google.ru to use it as host.

Getting ETIMEDOUT error when I try to do a simple Get request?

Hi I am trying to call a simple web API which returns a string as response. I want to use node for this. Since I am new to node so I tried reffering to many blog post and got a code snippet which I used but I am getting same error for all urls whether its google.com or anything else.
My Node code is as follows
var http = require('http');
//The url we want is: 'www.random.org/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
var options = {
host: 'www.random.org',
path: '/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
};
callback = function(response) {
var str = '';
//another chunk of data has been recieved, so append it to `str`
response.on('data', function (chunk) {
str += chunk;
});
//the whole response has been recieved, so we just print it out here
response.on('end', function () {
console.log(str);
});
}
http.request(options, callback).end();
Error:
F:\nodejs>node ..\NodeLearning\TestServer1\test.js
events.js:72
throw er; // Unhandled 'error' event
^
Error: connect ETIMEDOUT
at errnoException (net.js:901:11)
at Object.afterConnect [as oncomplete] (net.js:892:19)
Can Any one tell me what has gone wrong here?
Can you try one more time by setting a proxy like mentioned below
var options = {
host: 'www.random.org',
path: '/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new',
proxy:'add your proxy setting'
};

Resources