Is there a way in NodeJS to bypass linux's /etc/hosts file?
For example, if I have an entry in my hosts file like:
127.0.0.1 example.com
How would I access the 'real' example.com from NodeJS?
I don't want to temporarily modify /etc/hosts to do this, as it can bring about some problems and is not a clean solution.
Thanks in advance!
I didn't think was possible at first but then I stumbled upon a way to ignore the hosts file on Linux. Additionally, I found a DNS API built into Node. Basically, by default, Node defers to the operating system to do a DNS lookup (which will read from /etc/hosts and not make a DNS query at all if it doesn't have to). However, Node also exposes a method to resolve hostnames by explicitly making DNS requests. This will give you the IP address you're looking for.
$ cat /etc/hosts
<snip>
127.0.0.1 google.com
$ ping google.com
PING google.com (127.0.0.1) 56(84) bytes of data.
...
That shows the host file is definitely working.
const dns = require('dns')
// This will use the hosts file.
dns.lookup('google.com', function (err, res) {
console.log('This computer thinks Google is located at: ' + res)
})
dns.resolve4('google.com', function (err, res) {
console.log('A real IP address of Google is: ' + res[0])
})
This outputs different values as expected:
$ node host.js
This computer thinks Google is located at: 127.0.0.1
A real IP address of Google is: 69.77.145.253
Note that I tested this using the latest Node v8.0.0. However, I looked at some older docs and the API existed since at least v0.12 so, assuming nothing significantly changed, this should work for whatever version of Node you happen to running. Also, resolving a hostname to an IP address might be only half the battle. Some websites will behave strangely (or not at all) if you try accessing them through an IP directly.
Based on #supersam654 and this: my solution (full example) with .get request (igrnore hosts with all requests):
const dns = require("dns");
const url = require("url");
const req = (urlString, cb) => {
const parsedUrl = url.parse(urlString);
const hostname = parsedUrl.hostname;
dns.resolve4(hostname, function(err, res) {
if (err) throw err;
console.log(`A real IP address of ${hostname} is: ${res[0]}`);
const newUrl = urlString.replace(`${parsedUrl.protocol}//${hostname}`, `${parsedUrl.protocol}//${res[0]}`);
https
.get(
newUrl,
{
headers: { host: hostname },
servername: hostname
},
resp => {
let data = "";
// A chunk of data
resp.on("data", chunk => {
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on("end", () => {
cb(data);
});
}
)
.on("error", err => {
console.log("Error request " + url + ": " + err.message);
});
});
};
// Example
req("https://google.com/", data => console.log(data));
Related
I've try yo measure DNS latency in my docker-compose/kubernetes cluster.
setInterval(() => {
console.time('dns-test');
dns.lookup('http://my-service', (_, addresses, __) => {
console.log('addresses:', addresses);
console.timeEnd('dns-test');
});
}, 5000);
But get addresses: undefined, any ideas?
...dns.lookup('http://my-service'...
The lookup function (with example usage) takes the first parameter as the host name that you want to lookup, eg. google.com. You should remove "http://" from the name you passed in.
UPDATE:
So this works as expected if I add the following:
const options = {
...standardStuff,
family: 6,
}
Adding the family: 6 option means it works as expected. So I supposed my question changes to why?? From the docs it states:
IP address family to use when resolving host or hostname.
Valid values are 4 or 6. When unspecified, both IP v4 and v6 will be used.
Whixh would leave me to conclude I wouldnt need to as IPV6 is being used anyway. And why would curl etc not matter?
Have a zone lockdown rule on cloudflare for a cname we have registered with my IPV6 address added to the white list. I got this from googling whatsmyip.
I also added my companies VPN ip address with is in the ipv4 format.
When I curl this endpoint I receive the expeted 200 - however when I run a request via nodejs I receive a 403.
This is even stranger as I am able to access the endpoint via golang, insomnia, curl and I am also able to access it via nodejs when I am connected to an ipv4 network - e.g VPN or if I tether my phone to my laptop.
curl https://my-restricted-endpoint.com
# 200
package main
import (
"fmt"
"log"
"net/http"
)
func main() {
resp, err := http.Get("https://my-restriced-endpoint.com")
if err != nil {
log.Fatal(err)
}
fmt.Println(resp.StatusCode)
// 200
}
const requestAsync = (options: RequestOptions | string | URL): Promise<number> => {
return new Promise((resolve, reject) => {
const req = request(options, (res) => {
if (typeof res.statusCode !== 'number') {
reject('no status code returned');
return;
}
resolve(res.statusCode);
});
req.on('error', (error) => {
reject(error);
});
req.on('timeout', () => {
reject('request timed out');
});
req.end();
});
};
const statusCode = await requestAsync('https://my-restricted-endpoint.com')
// returns 200 on VPN or thetherd to phone with an ipv4 ip address
// returns 403 otherwise
My knowlege of netowkring and IPV4/6 is limited to nonexistent - but feel like this is the cause with something nodejs is doing with the request.
I have also tried using axios
So if I only whitelist my ipv6 address then I need to force nodejs to resolve the hostname to ipv6 by setting {family: 6}.
If I add both my ipv6 and ipv4 to whitelist then I can leave that option alone.
It seems that golang, curl, and insomnia in which I was using implement RFC6555 'Happy Eyeballs' which means that ipv6 will be used first and only on failure would ipv4 be used. It was why these worked and nodejs did not. From what I can gather nodejs does not implement this which means that due no whitelisting my ipv4 address on cloudflare it would fail.
I'd like to restrict the connection to the socket server only to a list of IP, is there any simple way to do that? I couldn't find any so far.
Thanks
This can be done a few ways..
Put some middleware on the connect to read the incoming request IP, check it against an IP list (an array), if it does not work then respond with an error code.
Use an ACL via NGINX, if their IP is not listed, NGINX will block the request
Not really based on IP but, control access via JWT. This would require you to generate the token and send it with each request but it does provide a little more security and you won't have to manage IP ranges. In this architecture your user would request the token, once the token is sent back you make the request to the socket server with the token in the head or as a req param. The socket server then decodes it, if it decodes correctly (meaning it valid) then let them connect, otherwise don't
Following #proxim0 post, I used a middleware :
io.use((socket, next) => {
fs.readFile(__dirname + '/config.json', function(err, file) {
if(err) return next(new Error("Internal error : " + err.message));
const config = JSON.parse(file.toString('utf8'));
if(!config.restrictAccess) return next();
var ip = socket.request.headers['x-forwarded-for'] || socket.request.socket.remoteAddress || null;
if(ip == null || config.allowedAddresses.indexOf(ip) == -1) {
console.log("Access denied from remote IP " + ip);
return next(new Error("Your IP address is not allowed to access this resource."));
}
next();
});
});
With a config.json file with the following format :
{
"restrictAccess": true,
"allowedAddresses": [
"127.0.0.1",
"123.456.789"
]
}
Any comment on improvment is welcome.
I know on a Mac OSX I can run this command: dns-sd -q a5b3ef18-2e66-4e24-91d2-893b93bbc1c1.local and it returns an IP address. Can I do this in Node.js? It seems like the dns module is mainly used for website -> IP, not IP -> IP (resolved) conversions. Any help is appreciated. Thanks!
Note: The imputted addresses will be mDNS, converted by Bonjour. I found the Bonjour npm package/library, but don't think it works in this case. Also, I found mdns which has the mdns.dns_sd function but I cannot seem to figure out how to use it in my case.
Thanks!
I found a Node module that does exactly what you need.
Multicast-DNS is capable of querying mDNS IPs to the standard IP format. The snippet on their README does what you need:
var mdns = require('multicast-dns')()
mdns.on('response', function(response) {
console.log('got a response packet:', response)
})
mdns.on('query', function(query) {
console.log('got a query packet:', query)
})
// lets query for an A record for 'brunhilde.local'
mdns.query({
questions:[{
name: 'brunhilde.local',
type: 'A'
}]
})
Obviously you need to replace brunhilde.local with a valid mDNS ip. I simplified the code into this:
function query(mdns_ip){
return new Promise((resolve, reject)=>{
mdns.on('response', function(response) {
if(response.rcode === 'NOERROR'){
resolve(response.answers[0].data)
mdns.destroy()
} else {
reject(response.rcode)
mdns.destroy()
}
})
mdns.query({
questions:[{
name: mdns_ip,
type: 'A'
}]
})
})
}
there I want to extract the domain name of incoming request using request module. I tried by looking at
request.headers,
request.headers.hostname
with no luck. Any help please?
I figured it out. Client domain is available at origin.
request.headers.origin
for ex:
if(request.headers.origin.indexOf('gowtham') > -1) {
// do something
}
Thanks!
You should use request.headers.host .
So you want the domain name of the client that is making the request?
Since Express only provides you with their IP-address, you have to reverse-lookup that IP-address to get the hostname. From there, you can extract the domain name.
In a middleware
const dns = require('dns');
...
app.use(function(req, res, next) {
dns.reverse(req.ip, function(err, hostnames) {
if (! err) {
console.log('domain name(s):', hostnames.map(function(hostname) {
return hostname.split('.').slice(-2).join('.');
}));
}
next();
});
});
However, very big disclaimer: making a DNS lookup for every request can have a severe performance impact.