I have created a cluster using the CloudBreak and that all works and I can log into the servers just fine. The problem that I am having is the network setups on the host os and the docker containers seems to not be setup right. The host os and the containers /etc/hosts file like like this
cloudbreak# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
This causes a problem for the hadoop cluster because all the nodes then don’t know how to communicate. If I change the hosts files to contain the other nodes then things start to work. However this does not seems like something I should have to do. This will also be a problem when trying make new clusters, as I will have to go in and make changes, and the auto scaling will not work if i have to change the host file on every host and docker container.
Any help would be helpful, thanks.
CloudBreak does not use the host file to resolve the other nodes in the cluster, It uses swarm and consul for discovery.
Related
I'm not a networking guru so could use some help. I am running a RHEL7 (Red Hat Enterprise Linux) VM (Virtual Machine) where some component on the OS is sending frequent DNS queries to resolve it's own local hostname to our main DNS server (which shouldn't be happening because the DNS server won't know anything about its address). Can anyone provide guidance as to how I can find out what component/service/process this is? It's filling our logs with 19k records over just hours and I need to find a way to fix this.
The hostname for the RHEL VM is spe1.2v29999999.dev.local , there is a static IP on this VM and it is 10.70.49.61. The /etc/hosts looks like:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost4 localhost4.localdomain4
I suspected it might be a java jar we have running on the VM, but I stopped it via systemctl stop MyJavaJar but after running a tcp dump via tcpdump -i any udp port 53, I could still see the queries happening.
Here are some examples from different days/times in the logs (both A and TXT records):
2020-05-05T13:53:50.189178+00:00 dns.green.blue.mycompany.com 127.0.0.1 <daemon.info> dnsmasq[20886]: 739 10.70.49.61/65078 query[A] spe1.2v29999999.dev.local from 10.70.49.61
2020-05-07T00:01:39.934899+00:00 dns.green.blue.mycompany.com 127.0.0.1 <daemon.info> dnsmasq[8615]: 27827 10.70.49.61/57348 reply spe1.2v29999999.dev.local is NXDOMAIN
2020-05-11T00:01:20.674688+00:00 dns.green.blue.mycompany.com 127.0.0.1 <daemon.info> dnsmasq[8615]: 130345 10.70.49.61/53321 query[TXT] bootstrap.spe1.2v29999999.dev.local from 10.70.49.61
Would making any changes to /etc/hostname, /etc/sysconfig, /var/named .zone files, /var/named.conf or /etc/named help? Can I do more with tcpdump? Thanks
Put:
127.0.0.1 spe1.2v29999999.dev.local
in /etc/hosts. This is done by default on many distros.
So I want to connect to my mongodb running on my host machine (DO droplet, Ubuntu 16.04). It is running on the default 27017 port on localhost.
I then use mup to deploy my Meteor app on my DO droplet, which is using docker to run my Meteor app inside a container. So far so good.
A standard mongodb://... connection url is used to connect the app to the mongodb.
Now I have the following problem:
mongodb://...#localhost:27017... obviously does not work inside the docker container, as localhost is not the host's localhost.
I already read many stackoverflow posts on this, I already tried using:
--network="host" - did not work as it said 0.0.0.0:80 is already in use or something like that (nginx proxy)
--add-host="local:<MY-DROPLET-INTERNET-IP>" and connect via mongodb://...#local:27017...: also not working as I can access my mongodb only from localhost, not from the public IP
This has to be a common problem!
tl;dr - What is the proper way to expose the hosts localhost inside a docker container so I can connect to services running on the host? (including their ports, e.g. 27017).
I hope someone can help!
You can use: 172.17.0.1 as it is the default host ip that the containers can see. But you need to configure Mongo to listen to 0.0.0.0.
From docker 18.03 onwards the recommendation is to connect to the special DNS name host.docker.internal
For previous versions you can use DNS names docker.for.mac.localhost or docker.for.windows.localhost.
change the bindIp from 127.0.0.1 to 0.0.0.0 in /etc/mongod.conf. Then it will work
or start mongod on ubuntu with a flag to bind all ip address as a temporary workaround (dev/learning purposes)
$ mongod --bind_ip_all
Tried 100500 variants for Windows (using docker desktop), but without any result...
Unfortunately, currently, Windows (at least docker desktop) is not supporting --net=host
Quoted from: https://docs.docker.com/network/network-tutorial-host/#prerequisites
The host networking driver only works on Linux hosts, and is not supported on Docker for Mac, Docker for Windows, or Docker EE for Windows Server.
You can try to use https://docs.docker.com/toolbox/
I started an instance on AWS ec2 and am trying to connect via my web browser to the app on the server running on port 3000. I've also turned off iptables...
I can telnet via telnet localhost 3000 and telnet 127.0.0.1 3000 but can't telnet via the hostname or ip like telnet ipaddress 3000.
When i do that, I get a connection refused. I think this has somethign to do with my hosts file but can't figure out what. My host file looks like this:
127.0.0.1 ip-108-205-72-168
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost6 localhost6.localdomain6
Provided that you gave the instance a public ip, have you checked the security groups? AWS security groups are associated with instances and apply inbound/outbound rules.
If you have already done that then my next step would probably be to make sure that the port is bound to the correct interface(s). ss -tupan | grep 3000
please make sure you iptable rules is right and can accessed by ip.
Could you confirm whether you using elastic IP over the amazon VM?
If yes, then it will do the entry in the host file automatically when you associate elastic IP to the EC2.
But if not, then need to do a manual entry.
Thanks,
SIM
I am using a LAMP stack. Will there be problems if I update the /etc/host file to reflect 127.0.0.1 as somename from localhost?
Thanks!
Edit:
I sometimes work in remote sites with no network. I have the same setup on different machines and I need the server name to know dynamically where to do changes, etc.
I edited /etc/hosts to show
127.0.0.1 localhost somename
Now, with my Wifi off I am trying http://somename and it is not connecting. If I turn my wifi on, it works. But I need it to work with no connection. How can I resolve this?
No. Just add somename after localhost separated by space in /etc/hosts. You can add as many aliases as you like, as long as you don't delete localhost.
Linux terminal showing hostname dhcppc4 that i never configured. As per configuration files hostname must be localhost :
[root#dhcppc4 ~]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=localhost.localdomain
[root#dhcppc4 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
Note : I attached a new monitor to my laptop.
Don't know how but sometimes terminal shows localhost as hostname.
what is dhcppc4? How to resolve the issue?
This question is not suitable for stackoverflow (this should be on serverfault i guess)
Anyway, DHCP server could even assign an hostname to your machine
Look here: RFC 1553 - DHCP
Maybe this is your case?
You are most assuredly allowed to edit the HOSTNAME line of that config' file you refer to in your question just fine.
It will not show up until you logout & back in again. You will also want to run /etc/init.d/rsyslogd restart (or whatever syslog daemon you are using) to make that name take effect in the log files.
You may or may not need to run a vi /etc/hosts to adjust the lines in there as you need for your specific applications. Most will be fine, but some are very very picky, like the otherwise very awesome software, Zimbra, for one example, wont work if those lines are not perfect. Asterisk could be another one that might require that file to be perfect match with your configuration.
You can or can not work with your DHCP server to set this, but most people dont, most edit the HOSTNAME and move on.
It is actually important, though, and do in fact resolve this, dont just leave it, as we sysadmins do not want to be typing mission critical commands at the BASH prompt on the wrong machine when all 6 of your servers all say "localhost".