I'm having a problem building Docker images on my corporate network. I'm just getting started with Docker, so I have the following Dockerfile for a hello-world type app:
# DOCKER-VERSION 0.3.4
FROM centos:6.4
# Enable EPEL for Node.js
RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# Install Node.js and npm
RUN yum install -y npm
# Bundle app source
ADD . /src
# Install app dependencies
RUN cd /src; npm install
EXPOSE 8080
CMD ["node", "/src/index.js"]
This works fine when I build it on my laptop at home, on my own wireless network. It pulls down the requisite dependencies and builds the image correctly.
However, when I'm on my corporate network at work, this same docker build fails when trying to pull down the RPM from download.fedoraproject.org, with this error message:
Step 2 : RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
---> Running in e0c26afe9ed5
curl: (5) Couldn't resolve proxy 'some.proxy.address'
error: skipping http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm - transfer failed
On my corporate network, I can access that URL just fine from my laptop. But once Docker is trying to build the container, all of a sudden it can't resolve at all. This behavior is the same for a variety of external resources (apt-get, etc.): They all can resolve just fine on my laptop on the corporate network, but Docker can't resolve them.
I don't have the network know-how to figure out what's going on here. Does anyone know why this strange behaviour would be occurring when building Docker containers?
I was able to figure out the issue. On Ubuntu, Docker sets the DNS servers for container to Google's servers at 8.8.8.x. As I understand it, this is a workaround on Ubuntu due to the fact that Ubuntu sets /etc/resolv.conf to be 127.0.0.1.
Those Google servers weren't accessible from behind our firewall, which is why we couldn't resolve any URLs.
The fix is to tell Docker which DNS servers to use. This fix depends on how you installed Docker:
Ubuntu Package
If you have the Ubuntu package installed, edit /etc/default/docker and add the following line:
DOCKER_OPTS="--dns <your_dns_server_1> --dns <your_dns_server_2>"
You can add as many DNS servers as you want to this config. Once you've edited this file you'll want to restart your Docker service:
sudo service docker restart
Binaries
If you've installed Docker via the binaries method (i.e. no package), then you set the DNS servers when you start the Docker daemon:
sudo docker -d -D --dns <your_dns_server_1> --dns <your_dns_server_2> &
I advise changing the DNS settings of the Docker daemon. You can set the default options for the docker daemon by creating a daemon configuration file at /etc/docker/daemon.json. Set DNS server according to your host machine, e.g. my DNS server is 10.0.0.2:
{"dns": ["10.0.0.2", "8.8.8.8"] }
Then you need just restart docker service:
sudo service docker restart
Step-by-step explanation is available here Fix Docker's networking DNS config
The following steps works for me ( for both docker build and docker run command). My linux version is Ubuntu 14.04.
Identify DNS using following command.
nm-tool | grep DNS
This result DNS:192.168.1.1 in my case
Create entry in /etc/default/docker.io. My current entry looks like this
DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 --dns 192.168.1.1"
Restart docker service
sudo service docker.io restart
For any Linux distribution working with SystemD (Ubuntu 16, RHEL 7...), the path will be displayed with the following command:
$ systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2016-06-29 08:10:33 PDT; 2min 34s ago
Docs: https://docs.docker.com
Main PID: 1169 (dockerd)
Tasks: 19
Memory: 85.0M
CPU: 1.779s
CGroup: /system.slice/docker.service
├─1169 /usr/bin/dockerd --dns 172.18.20.11 --dns 172.20.100.15 --dns 8.8.8.8 --dns 8.8.4.4 -H fd://
└─1232 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --met
The path would be /lib/systemd/system/docker.service. Add the DOCKER_OPTS values, which can have any of the --dns, in the line where the daemon is started.
cat /lib/systemd/system/docker.service | grep dns
ExecStart=/usr/bin/dockerd --dns 172.18.20.11 --dns 172.20.100.15 --dns 8.8.8.8 --dns 8.8.4.4 -H fd://
Docker (at least >=1.13, probably earlier) on Mac and Windows allow you configure the DNS in Preferences -> Daemon -> Advanced:
The following config sets two corporate DNS servers (use your own values here) with fallback to Google public DNS servers.
Specify your DNS to the Docker daemon.
First of all get your DNS address
$ nmcli dev show | grep 'IP4.DNS'
IP4.DNS[1]: 10.0.0.2
Test if the problem is really with the DNS by launching a docker container forcing this new DNS
$ docker run --dns 10.0.0.2 <image_name> <command_name>
If this solves the problem, you can apply this fix for all the docker daemons in the following way
Edit or create a file /etc/docker/daemon.json
Add the following line to this file
{
"dns": ["10.0.0.2", "8.8.8.8"]
}
Restart docker
$ sudo service docker restart
A very nice guide for doing ALL this process can be found here.
https://development.robinwinslow.uk/2016/06/23/fix-docker-networking-dns/
Solution without restarting Docker service
It is possible to modify the DNS settings for a single Docker image without affecting other docker build calls (and without restarting the Docker service) by overriding the resolv.conf at build time:
FROM ubuntu:18.04
RUN echo "nameserver 123.123.123.123" > /etc/resolv.conf && apt update
Replace the IP 123.123.123.123 with the one which is used within your corporate network (use nmcli dev show | grep 'IP4.DNS' to get the currently used DNS server).
Downsides:
This does not affect any other line from the Dockerfile. Hence, you have to prefix every line with the fix, if it depends on DNS resolution
On my Ubuntu 16.04 machine, sometimes, Google's DNS do not work for building Docker images.
cat /etc/docker/daemon.json
{"dns": [""8.8.8.8"] }
I have to manually find out my Service Providers DNS using the following command
nmcli device show <interfacename> | grep IP4.DNS
125.22.47.102
and add it to my daemon.json as show below
cat /etc/docker/daemon.json
{"dns": ["125.22.47.102","8.8.8.8"] }
restart docker
sudo service docker restart
(PS nm-tool is deprecated from Ubuntu 15.04)
Updated info September 2021
Inspired by Jason's answer; setting DNS server in the JSON didn't work for me in the current version, but there's now another place to set it:
When you turn on the toggle, the 8.8.8.8 is already there, so I just left it and it works well enough for me in my dev environment. I didn't research it but if wanted, there may be a way to add a list, perhaps separated by commas/semicolons/spaces etc.
Related
Continue on with Docker host network container service access under Windows,
I am having a hard time trying to expose services in the Linux container (i.e. their ports) so that they can be directly accessed from host.
If the host is Linux, I know a whole spectrum of tools to troubleshoot the situation. But when it comes to Windows host, I don't know where to start and how to troubleshoot step by step.
I'm starting docker with -p 3999:3999 on Windows, and within the Linux container I'm starting a Go based web service listening on 0.0.0.0:3999. These are the things I have been doing without any issue when the host is Linux. Now the only difference is,
The host is now Windows
It may not have any relevance, but the Windows is using corporation's transparent proxy.
How can I troubleshoot the situation step by step?
Update:
I don't have firewall installed --
% iptables -L
-bash: iptables: command not found
% sudo iptables -L
sudo: iptables: command not found
% ufw status
-bash: ufw: command not found
% dpkg -l | grep fire || echo no firewall found
no firewall found
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux bullseye/sid
Release: testing
Codename: bullseye
I had the same situation. I deployed react service in the container in the centos 8. The host from which I was trying to access the service was windows. I was unable to access the service from my windows host browser. And stopping the firewall did the thing for me.
systemctl stop firewalld
systemctl restart docker
Is that possible to have multiple IPs on eth0 in a Docker container?
I would like having 5 IPs on eth0 in a Docker container interface. I am using "ip" utility. Executing ip address add 172.20.0.200/16 dev eth0 in the container give "Operation not permited.
I tried manually log to the container as root user using "sudo exec
-u root ..".
I have even tried apt-install sudo in the container. Result is same "Operation not permitted"
I have found the answer. There must be added --cap-add option
docker run --cap-add=NET_ADMIN image
But as I understand docker know nothing about static IPs since docker network inspect shows only one IP. Hence think of using custom network
I have a docker container which has NVM installed by default. when I try to install any version of node, or running command nvm ls-remote it fails to connect to it's server every time.
the message is:
Version '6.11.2' not found - try nvm ls-remote to browse available versions.
this error occurs just in this network I am joining to.
it is my /etc/resolve.conf file content:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
search SAD.UM.AC.IR
nameserver 8.8.8.8
nameserver 8.8.4.4
result of docker network ls command is:
NETWORK ID NAME DRIVER SCOPE
dc8cffbb2789 bridge bridge local
5efb2b5fb44e host host local
8c19a2b28c14 none null local
It is just a network problem!! Is there any thing to solve this??
Finally after a discussion with #TarunLalwani, I find what is wrong in this case. The problem is that my container does not use my host machine internet, so it has no internet access. I use this command when I run the container:
docker run -it -v somewhere/:/somewhere --net=host -p 8585:8585 --name test docker-image
--net=host added to command.
I set up a one-click docker server on DigitalOcean, then ssh'd into it as root#[Server IP] and ran the following
docker pull continuumio/memex-explorer
docker run -p 80:5000 continuumio/memex-explorer
Which outputs:
* Starting OpenBSD Secure Shell server sshd
...done.
* Running on http://0.0.0.0:5000/
* Restarting with reloader
Then when I navigate to [Server IP]:5000 it doesn't display anything. I expected it to present the landing page of the app.
I then ran
ufw allow 5000/tcp
ufw allow 80/tcp
ufw enable
but it didn't help.
Can anyone install and set up this app? There's a link to the source of the app I'm trying to run: Here, and the docker image: Here
It looks like you just needed to run it in detached mode.
I just provisioned a droplet on Digital Ocean and spun up the Docker image with this run command:
sudo docker run -d -p 80:5000 --name memex continuumio/memex_explorer
There is no need to change any firewall settings.
Make sure the container is active:
sudo docker ps
It should display something like this:
64242f576c16 continuumio/memex_explorer:latest "/root/memex-explore 35 minutes ago Up 35 minutes 22/tcp, 80/tcp, 0.0.0.0:80->5000/tcp memex
To see the application running, just type the Digital Ocean [Server IP] into the URL for the browser. The port is redirected to port 80 so no need to type it in.
You can attach to the container and look around (the Dockerfile is available in the image as well).
sudo docker exec memex bash
I'm having an interesting problem running docker containers: out of the blue, I'm not able to resolve DNS from within the container.
Here's a rundown:
Nothing is resolving; apt-get, pip, one-off ping containers, etc. Running docker run -it --dns=8.8.8.8 ubuntu ping www.google.com results in ping: unknown host www.google.com both with and without the --dns flag.
I can reach 8.8.8.8 both from inside and outside the containers. docker run -it ubuntu ping 8.8.8.8 works.
I've configured the containers to use both 8.8.8.8, 8.8.8.4, and my local network DNS servers (in various permutations) both by editing /etc/sysconfig/docker to add DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.8.4" and by setting the --dns 8.8.8.8 flag on the containers at runtime. systemctl restart docker was run between each change.
ip_forward is enabled. (see here)
I've done a full reset as specified here.
I'm running Fedora 21, docker client version 1.5.0.
Any ideas? I'm at a complete loss as to what's preventing docker from accessing the Internet successfully.
They have fixed the issue in 1.8: https://github.com/docker/docker/issues/13381 Cheers.
After all this, a full reboot solved the problem - although, that still doesn't answer what it was.
If anyone knows what the actual cause was, I'm still curious, but for now the problem is gone.
You could install tools like dig inside the docker image to alanyse the issue.
use static hosts in your /etc/hosts file te make apt-get run inside the docker image. add these lines to your hosts file:
213.32.5.7 debian.mirrors.ovh.net
141.76.2.4 ftp.de.debian.org
217.196.149.233 mirror-conova-security.debian.org
212.211.132.250 lobos.debian.org
212.211.132.250 security.debian.org
5.153.231.4 http.debian.net
151.101.12.204 cdn-fastly.deb.debian.org
151.101.12.204 security-cdn.debian.org