docker-machine breaks docker native client on linux - linux

I am on Ubuntu and decided to use docker-machine to run some docker swarm tests. Here you execute
eval $(docker-machine env xxxxx)
and with that your native docker client points to that machine/vm. However, after the tests I wan't the docker command to point to my local docker client/daemon/whatever and executed
eval $(docker-machine env -u)
which is supposed to unset the environment variables. But now I get this error
docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
See 'docker run --help'.
I've had to create a docker machine on VirtualBox called default, point to that machine and run my commands there. But its pretty lame since I feel like I am back on Windows and one of the reason I came to Ubuntu was better docker integration.
Is there any fix for this?


unset all docker variables
unset ${!DOCKER_*}
regarding the 'can't connect to daemon', ensure you're prepending each docker command with sudo, or to allow your current user to interact with docker use:
sudo groupadd docker
sudo usermod -aG docker $(whoami)
restart docker and
re-login to the terminal

Related

Run sshd in Docker container

I found this Dockerfile sample here:
// version 1
FROM ubuntu:latest
RUN apt update && apt install ssh -y
RUN service ssh start
CMD ["/usr/sbin/sshd","-D"]
When I build and run this Dockerfile, it runs an SSH server in the foreground, which is great.
If I use the following Dockerfile though:
// version 2
FROM ubuntu:latest
RUN apt update && apt install ssh -y
RUN service ssh start
# CMD ["/usr/sbin/sshd","-D"] // without this line
And then run the container:
~$ docker run -p 2222:22 -it ssh_server
And try to connect to it from another terminal, it doesn't work. Seemingly this call to sshd is necessary. On the other hand, If I just install SSH in the Dockerfile:
// version 3
FROM ubuntu:latest
RUN apt-get update && apt-get install -y ssh
And run the container like this:
~$ docker run -p 2222:22 -it ssh:test
~$ service ssh start
* Starting OpenBSD Secure Shell server sshd
Now I'm able to connect to the container. So I wonder: If the line RUN ssh service start
in version 1 is necessary, why isn't necessary for version 3?
To add more to the confusion, if I build and run version 4:
// version 4
FROM ubuntu:latest
RUN apt update && apt install ssh -y
#RUN service ssh start // without this line
CMD ["/usr/sbin/sshd","-D"]
It also doesn't work either.
Can someone please explain those behaviours? What is the relation between service ssh start and /usr/sbin/sshd?

OK everything is clear now:
Basically running the /usr/sbin/sshd is what runs the ssh server. The reason it didn't work out on it's own (version 4) is because the script that runs when you run service ssh start - which is the script /etc/init.d/ssh - creates a directory /run/sshd which is required for the run of sshd.
This script also calls the executable /usr/sbin/sshd, but since this is run as part of the build, it didn't sustain beyond the temporary container that the layer was made of. W
What did sustain is the /run/sshd directory! That's why when we run /usr/sbin/sshd as the CMD it works!
Thanks all!

To build on #YoavKlein's answer, service ssh start can take arguments which are passed to sshd, so rather than
# Incidentally creates /run/sshd
RUN service ssh start
# Run the service in the foreground when starting the container
CMD ["/usr/sbin/sshd", "-D"]
you can just do
# Run the service in the foreground when starting the container
CMD ["service", "ssh", "start", "-D"]
which will start the SSH server through service, but run it in the foreground, avoiding having to have a separate RUN to do first time setup.

I have taken the idea from #mark-raymond :)
Following docker run command with the -D flag worked for me!:
docker run -itd -p 2222:22 <dockerImageName:Tag> /usr/sbin/sshd -D

Running desktop enviroment in docker in headless linux

is it possible to run in headless linux, to be exact, linux with no desktop enviroment with GUI from inside docker.
(only if couldt be done differently with x server of some sort, but I would rather run everything within docker)
I want to run GUI only on occasions and I dont want it to share the userspace with the base system programs. Also I dont want to preserve the DE till the next occasion that is needed.

Sure it's possible!
First let's create a docker volume to store the X11 socket:
docker volume create --name xsocket
Now we can create an image with X Server:
FROM ubuntu
RUN apt-get update && \
DEBIAN_FRONTEND='noninteractive' apt-get install -y xorg
CMD /usr/bin/X :0 -nolisten tcp vt1
Let us build it and start it and store the X11 socket in xsocket docker volume:
docker build . -t docker-x-server:latest
docker run --privileged -v xsocket:/tmp/.X11-unix -d docker-x-server:latest
Now we can run a GUI application in another docker container (yay!) and point it to our X server using xsocket volume:
docker run --rm -it -e DISPLAY=:0 -v xsocket:/tmp/.X11-unix:ro stefanscherer/xeyes
If you need input (like keyboard) install xserver-xorg-input-evdev package and add -v /run/udev/data:/run/udev/data since there's no udev in containers by default.
You can even get rid of --privileged flag by granting SYS_TTY_CONFIG capability and binding some devices into container:
docker run --name docker-x-server --device=/dev/input --device=/dev/console --device=/dev/dri --device=/dev/fb0 --device=/dev/tty --device=/dev/tty1 --device=/dev/vga_arbiter --device=/dev/snd --device=/dev/psaux --cap-add=SYS_TTY_CONFIG -v xsocket:/tmp/.X11-unix -d docker-x-server:latest

How to SSH into running docker container from jenkins execute shell

I am running a docker container (dind) from jenkins execute shell
CONTAINER_ID="$(sudo docker run --privileged -i -d jpetazzo/dind)"
To execute docker commands inside container I get into container shell
sudo docker exec -it --privileged ${CONTAINER_ID} bash
and than I am trying to execute these commands inside dind container.
sudo docker pull hubuser/hello-world
sudo docker run hubuser/hello-world
sudo docker tag imageId hubuser/hello-world:123
sudo docker login --username=hubuser --password=password
sudo docker push hubuser/hello-world
All of these 7 commands I have written in jenkins execute shell. Below given 5 commands are executing outside dind container, not inside. If I am trying from terminal than it is attaching to the container shell and executing properly. I want to execute them inside container but from jenkins. I also tried adding exec before every command like this:
sudo docker exec -it --privileged ${CONTAINER_ID} sudo docker pull hubuser/hello-world
sudo docker exec -it --privileged ${CONTAINER_ID} sudo docker run hubuser/hello-world
and so on. This executes commands inside dind container but all commands executes in parallel, so before pulling and running the hello-world image, it tries to tag it and push it. There it is not finding any hello-world image to tag and it does't do anything.
I want all my below 5 commands to execute serially inside dind container, that too from jenkins execute shell.

The title of your post is "how to ssh into running docker". I just want to point out this article, wrote by a Docker engineer: If you run SSHD in your Docker containers, you're doing it wrong!
After I read your post, that is not treating issue with ssh, I just thought about: why not execute a bash-script that does sequentially what you wanted to?
I'm not sure if I understood well btw, is Jenkins inside a docker? Are you running a docker in a docker?

Tips on getting docker to work without having to run `sudo docker -d` on Ubuntu 15.04

After upgrading my system from 14.10 to 15.04 I can't seem to use docker like I used to. I already have a docker group that my user is part of and I used to be able to use docker without sudo just fine. Now I can't use it unless I have sudo docker -d running in another terminal. Simply running docker ps gives me this error:
FATA[0000] Get http:///var/run/docker.sock/v1.18/containers/json: dial unix /var/run/docker.sock: no such file or directory. Are you trying to connect to a TLS-enabled daemon without TLS?
I've tried reinstalling, rebooting, restarting services, and blowing out configurations to no avail. Any tips would be appreciated. As a side note, I installing 15.04 in a vm to see if I could get docker working there and I was able to set it up no problem. seems like an issue specific to those who have upgraded from 14.10.

Did u checked this http://docs.docker.com/articles/systemd/? This helped me to start docker under Ubunu 15.04.

What to do if this fails...
$ sudo usermod -aG docker $USER
..and you have added user to docker group and Ubuntu still requires sudo:
If you initially ran Docker CLI commands using sudo before adding your user to the docker group, you may see the following error, which indicates that your ~/.docker/ directory was created with incorrect permissions due to the sudo commands.
To fix this problem, either remove the ~/.docker/ directory (it is recreated automatically, but any custom settings are lost), or change its ownership and permissions using the following commands:
$ sudo chown "$USER":"$USER" /home/"$USER"/.docker -R
$ sudo chmod g+rwx "$HOME/.docker" -R

What the link mafahand provided tells is how to use docker on a systemd based host. Ubuntu 15.04 uses systemd now while older version used upstart. That might explain why upgraded systems show erratic behavior. Check out the Ubuntu wiki for some help on that regard.
After installing docker via
sudo apt install docker.io
you might have to reboot your system or start the docker.socket unit manually. For some reason that did not happen on my machine after installing it.
Type
systemctl status docker
to check whether docker is up and running. If it is not enabled use
sudo systemctl enable docker
to enable it permanently and/or
sudo systemctl start docker
to run the service.

Docker can't connect to docker daemon

After I update my Docker version to 0.8.0, I get an error message while entering sudo docker version:
Client version: 0.8.0
Go version (client): go1.2
Git commit (client): cc3a8c8
2014/02/19 12:54:16 Can't connect to docker daemon. Is 'docker -d' running on this host?
And I've followed the instructions and entered command sudo docker -d, and I got this:
[/var/lib/docker|2462000b] +job initserver()
[/var/lib/docker|2462000b.initserver()] Creating server
open /var/lib/docker/aufs/layers/cf2414da53f9bcfaa48bc3d58360d7f1cfd3784e4fe51fbef95197709dfc285d: no such file or directory[/var/lib/docker|2462000b] -job initserver() = ERR (1)
2014/02/19 12:55:57 initserver: open /var/lib/docker/aufs/layers/cf2414da53f9bcfaa48bc3d58360d7f1cfd3784e4fe51fbef95197709dfc285d: no such file or directory
How do I solve the problem?

Linux
The Post-installation steps for Linux documentation reveals the following steps:
Create the docker group.
sudo groupadd docker
Add the user to the docker group.
sudo usermod -aG docker $(whoami)
Log out and log back in to ensure docker runs with correct permissions.
Start docker.
sudo service docker start
Mac OS X
As Dayel Ostraco says is necessary to add environments variables:
docker-machine start # Start virtual machine for docker
docker-machine env # It's helps to get environment variables
eval "$(docker-machine env default)" # Set environment variables
The docker-machine start command outputs the comments to guide the process.

Linux
To run docker daemon on Linux (from CLI), run:
$ sudo service docker start # Ubuntu/Debian
Note: Skip the $ character when copy and pasting.
On RedHat/CentOS, run: sudo systemctl start docker.
To initialize the "base" filesystem, run:
$ sudo service docker stop
$ sudo rm -rf /var/lib/docker
$ sudo service docker start
or manually like:
$ sudo docker -d --storage-opt dm.basesize=20G
Install docker-machine on Linux
To install machine binaries on Linux:
locally:
install -vm755 <(curl -L https://github.com/docker/machine/releases/download/v0.5.3/docker-machine_linux-amd64) $HOME/bin/docker-machine
global:
sudo bash -c 'install -vm755 <(curl -L https://github.com/docker/machine/releases/download/v0.5.3/docker-machine_linux-amd64) /usr/local/bin/docker-machine'
macOS
On macOS the docker binary is only a client and you cannot use it to run the docker daemon, because Docker daemon uses Linux-specific kernel features, therefore you can’t run Docker natively in OS X. So you have to install docker-machine in order to create VM and attach to it.
Install docker-machine on macOS
If you don't have docker-machine command yet, install it by using one of the following methods:
Using Brew command: brew install docker-machine docker.
manually from GitHub:
install -v <(curl https://github.com/docker/machine/releases/download/v0.5.3/docker-machine_linux-amd64) /usr/local/bin/docker-machine
See: Get started with Docker for Mac.
Configure docker-machine on macOS
To start Docker Machine via Homebrew, run:
brew services start docker-machine
To create a default machine (if you don't have one, see: docker-machine ls):
docker-machine create --driver virtualbox default
Then set-up the environment for the Docker client:
eval "$(docker-machine env default)"
Then double-check by listing containers:
docker ps
See: Get started with Docker Machine and a local VM.
Install Docker.app on macOS
Alternatively to above solution, you can install a Docker app by:
brew cask install docker
Check this post for more details. See also: Cannot connect to the Docker daemon on macOS

If you are running Docker on OS X, running the following eval has worked for me.
eval "$(docker-machine env default)"
If you'd prefer not to have to run this eval statement on every terminal session, you can add this to your bash_profile:
#Docker
eval "$(docker-machine env default)"
Be sure to restart the terminal session or run source on bash_profile for the changes to take effect.

After a detailed investigation, this issue seems to happen every time after Mac OS X is rebooted (or the Docker virtual machine is restarted) which prevents the Docker client from connecting to the Docker daemon.
To solve the issue, you can either:
A) Reinstall Docker Toolbox using the official installer (https://www.docker.com/products/docker-toolbox), or simply
B) Run the following commands in order:
# First make sure that the virtual machine is running
docker-machine start default
# Regenerate TLS connection certs, requires confirmation
docker-machine regenerate-certs default
# Finally, set env
eval "$(docker-machine env default)"
C) Same as (B), you can also copy and paste the following line to run all of the three commands:
docker-machine start default; docker-machine regenerate-certs default; eval "$(docker-machine env default)"
In case you get the following error:
Error getting SSH command: Something went wrong running an SSH command!
command : cat /etc/os-release
err : exit status 255
output :
just re-run the three commands another time, and it should work the second time.

This usually happens when you are not in the docker group. You can add yourself to the docker group with:
sudo usermod -aG docker yourusername
or
sudo usermod -aG docker $(whoami)
After this, you need to logout and log back into the server.
Alternatively, you can sudo every Docker command.

If all the other solutions above don't work you can try checking the ownership of /var/run/docker.sock:
ls -l /var/run/docker.sock
If you're not the owner then change ownership with the command:
sudo chown *your-username* /var/run/docker.sock
Then you can go ahead and try executing the Docker commands hassle-free :D

You can use the command
sudo service docker stop && sudo service docker start
OR
sudo service docker restart
to simply restart it.

The best way to find out why Docker isn't working will be to run the daemon manually.
$ sudo service docker stop
$ ps aux | grep docker # do this until you don't see /usr/bin/docker -d
$ /usr/bin/docker -d
The Docker daemon logs to STDOUT, so it will start spitting out whatever it's doing.
Here was what my problem was:
[8bf47e42.initserver()] Creating pidfile
2015/01/11 15:20:33 pid file found, ensure docker is not running or delete /var/run/docker.pid
This was because the instance had been cloned from another virtual machine. I just had to remove the pidfile, and everything worked afterwards.
Of course, instead of blindly assuming this will work, I'd suggest running the daemon manually one more time and reviewing the log output for any other errors before starting the service back up.

Do a ps aux | grep docker to see if the daemon is running. If not run /etc/init.d/docker start

If you get the message Can't connect to docker daemon. Is 'docker -d' running on this host?, you can check it by docker version.
If you see the information like Docker Client is running. but Docker Server is not, it's obviously you need to start the Docker server.
In CentOS, you can use service to start or stop the Docker server.
$ sudo service docker stop
$ sudo service docker start
Then, after you type docker version, you will get the information of Docker Client and Docker Server, and the Docker daemon has been started.

Use Docker CE app
macOS
Use the new Docker Community Edition app for macOS. For example:
Uninstall all Docker Homebrew packages which you've installed so far:
brew uninstall docker-compose
brew uninstall docker-machine
brew uninstall docker
Install an app manually or via Homebrew-Cask:
brew install --cask docker
Note: This app will create necessary links to docker, docker-compose, docker-machine, etc.
After running the app, checkout the a Docker whale icon in the status menu.
Now you should be able to use docker, docker-compose, docker-machine commands as usual in the Terminal.
Related:
Brew install docker does not include docker engine?
Cannot connect to the Docker daemon on macOS
Linux/Windows
Download the Docker CE from the download page and follow the instructions.

I have similar problem. I had to logout and login again to shell because I have just installed Docker and following command didn't show in my environment.
export DOCKER_HOST=127.0.0.1:4243 >> ~/.bashrc

I restart Docker after installing it:
$ sudo service docker stop
$ sudo service docker start
And it works.

I have faced this problem, and I restarted Docker using these commands:
$ sudo service docker stop
$ sudo service docker start
But I did not solve my problem, because I forgot to execute my Docker commands without sudo. For those who faces this problem, try to check that.
Try
$ sudo docker info
instead of this:
$ docker info

I have the same error and trying docker-machine regenerate-certs or eval.. did not work for me.
This on OS X 10.11.3 (El Capitan) and Docker v1.10.1. I was able to fix it only by deleting and recreating docker-machine again. Source
If running docker-machine ls, it shows you a similar output to the one below;
DOCKER
Unknown
ERRORS
Unable to query docker version: Cannot
connect to the docker engine endpoint
Try removing your Docker machine with;
docker-machine rm -f default
Where default is your Docker machine name. Then;
docker-machine create -d virtualbox default
Creates a new Docker machine.
Double check that everything looks normal now (no errors or unknown Docker) with:
docker-machine ls
Finally don't forget to run "$(docker-machine env default)" before you continue or run the Docker Quickstart Terminal which does it for you...

I knew that there are plenty of answers already in this post. Just I would like to add one simple answer that is solved the above mentioned problem .
sudo systemctl start docker
Run the above command and it will start all the docker related threads/services.

Try adding the current user to docker group:
sudo usermod -aG docker $USER
Then log out and login.

At April 2020 on MacOS Catalina, you just need to open the desktop application:

I had the same problem - "Can't connect to docker daemon." (except I didn't get any 'file not found' errors on trying to start the server.)
'ps' showed that "/usr/bin/docker -d" was still running
I realised that I'd never actually succeeded in running the server myself though. Every attempt had produced
...
2014/03/24 21:57:29 pid file found, ensure docker is not running or delete /var/run/docker.pid
So I belatedly realised that installing docker had maybe registered the daemon with upstart, which had started it for me. Hence, trying to kill the daemon to manually restart it fails (operation not permitted). So I did a
sudo kill -9 <PID>
on the daemon process. Another daemon immediately took its place, and this new one DOES now let my CLI client connect:
$ sudo docker info
Containers: 0
Images: 0
Driver: aufs
Root Dir: /var/lib/docker/aufs
Dirs: 0
WARNING: No memory limit support
WARNING: No swap limit support

Following Docker's DOC site: Manage Docker as a non-root user
1) Create Docker Group
sudo groupadd docker
2) Make user belong to docker group to get the group's privileges.
sudo usermod -aG docker $USER
Check whether the DOCKER_HOST environment variable is set for your shell.
env | grep DOCKER_HOST
If it exists,
unset DOCKER_HOST
Then this should work:
docker run hello-world

I just had the same issue, running on Amazon AWS.
Here's what I attempted:
Set up docker-machine locally with already existing AWS instance
Used generic setup
It kind of connected, but since the remote port was closed, it failed
After that, the Docker daemon refused to start up, but running dockerd did work...
It was tested following on the remote machine:
service docker start # Also restart, no success
systemctl start docker # Also restart, no success
dockerd # Success
I removed /var/lib/docker and uninstalled everything, but there was no success after reinstallation. Unfortunately I have no logs stored from failures, but docker.service just refused to start.
However, what finally solved my issue was basically:
sudo usermod -aG docker $(whoami)

I got the same problem. In CentOS 6.5:
ps aux |grep `cat /var/run/docker.pid`
If it shows no Docker daemon process exists, then I type:
docker -d
Then Ctrl + D to stop Docker. Because we use the -d option, Docker will run as daemon. Now we can do:
service docker start
Then I can do a docker pull centos. That's all.
NOTE: If these do not work, you can try yum update, and then repeat these again, because I yum install before these.

If you are running on OS X using Docker tool, follow this.
Restart the daemon and configure your environment:
docker-machine restart
And then
docker-machine env
Finally,
eval $(docker-machine env)
To test the daemon is running:
docker ps -a or docker-machine ls. This will list all containers.

The Docker Service may not be running.
If you are on a RedHat/Fedora/CentOS, please try this:
sudo systemctl start docker
If you are on Ubuntu/Debian:
sudo service start docker
Docker will start running on your host and respective port.

Run the following command:
docker context use default

To fix this issue, I had to enable the docker service:
sudo systemctl enable /usr/lib/systemd/system/docker.service

Check if you are using Docker Machine :)
Run docker-machine env default should do the trick.
Because according to documentation:
Docker Machine is a tool that lets you install Docker Engine on
virtual hosts, and manage the hosts with docker-machine commands. You
can use Machine to create Docker hosts on your local Mac or Windows
box, on your company network, in your data center, or on cloud
providers like AWS or Digital Ocean.
Using docker-machine commands, you can start, inspect, stop, and
restart a managed host, upgrade the Docker client and daemon, and
configure a Docker client to talk to your host.
Point the Machine CLI at a running, managed host, and you can run
docker commands directly on that host. For example, run
docker-machine env default to point to a host called default, follow on-screen
instructions to complete env setup, and run docker ps,
docker run hello-world, and so forth.
https://docs.docker.com/machine/overview/

I also had the same issue. The problem was in sockets allocated to docker-daemon and docker-client.
First, permission was not set for the docker-client on docker.sock You can set it using "sudo usermod -aG docker $USER"
Then check your bash file where the docker-client is running, For me it was on 0.0.0.0:2375, while docker-daemon was running on unix socket.(It was set in the configuration file of dockerd).
Just comment the bash-line and it'll work fine.
But if you want to make it work on TCP port instead of unix socket, change the configuration file of dockerd and set it on 0.0.0.0.2375 and keep the line in bash as it is if present or set it to 0.0.0.0:2375.

To fix, you need to issue the following commands in the terminal. I'll explain each step:
# Uninstall Docker from apt packages
$ sudo apt-get remove docker docker.io
# Remove it from the libraries just to be
# sure it's gone forever
$ sudo rm -rf /var/lib/docker/*
Now, if you want to simplify things and get more time, you can run my init script with the parameter installDocker:
# Pull the init script from GitHub
$ wget https://github.com/dminca/dotfiles/blob/master/init
# Add rights to run the script
$ chmod 755 init
# Just run the script with the installDocker parameter
$ ./init installDocker
A reboot is optional, but I suggest you do it to be sure all runs smoothly.

I had the same problem running Docker 1.10 on Ubuntu 14.04 and none of the given answers worked. For me, the fix was to specify the storage driver when running the Docker daemon.
sudo docker daemon --storage-driver=devicemapper

Resources