I would like to know why --group-add docker in the following doesn't work.But I have another priority image that it does work.
docker run \
--rm \
-it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /etc/docker/daemon.json:/etc/docker/daemon.json \
-v /etc/shadow:/etc/shadow \
-v /etc/passwd:/etc/passwd \
-v /etc/sudoers:/etc/sudoers \
-v /etc/group:/etc/group \
-u $(id -u):$(id -g) \
--group-add docker \
docker/compose:debian-1.27.4 \
bash
it errors out
docker: Error response from daemon: Unable to find group docker.
I have the same issue with other images like ubuntu, hello-world and so on. What is needed in the image to be able to add docker group?
My system:
Ubuntu 18.04
docker 19.03.13
I know I have the docker group in the host. I can see it in the output of groups.
With you command, you are trying to add current user to the group docker, but inside docker, not in the host.
As group docker does not exist inside docker, reason why you've got error message.
If it's really what you want, you can use groupadd docker to create the group, then add the user to the group.
Unfortunately, docker tries to set the groups (as well as uid/gid) before it mounts volumes, so the group docker (which is defined in /etc/group on your host) is not visible at that point inside the container since it is not it the /etc/group file contained in the image.
[Edit] In modern docker (e.g. 20.10.23) using numerical gid for docker group will do the trick. For example if docker gid is 999 use --group-add 999 instead of --group-add docker.
One possible solution could be to create your own image and put the appropriate /etc/group (and possibly passwd) file inside it. It's not that convenient as manipulating the compose file only but certainly would do the trick.
Another alternative could be to use setpriv as container entry point. Since entry point is executed after the volumes are mounted it can access the content of mounted /etc/group file. Other similar tools like gosu or su-exec do not support auxiliary groups ARAIK, but you may find some security-related information in their documentation that you may need to take into consideration, depending on your use cases.
Related
I am trying to use a remote server to run experiments on Docker. The problem is that I have scripts that actively modify configuration files within the Docker container to run experiments, which I can only do if the user/group that owns the files does it (I do not have root access on the remote server).
On my local system, the user / group permissions are my personal user when accessed without launching the docker container. And as soon as the docker container is launched, the user / groups change to alice / alice by configuration as seen in the Dockerfile. But on the remote server, it shows as root / root even after launching in the docker container. Any suggestions?
Within my build/run shell script for Docker, I have the lines:
.
.
.
CURR_UID=$(id -u)
CURR_GID=$(id -g)
RUN_OPT="-u $CURR_UID:$CURR_GID --net=host --env DISPLAY=$DISPLAY \
--volume $XAUTHORITY:/home/alice/.Xauthority \
--volume /tmp/.X11-unix:/tmp/.X11-unix \
--privileged $MOUNT_DEVEL $MOUNT_LEARN \
--shm-size $SHM_SIZE $GPU_OPT $CONT_NAME \
-it $DETACH --rm $IMAGE_NAME:latest"
docker run $RUN_OPT
.
.
.
The run option -u $CURR_UID:$CURR_GID is supposed to set the user/group permissions to whatever user/group is running it at the moment. And within my Dockerfile:
.
.
.
# Working user
RUN groupadd --gid ${GROUP_ID} alice && \
useradd -m -s /bin/bash -u ${USER_ID} -g ${GROUP_ID} alice && \
echo "alice:alice" | chpasswd && adduser alice sudo
.
.
.
I can provide more information if needed, I really just need any help at all. Been at this for days. Please advise. Thank you.
In your docker container you can set the effective user with the directive:
USER alice
It is documented here: https://docs.docker.com/engine/reference/builder/#user
The scenario is docker inside/beside docker via a sock binding for the purpose of having an easily deployable and scalable runner agent for C.I./C.D. tools (in this particular case, VSTS). The reason for this set up is that the various projects that I want to test use docker/compose to run tests, and configuring a C.I./C.D. worker to be compatible with docker/compose a bunch of times gets cumbersome and time consuming. (This'll eventually be deployed to 4+ Kubernetes Clusters)
Anyway, the problem:
Steps to replicate
Run the vsts-agent image
docker run \
-it \
-v /var/run/docker.sock:/var/run/docker.sock \
nullvoxpopuli/vsts-agent-with-aws-ecr:latest \
/bin/bash
Run another image (to emulate docker/compose running tests)
echo 'test' > test-file.txt
docker run -it -v file-test.txt:/file-test.txt busybox /bin/sh
Check for existence of test-file.txt
cd /
ls -la # shows that test-file.txt is a directory
So,
- why are files being mounted as folders inside containers?
- what do I need to do to make the volumes mount correctly?
Solution A - thanks to #BMitch
# On Host machine
docker run -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp/vsts/work/:/tmp/vsts/work \
nullvoxpopuli/vsts-agent-with-aws-ecr:latest \
/bin/bash
# In vsts-agent-with-aws-ecr
cd /tmp/vsts/work/
git clone https://NullVoxPopuli#bitbucket.org/group/project.git
cd project/
./scripts/run/eslint.sh
# Success! (this uses docker-compose to map files to the node-based docker image)
Docker creates containers and mounts volumes from the docker host. Any time a file or directory in a volume mount doesn't exist, it gets initialized as an empty directory. So if you are running docker commands from inside of a container to the docker socket those commands get interpreted outside the container on the docker host, where the file doesn't exist. Additionally, the docker run command requires a full path to the volume being mounted when you want a host volume, otherwise it's interpreted as a named volume.
What you likely want to do at this point is:
docker volume rm file-test.txt
docker run -it -v $(pwd)/file-test.txt:/file-test.txt busybox /bin/sh
If instead you are trying to include a file from inside the container to another container, you can initialize a named volume with input redirection like this:
tar -cC . . | docker run -i --rm -v file-test:/target busybox tar -xC /target
docker run -it -v file-test:/data busybox /bin/sh
That uses tar to copy the contents of the current directory to stdout which is processed by the interactive docker command which then extracts those directory contents into /target inside the container which is a named volume. Note that I didn't mount the volume in root in this second example since named volumes are directories and I didn't want to replace the root filesystem.
Another option is to share a volume mount point between multiple containers on the docker host so that files you edit inside one container go to the host where they are mounted into the other container and visible there:
docker run \
-it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /container-data:/container-data \
nullvoxpopuli/vsts-agent-with-aws-ecr:latest \
/bin/bash
echo 'test' > /container-data/test-file.txt
docker run -it -v /container-data:/container-data busybox /bin/sh
I don't recommend mounting individual files into a container if these files may be modified while the container is running. File changes often result in a changed inode and docker will have the old inode mounted into the container. As a result, changes either inside or outside of the container to the file may not be seen on the other side, and if you modify the file inside the container, that change may be lost when you delete the container. The solution to the inode issue is to mount the entire directory into the container.
I am logged in in my PC (Fedora 24) as rperez. I have setup Docker for being able to run through this user, so I am running a container as follow:
$ docker run -d \
-it \
-e HOST_IP=192.168.1.66 \
-e PHP_ERROR_REPORTING='E_ALL & ~E_STRICT' \
-p 80:80 \
-v ~/var/www:/var/www \
--name php55-dev reypm/php55-dev
Notice the $ sign meaning I am running the command as a non root user (which uses #). The command above creates the following directory: /home/rperez/var/www but owner is set to root I believe this is because docker run as root user behind scenes.
Having this setup I am not able to create a file under ~/var/www as rperez because the owner is root so ...
What is the right way to deal with this? I have read this and this but is not so helpful.
Any help?
As discussioned here, this is an expected behavior of docker. You can create the target volume directory before running docker command or change the owner to your current user after the directory is created by docker:
chown $(whoami) -R /path/to/your/dir
I hit this same issue (also in a genomics context for the very same reason) and also found it quite unintuitive. What is the recommended way to "inherit ownership". Sorry if this described elsewhere, but I couldn't find it. Is it something like:
docker run ... -u $(id -u):$(id -g) ...
Running a docker container with the --rm option deletes a mounted volume post exit. I'm wondering whether this is intended behavior?
Here is the exact sequence.
ole#MKI:~$ docker volume create --name a-volume-test
ole#MKI:~$ sudo ls /var/lib/docker/volumes/ | grep a-
a-volume-test
ole#MKI:~$ docker run --rm -it -v a-volume-test:/data alpine /bin/ash
/ # touch /data/test
/ # ls /data
test
/ # exit
ole#MKI:~$ sudo ls /var/lib/docker/volumes/ | grep a-
After I exit the the volume is gone.
This was a bug that will be fixed in docker 1.11 - https://github.com/docker/docker/pull/19568
According to the Docs, no that is not intended, because you are mounting a named volume it should not be deleted. Maybe submit a github issue?
Note: When you set the --rm flag, Docker also removes the volumes associated with the container when the container is removed. This is similar to running docker rm -v my-container. Only volumes that are specified without a name are removed. For example, with docker run --rm -v /foo -v awesome:/bar busybox top, the volume for /foo will be removed, but the volume for /bar will not. Volumes inheritted via --volumes-from will be removed with the same logic -- if the original volume was specified with a name it will not be removed.
Source: Docker Docs
I am using the docker-solr image with docker, and I need to mount a directory inside it which I achieve using the -v flag.
The problem is that the container needs to write to the directory that I have mounted into it, but doesn't appear to have the permissions to do so unless I do chmod 777 on the entire directory. I don't think setting the permission to allows all users to read and write to it is the solution, but just a temporary workaround.
Can anyone guide me in finding a more canonical solution?
Edit: I've been running docker without sudo because I added myself to the docker group. I just found that the problem is solved if I run docker with sudo, but I am curious if there are any other solutions.
More recently, after looking through some official docker repositories I've realized the more idiomatic way to solve these permission problems is using something called gosu in tandem with an entry point script. For example if we take an existing docker project, for example solr, the same one I was having trouble with earlier.
The dockerfile on Github very effectively builds the entire project, but does nothing to account for the permission problems.
So to overcome this, first I added the gosu setup to the dockerfile (if you implement this notice the version 1.4 is hardcoded. You can check for the latest releases here).
# grab gosu for easy step-down from root
RUN mkdir -p /home/solr \
&& gpg --keyserver pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
&& curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.4/gosu-$(dpkg --print-architecture)" \
&& curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.4/gosu-$(dpkg --print-architecture).asc" \
&& gpg --verify /usr/local/bin/gosu.asc \
&& rm /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu
Now we can use gosu, which is basically the exact same as su or sudo, but works much more nicely with docker. From the description for gosu:
This is a simple tool grown out of the simple fact that su and sudo have very strange and often annoying TTY and signal-forwarding behavior.
Now the other changes I made to the dockerfile were these adding these lines:
COPY solr_entrypoint.sh /sbin/entrypoint.sh
RUN chmod 755 /sbin/entrypoint.sh
ENTRYPOINT ["/sbin/entrypoint.sh"]
just to add my entrypoint file to the docker container.
and removing the line:
USER $SOLR_USER
So that by default you are the root user. (which is why we have gosu to step-down from root).
Now as for my own entrypoint file, I don't think it's written perfectly, but it did the job.
#!/bin/bash
set -e
export PS1="\w:\u docker-solr-> "
# step down from root when just running the default start command
case "$1" in
start)
chown -R solr /opt/solr/server/solr
exec gosu solr /opt/solr/bin/solr -f
;;
*)
exec $#
;;
esac
A docker run command takes the form:
docker run <flags> <image-name> <passed in arguments>
Basically the entrypoint says if I want to run solr as per usual we pass the argument start to the end of the command like this:
docker run <flags> <image-name> start
and otherwise run the commands you pass as root.
The start option first gives the solr user ownership of the directories and then runs the default command. This solves the ownership problem because unlike the dockerfile setup, which is a one time thing, the entry point runs every single time.
So now if I mount directories using the -d flag, before the entrypoint actually runs solr, it will chown the files inside of the docker container for you.
As for what this does to your files outside the container I've had mixed results because docker acts a little weird on OSX. For me, it didn't change the files outside of the container, but on another OS where docker plays more nicely with the filesystem, it might change your files outside, but I guess that's what you'll have to deal with if you want to mount files inside the container instead of just copying them in.