How to point to initial runtime when restarting an existing docker container - linux

I would like to restart a docker container, after exiting it and rebooting, with the same runtime with which it was initially created.
Here's what I did so far.
Create the container:
sudo docker run --runtime=nvidia [...]
Restart Docker after exiting the container and rebooting:
service docker restart
Restart the container previously created:
sudo docker start my_container
Reopen the container.
docker exec -it my_container [...]
The program which is then launched in the container doesn't use the Nvidia GPU as expected. It instead uses the system CPU.
Any help would be greatly appreciated.

I got the expected result by creating a new container with the --restart=unless-stopped policy, which lets Docker restart the container by itself when the Docker service is restarted. There seems to be more that's being done in this process than the start/exec sequence that I was using.

Related

Unused Docker containers stuck in Removal in Progress state. Device or Resource Busy

I'm running Docker version 20.10.5 on a Centos 7 Box. I stopped my project with docker-compose down and every container had the same message -
Error response from daemon: container <container ID>: driver "overlay2" failed to remove root filesystem: unlinkat /var/lib/docker/overlay2/<long number>/merged: device or resource busy
I've stopped the daemon, I've reinstalled Docker, I've tried umount, lsof, kill, and all the docker go-away commands including system prune but still they hang on.
(After re-installing Docker the status changes to Dead. When I try to delete the zombie containers their status changes to Removal In Progress)
How can I get rid of these containers?
For people who have similar issues:
So I had similar issue where
docker rm -f <docker name> was hanging
The only thing that helped me was:
service docker restart
On Ubuntu 22.04 with Docker engine 23.0.0 neither stopping/starting the Docker service nor a docker system prune removed the containers (still showing Removal in progress). The solution as outlined here was to manually remove the volumes associated with the container(s):
sudo service docker stop
sudo -i
cd /var/lib/docker/containers
rm -rf <container id>
sudo service docker start

How to debug docker restart not restarting in node.js app?

I have a container with a docker-compose like this
services:
app:
build:
context: app
restart: always
version: '3.5'
It launches a node app docker-compose run -d --name my-app app node myapp.js
the app is made to either run to completion or throw, and then the goal would be to have docker restart it in an infinite loop, regardless of the exit code. I'm unsure why but it doesn't restart it.
How can I debug this? I have no clue what exit code node is sending, nor do I know which exit code docker uses to decide to restart or not.
I am also on mac, haven't tested on linux yet. Edit: It does restart on linux, don't have another mac to see if the behavior is isolated to my mac only.
It is important to understand the following two concepts:
Ending your Node app doesn't mean the end of your container. Your container runs a shared process from your OS and your Node app is only a sub process of that. (Assuming your application runs with the Deamon)
The restart indicates the "starting" policy - it will never terminate and start your container again.
Having said that, what you need is a way you can really restart your container from within the application. The best way to do this is via Docker healthchecks:
https://docs.docker.com/engine/reference/builder/#healthcheck
Or, here are some answers on restarting a container from within the application.
Stopping docker container from inside
From Github Issue seems like it does not respect `--restart``, or from the #Charlie comment seems like its vary from platform to platform.
The docker-compose run command is for running “one-off” or “adhoc” tasks.The run command acts like docker run -ti in that it opens an interactive terminal to the container and returns an exit status matching the exit status of the process in the container.
docker-compose run
Also if its like docker run -it then I am not seeing an option for restart=always but it should then respect ``restart` option in compose.
Usage:
run [options] [-v VOLUME...] [-p PORT...] [-e KEY=VAL...] [-l KEY=VALUE...]
SERVICE [COMMAND] [ARGS...]
Options:
-d, --detach Detached mode: Run container in the background, print
new container name.
--name NAME Assign a name to the container
--entrypoint CMD Override the entrypoint of the image.
-e KEY=VAL Set an environment variable (can be used multiple times)
-l, --label KEY=VAL Add or override a label (can be used multiple times)
-u, --user="" Run as specified username or uid
--no-deps Don't start linked services.
--rm Remove container after run. Ignored in detached mode.
-p, --publish=[] Publish a container's port(s) to the host
--service-ports Run command with the service's ports enabled and mapped
to the host.
--use-aliases Use the service's network aliases in the network(s) the
container connects to.
-v, --volume=[] Bind mount a volume (default [])
-T Disable pseudo-tty allocation. By default `docker-compose run`
allocates a TTY.
-w, --workdir="" Working directory inside the container

How to kill dockerd -h fd// processes?

Everytime I restart, I see loads of dockerd -h fd// processes when I run htop on ubuntu
I want to kill all of these processes (I have no idea why they're being run upon restart, but they're massively reducing my cpu performance.)
I tried every suggestion here Stop and remove all docker containers but nothing worked: the processes are still running.
How do I fix this?
First verify if you do not have containers that are still running :docker container ps, if you have containers that are still running stop them docker stop container1 container2 or even docker stop $(docker ps -q)and verify if the cpu performance issue persists.
Then verify the status of docker service: sudo systemctl status docker here you can see different things that may help you :
Afterwards you can restart/stop and even disable the auto-restart of the docker daemon if you want :
sudo systemctl restart docker
sudo systemctl stop docker
sudo systemctl disable docker
And if you want to start the service again at boot time sudo systemctl enable docker and verify if the problem persists.
PS: If you do not have systemctl, use the service command.
you can see the proccess of the docker containers by using :
1 - docker ps -a
and then stop :
2- docker stop Container-ID
and then you can remove it :
3- docker rm Container-ID

How can I attach to docker container running in detached mode?

If I start up a docker container in detached mode using the command:
docker run -d ubuntu
the container immediately exits. docker ps -a yields the output (selected columns shown):
CONTAINER ID IMAGE COMMAND STATUS
245fe871a1e3 ubuntu "/bin/bash" Exited (0) 4 minutes ago
Is it possible to start the same container (container ID 245fe871a1e3) in interactive mode with a terminal session?
I'm afraid there's no such a way to archive this. docker attach and docker exec are working against running container only, but if you docker start the container in your case, it'll exited immediately again because the CMD is /bin/bash.
There's also a discussion about this, post some key info here:
It's not possible to enter a stopped container, because the processes are gone, and therefore, the namespaces are gone as well.

Automatically Start Services in Docker Container

I'm doing some initial tests with docker. At moment i have my images and I can put some containers running, with:
docker ps
I do docker attach container_id and start apache2 service.
Then from the main console I commit the container to the image.
After exiting the container, if I try to start the container or try to run one new container from the committed image, the service is always stopped.
How can create or restart one container with the services started, for example apache?
EDIT:
I've learned a lot about Docker since originally posting this answer. "Starting services automatically in Docker containers" is not a good usage pattern for Docker. Instead, use something like fleet, Kubernetes, or even Monit/SystemD/Upstart/Init.d/Cron to automatically start services that execute inside Docker containers.
ORIGINAL ANSWER:
If you are starting the container with the command /bin/bash, then you can accomplish this in the manner outlined here: https://stackoverflow.com/a/19872810/2971199
So, if you are starting the container with docker run -i -t IMAGE /bin/bash and if you want to automatically start apache2 when the container is started, edit /etc/bash.bashrc in the container and add /usr/local/apache2/bin/apachectl -f /usr/local/apache2/conf/httpd.conf (or whatever your apache2 start command is) to a newline at the end of the file.
Save the changes to your image and restart it with docker run -i -t IMAGE /bin/bash and you will find apache2 running when you attach.
An option that you could use would to be use a process manager such as Supervisord to run multiple processes. Someone accomplished this with sshd and mongodb: https://github.com/justone/docker-mongodb
I guess you can't. What you can do is create an image using a Dockerfile and define a CMD in that, which will be executed when the container starts. See the builder documentation for the basics (https://docs.docker.com/reference/builder/) and see Run a service automatically in a docker container for information on keeping your service running.
You don't need to automate this using a Dockerfile. You could also create the image via a manual commit as you do, and run it command line. Then, you supply the command it should run (which is exactly what the Dockerfile CMD actually does). You can also override the Dockerfiles CMD in this way: only the latest CMD will be executed, which is the command line command if you start the container using one. The basic docker run -i -t base /bin/bash command from the documentation is an example. If your command becomes too long you could create a convenience script of course.
By design, containers started in detached mode exit when the root process used to run the container exits.
You need to start a Apache service in FOREGROUND mode.
docker run -p 8080:80 -d ubuntu/apache apachectl -D FOREGROUND
Reference: https://docs.docker.com/engine/reference/run/#detached-vs-foreground
Try to add start script to entrypoint in dockerfile like this;
ENTRYPOINT service apache2 restart && bash

Resources