First: I have searched the forum and also went through documentation, but still cannot get it right.
So, I have a docker command I want to run on a remote server, from my bash script. I want to pass an environment variable – on the local machine running the script – to the remote server. Furthermore, I need a response from the remote command.
Here is what I actually am trying to do and what I need: the script is a tiny wrapper around our Traefik/Docker/Elixir/Phoenix app setup to be able to connect easily to the running Elixir application, inside the Erlang observer. With the script, the steps would be:
ssh into the remote machine
docker ps to see all running containers, since in our blue/green deploy the active one changes name
docker exec into the correct container
execute a command inside the docker container to connect to the running Elixir application
The command I am using now is:
CONTAINER=$(ssh -q $USER#$IP 'sudo docker ps --format "{{.Names}}" | grep ""$APP_NAME"" | head -n 1')
The main problem is the part with the grep and the ENV var... It is empty, and does not get replaced. It makes sence, since that var does not exist on the remote machine, it does on my local machine. I tried single quotes, $(), ... Either it just does not work, or the solutions I find online execute the command but then I have no way of getting the container name, which I need for the subsequent command:
ssh -o 'RequestTTY force' $USER#$IP "sudo docker exec -i -t $CONTAINER /bin/bash -c './bin/app remote'"
Thanks for your input!
First, are you sure you need to call sudo docker stop? as stopping the containers did not seem to be part of the workflow you mentioned. [edit: not applicable anymore]
Basically, you use a double-double-quote, grep ""$APP_NAME"", but it seems this variable is not substituted (as the whole command 'sudo docker ps …' is singled-quoted); according to your question, this variable is available locally, but not on the remote machine, so you may try writing:
CONTAINER=$(ssh -q $USER#$IP 'f() { sudo docker ps --format "{{.Names}}" | grep "$1" | head -n 1; }; f "'"$APP_NAME"'"')
You can try this single command :
ssh -t $USER#$IP "docker exec -it \$(docker ps -a -q --filter Name=/$APP_NAME) bash -c './bin/app remote'"
You will need to redirect the command with the local environmental variable (APP_NAME) into the ssh command using <<< and so:
CONTAINER=$(ssh -q $USER#$IP <<< 'sudo docker ps --format "{{.Names}}" | grep "$APP_NAME" | head -n 1 | xargs -I{} sudo docker stop {}')
Related
we have a list of docker images, and we are trying to see what version of node/java/mongo.. is installed inside every image, using the following bash script:
#!/bin/bash
version=""
file='images.txt'
while read line; do
sudo docker pull $line
sudo docker run $line bash
version=$(node -v)
exit
$version >> "version.txt"
sudo docker image rm $line
done < $file
but we have an error on the "docker run" line which says "command not found"
the docker pull comannd works fine
we have also noticed that the while loop continues without waiting for the first docker pull to finish
and when we do the "docker run" command without the script it works fine and gets into the shell of the image.
we would appreciate any suggestions :)
I'm guessing you probably mean
#!/bin/bash
while read -r line; do
sudo docker pull "$line"
sudo docker run -it --rm "$line" node -v
done <images.txt >version.txt
... but code which doesn't do what you want to do is a terrible way to tell us what you actually want to do. This will run node -v inside each container, where the -it option is crucial for getting output on standard output, and --rm takes care of what (I'm guessing) you were doing with a separate rm. This still obviously depends on what exactly is in the input file.
I can't understand some thing, namely as we know we can pass to docker run argument -e SOME_VAR=13.
Then each process launched (for example using docker exec ping localhost -c $SOME_VAR=13) can see this variable.
How does it work ? After all, enviroment are about bash, we don't launched bash. I can't understand it. Can you explain me how -e does work without shell ?
For example, let's look at following example:
[user#user~]$ sudo docker run -d -e XYZ=123 ubuntu sleep 10000
2543e7235fa9
[user#user~]$ sudo docker exec -it 2543e7235fa9 echo test
test
[user#user~]$ sudo docker exec -it 2543e7235fa9 echo $XYZ
<empty row>
Why did I get <empty row> instead of 123 ?
The problem is that your $XYZ is getting interpolated in the host shell environment, not your container.
$ export XYZ=456
$ docker run -d -e XYZ=123 ubuntu sleep 10000
$ docker exec -it $(docker ps -ql) echo $XYZ
$ 456
$ docker exec -it $(docker ps -ql) sh -c 'echo $XYZ'
$ 123
You have to quote it so it's passed through as a string literal to the container. Then it works fine.
The environment is not specific to shells. Even ordinary processes have environments. They work the same for both shells and ordinary processes. This is because shells are ordinary processes.
When you do SOMEVAR=13 someBinary you define an environment variable called SOMEVAR for the new process, someBinary. You do this with -e in docker because you ask another process to start your process, the docker daemon.
I'm trying to do two commands in docker exec. Concretely, I have to run a command inside a specific directory.
I tried this, butit didn't work:
docker exec [id] -c 'cd /var/www/project && composer install'
Parameter -c is not detected.
I also tried this:
docker exec [id] cd /var/www/project && composer install
But the command composer install is executed after the docker exec command.
How can I do it?
In your first example, you are giving the -c flag to docker exec. That's an easy answer: docker exec does not have a -c flag.
In your second example, your shell is parsing this into two commands before Docker even sees it. It is equivalent to this:
if docker exec [id] cd /var/www/project
then
composer install
fi
First, the docker exec is run, and if it exits 0 (success), composer install will try to run locally, outside of Docker.
What you need to do is pass both commands in as a single argument to docker exec using a string. Then they will not be interpreted by a shell until already inside the container.
docker exec [id] "cd /var/www/project && composer install"
However, as you noted in the comments, this also does not work. That's because cd is a shell builtin, and doesn't exist on its own. Trying to execute it as the initial command will fail. So the next step is to hand this off to a shell to execute.
docker exec [id] "bash -c 'cd /var/www/project && composer install'"
And finally, at this point the && has moved into an inner set of quote marks, so we don't really need the quotes around the bash command... you can drop them if you prefer.
docker exec [id] bash -c 'cd /var/www/project && composer install'
Everything after the container id is the command to run, so in the first example -c isn't an option to exec, but a command docker tries to run and fails since that command doesn't exist.
Most likely you found this syntax from a docker run command where the entrypoint was set to /bin/sh. However, exec bypasses the entrypoint, so you need to include the full command to run. As others have pointed out, that command includes a shell like bash or in the below example, sh:
docker exec [id] /bin/sh -c 'cd /var/www/project && composer install'
The other answers are fine if you want to run 2 arbitrary commands. But if the first command is simply cd, then you should use the -w option to set the working directory instead.
docker exec -w {dir} {container} {commands}
So in your example:
docker exec -w /var/www/project {container} composer install
as Nehal J Wani said in his commentary, the correct syntax is the following:
docker exec [id] /bin/bash -c 'cd /var/www/project && composer install'
many thanks!
I would like to add my example here because it is a bit more complex then the ones thate were shown above. This example also illustrates on how to find a container id that should be used in the docker exec command.
I needed to execute a composite docker exec command against docker container over ssh.
I managed to achieve this in 2 steps:
-definition of the variable that contains a command expored as an environment variable
-ssh command that runs it
environment varialbe definition:
export COMMAND="bash -c 'php bin/console --version && composer --version'"
ssh command that runs on a remote system:
ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i keyfile.pem ec2-user#111.222.111.222 'docker exec docker ps|grep php|grep api|grep -v cron|awk '"'"'{print $1}'"'"' '$COMMAND
As you can see I left the command out of the single quotes to pass its actual value to the SSH process
The output of the command execution is:
Warning: Permanently added '111.222.111.222' (ECDSA) to the list of known hosts.
Cannot load Xdebug - it was already loaded
Symfony 4.3.5 (env: dev, debug: true)
Cannot load Xdebug - it was already loaded
Composer version 1.9.1 2019-11-01 17:20:17
Connection to 111.222.111.222 closed.
If you wish to execute this command in a single line you can use a bit modified version of my first example:
COMMAND="bash -c 'php bin/console --version && composer --version'" ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i keyfile.pem ec2-user#111.222.111.222 'docker exec `docker ps|grep php|grep api|grep -v cron|awk '"'"'{print $1}'"'"'` '$COMMAND
The following command issued on a Mac terminal is failing the docker command on the remote shell.
However it works if I log in to the server and issue the command there with replacing ";" with "&&"
ssh -i "myKey.pem" user#host ‘docker stop $(docker ps -a -q --filter ancestor=name/kind); docker rm $(docker ps -a -q --filter ancestor=name/kind); docker rmi name/kind; docker build -t name/kind .; sudo docker run -it -d -p 80:80 name/kind’
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
I need to run this command form the local terminal because if is part of bigger command which first build the project locally and scp it to the server.
`$bigger-command && then-the-ssh-as-shown-above.
How do I go about it? Thanks
The best way to pass very complex commands to ssh is the create a script on the server side.
If you need to pass some parameters, proceed this way:
create a .sh file on your localhost
scp it to your remote host
run `ssh user#remotehost 'bash scriptfile.sh'
This should do the trick without giving you headaches about escapement.
To be honest, I have always been confused about docker exec -it …, docker exec -i … and docker exec -t …, so I decide to do a test:
docker exec -it …:
# docker exec -it 115c89122e72 bash
root#115c89122e72:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
It works normally.
docker exec -i …:
# docker exec -i 115c89122e72 bash
^C
The command hangs and I have to use Ctl + c to interrupt it.
docker exec -t …:
# docker exec -t 115c89122e72 bash
root#115c89122e72:/# ls
^C
It enters the container successfully but hangs on executing the first command.
So it seems there is no point in having the docker exec -i … and docker exec -t … commands. Could anyone elaborate on why there exist -i and -t options for the docker exec command?
-i, --interactive keeps STDIN open even if not attached, which you need if you want to type any command at all.
-t, --tty Allocates a pseudo-TTY, a pseudo terminal which connects a user's "terminal" with stdin and stdout. (See container/container.go)
If you do an echo, only -t is needed.
But for an interactive session where you enter inputs, you need -i.
Since -i keeps stdin open, it is also used in order to pipe input to a detached docker container. That would work even with -d (detach).
See "When would I use --interactive without --tty in a Docker container?":
$ echo hello | docker run -i busybox cat
hello
-i keeps STDIN open even if not attached, what is the status of STDOUT in this case?
It is, for docker exec, the one set by docker run.
But, regarding docker exec, there is a current issue (issue 8755: Docker tty is not a tty with docker exec
unfortunately your discovery only amounts to a difference between the behaviour of tty in centos6 vs ubuntu:14.04. There is still not a functional tty inside the exec - just do ls -la /proc/self/fd/0 and see that it's a broken link pointing to a pts which doesn't exist.
the actual bug we're dealing with is that certain standard libraries assume that the symlinks in /proc/self/fds/ must be valid symlinks
The problem is that the tty is created outside on the host and there is no reference to it in the container like how /dev/console is setup in the primary container.
One options to fix this would be allocate and bind mount the devpts from the host in to the containers.
Note (Q4 2017): this should been fixed by now (docker 17.06-ce).
See PR 33007.
That PR now allows (since 17.06):
zacharys-pro:dev razic$ docker run --rm -t -d ubuntu bash
83c292c8e2d13d1b1a8b34680f3fb95c2b2b3fef71d4ce2b6e12c954ae50965a
zacharys-pro:dev razic$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
83c292c8e2d1 ubuntu "bash" 2 seconds ago Up 1 second xenodochial_bardeen
zacharys-pro:dev razic$ docker exec -ti xenodochial_bardeen tty
/dev/pts/1
(before 17.06, tty was returning "not a tty")