No such file when doing `ls /mnt` in docker run - linux

I have a super simple test Dockerfile:
FROM ubuntu:18.04
RUN apt-get update -y && apt-get upgrade -y
CMD ["/bin/bash"]
I build it with docker build . -t dockertest.
Then I try to run it with a test command and get a weird error:
> docker run -it dockertest "ls /mnt"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "ls /mnt": stat ls /mnt: no such file or directory: unknown.
But when I do just ls, everything is fine:
> docker run -it dockertest "ls"
bin dev home lib64 mnt proc run srv tmp var
boot etc lib media opt root sbin sys usr
Doing docker run -it dockertest "/bin/bash -c ls /mnt" yields the same error.
What exactly am doing wrong? Thanks!

The documentation for the run command can be found here: https://docs.docker.com/engine/reference/commandline/container_run/
Essentially it states
docker container run [OPTIONS] IMAGE [COMMAND] [ARG...]
It works with "ls" since ls is a valid unix command. However you are passing the command and the args together in the command value. Docker is failing since there is no command "ls /mnt" You need to pass this as command and arg "ls" "/mnt"
ubuntu#vps-f116ed9f:/opt/docker_projects/stack_example$ docker container run -it stack_test "ls /bin"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "ls /bin": stat ls /bin: no such file or directory: unknown.
ubuntu#vps-f116ed9f:/opt/docker_projects/stack_example$ docker container run -it stack_test "ls" "/bin"
bash chmod findmnt mount sleep zcat
bunzip2 chown grep mountpoint stty zcmp
bzcat cp gunzip mv su zdiff
bzcmp dash gzexe nisdomainname sync zegrep
bzdiff date gzip pidof tar zfgrep
bzegrep dd hostname ps tempfile zforce
bzexe df kill pwd touch zgrep
bzfgrep dir ln rbash true zless
bzgrep dmesg login readlink umount zmore
bzip2 dnsdomainname ls rm uname znew
bzip2recover domainname lsblk rmdir uncompress
bzless echo mkdir run-parts vdir
bzmore egrep mknod sed wdctl
cat false mktemp sh which
chgrp fgrep more sh.distrib ypdomainname

Related

How to edit the mosquitto.conf in a mosquitto Docker container?

I have a linux system running with several Docker containers. One of them is mosquitto container which runs from mosquitto 1.6.7 docker image.
I do not have control how the Mosquitto container is created as it is given by default from a supplier/client.
I need to make changes in the mosquitto/config/mosquitto.conf file. This is the output when I run ls -l
/mosquitto/config # ls -l
total 4
-rwxrwxr-x 1 nobody nobody 210 May 24 05:35 mosquitto.conf
I tried the codes below to add a comment in the mosquitto.conf, but I am not successful.
/mosquitto/config # echo '#test' | su nobody -c 'tee -a mosquitto.conf'
nologin: this account is not available
/mosquitto/config # echo '#test' | su nobody -s sh -c 'tee -a mosquitto.conf'
su: can't execute 'sh': No such file or directory
/mosquitto/config # echo '#test' | su nobody -s bin/sh -c 'tee -a mosquitto.conf'
su: can't execute 'bin/sh': No such file or directory
/mosquitto/config # echo '#test' | su nobody -s /bin/sh -c 'tee -a mosquitto.conf'
tee: mosquitto.conf: Permission denied
#test
Is it possible to change the mosquitto.conf?
If yes, how? Thanks.
You don't.
You make a copy of it on the host machine, edit there and then mount that edited copy into the container when you start it.
e.g.
docker run -d -v /path/to/local/mosquitto.conf:/mosquitto/config/mosquitto.conf mosquitto

Dockerfile set runtime ENV dinamically by sourcing a script

Basically, I need to keep the functionality of an ubuntu:18.04 image but with some environment variables set every time I execute a docker run or a docker exec this variables are dynamic, so I can't use the keyword ENV in the Dockerfile, I will need to use a script that should be sourced, for simplicity the file I will be using for this post is:
$ cat setenv.sh
#!/usr/bin/env bash
# Set some dynamic variables
export TEST="Hello World"
I have tried different approaches without success, here is my research:
Using an entrypoint
The files I used for this example:
$ cat entrypoint.sh
#!/usr/bin/env bash
echo "Setting environment"
. /setenv.sh
exec $#
$ cat Dockerfile
FROM ubuntu:18.04
COPY setenv.sh /
COPY entrypoint.sh /
ENTRYPOINT [ "/entrypoint.sh" ]
I built this Dockerfile the following command: docker build -f Dockerfile -t test_img .
This works fine except by two problems:
1. exec does not support double ampersand && nor pipes | nor escaping chars \
As I previously stated, I require my container to have the same functionality as the ubuntu image, for example, in ubuntu I can totally execute the following container:
$ docker run --rm ubuntu:18.04 bash -c "echo \"Hello World\" && ls | head -n1 "
Hello World
bin
But if I use the image I created:
$ docker run --rm test_img bash -c "echo \"Hello World\" && ls | head -n1"
Setting environment
It truncates the command every time it finds a quote (doesn't honor the escape character) a double ampersand or a pipe, here is an example of the commands in different order:
$ docker run --rm ubuntu:18.04 bash -c "ls | head -n1 && echo \"Hello World\""
bin
Hello World
$ docker run --rm test_img bash -c "ls | head -n1 && echo \"Hello World\""
Setting environment
bin
boot
dev
entrypoint.sh
etc
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
setenv.sh
srv
sys
tmp
usr
var
In this case, the command truncates when finding the pipe |.
2. Entrypoint is only called for the parent shell.
If I run a ephemeral container I can see that my env variable is there:
$ docker run --rm test_img env | grep TEST
TEST=Hello World
But if I want a keep-alive container, the env var is not set:
$ docker create -ti --name=test test_img bash
e0e5278c46bdcf33195661fac5911326b701586e9a9c638f71a6e08021ee2f57
$ docker start test
test
$ docker exec test env | grep TEST
What is happening here is that the shell I create when running docker create is calling the entrypoint, but the shell I create when running docker exec is a different one.
If you login to the container you can see shells are different:
$ docker exec -ti test bash
root#e0e5278c46bd:/# ps -fe
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 15:21 pts/0 00:00:00 bash
root 15 0 0 15:29 pts/1 00:00:00 bash
root 29 15 0 15:29 pts/1 00:00:00 ps -fe
root#e0e5278c46bd:/# env | grep TEST
If instead of having an entrypoint script to set the environment variable TEST I had used the keyword ENV in my Dockerfile: ENV TEST "Hello World" this would set the variable in every shell created by the commands docker run and docker exec. Here is the example:
$ cat Dockerfile
FROM ubuntu:18.04
ENV TEST "Hello World"
$ docker build -f Dockerfile -t test_img .
Sending build context to Docker daemon 4.096kB
Step 1/2 : FROM ubuntu:18.04
---> 6526a1858e5d
Step 2/2 : ENV TEST "Hello World"
---> Using cache
---> eebe9952bb76
Successfully built eebe9952bb76
Successfully tagged test_img:latest
$ docker create -ti --name=test test_img bash
c1e508dae0f398a40c4c5534cf2811cdfe284a4f6601198f0ca97fdea100c376
$ docker start test
test
$ docker exec test env | grep TEST
TEST=Hello World
$ docker exec -ti test bash
root#c1e508dae0f3:/# env | grep TEST
TEST=Hello World
Sourcing in bashrc
I modify the Dockerfile to look like this, and built the image with the same build command:
$ cat Dockerfile
FROM ubuntu:18.04
COPY setenv.sh /
RUN echo ". /setenv.sh" >> /etc/bash.bashrc
The problem with this approach is the shell used to execute docker run, the bashrc file is not called, only on interactive bash shells, here is the output:
$ docker run --rm test_img echo $SHELL
/bin/bash
$ docker run --rm test_img env | grep TEST
$ docker run --rm test_img bash -c "env" | grep TEST
$ docker run --rm -ti test_img bash
root#1187568e1bec:/# env | grep TEST
TEST=Hello World
First I tried to add the setenv.sh to /etc/profile.d directory, but the problem with this is that /etc/profile is called only for login shells, and I will need to change the commands to explicitly use a login shell, in other words, instead of docker run test_img env I would need it to be docker run test_img bash -lc "env" (The -l is for login).
Create Dockerfile dinamically
This is the best solution so far, but is not the cleaner, I have to have a Dockerfile.pre file to create a container and save the generated variables to a file, then use this file to create a final Dockerfile and write all those ENV lines into the Dockerfile.
Combining two approaches
By using an entrypoint and sourcing in bashrc file I was able to get the variables set in all cases, the problem is the exec $# command that doesn't support full bash scripts. Is any way to modify my entrypoint script? or is there other approach for this problem?
you can create an enviroment file and just pass it to your container with the --env-file flag. This will make all the variables in the file available in the container.
ubuntu#vps-f116ed9f:~$ cat my_env_file
TEST=Hello World
ubuntu#vps-f116ed9f:~$ docker container run -it --rm --env-file my_env_file ubuntu bash -c "echo \$TEST"
Hello World
ubuntu#vps-f116ed9f:~$ docker container run -it --rm --env-file my_env_file ubuntu bash -c "echo \$TEST | wc -c"
12
here you can see i have used the latest ubuntu image, i pass my_env_file to it and then using the bash shell i print the value of this variable (Note i have to escape the $ other wise the shell will interpolate this before passing it to docker, this could be avoided by using single qoutes as the shell wont interpolate variables in single qoutes.)
I also dont see any issues using pipe or &&
ubuntu#vps-f116ed9f:~$ docker container run -it --rm --env-file my_env_file ubuntu bash -c 'ls | head -n1 && echo "$TEST"'
bin
Hello World
This also will persist in detached containers
ubuntu#vps-f116ed9f:~$ docker container run -itd --rm --name=c1 --env-file my_env_file ubuntu bash
3d7705f2f91f3f30c45e855778bd80f08a35616bbe822545c20d5a8886139693
ubuntu#vps-f116ed9f:~$ docker container exec c1 sh -c "ls | head -1 && echo \$TEST"
bin
Hello World

Executing a command inside docker shows wrong $PATH

I am trying to run a bash command inside docker from host:
$ docker exec -it -u weiss apollo_dev /bin/bash -c "rosbag"
/bin/bash: rosbag: command not found
So I tried:
$ docker exec -it -u weiss apollo_dev /bin/bash -c "echo \$PATH"
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
But when I run docker interactively:
$ docker exec -it -u weiss apollo_dev /bin/bash
weiss#docker$ echo $PATH
/usr/local/cuda-8.0/bin:/home/tmp/ros/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Any reason why I am getting different results for $PATH?
This path is most likely changed in your .bashrc file, and this file is not loaded when the shell is non interactive (see https://www.gnu.org/software/bash/manual/bash.html#Bash-Startup-Files)
So /bin/bash will load it, /bin/bash -c will not
Here you are getting de $PATH of your Host. Before you run the container the variable is replace for the host's $PATH.
$ docker exec -it -u weiss apollo_dev /bin/bash -c "echo \$PATH"
You need to pass the command without replace the variable, so when run the command in the container just invoke the $PATH variable.
$ docker exec -it -u weiss apollo_dev /bin/bash -c 'echo \$PATH'
Te 'apostrophe' is the key. Bye

Exception while executing Docker command in Jenkinsfile

I have a test project for end2end tests based on Nightwatch.js that is an NodeJS framework. I want to use 'Jenkinsfile' for my project to build a pipeline for my end2end tests to execute them over a Jenkins in a Docker container. So, I want to start a Docker container and execute the tests inside this Docker container. And this should be realized over a Jenkinsfile. Everything is perfect when I don't use a Jenkinsfile but directly use shell commands in a manually created job. While using Jenkinsfile I get an MultipleCompilationErrorsException while running the pipeline and I don't know why.
This is my Jenkinsfile:
pipeline {
agent any
parameters {
text(defaultValue: 'grme/nightwatch-chrome-firefox:0.0.3', description: '', name: 'docker_image')
text(defaultValue: 'npm-test-chrome', description: '', name: 'run_script_method')
text(defaultValue: '/Applications/Docker.app/Contents/Resources/bin/docker', description: '', name: 'docker')
}
stages {
stage('Test') {
steps {
sh 'sudo chmod -R 777 $(pwd)'
echo "------ stop all Docker containers ------"
sh '(sudo ${params.docker} stop $(sudo ${params.docker} ps -a -q) || echo "------ all Docker containers are still stopped ------")'
echo "------ remove all Docker containers ------"
sh '(sudo ${params.docker} rm $(sudo ${params.docker} ps -a -q) || sudo echo "------ all Docker containers are still removed ------")'
echo "------ pull Docker image from Docker Cloud ------"
sh 'sudo ${params.docker} pull "${params.docker_image}"'
echo "------ start Docker container from image ------"
sh 'sudo ${params.docker} run -d -t -i -v $(pwd):/my_tests/ "${params.docker_image}" /bin/bash'
echo "------ execute end2end tests on Docker container ------"
sh 'sudo ${params.docker} exec -i $(sudo ${params.docker} ps --format "{{.Names}}") bash -c "cd /my_tests && xvfb-run --server-args='-screen 0 1600x1200x24' npm run ${params.run_script_method} || true && google-chrome --version && firefox --version"'
echo "------ cleanup all temporary files ------"
sh 'sudo rm -Rf $(pwd)/tmp-*'
sh 'sudo rm -Rf $(pwd)/.com.google*'
sh 'sudo rm -Rf $(pwd)/rust_mozprofile*'
sh 'sudo rm -Rf $(pwd)/.org.chromium*'
echo "------ stop all Docker containers again ------"
sh '(sudo ${params.docker} stop $(sudo ${params.docker} ps -a -q) || sudo echo "------ all Docker containers are still stopped ------")'
echo "------ remove all Docker containers again ------"
sh '(sudo ${params.docker} rm $(sudo ${params.docker} ps -a -q) || sudo echo "------ all Docker containers are still removed ------")'
}
}
}
}
And this is the exception I get when running the pipeline:
Started by user GRme
> git rev-parse --is-inside-work-tree # timeout=10
Setting origin to https://github.com/GRme/e2e-web-tests
> git config remote.origin.url https://github.com/GRme/e2e-web-tests # timeout=10
Fetching origin...
Fetching upstream changes from origin
> git --version # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --tags --progress origin +refs/heads/*:refs/remotes/origin/*
Seen branch in repository origin/master
Seen 1 remote branch
Obtained Jenkinsfile from 0eb7d8c437df1efc56e46171d945e7f2806b838b
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 23: Expected a symbol # line 23, column 9.
sh 'sudo ${params.docker} exec -i $(sudo ${params.docker} ps --format "{{.Names}}") bash -c "cd /my_tests && xvfb-run --server-args='-screen 0 1600x1200x24' npm run ${params.run_script_method} || true && google-chrome --version && firefox --version"'
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:129)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:123)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:516)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:479)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:269)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Finished: FAILURE
What do I wrong and how can I solve this exception?
After escaping the ' in the line, the pipeline has no syntax error anymore :)
sh 'sudo ${params.docker} exec -i $(sudo ${params.docker} ps --format "{{.Names}}") bash -c "cd /my_tests && xvfb-run --server-args=\'-screen 0 1600x1200x24\' npm run ${params.run_script_method} || true && google-chrome --version && firefox --version"'

How to write a bash script which automate entering "docker container" and doing other things?

I want to implement an automatic bash script which enters a running docker container, and do some stuffs:
# cat docker.sh
#!/bin/bash -x
docker exec -it hammerdb_net8 bash
cd /data/oracle/tablespaces/
pwd
Executing the script on terminal:
# ./docker.sh
+ docker exec -it hammerdb_net8 bash
[root#npar1 /]#
The output shows only login the docker container, but won't do other operations.
Is there any method to automate entering docker container and doing other things?
You can use bash -c:
docker exec -it hammerdb_net8 bash -c 'cd /data/oracle/tablespaces/; pwd; ls'
For running a series of commands use here-doc in BASH:
docker exec -i hammerdb_net8 bash <<'EOF'
cd /data/oracle/tablespaces/
pwd
ls
EOF

Resources