We have a set of variables in env file as given below
examples.env
A="/path1"
B="/path2":$A
Now, docker run cannot substitute $B for /path/path1, due to its limitations
So, I want to export the variable in launcher script and then call those variable using -e flag, as given below
mydocker.sh
input="examples.env"
while IFS= read -r line
do
export $line
done < "$input"
docker run --rm -e <Some code> centos8
Now how to create docker command to get all the variables?
Following docker command works
docker run --rm -e A -e B centos8
But If the number of variables in examples.env file is unknown, then how can we generate docker run command?
Source the variables file in your mydocker.sh script insted of export and concat each variable with --env, at the and eval the concatenated string to variable so the variables will interpreted.
Here is an example:
# Source the variables file so they will be available in current script.
. ./examples.env
# Define docker env string it will lokk like below:.
# --env A=/path1 --env B=/path1/path2
dockerenv=""
input="examples.env"
while IFS= read -r line
do
dockerenv="${dockerenv} --env $line"
done < "$input"
# Evaluate the env string so the variables in it will be interpreted
dockerenv=$(eval echo $dockerenv)
docker run --rm $dockerenv centos8
P.S.
You need the --env insted of -e becouse -e will be interpreted as echo command argument.
Related
I am trying to create a wrapper command that will:
move to a different folder
change the current user
clear env variables
So far I created this /bin/app:
#!/bin/bash
cd /var/www/backend
if [[ $# == 0 ]];
then
exec chpst -u application env -i PATH="$PATH" /bin/bash
else
exec chpst -u application env -i PATH="$PATH" "$#"
fi
But I keep getting errors when running this commands:
app bin/console d:m:m
app bin/console d:m:m && sleep 10
app bin/console acme:bar "multi word data" "another multi word data"
First command just runs bin/console without parameters.
The second command shows error && command not found.
I also need support for cases like in third command.
What is the correct way to create a wrapper that works with more complex commands?
If this changes anything I need those to work inside docker eg.
docker-compose exec app app bin/console d:m:m
docker run acme/foo app bin/console d:m:m
This would work for you
#!/bin/bash
cd /var/www/backend || exit 1
# Pass argument 1 as shell script path and default to /bin/bash if none
shell_path="${1:-/bin/bash}"
shift
# Collect argument 2 and following arguments as shell script
shell_script="$*"
exec chpst -u application env -i PATH="$PATH" "$shell_path" -c "$shell_script"
Example usage:
app bin/console 'acme:bar "multi word data" "another multi word data"'
Another much cleaner option would be to pass the shell script as stdinput or a filename:
#!/bin/bash
cd /var/www/backend || exit 1
# Pass argument 1 as shell script path and default to /bin/bash if none
shell_path="${1:-/bin/bash}"
shift
# collect file name as argument 2. Use - for stdin if no file name
shell_file="${1:--}"
exec chpst -u application env -i PATH="$PATH" "$shell_path" <(
cat "$shell_file"
)
Example usage:
# here-string stdin
app bin/console <<<'acme:bar "multi word data" "another multi word data"'
Stored in a file console.script:
acme:bar "multi word data" "another multi word data"
Then:
app bin/console console.script
I have searched the answer at stackoverflow(and google too) for 3 hours and I didnt find correct solution.
In my CI a need to dynamicly add docker build arguments. And I need to use it from env variables.
I have a string with docker arg:
DOCKER_ARGS="CURL_ARGS='-k https://example.invalid/'|ARG2=As|ARG3=asdas"
In my *.sh I wrote some code:
DOCKER_ARGS="CURL_ARGS='-k https://example.invalid/'|ARG2=As|ARG3=asdas"
if [ ! -z "$DOCKER_ARGS" ]; then
ARG_LIST=()
while read -d"|" ARG || [[ -n "$ARG" ]];
do
ARG_LIST+=("--build-arg $ARG")
done <<<$DOCKER_ARGS
fi
docker build -t test . ${ARG_LIST[#]}
And when I use echo ${ARG_LIST[#]}, I receive correct result:
--build-arg CURL_ARGS='-k https://example.invalid/' --build-arg ARG2=As --build-arg ARG3=asdas
But when I use docker build -t test . ${ARG_LIST[#]} it gives me an error:
"docker build" requires exactly 1 argument.
And if I use echo for command:
echo "docker build -t test . ${ARG_LIST[#]}" - I had correct result
I expect that will works as:
docker build -t image:tag . --build-arg CURL_ARGS='-k https://example.invalid/' --build-arg ARG2=As --build-arg ARG3=asdas
There are some problems with your script.
DOCKER_ARGS="CURL_ARGS='-k https://example.invalid/'|ARG2=As|ARG3=asdas"
With current handling CURL_ARGS will be set to '-k https://example.invalid/' including the quotes. Either remove the quotes from the input, or write your own un-quoting removal routine.
read -d"|" ARG
Removes trailing and leading whitespaces from input because of IFS and interprets missing slash sequences because or missing -r.
ARG_LIST+=("--build-arg $ARG")
Assigns a single element with the value --build-arg $ARG to the array. You would have to split later on the space between build-arg and $ARG. Instead assign two elements - one for --build-arg and the other for $ARG.
A better script might look like the following:
docker_args="CURL_ARGS=-k https://example.invalid/|ARG2=As|ARG3=asdas"
arg_list=()
if [[ -n "$docker_args" ]]; then
# split input on `|`
# Potentially use readarray to handle newlines
IFS='|' read -r -a args <<<"$docker_args"
# Add --built-arg to arrays
for arg in "${args[#]}"; do
# Proper quoting
arg_list+=(--build-arg "$arg")
done
fi
# proper quoting
docker build -t test . "${arg_list[#]}"
Check your scripts with http://shellcheck.net . Re-research how quoting works in shell and to use bash arrays. Prefer to use lower case variable names.
Basically, I need to keep the functionality of an ubuntu:18.04 image but with some environment variables set every time I execute a docker run or a docker exec this variables are dynamic, so I can't use the keyword ENV in the Dockerfile, I will need to use a script that should be sourced, for simplicity the file I will be using for this post is:
$ cat setenv.sh
#!/usr/bin/env bash
# Set some dynamic variables
export TEST="Hello World"
I have tried different approaches without success, here is my research:
Using an entrypoint
The files I used for this example:
$ cat entrypoint.sh
#!/usr/bin/env bash
echo "Setting environment"
. /setenv.sh
exec $#
$ cat Dockerfile
FROM ubuntu:18.04
COPY setenv.sh /
COPY entrypoint.sh /
ENTRYPOINT [ "/entrypoint.sh" ]
I built this Dockerfile the following command: docker build -f Dockerfile -t test_img .
This works fine except by two problems:
1. exec does not support double ampersand && nor pipes | nor escaping chars \
As I previously stated, I require my container to have the same functionality as the ubuntu image, for example, in ubuntu I can totally execute the following container:
$ docker run --rm ubuntu:18.04 bash -c "echo \"Hello World\" && ls | head -n1 "
Hello World
bin
But if I use the image I created:
$ docker run --rm test_img bash -c "echo \"Hello World\" && ls | head -n1"
Setting environment
It truncates the command every time it finds a quote (doesn't honor the escape character) a double ampersand or a pipe, here is an example of the commands in different order:
$ docker run --rm ubuntu:18.04 bash -c "ls | head -n1 && echo \"Hello World\""
bin
Hello World
$ docker run --rm test_img bash -c "ls | head -n1 && echo \"Hello World\""
Setting environment
bin
boot
dev
entrypoint.sh
etc
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
setenv.sh
srv
sys
tmp
usr
var
In this case, the command truncates when finding the pipe |.
2. Entrypoint is only called for the parent shell.
If I run a ephemeral container I can see that my env variable is there:
$ docker run --rm test_img env | grep TEST
TEST=Hello World
But if I want a keep-alive container, the env var is not set:
$ docker create -ti --name=test test_img bash
e0e5278c46bdcf33195661fac5911326b701586e9a9c638f71a6e08021ee2f57
$ docker start test
test
$ docker exec test env | grep TEST
What is happening here is that the shell I create when running docker create is calling the entrypoint, but the shell I create when running docker exec is a different one.
If you login to the container you can see shells are different:
$ docker exec -ti test bash
root#e0e5278c46bd:/# ps -fe
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 15:21 pts/0 00:00:00 bash
root 15 0 0 15:29 pts/1 00:00:00 bash
root 29 15 0 15:29 pts/1 00:00:00 ps -fe
root#e0e5278c46bd:/# env | grep TEST
If instead of having an entrypoint script to set the environment variable TEST I had used the keyword ENV in my Dockerfile: ENV TEST "Hello World" this would set the variable in every shell created by the commands docker run and docker exec. Here is the example:
$ cat Dockerfile
FROM ubuntu:18.04
ENV TEST "Hello World"
$ docker build -f Dockerfile -t test_img .
Sending build context to Docker daemon 4.096kB
Step 1/2 : FROM ubuntu:18.04
---> 6526a1858e5d
Step 2/2 : ENV TEST "Hello World"
---> Using cache
---> eebe9952bb76
Successfully built eebe9952bb76
Successfully tagged test_img:latest
$ docker create -ti --name=test test_img bash
c1e508dae0f398a40c4c5534cf2811cdfe284a4f6601198f0ca97fdea100c376
$ docker start test
test
$ docker exec test env | grep TEST
TEST=Hello World
$ docker exec -ti test bash
root#c1e508dae0f3:/# env | grep TEST
TEST=Hello World
Sourcing in bashrc
I modify the Dockerfile to look like this, and built the image with the same build command:
$ cat Dockerfile
FROM ubuntu:18.04
COPY setenv.sh /
RUN echo ". /setenv.sh" >> /etc/bash.bashrc
The problem with this approach is the shell used to execute docker run, the bashrc file is not called, only on interactive bash shells, here is the output:
$ docker run --rm test_img echo $SHELL
/bin/bash
$ docker run --rm test_img env | grep TEST
$ docker run --rm test_img bash -c "env" | grep TEST
$ docker run --rm -ti test_img bash
root#1187568e1bec:/# env | grep TEST
TEST=Hello World
First I tried to add the setenv.sh to /etc/profile.d directory, but the problem with this is that /etc/profile is called only for login shells, and I will need to change the commands to explicitly use a login shell, in other words, instead of docker run test_img env I would need it to be docker run test_img bash -lc "env" (The -l is for login).
Create Dockerfile dinamically
This is the best solution so far, but is not the cleaner, I have to have a Dockerfile.pre file to create a container and save the generated variables to a file, then use this file to create a final Dockerfile and write all those ENV lines into the Dockerfile.
Combining two approaches
By using an entrypoint and sourcing in bashrc file I was able to get the variables set in all cases, the problem is the exec $# command that doesn't support full bash scripts. Is any way to modify my entrypoint script? or is there other approach for this problem?
you can create an enviroment file and just pass it to your container with the --env-file flag. This will make all the variables in the file available in the container.
ubuntu#vps-f116ed9f:~$ cat my_env_file
TEST=Hello World
ubuntu#vps-f116ed9f:~$ docker container run -it --rm --env-file my_env_file ubuntu bash -c "echo \$TEST"
Hello World
ubuntu#vps-f116ed9f:~$ docker container run -it --rm --env-file my_env_file ubuntu bash -c "echo \$TEST | wc -c"
12
here you can see i have used the latest ubuntu image, i pass my_env_file to it and then using the bash shell i print the value of this variable (Note i have to escape the $ other wise the shell will interpolate this before passing it to docker, this could be avoided by using single qoutes as the shell wont interpolate variables in single qoutes.)
I also dont see any issues using pipe or &&
ubuntu#vps-f116ed9f:~$ docker container run -it --rm --env-file my_env_file ubuntu bash -c 'ls | head -n1 && echo "$TEST"'
bin
Hello World
This also will persist in detached containers
ubuntu#vps-f116ed9f:~$ docker container run -itd --rm --name=c1 --env-file my_env_file ubuntu bash
3d7705f2f91f3f30c45e855778bd80f08a35616bbe822545c20d5a8886139693
ubuntu#vps-f116ed9f:~$ docker container exec c1 sh -c "ls | head -1 && echo \$TEST"
bin
Hello World
I would like to automatically execute tasks inside a docker container. The task that should be executed should be run inside a specific, mounted directory. To do this, I am using the this command:
docker run --rm -v /a/dir/on/my/host:/tmp some_container /bin/bash -c "cd /tmp/dir/inside/volume && echo \"$PWD\""
, followed by the actual task, which I omit, due to brevity.
PWD should give me /tmp/dir/inside/volume, but prints /a/dir/on/my/host. Why is that?
$PWD is expanded before you run your container. Use single quotes instead of double quotes to defer expansion. Also it's simpler to use --workdir or -w instead of cd .. && and subshell.
docker run --rm -v /a/dir/on/my/host:/tmp some_container /bin/bash -c 'cd /tmp/dir/inside/volume && echo "$PWD"'
or I suggest:
docker run --rm -v /a/dir/on/my/host:/tmp -w /tmp/dir/inside/volume /some_container pwd
I need to input a variable into third linux system, here is the scheme:
From my laptop > docker server > a container,
#!/bin/bash
domain=$1
ssh -i $SSH_KEY docker#10.10.10.10 "docker run --rm=true 931967fb3e32 /bin/bash -c curl -Is $domain
Of course the variable reaches only the docker server, but not the container.
The first option to test is to pass $domain as an environment variable to your docker run command:
docker run -it --rm -e "domain=$domain" 931967fb3e32 /bin/bash -c curl -Is $domain
(note the use of -it, to be sure to have a tty in an interactive session)
If the curl somehow doesn't pick the right value, (you can test it by replacing /bin/bash -c curl -Is $domain with /bin/bash -c echo $domain), wrap it in a script (which mean your image should include that script)
As discussed in the comments, it seems to work without the bash -c:
ssh -i $SSH_KEY docker#10.10.10.10 "docker run --rm=true 931967fb3e32 curl -Is $domain