Where are the environment variables located on a docker container, more precisely, where is ENV=1 stored from the following:
$ docker run --rm -e ENV=1 -it ubuntu:16.04 bash
root#40e384fc9c1f:/# env
HOSTNAME=40e384fc9c1f
TERM=xterm
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
ENV=1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
HOME=/root
_=/usr/bin/env
All linux docs I read pointed to either: /etc/environment, ~/.bashrc, ~/.bash_profile, and ~/profile, but when dumped those files, I saw nothing related to declaration of ENV=1.
Is it perhaps something specific to docker and how it sets up the environment?
If a variable is specified as an option when executing a Docker command, the Docker image is read-only, so it is likely to be stored only in memory, not in the file.
When you run docker run -e VAR=value some-image, the container has a single process, and the environment variable is set in that process's environment. It's not in any particular file. The lowest-level Unix function to run a command is execve(2) and in that system call you just pass the new command's environment as parameters.
Locally, try running
export ANYTHING=value
env | grep ANYTHING
grep ANYTHING ~/.profile
and you'll similarly notice that the environment variable is set; running env(1) as a subprocess sees it; but it doesn't exist in any of the files you mention.
Normally in Docker, none of the files you mention are read at all. It's highly likely that the standard shell isn't GNU bash; if you're using an Alpine-based image, you may not even have a /bin/bash. None of this is a problem, but it means that you should usually ignore shell dotfiles.
Instead, to set an environment variable in an image, use the Dockerfile ENV directive. When you docker build the image, that value will be persisted, and you can see its value if you docker inspect the image, but it's not directly accessible on disk anywhere and it won't be visible in any of the dotfiles you mention.
As an example:
FROM alpine
# Set an environment variable using ENV
ENV VAR_1=from-dockerfile-env
# These won't work at all
RUN echo 'export VAR_2=from-etc-profile' >> /etc/profile
RUN echo 'export VAR_3=from-etc-bashrc' >> /etc/bashrc
# Also has no effect
RUN echo 'export VAR_4=from-source-script' > /source-script.sh
RUN echo 'source /source-script.sh' >> /etc/profile
# _Also_ has no effect
RUN export VAR_5=from-dockerfile-run
# When you run the container, see what is set (only $VAR_1)
CMD env | grep VAR
Related
To simplify test execution for several different container I want to create an alias to use the same command for every container.
For example for a backend container I want to be able to use docker exec -t backend test instead of docker exec -t backend pytest test
So I add this line in my backend Dockerfile :
RUN echo alias test="pytest /app/test" >> ~/.bashrc
But when I do docker exec -t backend test it doesn't work, otherwise it works when I do docker exec -ti backend bash then test.
I saw that it is because alias in .bashrc works only if we use interactive terminal.
How can I get around that?
docker exec does not run the shell, so .bashrc is just never used.
Create an executable in PATH, most probably in /usr/local/bin. Note that test is a very basic shell command, use a different unique name.
That alias will only work for interactive shells, if you want that alias to work on other programs:
RUN echo -e '#!/bin/bash\npytest /app/test' > /usr/bin/mypytest && \
chmod +x /usr/bin/mypytest
I am just experimenting with configuring Dockerfile-
FROM ubuntu:latest
RUN apt-get update
RUN echo VERSION_TAG="latest" >> /etc/environment
RUN cat /etc/environment
CMD echo $VERSION_TAG
I built the image using (I was in the req directory) -
docker build -t temp/testing:latest .
Ran it using-
docker run temp/testing:latest
Expected output-
latest
Actual Output-
While building the image, the output of cat /etc/environment
Step 4/5 : RUN cat /etc/environment
---> Running in 4grdc7b5165a
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
VERSION_TAG=latest
It is present inside env variables, still, its value is not being printed, any help will be appreciated.
Note - I want to do it without using
https://docs.docker.com/engine/reference/builder/#env
I need to know the reason.
Docker just exec() the command ENTRYPOINT+CMD with nothing in between.
pam is not loaded, so pam_env.so is not loaded, so /etc/environment is not read.
The intention is to use ENV for this, but you can set extra parameters on the command line at docker run with -e
docker run -it -e VERSION='latest' -e NAME='abcd' ubuntu:latest /bin/bash
Alternatively, you can provide a file with each variable per-line
docker run -it --env-file ./my_env ubuntu:latest /bin/bash
You can also do a volume mount with -v, which can smuggle further information and entire paths into the container, including information useful to whatever application you run within it (the intent of Docker containers is normally to run a single application within a known environment)
However, if you're trying to determine if the container is running the latest version of its own tag, this will be problematic
Specifically, the tag may change, but your explicit setting of it won't! .. for such a case, you should consider something else, such as having an outside process which replaces the running container with a new one when the tag changes (so you can assume what's running is always latest) or at some frequency (perhaps every day in the morning)
I'm running a set of commands on an ubuntu docker, and I need to set a couple of environment variables for my scripts to work.
I have tried several alternatives but none of them seem to solve my problem.
Alternative 1: Using --env or --env-file
On my already running container, I run either:
docker exec -it --env TESTVAR="some_path" ai_pipeline_new_image bash -c "echo $TESTVAR"
docker exec -it --env-file env_vars ai_pipeline_new_image bash -c "echo $TESTVAR"
The content of env_vars:
TESTVAR="some_path"
In both cases the output is empty.
Alternative 2: Using a dockerfile
I create my image using the following docker file
FROM ai_pipeline_yh
ENV TESTVAR "A_PATH"
With this alternative the variable is set if I attach to the docker (aka if I run an interactive shell), but the output is blank if I run docker exec -it ai_pipeline_new_image bash -c "echo $TESTVAR" from the host.
What is the clean way to do this?
EDIT
Turns out that if I check the state of the variables from a shell script, they are set, but not if check them directly in bash -c "echo $VAR". I would really like to understand why this is so. I provide a minimal example:
Run docker
docker run -it --name ubuntu_env_vars ubuntu
Create a file that echoes a VAR (inside the container)
root#afdc8c494e8a:/# echo "echo \$VAR" > env_check.sh
root#afdc8c494e8a:/# chmod +x env_check.sh
From the host, run:
docker exec -it -e VAR=BLA ubuntu_env_vars bash -c "echo $VAR"
(Blank output)
From the host, run:
docker exec -it -e VAR=BLA ubuntu_env_vars bash -c "/env_check.sh"
output: BLA
Why???????
I revealed my noobness. Answering my own question here:
Both options, --env-file file or -env foo=bar are okay.
I forgot to escape the $ character when testing. Therefore the correct command to test if the variable exists is:
docker exec -it my_docker bash -c "echo \$MYVAR"
Is a good option design your apps driven to environment variables at the runtime(start of application)
Don't use env variables at docker build stage.
Sometimes the problem is not the environment variables or docker, the problem is the app who reads the environment variables.
Anyway, you have these options to inject environment variables to a docker container at runtime:
-e foo=bar
This is the most basic way:
docker run -it -e foo=bar ubuntu
These variables will be available since the start of your container.
remote variables
If you need to pass several variables, using the -e will not be the best way.
Or if you don't want to use .env files or any kind of local file with variables, you should:
prepare your app to read environment variables
inject the variables in a docker entrypoint bash script, reading it from a remote variables manager
in the shell script you will get the remote variables and inject them using source /foo/bar/variables. A sample here
With this approach you will have a variables manager for all of your containers. These variables manager has these features:
login
create applications
create variables by application
create global variables if a variable is required for two or more apps
expose an http endpoint to be used in the client (apps) in order to get the variables
crypt
etc
You have these options:
spring cloud
zookeeper
https://www.vaultproject.io/
https://www.doppler.com/
Configurator (I'm the author)
Let me know if you want to use this approach to help you.
I am aware that we can specify the option -e during the run command to set environment variables in a docker. This only sets the PATH for the root user. Let us say if I have another user called admin and want to set the environment variables for that user as well, how can I achieve that?
This is the command I tried to set environment variables.
docker run -t -d -v /usr/hdp:/usr/hdp -v /usr/lib/jvm/:/usr/lib/jvm/ -e JAVA_HOME="${java_home}" -e HADOOP_HOME="${hadoop_home}" -e PATH=$PATH:$JAVA_HOME/bin -e PATH=$PATH:$HADOOP_HOME/bin gtimage
This only sets the PATH under root user but not for my admin user which a software that I installed during docker build has created.
I don't have a perfect solution for my question above but I tried something like below to login as user and set environment variables for that user. I don't recommend the below way unless you could not find a solution for your problem. Please let me know if you find a better approach than this
docker exec $containervalue bash -c 'env | grep PATH >> temp && chmod 775 temp && mv temp /opt/nagios'
docker exec --user ngadmin $containervalue bash -c 'cat ~/temp >> ~/.bashrc && source ~/.bashrc'
During docker build I need to run a bash script, which sets up some environment variables.
The script looks something like this:
#!/bin/bash
export ENVVAR=TEST
export HOST=local
export PORT=port
I try to call this script in my dockerfile in different ways but none of them are working. I tried these:
ADD ./myscript.sh
RUN chmod +x /myscript.sh;\
/bin/bash -c 'source ./myscript.sh';\
/bin/bash -c 'source /myscript.sh';\
/bin/bash -c source ./myscript.sh;\
/bin/bash -c source /myscript.sh;\
source ./myscript.sh;\
source /myscript.sh;\
/myscript.sh;\
./myscript.sh;\
ENTRYPOINT ["/bin/bash"]
Of course I only had one of these commands in my RUN and I just put them here grouped.
If I run the container and use source ./myscript.sh it works as expected.
Because of multiple restrictions and other reasons it is not possible for me to use docker compose, the -e argument, ENV KEY VALUE in dockerfile or similar approaches. I need to set up the environment variables during the docker build process.
You're simply not sourcing the shell that is specified in your ENTRYPOINT. Just add your myscript.sh to your image (use COPY instead of ADD).
COPY myscript.sh /usr/local/bin
Then source it on the shell that is actually started by your entrypoint.
docker run myimage source /usr/local/bin/myscript.sh
btw, myscript.sh is pretty non descriptive. You could use env.sh for instance.