Here is a sample dockerfile with a shell-form entrypoint:
FROM ubuntu
ENTRYPOINT /bin/echo "(shell) ENTRYPOINT#image ($0)"
Below, are some of the outputs that I see from various runs:
$ docker run -it sc-test:v4
(shell) ENTRYPOINT#image (/bin/sh)
$ docker run -it sc-test:v4 /bin/echo
(shell) ENTRYPOINT#image (/bin/echo)
$ docker run -it sc-test:v4 /bin/cat
(shell) ENTRYPOINT#image (/bin/cat)
$ docker run -it sc-test:v4 /bin/dog
(shell) ENTRYPOINT#image (/bin/dog)
$ docker run -it sc-test:v4 "/bin/dog ($0)"
(shell) ENTRYPOINT#image (/bin/dog (-bash))
Based on docker documentation here, we can see that the command args are ignored.
However, the value of $0 changes with the args provided. Can someone explain why this happens? Thanks!
The table in that part of the Docker documentation isn't technically correct: Docker doesn't actually drop or ignore the command part when there's a shell-form entrypoint. What actually happens here (and your example demonstrates) is:
If either the ENTRYPOINT or CMD (or both) is the shell form, it's wrapped in ["/bin/sh", "-c", "..."].
The ENTRYPOINT and CMD lists are concatenated to form a single command list.
Let's take your third example. This is, in Dockerfile syntax,
ENTRYPOINT /bin/echo "(shell) ENTRYPOINT#image ($0)"
CMD ["/bin/cat"]
and the resulting combined command is (in JSON array syntax, expanded for clarity)
[
"/bin/sh",
"-c",
"/bin/echo \"(shell) ENTRYPOINT#image ($0)\"",
"/bin/cat"
]
So, what does sh -c do if you give it multiple arguments? The POSIX spec for the sh command documents the syntax as
sh -c command_string [command_name [argument...]]
and further documents
-c: Read commands from the command_string operand. Set the value of special parameter 0 [...] from the value of the command_name operand and the positional parameters ($1, $2, and so on) in sequence from the remaining argument operands. No commands shall be read from the standard input.
That's what you're seeing in your examples. If ENTRYPOINT is a bare string and CMD is a JSON array, then within the ENTRYPOINT string command, the arguments in CMD can be used as $0, $1, and so on. If both are bare strings, both get wrapped in sh -c, and you'll get something like:
ENTRYPOINT /bin/echo "$0 is /bin/sh, $1 is -c, and then $2"
CMD the rest of the line
In your first example, the command part is empty, and in this case (still from the POSIX sh documentation)
If command_name is not specified, special parameter 0 shall be set to [...] normally a pathname used to execute the sh utility.
Your last example is slightly more subtle:
docker run -it sc-test:v4 "/bin/dog ($0)"
Since the string is double-quoted, your local shell expands the $0 reference in it, which is how bash gets in there; then since it's a single (quoted) word, it becomes the single command_name argument to sh -c.
There's two more normal patterns for using ENTRYPOINT and CMD together. The pattern I prefer has CMD be a full shell command, and ENTRYPOINT does some first-time setup and then runs a command like exec "$#" to run that command. There's also a "container as command" pattern where ENTRYPOINT has a complete command (perhaps with involved JVM arguments) and CMD additional options. In these cases the ENTRYPOINT must be JSON-array syntax:
ENTRYPOINT ["/script-that-exec-dollar-at.sh"]
CMD ["the_command", "--option"]
Since the ENTRYPOINT string doesn't directly reference $0, $1, etc. the CMD arguments are effectively ignored by the sh -c wrapper. If you had ENTRYPOINT script.sh it would be invoked by sh -c as a subprocess with no arguments, and the CMD would get lost.
It's probably clearer for the Docker documentation to say "if ENTRYPOINT is a string then CMD is ignored" than to try to explain the subtleties of this.
Related
I do this in a script:
read direc <<< $(basename `pwd`)
and I get:
Syntax error: redirection unexpected
in an ubuntu machine
/bin/bash --version
GNU bash, version 4.0.33(1)-release (x86_64-pc-linux-gnu)
while I do not get this error in another suse machine:
/bin/bash --version
GNU bash, version 3.2.39(1)-release (x86_64-suse-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
Why the error?
Does your script reference /bin/bash or /bin/sh in its hash bang line? The default system shell in Ubuntu is dash, not bash, so if you have #!/bin/sh then your script will be using a different shell than you expect. Dash does not have the <<< redirection operator.
Make sure the shebang line is:
#!/bin/bash
or
#!/usr/bin/env bash
And run the script with:
$ ./script.sh
Do not run it with an explicit sh as that will ignore the shebang:
$ sh ./script.sh # Don't do this!
If you're using the following to run your script:
sudo sh ./script.sh
Then you'll want to use the following instead:
sudo bash ./script.sh
The reason for this is that Bash is not the default shell for Ubuntu. So, if you use "sh" then it will just use the default shell; which is actually Dash. This will happen regardless if you have #!/bin/bash at the top of your script. As a result, you will need to explicitly specify to use bash as shown above, and your script should run at expected.
Dash doesn't support redirects the same as Bash.
Docker:
I was getting this problem from my Dockerfile as I had:
RUN bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
However, according to this issue, it was solved:
The exec form makes it possible to avoid shell string munging, and
to RUN commands using a base image that does not contain /bin/sh.
Note
To use a different shell, other than /bin/sh, use the exec form
passing in the desired shell. For example,
RUN ["/bin/bash", "-c", "echo hello"]
Solution:
RUN ["/bin/bash", "-c", "bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)"]
Notice the quotes around each parameter.
You can get the output of that command and put it in a variable. then use heredoc. for example:
nc -l -p 80 <<< "tested like a charm";
can be written like:
nc -l -p 80 <<EOF
tested like a charm
EOF
and like this (this is what you want):
text="tested like a charm"
nc -l -p 80 <<EOF
$text
EOF
Practical example in busybox under docker container:
kasra#ubuntu:~$ docker run --rm -it busybox
/ # nc -l -p 80 <<< "tested like a charm";
sh: syntax error: unexpected redirection
/ # nc -l -p 80 <<EOL
> tested like a charm
> EOL
^Cpunt! => socket listening, no errors. ^Cpunt! is result of CTRL+C signal.
/ # text="tested like a charm"
/ # nc -l -p 80 <<EOF
> $text
> EOF
^Cpunt!
do it the simpler way,
direc=$(basename `pwd`)
Or use the shell
$ direc=${PWD##*/}
Another reason to the error may be if you are running a cron job that updates a subversion working copy and then has attempted to run a versioned script that was in a conflicted state after the update...
On my machine, if I run a script directly, the default is bash.
If I run it with sudo, the default is sh.
That’s why I was hitting this problem when I used sudo.
In my case error is because i have put ">>" twice
mongodump --db=$DB_NAME --collection=$col --out=$BACKUP_LOCATION/$DB_NAME-$BACKUP_DATE >> >> $LOG_PATH
i just correct it as
mongodump --db=$DB_NAME --collection=$col --out=$BACKUP_LOCATION/$DB_NAME-$BACKUP_DATE >> $LOG_PATH
Before running the script, you should check first line of the shell script for the interpreter.
Eg:
if scripts starts with /bin/bash , run the script using the below command
"bash script_name.sh"
if script starts with /bin/sh, run the script using the below command
"sh script_name.sh"
./sample.sh - This will detect the interpreter from the first line of the script and run.
Different Linux distributions having different shells as default.
I have a script that consumes users input. It is run like this:
./script <<EOF
> input
> onther input
> more input
> EOF
I need to distribute the script as a Docker image.
I am able to run the script in two steps.
First, I get into the containers shell.
docker run -it my-docker-tag sh
And then, inside the shell, I execute the script itself.
Question: Is it possible to run the script in on shut (without having to navigate to the containers shall)?
I tried this:
docker run -it my-docker-tag ./script <<EOF
> input
> onther input
> more input
> EOF
But it fails with:
the input device is not a TTY
The Docker run reference notes:
Specifying -t is forbidden when the client is receiving its standard input from a pipe, as in:
$ echo test | docker run -i busybox cat
If you remove the -t option, shell pipes and redirections around a (foreground) docker run command will work as you expect.
# A heredoc as in the question should work too
sudo docker run --rm my-docker-tag ./script <script-input.txt
I have a docker image running java application, its main class is dynamic, in a file called start-class. Traditionally, I started the application like this.
java <some_options_ignored> `cat start-class`
Now I want to run these applications in docker containers. This is my Dockerfile.
FROM openjdk:8
##### Ignored
CMD ["java", "`cat /app/classes/start-class`"]
I built the image and run the containers. The command actually executed was this.
$ docker ps --no-trunc | grep test
# show executed commands
"java '`cat /app/classes/start-class`"
Single quotes was automatically wrapped outside the backticks. How can I fix this??
You're trying to run a shell command (expanding a sub-command) without a shell (the json/exec syntax of CMD). You need to switch to the shell syntax (or explicitly run a shell with the exec syntax). That would look like:
CMD exec java `cat /app/classes/start-class`
Without the json formatting, docker will run
sh -c "exec java `cat /app/classes/start-class`"
The exec in this case will replace the shell in pid 1 with the java process to improve signal handling.
I cannot get a script to return to bash.
The script is kicked off via the following Docker directives:
ENTRYPOINT ["/bin/bash", "-c"]
CMD ["set -e && /config/startup/init.sh"]
The init script looks like this:
#!/bin/bash
if [ -d /etc/postfix/init.d ]; then
for f in /etc/postfix/init.d/*.sh; do
[ -f "$f" ] && . "$f"
done
fi
echo "[x] Starting supervisord ..."
/usr/bin/supervisord -c /etc/supervisord.conf
bash
And this is the command I use to kick off the image into a container:
docker run -it --env-file ENV_LOCAL mailrelay
The init script runs as expected (and I see output from the scripts within the /etc/postfix/init.d/ directory and supervisord kicks off Postfix.
The problem is getting the script to return to the parent process (bash) instead of needing to start a new one. After it hits the supervisord the session sits there, requiring a Ctrl+C to get it to get back into a bash prompt.
If I leave off the call to bash at the end of the init.sh script, Ctrl+D exits the script AND the container, returning me to the host OS (osx). If I replace the bash call with exit, it returns to the host OS as well.
Is supervisord behaving the way it's supposed to, by running in the foreground this way? I'd like to be able to easily get back into the container shell session to check to see if things are running. Am I left with needing to Ctrl+D (into the secondary bash session) in order to do this?
UPDATE
Marc B
take out the bash line, so you don't start a new shell. and if
supervisord doesn't go into the background automatically, you could
try running it with & to force it into the background, or maybe
there's an extra cli option to force it to go into daemon mode
I've tried removing the last call to bash, but as I've mentioned it just sits there still, and Ctrl+D takes me to the host OS (exits the container).
I just tried /usr/bin/supervisord -c /etc/supervisord.conf & (and left off the call to bash at the end) and it just immediately returns to host OS, exiting the container. I assume because the container had nothing left to "do", and so stopped.
#!/bin/bash
if [ -d /etc/postfix/init.d ]; then
for f in /etc/postfix/init.d/*.sh; do
[ -f "$f" ] && . "$f"
done
fi
echo "[x] Starting supervisord ..."
/usr/bin/supervisord -c /etc/supervisord.conf
one
bash # You are spawning a new bash shell here. Remove this statement
At the end your're stuck in a child bash shell :(
Now if you're not returning to the parent shell, the last command that you have run is the culprit.
/usr/bin/supervisord -c /etc/supervisord.conf
You can either force the command to run in the background by
/usr/bin/supervisord -c /etc/supervisord.conf & #the & tells to run in background
A workaround for keeping the container open is mentioned here
I have some script I need to run during a Docker build which requires a tty (which Docker does not provide during a build). Under the hood the script uses the read command. With a tty, I can do things like (echo yes; echo no) | myscript.sh.
Without it I get strange errors I don't completely understand. So is there any way to use this script during the build (given that its not mine to modify?)
EDIT: Here's a more definite example of the error:
FROM ubuntu:14.04
RUN echo yes | read
which fails with:
Step 0 : FROM ubuntu:14.04
---> 826544226fdc
Step 1 : RUN echo yes | read
---> Running in 4d49fd03b38b
/bin/sh: 1: read: arg count
The command '/bin/sh -c echo yes | read' returned a non-zero code: 2
RUN <command> in Dockerfile reference:
shell form, the command is run in a shell, which by default is /bin/sh -c on Linux or cmd /S /C on Windows
let's see what exactly /bin/sh is in ubuntu:14.04:
$ docker run -it --rm ubuntu:14.04 bash
root#7bdcaf403396:/# ls -n /bin/sh
lrwxrwxrwx 1 0 0 4 Feb 19 2014 /bin/sh -> dash
/bin/sh is a symbolic link of dash, see read function in dash:
$ man dash
...
read [-p prompt] [-r] variable [...]
The prompt is printed if the -p option is specified and the standard input is a terminal. Then a line
is read from the standard input. The trailing newline is deleted from the line and the line is split as
described in the section on word splitting above, and the pieces are assigned to the variables in order.
At least one variable must be specified. If there are more pieces than variables, the remaining pieces
(along with the characters in IFS that separated them) are assigned to the last variable. If there are
more variables than pieces, the remaining variables are assigned the null string. The read builtin will
indicate success unless EOF is encountered on input, in which case failure is returned.
By default, unless the -r option is specified, the backslash ``\'' acts as an escape character, causing
the following character to be treated literally. If a backslash is followed by a newline, the backslash
and the newline will be deleted.
...
read function in dash:
At least one variable must be specified.
let's see read function in bash:
$ man bash
...
read [-ers] [-a aname] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [-t timeout] [-u fd] [name...]
If no names are supplied, the line read is assigned to the variable REPLY. The return code is zero,
unless end-of-file is encountered, read times out (in which case the return code is greater than
128), or an invalid file descriptor is supplied as the argument to -u.
...
So I guess your script myscript.sh is start with #!/bin/bash or something else but not /bin/sh.
Also, you can change your Dockerfile like below:
FROM ubuntu:14.04
RUN echo yes | read ENV_NAME
Links:
https://docs.docker.com/engine/reference/builder/
http://linux.die.net/man/1/dash
http://linux.die.net/man/1/bash
Short answer : You can't do it straightly because docker build or either buildx didn't implement [/dev/tty, /dev/console]. But there is a hacky solution where you can achieve what you need but I highly discourage using it since it break the concept of CI. That's why docker didn't implement it.
Hacky solution
FROM ubuntu:14.04
RUN echo yes | read #tty requirement command
As mentioned in docker reference document the RUN consist of two stage, first is execution of command and the second is commit to the image as a new layer. So you can do the stages manually on your own where we will provide tty to first stage(execution) and then commit the result.
Code:
cd
cat >> tty_wrapper.sh << EOF
echo yes | read ## Your command which needs tty
rm /home/tty_wrapper.sh
EOF
docker run --interactive --tty --detach --privileged --name name1 ubuntu:14.04
docker cp tty_wrapper.sh name1:/home/
docker exec name1 bash -c "cd /home && chmod +x tty_wrapper.sh && ./tty_wrapper.sh "
docker commit name1 your:tag
Your new image is ready.
Here is a description about the code.
At first we make a bash script which wrap our tty to it and then remove itself after fist execute. Then we run a container with provided tty option(you can remove privileged if you don't need). Next step we copy wrapped bash script inside container and do the execution & commit stage on our own.
You don't need a tty for feeding your data to your script . just doing something like (echo yes; echo no) | myscript.sh as you suggested will do. also please make sure you copy your file first before trying to execute it . something like COPY myscript.sh myscript.sh
Most likely you don't need a tty. As the comment on the question shows, even the example provided is a situation where the read command was not properly called. A tty would turn the build into an interactive terminal process, which doesn't translate well to automated builds that may be run from tools without terminals.
If you need a tty, then there's the C library call to openpty that you would use when forking a process that includes a pseudo tty. You may be able to solve your problem with a tool like expect, but it's been so long that I don't remember if it creates a ptty or not. Alternatively, if your application can't be built automatically, you can manually perform the steps in a running container, and then docker commit the resulting container to make an image.
I'd recommend against any of those and to work out the procedure to build your application and install it in a non-interactive fashion. Depending on the application, it may be easier to modify the installer itself.