I'm trying to execute python scripts automatically generated by zc.buildout so I don't have control over them. My problem is that the shebang line (#!) is too long for either bash (80 character limit) or direct execution (some Linux kernel constant I don't know).
This is an example script to help you reproduce my problem:
#!/././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././bin/bash
echo Hola!
How can be bash or the kernel configured to allow for bigger shebang lines?
Limited to 127 chars on 99.9% of systems due to kernel compile time buffer limit.
It's limited in the kernel by BINPRM_BUF_SIZE, set in include/linux/binfmts.h.
If you don't want to recompile your kernel to get longer shebang lines, you could write a wrapper:
#!/bin/bash
if [[ $# -eq 0 ]]; then
echo "usage: ${0##*/} script [args ...]"
exit
fi
# we're going to expand a variable *unquoted* to use word splitting, but
# we don't want to have path expansion effects, so turn that off
set -f
shebang=$(head -1 "$1")
if [[ $shebang == '#!'* ]]; then
interp=( ${shebang#\#!} ) # use an array in case a argument is there too
else
interp=( /bin/sh )
fi
# now run it
exec "${interp[#]}" "$#"
and then run the script like: wrapper.sh script.sh
Updated #glenn jackman's script to support passing in command line arguments.
Incidentally, I ran into this problem when creating a python virtualenv inside of a very deep directory hierarchy.
In my case, this was a virtualenv created inside a Mesos framework dir.
The extra long shebang rendered calling xxx/.../venv/bin/pip useless.
The wrapper script proved most useful.
#!/usr/bin/env bash
script="$1"
shebang=$(head -1 "$script")
# use an array in case a argument is there too
interp=( ${shebang#\#!} )
# now run it, passing in the remaining command line arguments
shift 1
exec "${interp[#]}" "$script" "${#}"
Related
I'm writing a bash script and it throws an error when using "sh" command in Ubuntu (it seems it's not compatible with dash, I'm learning on this subject). So I would like to detect if dash is being used instead of bash to throw an error.
How can I detect it in a script context?. Is it even possible?
You can check for the presence of shell-specific variables:
For instance, bash defines $BASH_VERSION.
Since that variable won't be defined while running in dash, you can use it to make the distinction:
[ -n "$BASH_VERSION" ] && isBash=1
Afterthought: If you wanted to avoid relying on variables (which, conceivably, could be set incorrectly), you could try to obtain the ultimate name of the shell executable running your script, by determining the invoking executable and, if it is a symlink, following it to its (ultimate) target.
The shell function getTrueShellExeName() below does that; for instance, it would return 'dash' on Ubuntu for a script run with sh (whether explicitly or via shebang #!/bin/sh), because sh is symlinked to dash there.
Note that the function's goal is twofold:
Be portable:
Work with all POSIX-compatible (Bourne-like) shells,
across at least most platforms, with respect to what utilities and options are used - see caveats below.
Work in all invocation scenarios:
sourced (whether from a login shell or not)
executed stand-alone, via the shebang line
executed by being passed as a filename argument to a shell executable
executed by having its contents piped via stdin to a shell executable
Caveats:
On at least one platform - macOS - sh is NOT a symlink, even though it is effectively bash. There, the function would return 'sh' in a script run with sh.
The function uses readlink, which, while not mandated by POSIX, is present on most modern platforms - though with differing syntax and features. Therefore, using GNU readlink's -f option to find a symlink's ultimate target is not an option.
(The only modern platform I'm personally aware of that does not have a readlink utility is HP-UX - see https://stackoverflow.com/a/24114056/45375 for a recursive-readlink implementation that should work on all POSIX platforms.)
The function uses the which utility (except in zsh, where it's a builtin), which, while not mandated by POSIX, is present on most modern platforms.
Ideally, ps -p $$ -o comm= would be sufficient to determine the path of the executable underlying the process, but that doesn't work as intended when directly executing shell scripts with shebang lines on Linux, at least when using the ps implementation from the procps-ng package, as found on Ubuntu, for instance: there, such scripts report the script's file name rather than the underlying script engine's.Tip of the hat to ferdymercury for his help.
Therefore, the content of special file /proc/$$/cmdline is parsed on Linux, whose first NUL-separated field contains the true executable path.
Example use of the function:
[ "$(getTrueShellExeName)" = 'bash' ] && isBash=1
Shell function getTrueShellExeName():
getTrueShellExeName() {
local trueExe nextTarget 2>/dev/null # ignore error in shells without `local`
# Determine the shell executable filename.
if [ -r /proc/$$/cmdline ]; then
trueExe=$(cut -d '' -f1 /proc/$$/cmdline) || return 1
else
trueExe=$(ps -p $$ -o comm=) || return 1
fi
# Strip a leading "-", as added e.g. by macOS for login shells.
[ "${trueExe#-}" = "$trueExe" ] || trueExe=${trueExe#-}
# Determine full executable path.
[ "${trueExe#/}" != "$trueExe" ] || trueExe=$([ -n "$ZSH_VERSION" ] && which -p "$trueExe" || which "$trueExe")
# If the executable is a symlink, resolve it to its *ultimate*
# target.
while nextTarget=$(readlink "$trueExe"); do trueExe=$nextTarget; done
# Output the executable name only.
printf '%s\n' "$(basename "$trueExe")"
}
Use $0 (that is the name of the executable of the shell being called).The command for example
echo $0
gives
/usr/bin/dash
for the dash and
/bin/bash
for a bash.The parameter substitution
${0##*/}
gives just 'dash' or 'bash'. This can be used in a test.
An alternative approach might be to test if a shell feature is available, for example to give an idea...
[[ 1 ]] 2>/dev/null && echo could be bash || echo not bash, maybe dash
echo $0 and [[ 1 ]] 2>/dev/null && echo
could be bash || echo not bash, maybe bash worked for me running Ubuntu 19.
Done slight Pascal, Fortran and C in school, but need to become fluent in shell script.
This question already has answers here:
Pass commands as input to another command (su, ssh, sh, etc)
(3 answers)
Closed 6 years ago.
I'm very very new to Linux(coming from windows) and trying to write a script that i can hopefully execute over multiple systems. I tried to use Python for this but fount it hard too. Here is what i have so far:
cd /bin
bash
source compilervars.sh intel64
cd ~
exit #exit bash
file= "~/a.out"
if[! -f "$file"]
then
icc code.c
fi
#run some commands here...
The script hangs in the second line (bash). I'm not sure how to fix that or if I'm doing it wrong. Please advice.
Also, any tips of how to run this script over multiple systems on the same network?
Thanks a lot.
What I believe you'd want to do:
#!/bin/bash
source /bin/compilervars.sh intel64
file="$HOME/a.out"
if [ ! -f "$file" ]; then
icc code.c
fi
You would put this in a file and make it executable with chmod +x myscript. Then you would run it with ./myscript. Alternatively, you could just run it with bash myscript.
Your script makes little sense. The second line will open a new bash session, but it will just sit there until you exit it. Also, changing directories back and forth is very seldom required. To execute a single command in another directory, one usually does
( cd /other/place && mycommand )
The ( ... ) tells the shell that you'd like to do this in a sub-shell. The cd happens within that sub-shell and you don't have to cd back after it's done. If the cd fails, the command will not be run.
For example: You might want to make sure you're in $HOME when you compile the code:
if [ ! -f "$file" ]; then
( cd $HOME && icc code.c )
fi
... or even pick out the directory name from the variable file and use that:
if [ -f "$file" ]; then
( cd $(dirname "$file") && icc code.c )
fi
Assigning to a variable needs to happen as I wrote it, without spaces around the =.
Likewise, there needs to be spaces after if and inside [ ... ] as I wrote it above.
I also tend to use $HOME rather than ~ in scripts as it's more descriptive.
A shell script isn't a record of key strokes which are typed into a terminal. If you write a script like this:
command1
bash
command2
it does not mean that the script will switch to bash, and then execute command2 in the different shell. It means that bash will be run. If there is a controlling terminal, that bash will show you a prompt and wait for a command to be typed in. You will have to type exit to quit that bash. Only then will the original script then continue with command2.
There is no way to switch a script to a different shell halfway through. There are ways to simulate this. A script can re-execute itself using a different shell. In order to do that, the script has to contain logic to detect that it is being re-executed, so that it can prevent re-executing itself again, and to skip some code that shouldn't be run twice.
In this script, I implemented such a re-execution hack. It consists of these lines:
#
# The #!/bin/sh might be some legacy piece of crap,
# not even up to 1990 POSIX.2 spec. So the first step
# is to look for a better shell in some known places
# and re-execute ourselves with that interpreter.
#
if test x$txr_shell = x ; then
for shell in /bin/bash /usr/bin/bash /usr/xpg4/bin/sh ; do
if test -x $shell ; then
txr_shell=$shell
break
fi
done
if test x$txr_shell = x ; then
echo "No known POSIX shell found: falling back on /bin/sh, which may not work"
txr_shell=/bin/sh
fi
export txr_shell
exec $txr_shell $0 ${#+"$#"}
fi
The txr_shell variable (not a standard variable, my invention) is how this logic detects that it's been re-executed. If the variable doesn't exist then this is the original execution. When we re-execute we export txr_shell so the re-executed instance will then have this environment variable.
The variable also holds the path to the shell; that is used later in the script; it is passed through to a Makefile as the SHELL variable, so that make build recipes use that same shell. In the above logic, the contents of txr_shell don't matter; it's used as Boolean: either it exists or it doesn't.
The programming style in the above code snippet is deliberately coded to work on very old shells. That is why test x$txr_shell = x is used instead of the modern syntax [ -z "$txr_shell" ], and why ${#+"$#"} is used instead of just "$#".
This style is no longer used after this point in the script, because the
rest of the script runs in some good, reasonably modern shell thanks to the re-execution trick.
I'm trying to execute python scripts automatically generated by zc.buildout so I don't have control over them. My problem is that the shebang line (#!) is too long for either bash (80 character limit) or direct execution (some Linux kernel constant I don't know).
This is an example script to help you reproduce my problem:
#!/././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././././bin/bash
echo Hola!
How can be bash or the kernel configured to allow for bigger shebang lines?
Limited to 127 chars on 99.9% of systems due to kernel compile time buffer limit.
It's limited in the kernel by BINPRM_BUF_SIZE, set in include/linux/binfmts.h.
If you don't want to recompile your kernel to get longer shebang lines, you could write a wrapper:
#!/bin/bash
if [[ $# -eq 0 ]]; then
echo "usage: ${0##*/} script [args ...]"
exit
fi
# we're going to expand a variable *unquoted* to use word splitting, but
# we don't want to have path expansion effects, so turn that off
set -f
shebang=$(head -1 "$1")
if [[ $shebang == '#!'* ]]; then
interp=( ${shebang#\#!} ) # use an array in case a argument is there too
else
interp=( /bin/sh )
fi
# now run it
exec "${interp[#]}" "$#"
and then run the script like: wrapper.sh script.sh
Updated #glenn jackman's script to support passing in command line arguments.
Incidentally, I ran into this problem when creating a python virtualenv inside of a very deep directory hierarchy.
In my case, this was a virtualenv created inside a Mesos framework dir.
The extra long shebang rendered calling xxx/.../venv/bin/pip useless.
The wrapper script proved most useful.
#!/usr/bin/env bash
script="$1"
shebang=$(head -1 "$script")
# use an array in case a argument is there too
interp=( ${shebang#\#!} )
# now run it, passing in the remaining command line arguments
shift 1
exec "${interp[#]}" "$script" "${#}"
I am using bash and this works on Linux:
read -r -d '' VAR<<-EOF
Hello\nWorld
EOF
echo $VAR > trail
i.e the contents of the file on Linux is
Hello\nWorld
When i run on Solaris
trial file has
Hello
World
The newline(\n) is being replaced with a newline. How can i avoid it?
Is it a problem with heredoc or the echo command?
[UPDATE]
Based on the explanation provided here:
echo -E $VAR > trail
worked fine on Solaris.
The problem is with echo. Behavior is defined in POSIX, where interpretting \n is part of XSI but not basic POSIX itself.
You can avoid this on all platforms using printf (which is good practice anyways):
printf "%s\n" "$VAR"
This is not a problem for bash by the way. If you had used #!/usr/bin/env bash as the shebang (and also not run the script with sh script), behavior would have been consistent.
If you use #!/bin/sh, you'll get whichever shell the system uses as a default, with varying behaviors like this.
To complement #that other guy's helpful answer:
Even when it is bash executing your script, there are several ways in which the observed behavior - echo by default interpreting escape sequences such as \n - can come about:
shopt -s xpg_echo could be in effect, which makes the echo builtin interpret \ escape sequences by default.
enable -n echo could be in effect, which disables the echo builtin and runs the external executable by default - and that executable's behavior is platform-dependent.
These options are normally NOT inherited when you run a script, but there are still ways in which they could take effect:
If your interactive initialization files (e.g., ~/.bashrc) contain commands such as the above and you source (.) your script from an interactive shell.
When not sourcing your script: If your environment contains a BASH_ENV variable that points to a script, that script is sourced before your script runs; thus, if that script contains commands such as the above, they will affect your script.
I have a problem implementing a for loop. I get this error when I execute my script
test1.sh: 2: Syntax error: Bad for loop variable
I don't understand this error.
This is my script
#!/bin/bash
for (( c=1; c<=5; c++ ))
do
echo "Welcome $c times..."
done
can any one tell me syntax for for loop in sh(in ubuntu it links to dash shell) shell in ubuntu?
You probably run it with sh, not bash. Try bash test1.sh, or ./test1.sh if it's executable, but not sh test1.sh.
A standard POSIX shell only accepts the syntax for varname in list
The C-like for-loop syntax for (( expr1; expr2; expr3 )) is a bashism.
You can get similar behavior in the standard POSIX shell using for c in $(seq 1 5)
What does
ls -l /bin/sh
give on your machine ?
Make sh a symbolic link to bash and then you can do sh ./test1.sh
Your shell script (as shown) runs in both Korn shell and Bash. Some thoughts:
You might need a space after the shebang (#! /bin/bash and not #!/bin/bash). However, Dennis Ritchie had originally specified the space is optional. Besides, it isn't the error you get with Bourne shell (you get syntax error: '(' unexpected instead).
Are you on a Windows system? Just a stab in the dark. This doesn't look like a Windows error.
Is this Solaris or HP/UX system? They might not be running true versions of Bash, or maybe an older version. However, even the oldest version of Bash recognizes the for ((x;y;z)) construct.
Try this:
#! /bin/bash
set -vx
echo "Random = $RANDOM" #Test for bash/Kornshell. Will be blank in other shells
echo \$BASH_VERSINFO[0] = ${BASH_VERSINFO[0]} #Should only work in BASH
echo \$BASH_VERSINFO[1] = ${BASH_VERSINFO[1]}
echo \$BASH_VERSINFO[2] = ${BASH_VERSINFO[2]}
echo \$BASH_VERSINFO[3] = ${BASH_VERSINFO[3]}
echo \$BASH_VERSINFO[4] = ${BASH_VERSINFO[4]}
echo \$BASH_VERSINFO[5] = ${BASH_VERSINFO[5]}
for ((c=0, c<=5, c++))
do
echo "Welcome $c times"
done
The set -xv will display all lines as they are executed.
The $RANDOM should display a value if this is either BASH or Kornshell (your for loop will work in either one).
The {$BASH_VERINFO[x]} should only be set if this is truly BASH. These aren't even set even if you run Korn shell after you're in BASH (unlike $SHELL which will still contain bash).
If the for loop still gives you trouble, just delete it. Somewhere in this script, we'll find out if you're really executing a bash shell or not.