How can I test my Bash script on older versions of Bash? - linux

I'm working on a Bash library and want to ensure I'm supporting as many environments as possible - including old installations of Bash. My development environment is Bash 4.3, but some of my users may well be running much older versions and presently I have no way to confirm or deny that my library will work for them. In particular I'd like to be compatible with OSX (which still ships with Bash 3.2, AFAIK).
I know Bash can run in POSIX-compliant mode; is there a similar setting to disable modern functionality? Or a way to run Bash in some sort of compatibility mode? I'm looking for any technique short of actually finding and booting up old operating systems and testing my library there.
Update
For example, I've avoided using associative arrays since they were introduced in Bash 4, but it's hard to be sure without testing that I'm not accidentally using some other Bash 4+ feature.

Finally coming back to this question, it's pretty easy to just compile (without installing) the bash version(s) you're interested in. Here's how I'm testing Bash 3.2.57:
$ mkdir ~/bash
$ cd ~/bash
$ wget http://ftp.gnu.org/gnu/bash/bash-3.2.57.tar.gz
$ tar xvzf bash-3.2.57.tar.gz
$ cd bash-3.2.57
$ ./configure
$ make
# if `make` fails due to yacc, run `sudo apt-get install byacc`
# No need to run `make install`
$ ./bash -version
GNU bash, version 3.2.57(1)-release (armv7l-unknown-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
Now you have a bash 3.2.57 binary you can run, without actually "installing" it or modifying your normal environment.
To run a shell script against this version:
$ ./bash your_script.sh
To enter a clean interactive prompt:
$ env -i PATH="$PWD:$PATH" ./bash --noprofile --norc
bash-3.2$ bash -version
GNU bash, version 3.2.57(1)-release (armv7l-unknown-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
bash-3.2$
Using env -i rather than just calling ./bash directly leaves you with a mostly-empty environment (run env from inside the shell to see what's still set). Updating the PATH allows calls to bash (e.g. bash -version) to invoke the local bash shell, not the system-wide installation (but note this pulls in your whole PATH). Adding --noprofile --norc avoids loading your .bashrc and associated scripts.
If you don't want to pick up any PATH modifications, just execute export PATH="$PWD:$PATH" once inside the subshell instead of as part of the env command.
I have a Docker image (repo) using these installation steps, if that's helpful for folks to reference. I wouldn't necessarily suggest using this image directly, but you're welcome to copy from the Dockerfile/install script. MIT licensed.

Although it's nice to know that it's possible to compile arbitrary versions of bash locally (as discussed in my other answer), these days there's a much simpler option - the official Docker bash images.
To test a script against multiple bash versions is often as simple as:
for v in 3 4 5; do # or whatever versions you're interested in
docker run -v "$PWD:/mnt" "bash:$v" \
bash /mnt/your_script.sh
done

Have a look at shenv: https://github.com/shenv/shenv. Just like rbenv, pyenv, goenv and others, but for shells, it lets you install different versions of Bash among others (zsh, fish, yash, etc.).
(Disclaimer: I'm the one who forked pyenv into shenv!)

You can use Bash own facilities to emulate older version of Bash. Look up "compat" options on shopt.
Mind you, while it does change behavior as described under each compatNN entry in the man page, it does not remove features that are otherwise present on the current version. For example, this doesn't cause any errors:
shopt -s compat31
shopt -s globstar
Even though globstar was only introduced on Bash 4.0.

Related

linux script - substring and append compatibility with unix

I am trying to make my following script compatible with other platforms (unix) and i am not sure if it will be. Especially [[test]] and %%# symbols are compatible. At lease this script works fine on linux.
It would be great if someone (who is familiar with unix) can make some suggestions or fixes to make the following script portable across the platforms (except windows).
#!/bin/sh
INSTALL_HOME=/opt/prod/install0308
export INSTALL_HOME
export CONF_INSTALL_ARGS="-Dinstall.ext.dir=/opt/prod/installExt -Dinstall.alternateExtDir=/opt/dev/installExt/lib -Dinstall.type=OSD"
INSTALL_ALTERNATIVE_TYPES_DIR=''
if [[ ${CONF_INSTALL_ARGS} == *'-Dinstall.alternateExtDir'* ]]; then
INSTALL_ALT_TYPE_DIR_TEMP=${CONF_INSTALL_ARGS#*-Dinstall.alternateExtDir=}
INSTALL_TYPE_DIR=${INSTALL_ALT_TYPE_DIR_TEMP%%-D*}
FINAL_INST_TYPE_DIR="$(echo -e "${INSTALL_TYPE_DIR}" | sed 's/ *$//g')"
INSTALL_ALTERNATIVE_TYPES_DIR=','$FINAL_INST_TYPE_DIR
fi
TOTAL_CONF_ARGS="-Dinstall.ext.dir=${INSTALL_HOME}/lib/provider,${INSTALL_HOME}/lib/security${INSTALL_ALTERNATIVE_TYPES_DIR}"
echo $TOTAL_CONF_ARGS
This is not a compatibility problem between Operating Systems, this is a compatibility problem between Shells.
Your script has been written for bash-like shells, so you just need to replace the first line #!/bin/sh by #!/bin/bash (or any path where bash is located) for it to work on other systems (do not forget to install bash on them).
NB: This script works on your Linux with the shebang #!/bin/sh probably because your Linux distribution has chosen to replace legacy sh by a link to bash or you are explicitly running the script with bash like this: bash ./script.sh.

Making Unix shell scripts POSIX compliant

I have been working on a shell script to automate some tasks. What is the best way to make sure the shell script would run without any issues in most of the platforms. For ex., I have been using echo -n command to print some messages to the screen without a trailing new line and the -n switch doesn't work in some ksh shells. I was told the script must be POSIX compliant. How do I make sure that the script is POSIX compliant. Is there a tool? Or is there a shell that supports only bare minimum POSIX requirements?
POSIX
One first step, which gives you indications of what works or not and why, is to set the shebang to /bin/sh and use shellcheck site to analyze your script.
For example, paste this script in the shellcheck editor window:
#!/bin/sh
read -r a b <<<"$1"
echo $((a+b))
to get an indication that: "In POSIX sh, here-strings are undefined".
As a second step, you can use a shell that is as compatible with POSIX as possible.
One shell that is compatible with most other simple shells, is dash, Debian default system shell, which is a derivative of the older BSD ash.
Another shell compatible with posix is posh.
However, dash and/or posh may not be available for some systems.
There is lksh (with a ksh flavor), with the goal to be compatible with legacy (old) shell scripts. From its manual:
lksh is a command interpreter intended exclusively for running legacy shell scripts.
But there is the need to use options when calling lksh, like -o posix and -o sh:
Note that it's strongly recommended to invoke lksh with at least the -o posix option, if not both that and -o sh, to fully enjoy better compatibility to the POSIX standard (which is probably why you use lksh over mksh in the first place) or legacy scripts, respectively.
You would call lksh -o posix -o sh instead of the simple lksh.
Using options is a way to make other shells become POSIX compatible. Like lksh, using the option -o posix, like bash -o posix.
In bash, it is even possible to turn the POSIX option inside an script, with:
shopt -o posix # also with: set -o posix
It is also possible to make a local link to bash or zsh that makes both act like an old sh shell. Like this:
$ ln -s /bin/bash ./sh
$ ./sh
There are plenty of alternatives (dash, posh, lksh, bash, zsh, etc.) to get a shell that will work as a POSIX shell.
Portable
However, even so, all the above does not ensure "portability".
Unfortunately, making a shell script 'POSIX-compliant' is usually easier than making it run on any real-world shell.
The only real-world sensible recommendation is test your script in several shells.
Like the list above: dash, posh, lksh, and bash --posix.
Solaris is a world on its own, probably you will need to test against /bin/sh and xpg4/sh.
Followup:
How can I test for POSIX compliance for shell scripts?
Starting Bash with the --posix command-line option or executing ‘set -o posix’ while Bash is running will cause Bash to conform more closely to the POSIX standard by changing the behavior to match that specified by POSIX in areas where the Bash default differs.
Reference
Note:
This answer complements user8017719's great answer.
As requested in the question, a tool is discussed below: while it does not directly check for POSIX compliance, it runs a given script in multiple shells, notably including /bin/sh.
/bin/sh, the system default shell, should not be assumed to support any features other than POSIX-prescribed ones, though in practice it does, to varying degrees, depending on the specific implementation. Therefore, successfully running via /bin/sh on one platform does not guarantee that the script will work on another. Among widely used shells, dash comes closest to being a POSIX-features-only shell.
Running successfully in multiple shells is important:
if you're authoring a script that needs to be sourced in various shells.
if you know that your script will encounter only a limited set of known-in-advance shells.
For a proof-of-the-pudding-is-in-the-eating approach, consider using shall (a utility I wrote), which allows you to invoke a given script or command with multiple shells at once, with feedback about which of the targeted shells the script/command executed successfully with.
If you have Node.js installed, you can easily install it with npm install -g shall (if not, follow the above link to the GitHub repo for manual installation instructions) and then use it as follows:
shall scriptFile
or, with an ad-hoc command:
shall -c '<shell-commands>'
By default, it invokes sh, and, if installed, dash, bash, zsh, and ksh, but you can target any set of shells that you have installed by using the SHELLS environment variable.
Using the example of the echo -n command on macOS to only target shells sh and bash:
$ SHELLS=sh,bash shall -c 'echo -n hi'
✓ sh (bash variant) [0.00s]
-n hi
✓ bash [0.00s]
hi
OK - All 2 shells (sh, bash) report success.
On macOS, bash (effectively) acts as sh, and while echo -n didn't fail when used with sh, you can also see that -n wasn't recognized as an option when bash ran as sh.
Another macOS example that shows that bash permits certain Bash-specific extensions even when running as sh, such as using nonstandard [[ ... ]] conditionals (assumes that dash - which acts as sh on Ubuntu systems - was installed via Homebrew):
$ SHELLS=sh,bash,dash shall -c '[[ -n nonempty ]] && echo nonempty'
✓ sh (bash variant) [0.00s]
nonempty
✓ bash [0.00s]
nonempty
✗ dash [0.01s]
dash: 1: [[: not found
FAILED - 1 shell (dash) reports failure, 2 (sh, bash) report success.
As you can see, Bash running as sh still accepted [[ ... ]], whereas dash, which is a (mostly) POSIX-features-only shell, failed, because POSIX only mandates [ ... ] conditionals (as an alias of test ... commands).

Can I run a Linux shell script in Windows?

I created Shell Script on Linux and it runs fine.
Now I want to run the same script on Windows using gitbash. (Only filepath changed for windows)
I am confused:
do I need to write a new shell script again according to Windows syntax shell script?
or
can a Linux syntax shell script run on Windows without changes?
According TO differences-between-windows-batch-and-linux-bash-shell-script-syntax
Of course you can. There is a tool called cygwin that allows you to do so.
Note that you have to check what the paths are. If so, go to the path you are willing to work on and do pwd. This way, you will get the Windows\kind\of\path.
I use it all the time and it works pretty fine.
You can use Git Bash
It depends on how advanced the scripts are, but simple scripts can be executed in Git Bash.
test.sh contains:
#!/bin/bash
echo Hello World!
Execute script:
./test.sh
Output:
Hello World!
Git Bash vs Cygwin
To answer your question:
#fedorqui in my learning 'cygwin' And 'gitbash' do same stuff for
windows
Git Bash
Git Bash is lightweight and only aims to handle:
version control
a shell that runs commands
Read more: http://openhatch.org/missions/windows-setup/install-git-bash
Cygwin
a large collection of GNU and Open Source tools which provide
functionality similar to a Linux distribution on Windows.
a DLL (cygwin1.dll) which provides substantial POSIX API
functionality.
Read more: https://www.cygwin.com/

Multiple GNU-Bash commands behave differently when run on aix, any solutions

I have written a bash script on linux, and it works well, as a part of migration I moved [rather added] an aix 7.2 node to my cluster. When I tried running bash scripts on aix it failed with multiple errors on different gnu bash commands.
[ps: I have installed gnu bash on this aix node, IBM calls it a toolbox made for aix, which contains a collection of open source and GNU software built for AIX IBM Systems]
For example :
- grep -oP isn't supported
- ls -h doesn't work
- getopts fails to get parameter passed and $# as well.
I am not sure if I am doing it right with just installing the gnu bash on aix. Have anyone had any experience porting bash scripts over to ssh?
Are there any pointer community can suggest to get bash script work on aix?
The issue is that these commands are not part of bash. What you need is the GNU versions of all these utilities, that is grep and ls. As for getopts builtin, please check which version of bash you developed the script against as compared to which version you're running it against:
$ bash --version
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
That's my bash version. If your production environment has a really old version of bash, you'll need to use bourne shell scripting, instead of bash scripting (Bourne Again SHell) to ensure portable scripts.
Edit: Plagarizing from Olivier Dulac's answer, please take a look at the POSIX page on shell command language for portable Bourne shell scripting. Do take a look at the POSIX standard page for ls and grep for portable options.
Another Edit: See the page on AIX Toolbox for Linux for GNU variants of the standard utilities, which are installed into /usr/linux/bin
Yet another Edit: According to pedz, this link shows better (100% compatible) replacements for the AIX Toolbox
stick to standards, as much as possible...
write sh-compatible scripts, if you need to use them on various systems.
Stick to ancient options that are widely suppotred (-h option for ls, and other gnu introduced niceties, are nice to have, but NOT portable enough)

How to overcome an incompatibility between the ksh on Linux vs. that installed on AIX/Solaris/HPUX?

I am involved in the process of porting a system containing several hundreds of ksh scripts from AIX, Solaris and HPUX to Linux. I have come across the following difference in the way ksh behaves on the two systems:
#!/bin/ksh
flag=false
echo "a\nb" | while read x
do
flag=true
done
echo "flag = ${flag}"
exit 0
On AIX, Solaris and HPUX the output is "flag = true" on Linux the output is "flag = false".
My questions are:
Is there an environment variable that I can set to get Linux's ksh to behave like the
other Os's'? Failing that:
Is there an option on Linux's ksh to get the required behavior? Failing that:
Is there a ksh implementation available for Linux with the desired behavior?
Other notes:
On AIX, Solaris and HPUX ksh is a variant of ksh88.
On Linux, ksh is the public domain ksh (pdksh)
On AIX, Solaris and HPUX dtksh and ksh93 (where I have them installed) are consistent with ksh
The Windows NT systems I have access to: Cygwin and MKS NT, are consistent with Linux.
On AIX, Solaris and Linux, bash is consistent, giving the incorrect (from my perspective) result of "flag = false".
The following table summarizes the systems the problem:
uname -s uname -r which ksh ksh version flag =
======== ======== ========= =========== ======
Linux 2.6.9-55.0.0.0.2.ELsmp /bin/ksh PD KSH v5.2.14 99/07/13.2 false
AIX 3 /bin/ksh Version M-11/16/88f true // AIX 5.3
/bin/ksh93 Version M-12/28/93e true
SunOS 5.8, 5.9 and 5.10 /bin/ksh Version M-11/16/88i true
/usr/dt/bin/dtksh Version M-12/28/93d true
HP-UX B.11.11 and B.11.23 /bin/ksh Version 11/16/88 true
/usr/dt/bin/dtksh Version M-12/28/93d true
CYGWIN_NT-5.1 1.5.25(0.156/4/2) /bin/ksh PD KSH v5.2.14 99/07/13.2 false
Windows_NT 5 .../mksnt/ksh.exe Version 8.7.0 build 1859... false // MKS
Update
After some advice from people in my company we decided to make the following modification to the code. This gives us the same result whether using the "real" ksh's (ksh88, ksh93) or any of the ksh clones (pdksh, MSK ksh). This also works correctly with bash.
#!/bin/ksh
echo "a\nb" > junk
flag=false
while read x
do
flag=true
done < junk
echo "flag = ${flag}"
exit 0
Thanks to jj33 for the previously accepted answer.
Instead of using pdksh on linux, use the "real" ksh from kornshell.org. pdksh is a blind re-implementation of ksh. kornshell.org is the original korn shell dating back 25 years or so (the one written by David Korn). AIX and Solaris use versions of the original ksh, so the kornshell.org version is usually feature- and bug- complete. Having cut my teeth with SunOS/Solaris, installing kornshell.org ksh is usually one of the first things I do on a new Linux box...
After some advice from people in my company we decided to make the following modification to the code. This gives us the same result whether using the "real" ksh's (ksh88, ksh93) or any of the ksh clones (pdksh, MSK ksh). This also works correctly with bash.
#!/bin/ksh
echo "a\nb" > junk
flag=false
while read x
do
flag=true
done < junk
echo "flag = ${flag}"
exit 0
Thanks to jj33 for the previous accepted answer.
I installed 'ksh' and 'pdksh' on my local Ubuntu Hardy system.
ii ksh 93s+20071105-1 The real, AT&T version of the Korn shell
ii pdksh 5.2.14-21ubunt A public domain version of the Korn shell
ksh has the "correct" behavior that you're expecting while pdksh does not. You might check your local Linux distribution's software repository for a "real" ksh, instead of using pdksh. The "Real Unix" OS's are going to install the AT&T version of Korn shell, rather than pdksh, by default, what with them being based off AT&T Unix (System V) :-).
Do you have to stay within ksh?
Even if you use the same ksh you'll still call all kinds of external commands (grep, ps, cat, etc...) part of them will have different parameters and different output from system to system. Either you'll have to take in account those differences or use the GNU version of each one of them to make things the same.
The Perl programming language originally was designed exactly to overcome this problem.
It includes all the features a unix shell programmer would want from he shell program but
it is the same on every Unix system. You might not have the latest version on all those
systems, but if you need to install something, maybe it is better to install perl.
The reason for the differences is whether the inside block is executed in the original shell context or in a subshell. You may be able to control this with the () and {} grouping commands. Using a temporary file, as you do in your update, will work most of the time but will run into problems if the script is run twice rapidly, or if it executes without clearing the file, etc.
#!/bin/ksh
flag=false
echo "a\nb" | { while read x
do
flag=true
done }
echo "flag = ${flag}"
exit 0
That may help with the problem you were getting on the Linux ksh. If you use parentheses instead of braces, you'll get the Linux behavior on the other ksh implementations.
Here is the another solution for echo "\n" issue
Steps:
Find ksh package name
$ rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE}(%{ARCH})\n" | grep "ksh"
ksh-20100621-19.el6_4.3(x86_64)
uninstall ksh
$ sudo yum remove ksh-20100621-19.el6_4.3.x86_64
down load pdksh-5.2.14-37.el5_8.1.x86_64.rpm (Please check OS for 32-bit or 64-bit and choose correct pkg)
Install pdksh-5.2.14-37.el5_8.1.x86_64.rpm
$ sudo yum -y install /SCRIPT_PATH/pdksh-5.2.14-37.el5_8.1.x86_64.rpm
Output before PDKSH install
$ ora_db_start_stop.sh
\n==============
Usage: START
==============\n\n
./ora_db_start_stop.sh START ALL \n
OR \n
./ora_db_start_stop.sh START ONE_OR_MORE \n
\n==============
Usage: STOP
==============\n\n
./ora_db_start_stop.sh STOP ALL \n
OR \n
./ora_db_start_stop.sh STOP ONE_OR_MORE \n\n
After PDKSH install
==============
Usage: START
./ora_db_start_stop.sh START ALL
OR
./ora_db_start_stop.sh START ONE_OR_MORE
==============
Usage: STOP
./ora_db_start_stop.sh STOP ALL
OR
./ora_db_start_stop.sh STOP ONE_OR_MORE
I don't know of any particular option to force ksh to be compatible with a particular older version. That said, perhaps you could install a very old version of ksh on your linux box, and have it behave in a compatible manner?
It might be easier to install a more modern version of amy shell on the AIX/HP-UX boxes, and just migrate your scripts to use sh. I know there are versions of bash available for all platforms.
Your script gives the correct (true) output when zsh is used with the emulate -L ksh option. If all else fails you may wish to try using zsh on Linux.

Resources