I have written a bash script on linux, and it works well, as a part of migration I moved [rather added] an aix 7.2 node to my cluster. When I tried running bash scripts on aix it failed with multiple errors on different gnu bash commands.
[ps: I have installed gnu bash on this aix node, IBM calls it a toolbox made for aix, which contains a collection of open source and GNU software built for AIX IBM Systems]
For example :
- grep -oP isn't supported
- ls -h doesn't work
- getopts fails to get parameter passed and $# as well.
I am not sure if I am doing it right with just installing the gnu bash on aix. Have anyone had any experience porting bash scripts over to ssh?
Are there any pointer community can suggest to get bash script work on aix?
The issue is that these commands are not part of bash. What you need is the GNU versions of all these utilities, that is grep and ls. As for getopts builtin, please check which version of bash you developed the script against as compared to which version you're running it against:
$ bash --version
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
That's my bash version. If your production environment has a really old version of bash, you'll need to use bourne shell scripting, instead of bash scripting (Bourne Again SHell) to ensure portable scripts.
Edit: Plagarizing from Olivier Dulac's answer, please take a look at the POSIX page on shell command language for portable Bourne shell scripting. Do take a look at the POSIX standard page for ls and grep for portable options.
Another Edit: See the page on AIX Toolbox for Linux for GNU variants of the standard utilities, which are installed into /usr/linux/bin
Yet another Edit: According to pedz, this link shows better (100% compatible) replacements for the AIX Toolbox
stick to standards, as much as possible...
write sh-compatible scripts, if you need to use them on various systems.
Stick to ancient options that are widely suppotred (-h option for ls, and other gnu introduced niceties, are nice to have, but NOT portable enough)
Related
I have a requirement where all the scripts on solaris needs to be copy pasted on linux. I need to do this keeping in mind that the scripts should be linux compatible. I have no idea how I can check the compatibility of these already existing scripts. Note that I do not have the linux environment ready and we are doing a data collection of all such types, please help.
Thanks
Abhinav
You will have no guarantee, there are too much potential differences:
* scripts calling Solaris-specific tools.
check (grep) your scripts for calls to /bin/* and /usr/bin
* scripts calling utilities with different options on Linux
Most times Linux utilities (grep, sed, awk, date) will have more options. Try to install and use the GNU utilities or have some hope that the basic options on Solaris are supported on Linux as well.
* ksh or bash
When you can, try to install and use bash on Solaris.
Pay attention to while-loops (see https://stackoverflow.com/a/5061255/3220113).
Update:
My own experience:
I have migrated a lot of scripts from Solaris to AIX, both ksh. Problems I had were mainly:
* if [ -z $var ] fails on AIX when $var is empty (use if [ -z
"$var"]).
* Not having the sed -i option I edited files on place
using vi file <<#
This did not work when called from a remote script, it was missing a correct $TERM
* crontab scripts did not look at /etc/environment (AIX specific)
* The DST in the home-brewn timezone for date on AIX wasn't working well.
* Different directories (database distribition!)
* Other Java classes and classpaths to be set before launching a java script
* Connection to a mailserver worked different.
Recently I am only working with bash. What a relief! find that supports mtime in minutes, date that can jump days, grep has beautiful options and even awk has found his way to my toolbox. For bash just remember that you never pipe into a while-loop.
Of course you would like an estimate of the work you will have. When you need to port scripts written by yourself during the last 3 years (so you understand what they do and the amount is about 2 years work - you must have been doing other things as well), my personal guess is about 4 months programming and testing.
I'm working on a Bash library and want to ensure I'm supporting as many environments as possible - including old installations of Bash. My development environment is Bash 4.3, but some of my users may well be running much older versions and presently I have no way to confirm or deny that my library will work for them. In particular I'd like to be compatible with OSX (which still ships with Bash 3.2, AFAIK).
I know Bash can run in POSIX-compliant mode; is there a similar setting to disable modern functionality? Or a way to run Bash in some sort of compatibility mode? I'm looking for any technique short of actually finding and booting up old operating systems and testing my library there.
Update
For example, I've avoided using associative arrays since they were introduced in Bash 4, but it's hard to be sure without testing that I'm not accidentally using some other Bash 4+ feature.
Finally coming back to this question, it's pretty easy to just compile (without installing) the bash version(s) you're interested in. Here's how I'm testing Bash 3.2.57:
$ mkdir ~/bash
$ cd ~/bash
$ wget http://ftp.gnu.org/gnu/bash/bash-3.2.57.tar.gz
$ tar xvzf bash-3.2.57.tar.gz
$ cd bash-3.2.57
$ ./configure
$ make
# if `make` fails due to yacc, run `sudo apt-get install byacc`
# No need to run `make install`
$ ./bash -version
GNU bash, version 3.2.57(1)-release (armv7l-unknown-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
Now you have a bash 3.2.57 binary you can run, without actually "installing" it or modifying your normal environment.
To run a shell script against this version:
$ ./bash your_script.sh
To enter a clean interactive prompt:
$ env -i PATH="$PWD:$PATH" ./bash --noprofile --norc
bash-3.2$ bash -version
GNU bash, version 3.2.57(1)-release (armv7l-unknown-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
bash-3.2$
Using env -i rather than just calling ./bash directly leaves you with a mostly-empty environment (run env from inside the shell to see what's still set). Updating the PATH allows calls to bash (e.g. bash -version) to invoke the local bash shell, not the system-wide installation (but note this pulls in your whole PATH). Adding --noprofile --norc avoids loading your .bashrc and associated scripts.
If you don't want to pick up any PATH modifications, just execute export PATH="$PWD:$PATH" once inside the subshell instead of as part of the env command.
I have a Docker image (repo) using these installation steps, if that's helpful for folks to reference. I wouldn't necessarily suggest using this image directly, but you're welcome to copy from the Dockerfile/install script. MIT licensed.
Although it's nice to know that it's possible to compile arbitrary versions of bash locally (as discussed in my other answer), these days there's a much simpler option - the official Docker bash images.
To test a script against multiple bash versions is often as simple as:
for v in 3 4 5; do # or whatever versions you're interested in
docker run -v "$PWD:/mnt" "bash:$v" \
bash /mnt/your_script.sh
done
Have a look at shenv: https://github.com/shenv/shenv. Just like rbenv, pyenv, goenv and others, but for shells, it lets you install different versions of Bash among others (zsh, fish, yash, etc.).
(Disclaimer: I'm the one who forked pyenv into shenv!)
You can use Bash own facilities to emulate older version of Bash. Look up "compat" options on shopt.
Mind you, while it does change behavior as described under each compatNN entry in the man page, it does not remove features that are otherwise present on the current version. For example, this doesn't cause any errors:
shopt -s compat31
shopt -s globstar
Even though globstar was only introduced on Bash 4.0.
I have installed Git Bash on my Windows machine. It behaves very differently from the normal command prompt.
Why is it this different? It seems to behave like Linux. Is it just the style of writing commands that is similar to Linux or is Linux included in Git Bash?
What is the relationship between Git Bash and Linux
The only relation between Linux and Bash is that Bash is often used on Linux systems (among others). Git Bash is just a Bash that's set up to be able to find the Git tools. This is generally not necessary on Linux systems (so you wouldn't find anything called Git Bash on a Linux system).
It behaves very differently from the normal command prompt.
It's a different shell (and a different terminal emulator).
It seems to behave like Linux.
Bash does indeed behave the same regardless of whether it runs on Windows or Linux (or OS X, BSD or any other operating system). However that's a property of Bash, not Linux.
is Linux included in Git Bash
No.
git bash is a bash interpreter.
bash is a shell (bash is really an acronym for Bourne Again SHell), and it's a very popular shell on unix-like systems, uncluding Linux but also on other systems like OS-X.
But it doesn't include linux (it's basically a single binary; if you extract the cmd.exe from your w32 system, and give it to a friend, then you haven't given them Windows).
bash is very powerful, but it's real strength comes from the unix-philosophy of having many little tools that are good in doing (only) their job. the shell glues those tools together.
bash behaves very different than cmd because it is an interpreter for a different language. Just like python is different from java.
I am working on some bash scripts that I'd like to work across my Linux and FreeBSD systems.
Since I mostly work in Linux, I am used to starting my bash scripts with
#!/bin/bash
But this doesn't work on FreeBSD since bash lives at /usr/local/bin/bash. So on FreeBSD my scripts need to start with
#!/usr/local/bin/bash
So is there something else I could use that would be portable across both systems? I'd rather not maintain two versions of the scripts.
#!/usr/bin/env bash
should do the trick, provided that bash is on the path somewhere. See here for more details.
Honestly, if you want portability, invoke as /bin/sh and code to POSIX. It's less pretty, but you will run into fewer potential issues if you do.
Use #!/bin/sh on both systems if you want to be portable and avoid bashisms entirely.
I am involved in the process of porting a system containing several hundreds of ksh scripts from AIX, Solaris and HPUX to Linux. I have come across the following difference in the way ksh behaves on the two systems:
#!/bin/ksh
flag=false
echo "a\nb" | while read x
do
flag=true
done
echo "flag = ${flag}"
exit 0
On AIX, Solaris and HPUX the output is "flag = true" on Linux the output is "flag = false".
My questions are:
Is there an environment variable that I can set to get Linux's ksh to behave like the
other Os's'? Failing that:
Is there an option on Linux's ksh to get the required behavior? Failing that:
Is there a ksh implementation available for Linux with the desired behavior?
Other notes:
On AIX, Solaris and HPUX ksh is a variant of ksh88.
On Linux, ksh is the public domain ksh (pdksh)
On AIX, Solaris and HPUX dtksh and ksh93 (where I have them installed) are consistent with ksh
The Windows NT systems I have access to: Cygwin and MKS NT, are consistent with Linux.
On AIX, Solaris and Linux, bash is consistent, giving the incorrect (from my perspective) result of "flag = false".
The following table summarizes the systems the problem:
uname -s uname -r which ksh ksh version flag =
======== ======== ========= =========== ======
Linux 2.6.9-55.0.0.0.2.ELsmp /bin/ksh PD KSH v5.2.14 99/07/13.2 false
AIX 3 /bin/ksh Version M-11/16/88f true // AIX 5.3
/bin/ksh93 Version M-12/28/93e true
SunOS 5.8, 5.9 and 5.10 /bin/ksh Version M-11/16/88i true
/usr/dt/bin/dtksh Version M-12/28/93d true
HP-UX B.11.11 and B.11.23 /bin/ksh Version 11/16/88 true
/usr/dt/bin/dtksh Version M-12/28/93d true
CYGWIN_NT-5.1 1.5.25(0.156/4/2) /bin/ksh PD KSH v5.2.14 99/07/13.2 false
Windows_NT 5 .../mksnt/ksh.exe Version 8.7.0 build 1859... false // MKS
Update
After some advice from people in my company we decided to make the following modification to the code. This gives us the same result whether using the "real" ksh's (ksh88, ksh93) or any of the ksh clones (pdksh, MSK ksh). This also works correctly with bash.
#!/bin/ksh
echo "a\nb" > junk
flag=false
while read x
do
flag=true
done < junk
echo "flag = ${flag}"
exit 0
Thanks to jj33 for the previously accepted answer.
Instead of using pdksh on linux, use the "real" ksh from kornshell.org. pdksh is a blind re-implementation of ksh. kornshell.org is the original korn shell dating back 25 years or so (the one written by David Korn). AIX and Solaris use versions of the original ksh, so the kornshell.org version is usually feature- and bug- complete. Having cut my teeth with SunOS/Solaris, installing kornshell.org ksh is usually one of the first things I do on a new Linux box...
After some advice from people in my company we decided to make the following modification to the code. This gives us the same result whether using the "real" ksh's (ksh88, ksh93) or any of the ksh clones (pdksh, MSK ksh). This also works correctly with bash.
#!/bin/ksh
echo "a\nb" > junk
flag=false
while read x
do
flag=true
done < junk
echo "flag = ${flag}"
exit 0
Thanks to jj33 for the previous accepted answer.
I installed 'ksh' and 'pdksh' on my local Ubuntu Hardy system.
ii ksh 93s+20071105-1 The real, AT&T version of the Korn shell
ii pdksh 5.2.14-21ubunt A public domain version of the Korn shell
ksh has the "correct" behavior that you're expecting while pdksh does not. You might check your local Linux distribution's software repository for a "real" ksh, instead of using pdksh. The "Real Unix" OS's are going to install the AT&T version of Korn shell, rather than pdksh, by default, what with them being based off AT&T Unix (System V) :-).
Do you have to stay within ksh?
Even if you use the same ksh you'll still call all kinds of external commands (grep, ps, cat, etc...) part of them will have different parameters and different output from system to system. Either you'll have to take in account those differences or use the GNU version of each one of them to make things the same.
The Perl programming language originally was designed exactly to overcome this problem.
It includes all the features a unix shell programmer would want from he shell program but
it is the same on every Unix system. You might not have the latest version on all those
systems, but if you need to install something, maybe it is better to install perl.
The reason for the differences is whether the inside block is executed in the original shell context or in a subshell. You may be able to control this with the () and {} grouping commands. Using a temporary file, as you do in your update, will work most of the time but will run into problems if the script is run twice rapidly, or if it executes without clearing the file, etc.
#!/bin/ksh
flag=false
echo "a\nb" | { while read x
do
flag=true
done }
echo "flag = ${flag}"
exit 0
That may help with the problem you were getting on the Linux ksh. If you use parentheses instead of braces, you'll get the Linux behavior on the other ksh implementations.
Here is the another solution for echo "\n" issue
Steps:
Find ksh package name
$ rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE}(%{ARCH})\n" | grep "ksh"
ksh-20100621-19.el6_4.3(x86_64)
uninstall ksh
$ sudo yum remove ksh-20100621-19.el6_4.3.x86_64
down load pdksh-5.2.14-37.el5_8.1.x86_64.rpm (Please check OS for 32-bit or 64-bit and choose correct pkg)
Install pdksh-5.2.14-37.el5_8.1.x86_64.rpm
$ sudo yum -y install /SCRIPT_PATH/pdksh-5.2.14-37.el5_8.1.x86_64.rpm
Output before PDKSH install
$ ora_db_start_stop.sh
\n==============
Usage: START
==============\n\n
./ora_db_start_stop.sh START ALL \n
OR \n
./ora_db_start_stop.sh START ONE_OR_MORE \n
\n==============
Usage: STOP
==============\n\n
./ora_db_start_stop.sh STOP ALL \n
OR \n
./ora_db_start_stop.sh STOP ONE_OR_MORE \n\n
After PDKSH install
==============
Usage: START
./ora_db_start_stop.sh START ALL
OR
./ora_db_start_stop.sh START ONE_OR_MORE
==============
Usage: STOP
./ora_db_start_stop.sh STOP ALL
OR
./ora_db_start_stop.sh STOP ONE_OR_MORE
I don't know of any particular option to force ksh to be compatible with a particular older version. That said, perhaps you could install a very old version of ksh on your linux box, and have it behave in a compatible manner?
It might be easier to install a more modern version of amy shell on the AIX/HP-UX boxes, and just migrate your scripts to use sh. I know there are versions of bash available for all platforms.
Your script gives the correct (true) output when zsh is used with the emulate -L ksh option. If all else fails you may wish to try using zsh on Linux.