LD_LIBRARY_PATH fails in bash script - linux

I have a bash script that runs this line of code:
LD_LIBRARY_PATH=/tools/cluster/6.2/openbabel/2.3.2/lib ./xattr infile.txt outfile.txt
If I were to call this line directly from the shell, it works fine. However if I run it in the bash script I get this error:
update.sh: line 45: LD_LIBRARY_PATH=/tools/cluster/6.2/openbabel/2.3.2/lib: No such file or directory
Why doesn't LD_LIBRARY_PATH work when it's set in a bash script?
Here's more of the code around line 45:
BASE_DIR="/volatile/huanlab/bold/kendal/bioinformatics_database/tmp"
COMP_DIR="$BASE_DIR/compound"
# move to the current directory where xattr.cpp and other files are
cd /users/kharland/software/programs/BioDB-update/dev
# Compile xattr ('make xattr' is the same command I call from the shell
# to compile this program when this program actually works).
make xattr
# loop over each .sdf file in COMP_DIR
for INF in $(ls $COMP_DIR | grep sdf)
do
babel -isdf $COMP_DIR/$INF -ocan $SMILES_DIR/$INF.csv
LD_LIBRARY_PATH=/tools/cluster/6.2/openbabel/2.3.2/lib ./xattr $COMP_DIR/$INF $COMP_DIR/$COMP_FILE
done
The contents before these lines are just comments
edit
In My makefile, I am compiling with these options
LDLIBS=-lm -ldl -lz -lopenbabel
LDFLAGS=-Wl,-rpath,/tools/cluster/6.2/openbabel/2.3.2/lib:/tools/cluster/system/pkg/openbabel/openbabel-2.3.2/build/lib,-L/tools/cluster/6.2/openbabel/2.3.2/lib
and running ldd xattr shows that the libraries are indeed linked, so the program executes as expected when invoked from the shell. The only issue is with the bash script. If I remove the LD_LIBRARY_PATH option from the bash script I get an issue where the shared libraries for openbabel aren't found even though ldd shows that xattr knows where the libs are. That's why I have LD_LIBRARY_PATH added in the bash script, I'm attempting to use it as a workaround
edit
(corrected mistake: swapped 'libraries' with 'my code' below)
(had wrong file system name below)
Something just occurred to me. My source code is in the /users file system. If my libraries are on a different, mounted file system, would bash have trouble finding these documents?

Setting environment variables does work in bash scripts.
Try this:
#!/bin/bash
VAR1=VALUE1 env
...run that script, and you'll see output that includes VAR1 and its value.
Generally speaking, this also works with LD_LIBRARY_PATH:
#!/bin/bash
tempdir=$(mktemp -t -d testdir.XXXXXX)
LD_LIBRARY_PATH=$tempdir env
rm -rf "$tempdir"
If you can generate a minimal reproducer in which this doesn't occur, that would be helpful and appreciated.

Related

Anyway to change the default exec call length?

I have a bash script and will have the first line start with # and followed by the command to execute the script, and it seems the limitation is 80 characters due to the exec call has such limitation, is there anyway to change that ? because sometimes my path will be very long.
Update.
My case is that I use virtualenv to generate a clean python environment. And in this environment, there's one executable file called pip, the shebang line is python executable path and sometimes this path will be very long, e.g.
#!/Users/myname/github/myproject/virtualenv_python3.4/bin/python3.4
If you don't want to modify your path to include the directory in which the executable, you can create a simple wrapper:
#!/bin/bash
/Users/myname/github/myproject/virtualenv_python3.4/bin/python3.4 <(cat <<"EOF"
# Python script goes here
EOF) "$#"

Can't run a script

I tried to create a script in linux, on a Synology server over SSH
so I wrote a file test.sh
#!/bin/bash
echo "this is a test"
I saved the file.
after that I did
chmod 755 test.sh
the I did
./test.sh
then i got this error
-ash "./test.sh" is not found
the file was created in
/root
I don't understand
Your shell (ash?) is trying to execute your script and is getting an ENOENT (no such file or directory) error code back. This can refer to the script itself, but in this case it refers to the interpreter named in the #! line.
That is, /bin/bash does not exist and that's why the script couldn't be started.
Workaround: Install bash or (if you don't need any bash specific features) change the first line to #!/bin/sh.
This is one of the quirks with hash bang programs. If the interpreter is not found (i.e. the program interpreting the script), you don't get a completely useful error like /bin/bash: no such file, but a completely useless and misleading test.sh: not found.
If this isn't in the Unix Hater's Handbook, it should be. :-)
You can either use #!/bin/sh or #!/path/to/bash or #!/usr/bin/env bash (which searches PATH for bash).

Program runs fine from the terminal, but does not work from a shell script

I am on Linux Mint.
I have program called samtools, and it is stored in a folder on my desktop. I've added path to the executable into $PATH variable. In other words my local ~/.bashrc file has a line:
export PATH="~/Desktop/samtools/samtools-1.1:$PATH"
Executable file named samtools is in this folder.
So when I try to launch it from a command line like a simple command, by just typing "samtools" it works. It also works when I type direct path to the executable in the command line.
However when I try to launch it from a shell script, it does not launch and says either No such file or directory or not found.
Actually, I am trying to use another software which uses some shell scripts to preprocess some data. The error I am getting looks like this:
Indexing...
./RD_capture//process_one_capture.sh: 17: ./RD_capture//process_one_capture.sh: samtools: not found
Sorting...
./RD_capture//process_one_capture.sh: 20: ./RD_capture//process_one_capture.sh: samtools: not found
Piling up...
./RD_capture//process_one_capture.sh: 23: ./RD_capture//process_one_capture.sh: samtools: not found
Shell code looks like this:
echo "Indexing..."
samtools index $INPUTDIR/$sample.bam
#Then we sort them
echo "Sorting..."
samtools sort $INPUTDIR/$sample.bam $TMPDIR/$sample.sorted
#Finally we pile them up
echo "Piling up..."
samtools mpileup $TMPDIR/$sample.sorted.bam | cut -f 1-4 > $OUTPUT/$(basename $sample .bam).pile
Can anyone help me to solve this problem?
It looks like Linux is not able to find samtools.
To solve this use:
Declare this in the beginning (change pwd with the path that leads to executable from pwd)
export set CURRENT_DIR=`pwd`
and while calling samtools use
$CURRENT_DIR/samtools
Note 1: you are responsible to tell the exact path to shell script as shell scripts execute from /usr/bin/sh. try echoing the path on the console with the command, if this does not help, as follows:
echo `pwd`
Note 2: the use of back quote above that is located on left hand side top corner of your keyboard.
Note 3: the export set is used to store global variables in a shell script so that commands that appear after this command can use this variable and update it when ever required.

LD_LIBRARY_PATH: Does only work when startup call is also on same line

I can start my program with the following command in the makefile:
LD_LIBRARY_PATH=$(CUDAHOME)/lib64:$(LIBHOME)/boost_1_54_0/stage/lib mpiloader -np 2 ./program
When I switch to a two-line version of the same command (as anybody seems to do it)
export LD_LIBRARY_PATH=$(CUDAHOME)/lib64:$(LIBHOME)/boost_1_54_0/stage/lib
mpiloader -np 2 ./program
the libraries in boost are no longer found:
./program: error while loading shared libraries: libboost_chrono.so.1.54.0: cannot open shared object file: No such file or directory
As far as I understand, the one and the two line version of the startup code should do exactly the same. What's then wrong here?
By default every recipe line in a Makefile invokes a new shell instance, so your export is only valid up until when the first line finishes executing.
With GNU make you can change this behaviour by defining the .ONESHELL special target:
.ONESHELL:
all:
#export foo=bar
#echo $$foo
outputs
bar

Bash: Invalidated commands

I've incurred a worrisome issue with my bash shell. I was editing my bash_profile and accidentally exported an incomplete command (export PATH=/usr/local/bin). After I had reloaded my terminal, nearly all of my bash commands fail to work properly. When I try to run any one of them, the errors state: command not found.
How do I fix this? Is there an alternate way to open or find my bash_profile?
I would appreciate any immediate input I can get on this issue. Thank you in advance.
You can execute commands if you can give the directory name. Almost all the basic Unix commands are under the /bin or /usr/bin directory. For example, /bin/mv.
Fortunately, builtin commands are still recognizable.
Move your .bash_profile and .bashrc file out of the way for now, and see what the system default is.
You can manually edit your PATH on the command line to:
$ PATH="/bin:/usr/bin"
$ cd
$ mv .bash_profile .bash_profile.bak
$ mv .bashrc .bashrc.bak
$ mv .profile .profile.bak
$ mv .bash_login .bash_login.bak
NOTE: Some of these mv command may fail simply because that particular file may not exist.
which will give you access to most of the basic Unix commands. Or you can specify the commands with their full directory names:
$ PATH="/bin:/usr/bin"
$ cd
$ /bin/mv .bash_profile .bash_profile.bak
$ /bin/mv .bashrc .bashrc.bak
$ /bin/mv .profile .profile.bak
$ /bin/mv .bash_login .bash_login.bak
Now, log in again and see what your default $PATH is set to. This is set by the /etc/profile. You might find that's just fine, and remove setting PATH in your startup script.
The standard for PATH is something like this:
/usr/share/bin or /usr/local/bin - These contain non-standard Unix/Linux commands. For example, if you install Maven on your system, the mvn command will usually be located in one of these directories (maybe as a symbolic link). This directory is a place where commands not found in the /bin and /usr/bin directory are stored. This directory is first, so you can replace the version which came with your system with more recent versions. For example, I might have VIM 6.4 installed, but I want to use version 7.3 instead.
/bin:/usr/bin - The standard directories where 99% of the Unix commands live.
$HOME/bin - These are executables you wrote -- either scripts or binaries. This is at the end of the PATH list because it makes sure that you don't accidentally execute the wrong version of the command. Imagine if some joker wrote a shell script called cp that executed /bin/rm instead and placed it in your $HOME/bin directory.
Other directories you'll see may include /sbin/ and /usr/sbin which are administrator commands (ping and ifconfig are sometimes in one of these directories.) /opt/bin/X11 (or wherever the X11 binaries are stored). Sometimes other commands will futz around with your PATH, for example Perlbrew.
#fedorqui's comment provides a quick fix.
The OP could also have used the following to quickly get to a shell with default values for $PATH:
To create a bash shell with a pristine default environment:
without running profile/initialization scripts
without inheriting any environment variables from the current shell
run:
/usr/bin/env -i bash --norc
Note:
Due to use of env's -i option, many environment variables that are normally set will NOT be set in the resulting shell , such as USER, HOME and LANG.
Similarly, the $PATH value you'll get is presumably one hard-coded into bash itself, but it should provide access to at least the standard utilities.
--norc suppresses loading of ~/.bashrc, which normally happens by default for interactive non-login bash shells (bash also supports the --noprofile option to suppress loading of /etc/profile and ~/.bash_profile, but it doesn't apply here, since the shell created is a non-login shell).
If env is in the current shell's $PATH, env -i bash --norc will do.
env is in /usr/bin/ on at least Linux and on FreeBSD/OSX, probably also on other platforms.

Resources