I have a directory with the following contents:
bin/busybox
lib/ld-linux.so.2
lib/libc.so.6
and when I invoke:
chroot . bin/busybox sh
it fails with the following:
/bin/busybox: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory
When I move lib/libc.so.6 to usr/lib, it works fine.
Why is libc required to be in /usr/lib? When I invoke:
objcdump -p bin/busybox | grep NEEDED
I get:
NEEDED libc.so.6
So I thought, as only the soname of the library is used without slashes etc. the loaded will be able to find it in the standard folders, which is /lib and /usr/lib. Apparently, this is not the case.
To make matters even more confusing, ld-linux.so.2 seems to have to be in /lib because when it is moved to /usr/lib, chroot fails with:
chroot: failed to run command '/bin/busybox': No such file or directory
which I learned is actually an error that the loader cannot be found, not the busybox binary.
Is the issue with libc.so.2 distro specific? If this is important, I'm using Arch Linux.
The location of the loader (typically something like /lib/ld-linux.so) is hard-coded in the binary. There's no search process for this component — if it cannot be found, the binary won't run at all.
(The exact path depends on what libc and architecture you're using. It's at /lib64/ld-linux-x86-64.so.2 for glibc on x86_64, for instance.)
The locations that will be searched for dynamic libraries are configurable in /etc/ld.so.conf. If you don't have that file in the chroot, though, some of the standard paths may not be configured!
Related
I am currently working on a userland ELF File loader in C. LD_LIBRARY_PATH does not seem to be an option for me, as it does not seem to be set by default on my system(x86_64 openSUSE). What is the best way to get all of the directories where libraries are stored ?
/usr/lib64 and /lib64 for 64-bit binaries or /usr/lib and /lib for 32-bit binaries, than paths taken from /etc/ld.so.conf and included configs
From man ldconfig
ldconfig creates the necessary links and cache to the most recent shared libraries found in the directories specified on the command line, in the file /etc/ld.so.conf, and in the trusted directories, /lib and /usr/lib (on some 64-bit architectures such as x86-64, /lib and /usr/lib are the trusted directories for 32-bit libraries, while /lib64 and /usr/lib64 are used for 64-bit libraries).
The cache is used by the run-time linker, ld.so or ld-linux.so.
...
/etc/ld.so.conf File containing a list of directories, one per line, in which to search for libraries.
Note that this info is for openSUSE, other distros may use different paths.
LD_LIBRARY_PATH is the standard environment variable used for users to add and load their own libraries when they are unable or have no access to the system directories to install shared libraries.
There's a file which is normally read by ldconfig at boot time (it reads /etc/ld.so.conf to create a binary DBMsomewhat file /etc/ld.so.cache, with a hash table to quick access the paths to use when loading library shared objects, and that is used by the dynamic loader (there's only one such thing, as a kernel tool, so it is not dependant on which distribution you run, but just on the kernel version you use ---it has changed somewhat, but not as much as the kernel does---)
To know which sonames (soname is the common name used by a shared object to refer to an interface, which is what is required to warranty that a shared object will be compatible with the library) are being used by the dynamic loader, just run
ldconfig -p
and you'll get all the sonames registered, and the path to the library actually loaded for that soname.
If you want to know which libraries will be loaded by some specific executable by the dynamic loader, just execute this:
ldd your_executable
and It will print the sonames that executable needs and where in the system they are located.
What ldconfig(8) does, is to search al the directories included in the file /etc/ld.so.conf for shared object files, and look all the ones whose name matches the soname stored in the file, and include a reference to the file named for the soname found. Once the table is completed, the file /etc/ld.so.cache is created and used by /lib64/ld-linux-x86-64.so.2 which is the shared module in charge of user mode loading the rest of shared libraries used by a program.
There's no problem in having a local $HOME/lib directory to store your locally developed shared libraries, but as that directory will not be normally included in /etc/ld.so.conf, you'll need to create LD_LIBRARY_PATH=${HOME}/lib and be careful to export it, and never try to use it as root user, as for root user that env variable is disabled.
EDIT 1
By the way, if you need to load on demand a shared library (this is possibly what you need, probably), read about dlopen(3) and friend functions, as that is the method used by most programs to dynamically load modules you haven't heard about before compilation of main program. You'll need to load the module, look for the symbols you need (dlsym(3) or dlfunc(3)) store the references given by the module, and finally call them.
My Machine is slackware linux 64 kernel 3.0.4.
For this machine I don't have root access and the administrator is not available.
I am trying to run a program that requires the library file libc version 2.14 and the one installed in /lib64 is libc-2.13.
I have an identical machine where I have root access. I tried copying the libc-2.14 file from this machine to the first one then place it into a $HOME/lib64 folder and adding this folder to LD_LIBRARY_PATH, then make a new symbolic link libc.so.6 to point to the libc-2.14 file but the program keeps reading the libc.so.6 file in the /lib64 which points to libc-2.13. I can't modify anything in the /lib64 because I am not root.
Is there anyway to get around this?
Thanks in advance
You need to copy other files from glibc, too. You will need the program interpreter /lib64/ld-linux-x86-64.so.2, and perhaps also libdl.so.2, libm.so.6, libpthread.so.0 and more of these helper libraries.
Once you have these, you can try to launch arbitrary programs with the other glibc using an explicit dynamic linker invocation. Assuming that you have copied the files into the current directory, you can try this:
./ld-linux-x86-64.so.2 --library-path . --inhibit-cache /bin/bash
Note that this only applies to the directly launched binaries (bash in the example). Child processes (commands launched from the shell) will again use the system glibc.
There could still be problems if you did not copy all the required glibc libraries, or if there have been incompatible changes in the locale format, so that the new glibc cannot use the system locales.
Edit: I resolved this issue, the solution is below.
I am building a code in a shared computing cluster dedicated for scientific computing, thus I can only control files in my home folder. Although I am using fftw as an example, I would like to understand the specific reason, why my attempt to setup LD_LIBRARY_PATH does not work.
I build the fftw and fftw_mpi libraries in my home folder like this
./configure --prefix=$HOME/install/fftw --enable-mpi --enable-shared
make install
It builds fine, but in install/fftw/lib, I find that the freshly built libfftw3_mpi.so links to wrong version of fftw library.
$ ldd libfftw3_mpi.so |grep fftw
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007f7df0979000)
If I now try to set the LD_LIBRARY_PATH correctly pointing to this directory, it still prefers the wrong library:
$ export LD_LIBRARY_PATH=$HOME/install/fftw/lib
$ ldd libfftw3_mpi.so |grep fftw
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007f32b4794000)
Only if I explicitly use LD_PRELOAD, I can override this behavior. I don't think LD_PRELOAD is a proper solution though.
$ export LD_PRELOAD=$HOME/install/fftw/lib/libfftw3.so.3
$ ldd libfftw3_mpi.so |grep fftw
$HOME/install/fftw/lib/libfftw3.so.3 (0x00007f5ca3d14000)
Here is what I would have expecting, a small test done in Ubuntu desktop, where I installed fftw to /usr/lib first, and then override this search path with LD_LIBRARY_PATH.
$ export LD_LIBRARY_PATH=
$ ldd q0test_mpi |grep fftw3
libfftw3.so.3 => /usr/lib/x86_64-linux-gnu/libfftw3.so.3
$ export LD_LIBRARY_PATH=$HOME/install/fftw-3.3.4/lib
$ ldd q0test_mpi |grep fftw3
libfftw3.so.3 => $HOME/install/fftw-3.3.4/lib/libfftw3.so.3
In short: Why is libfft3_mpi library still finding the wrong dynamic fftw3 library? Where is this searchpath hard coded in a such way that it is prioritized over LD_LIBARY_PATH? Why is this is not the case in another computer?
I am using intel compilers 13.1.2, mkl 11.0.4.183 and openmpi 1.6.2 if this matters.
Edit: Thanks for all the answers. With help of those, we were able to isolate the problem to RPATH, and from there, the cluster support was able to figure out the problem. I accepted the first answer, but both answers were good.
The reason, why this was so hard to figure out, is that we did not know that the compilers were actually wrapper scripts, adding things to compiler command line. Here a part of a reply from the support:
[The] compilation goes through our compiler wrapper. We do RPATH-ing
by default as it helps most users in correctly running their jobs
without loading LD-LIBRARY_PATH etc. However we exclude certain
library paths from default RPATH which includes /lib, /lib64 /proj
/home etc. Earlier the /usr/lib64 was not excluded by mistake
(mostly). Now we have added that path in the exclusion list.
From http://man7.org/linux/man-pages/man8/ld.so.8.html
When resolving shared object dependencies, the dynamic linker first
inspects each dependency string to see if it contains a slash (this
can occur if a shared object pathname containing slashes was
specified at link time). If a slash is found, then the dependency
string is interpreted as a (relative or absolute) pathname, and the
shared object is loaded using that pathname.
If a shared object dependency does not contain a slash, then it is
searched for in the following order:
o (ELF only) Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH
attribute does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH. Except if the
executable is a set-user-ID/set-group-ID binary, in which case it
is ignored.
o (ELF only) Using the directories specified in the DT_RUNPATH
dynamic section attribute of the binary if present.
o From the cache file /etc/ld.so.cache, which contains a compiled
list of candidate shared objects previously found in the augmented
library path. If, however, the binary was linked with the -z
nodeflib linker option, shared objects in the default paths are
skipped. Shared objects installed in hardware capability
directories (see below) are preferred to other shared objects.
o In the default path /lib, and then /usr/lib. (On some 64-bit
archiectures, the default paths for 64-bit shared objects are
/lib64, and then /usr/lib64.) If the binary was linked with the
-z nodeflib linker option, this step is skipped.
with readelf readelf -d libfftw3_mpi.so you can check if your lib contains such a attribute in the dynamic section.
with export LD_DEBUG=libs you can debug the search path used to find your libs
with chrpath -r<new_path> <executable> the rpath can be changed
I see two possible reasons for this.
First, libfftw3_mpi.so may be linked with /usr/lib64/ as RPATH. In that case, providing LD_LIBRARY_PATH will have no effect. To check if it is your case, run readelf -d libfftw3_mpi.so | grep RPATH and see if it has /usr/lib64/ as a library path. If it does, use chrpath utility to change or remove it.
Alternatively, you may be running a system that does not support LD_LIBRARY_PATH at all (like HP-UX).
On my UNIX machine I observed that the binaries are looking for the dependent shared libraries in '/lib' directory by default even though the '/lib' directory is not configured in 'PATH' and 'LD_LIBRARY_PATH' variables.
I the below see that the library 'libssl.so.4' is found from the '/lib' directory.
bash-3.00$ ldd openssl
/lib/libcwait.so (0x00f86000)
libssl.so.4 => /lib/libssl.so.4 (0x00408000)
My 'PATH' and 'LD_LIBRARY_PATH' are below:
bash-3.00$ echo $LD_LIBRARY_PATH
:/opt/oracle/product/11.2.0/client32/lib:
bash-3.00$ echo $PATH
/opt/pure/releases/purify.hp.2003a.06.15.FixPack.0214/cache/opt/star-ncf-prod/ep_patch/usr/lib:/usr/ccs/bin:/usr/bin:/usr/ucb:/etc:/bin:.:/opt/ccm71/bin:/opt/oracle/product/11.2.0/client32/bin:/opt/tools/bin:/usr/local/bin
Please let mw know if the binaries refer to '/lib' directory by default ?
Read ld.so(8), ldd(1) and dlopen(3) man pages and Drepper's paper: How To Write Shared Libraries
You'll see that "If a library dependency does not contain a slash, then it is searched
for" at last in /lib then /usr/lib
These two directories are builtin, i.e. wired in the code of the dynamic linker, e.g. of /lib64/ld-linux-x86-64.so.2 on my Debian system (I guess -but I am not sure- that on my Debian /lib64/ is also wired in).
In order for an executable to find the required libraries to link with during run time, one must configure the system so that the libraries can be found. Methods available: (Do at least one of the following)
Add library directories to be included during dynamic linking to the file /etc/ld.so.conf
Sample: /etc/ld.so.conf
/usr/X11R6/lib
/usr/lib
...
/usr/lib/sane
/usr/lib/mysql
/opt/lib
Add the library path to this file and then execute the command (as root) ldconfig to configure the linker run-time bindings.
You can use the "-f file-name" flag to reference another configuration file if you are developing for different environments.
See man page for command ldconfig
OR
Add specified directory to library cache: (as root)
ldconfig -n /opt/lib
Where /opt/lib is the directory containing your library libctest.so
(When developing and just adding your current directory: ldconfig -n . Link with -L.)
This will NOT permanently configure the system to include this directory. The information will be lost upon system reboot.
OR
Specify the environment variable LD_LIBRARY_PATH to point to the directory paths containing the shared object library. This will specify to the run time loader that the library paths will be used during execution to resolve dependencies.
(Linux/Solaris: LD_LIBRARY_PATH, SGI: LD_LIBRARYN32_PATH, AIX: LIBPATH, Mac OS X: DYLD_LIBRARY_PATH, HP-UX: SHLIB_PATH)
Example (bash shell): export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH or add to your ~/.bashrc file:
`...
if [ -d /opt/lib ];
then
LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
fi
...
export LD_LIBRARY_PATH`
This instructs the run time loader to look in the path described by the environment variable LD_LIBRARY_PATH, to resolve shared libraries. This will include the path /opt/lib.
Library paths used should conform to the "Linux Standard Base" directory structure.
I'm using eclipse cdt to compile and run C++ application.
My_main_program needs specifically libjpeg.so.62.
My Ubuntu system previously have libjpeg.so.9 at /usr/local/lib/. I happened to compiled and run using libjpeg.so.9 before run-time compatibility errors was raised.
Then I deleted all libjpeg.* and installed libjpeg.la, libjpeg.so, libjpeg.so.62 and libjpeg.so.62.0.0 from source. Then I run ldconfig.
I can build the project. The problem is the dynamic linker keeps searching for libjpeg.so.9 and throwing
'error while loading shared libraries: libjpeg.so.9: cannot open shared object file: No such file or directory'
at run-time.
This problem is killing me.
I have checked that the symlink of libjpeg.so is correct.
Please help!
I can build the project. The problem is the dynamic linker keeps searching for libjpeg.so.9 and throwing
'error while loading shared libraries: libjpeg.so.9: ... No such file ...
You need to understand a couple of things:
A shared library may have SONAME dynamic tag (visible with readelf -d foo.so | grep SONAME).
If an executable is linked against such a library, the SONAME is recorded as a NEEDED dynamic tag (in the executable), regardless of what the library itself is called. That is, you can name the library foo.so, foo.so.1234, or anything else. IF the library has SONAME of libbar.so.7, then the executable will require libbar.so.7, no matter what [1].
On to your problem. Your executable fails to load libjpeg.so.9, therefor we conclude that it is being linked (at build time) with a shared library which has SONAME: libjpeg.so.9.
I deleted all libjpeg.* and installed libjpeg.so.62
You must not have deleted the libjpeg.so that is used at executable build time (which is somewhere other than /usr/local/lib). That library still has SONAME: libjpeg.so.9, and is causing you grief.
You can find out which libraries are being used at link time by passing -Wl,-t flag on the link line.
[1] Not strictly true: if the executable doesn't need any symbols from foo.so, and if --as-needed linker option is in effect, then NEEDED: libbar.so.7 will not be recorded after all.
Update:
I have also check ldd executable and it returns libjpeg.so.62
This means that the executable that you run ldd on is correct, but the executable that actually run is not, and they must be different executables.
Update 2:
You're right. ldd executable shows both libjpeg.so.62 and libjpeg.so.9 are included
Actually, no, I wasn't. But I will be right this time.
What's happening is that your executable correctly records NEEDED: libjpeg.so.62 (you can verify this with the following command: readelf -d /path/to/exe | grep 'NEEDED.*libjpeg').
But you also have some other shared library (one of the ones listed in ldd output), which has not been rebuilt, and still has a dependency on libjpeg.so.9.
You can find that library by running readelf -d /path/to/libXXX.so | grep 'NEEDED.*libjpeg\.so\.9' on all libraries listed in ldd output.
Once you find it, you'll have to rebuild it so it also depends on libjpeg.so.62.