According on the ldd linux command on /bin/chgrp the only necessary library for its execution is libc.so. When chgrp is debugged with gdb I've noticed that other libraries are actually loaded, such as:
/lib/x86_64-linux-gnu/libnss_compat.so.2
/lib/x86_64-linux-gnu/libnsl.so.1
/lib/x86_64-linux-gnu/libnss_nis.so.2
/lib/x86_64-linux-gnu/libnss_files.so.2
Why these libraries do not compare in the ldd output?
Also: they do not compare in the DYNAMIC section of libc.so, how are they located and loaded? (I've checked the DYNAMIC section of libc.so with readelf and these libraries are not there)
The various libnss libraries are loaded dynamically by libc, via dlopen, when you run the program. Which libraries are loaded is determined by the contents of /etc/nsswitch.conf. See the Name Service Switch documentation for more details.
Related
I've edited my CMake script to link libcurl with my C++ application on Linux.
Besides adding libcurl to my target link libraries, I've set the libcurl flag DCURL_STATICLIB.
What is the definitive way of checking this has linked statically, rather than dynamically?
What is the definitive way of checking this has linked statically, rather than dynamically?
Run the ldd program on the generated executable(s). This will give you a list of all the shared libraries that are dynamically linked to it. Verify that libcurl.so is not on the list.
ldd my_executable
It may be useful to engage grep to filter out other shared libs:
ldd my_executable | grep libcurl
, but because that will produce no output when libcurl indeed is not dynamically linked to the executable, the unfiltered form is useful at least for confirming that ldd actually worked.
I have built some shared libraries on Ubuntu Linux 16.0.2 from source.
They are 64-bit libs.
I manually copied them to /usr/local/lib.
I verified that the /usr/local/lib path is indeed in one of the .conf files that ld.so.conf includes.
I then ran: sudo ldconfig to update the cache.
But then when I try to run my executable which tries to dynamically load one of the .so files that I previously copied into /usr/local/lib using dlopen, it fails.
In my code, I have:
dlopen ("foobar.so", RTLD_LAZY);
Can anybody tell me what I am doing wrong?
The dynamic linker normally doesn't access the paths recursively included from /etc/ld.so.conf directly, but it uses a cache.
You can update the cache with
sudo ldconfig
See ldconfig(8) for more details.
for dlopen to work, there's no directory list of places to find the shared object. So doing dlopen("somefile", ...); probably won't work.
You don't need to use any path or to put the shared object (or to comply with naming conventions) to use a shared object via dlopen(3). That's only a requirement of the dynamic linker that loads and links all the shared libraries at launch time: linux-vdso.so.1 (in 64bit)
To test, just put the shared in your local dir and try to open it with it's basename, like you post.
For a system library, there are more requirements, like defining a soname for the library, which is used by the loader to load the library and to construct the cache database index, so if you have no idea of what I'm talking about, you will not be able to use automatic loading procedure. If you want to see if an executable file has all the libraries it needs and where the loader finds them out, just run ldd(1) with the executable as argument, and you'll se the dependencies for automatic loading and how the dynamic linker resolves the paths.
Edit: I resolved this issue, the solution is below.
I am building a code in a shared computing cluster dedicated for scientific computing, thus I can only control files in my home folder. Although I am using fftw as an example, I would like to understand the specific reason, why my attempt to setup LD_LIBRARY_PATH does not work.
I build the fftw and fftw_mpi libraries in my home folder like this
./configure --prefix=$HOME/install/fftw --enable-mpi --enable-shared
make install
It builds fine, but in install/fftw/lib, I find that the freshly built libfftw3_mpi.so links to wrong version of fftw library.
$ ldd libfftw3_mpi.so |grep fftw
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007f7df0979000)
If I now try to set the LD_LIBRARY_PATH correctly pointing to this directory, it still prefers the wrong library:
$ export LD_LIBRARY_PATH=$HOME/install/fftw/lib
$ ldd libfftw3_mpi.so |grep fftw
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007f32b4794000)
Only if I explicitly use LD_PRELOAD, I can override this behavior. I don't think LD_PRELOAD is a proper solution though.
$ export LD_PRELOAD=$HOME/install/fftw/lib/libfftw3.so.3
$ ldd libfftw3_mpi.so |grep fftw
$HOME/install/fftw/lib/libfftw3.so.3 (0x00007f5ca3d14000)
Here is what I would have expecting, a small test done in Ubuntu desktop, where I installed fftw to /usr/lib first, and then override this search path with LD_LIBRARY_PATH.
$ export LD_LIBRARY_PATH=
$ ldd q0test_mpi |grep fftw3
libfftw3.so.3 => /usr/lib/x86_64-linux-gnu/libfftw3.so.3
$ export LD_LIBRARY_PATH=$HOME/install/fftw-3.3.4/lib
$ ldd q0test_mpi |grep fftw3
libfftw3.so.3 => $HOME/install/fftw-3.3.4/lib/libfftw3.so.3
In short: Why is libfft3_mpi library still finding the wrong dynamic fftw3 library? Where is this searchpath hard coded in a such way that it is prioritized over LD_LIBARY_PATH? Why is this is not the case in another computer?
I am using intel compilers 13.1.2, mkl 11.0.4.183 and openmpi 1.6.2 if this matters.
Edit: Thanks for all the answers. With help of those, we were able to isolate the problem to RPATH, and from there, the cluster support was able to figure out the problem. I accepted the first answer, but both answers were good.
The reason, why this was so hard to figure out, is that we did not know that the compilers were actually wrapper scripts, adding things to compiler command line. Here a part of a reply from the support:
[The] compilation goes through our compiler wrapper. We do RPATH-ing
by default as it helps most users in correctly running their jobs
without loading LD-LIBRARY_PATH etc. However we exclude certain
library paths from default RPATH which includes /lib, /lib64 /proj
/home etc. Earlier the /usr/lib64 was not excluded by mistake
(mostly). Now we have added that path in the exclusion list.
From http://man7.org/linux/man-pages/man8/ld.so.8.html
When resolving shared object dependencies, the dynamic linker first
inspects each dependency string to see if it contains a slash (this
can occur if a shared object pathname containing slashes was
specified at link time). If a slash is found, then the dependency
string is interpreted as a (relative or absolute) pathname, and the
shared object is loaded using that pathname.
If a shared object dependency does not contain a slash, then it is
searched for in the following order:
o (ELF only) Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH
attribute does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH. Except if the
executable is a set-user-ID/set-group-ID binary, in which case it
is ignored.
o (ELF only) Using the directories specified in the DT_RUNPATH
dynamic section attribute of the binary if present.
o From the cache file /etc/ld.so.cache, which contains a compiled
list of candidate shared objects previously found in the augmented
library path. If, however, the binary was linked with the -z
nodeflib linker option, shared objects in the default paths are
skipped. Shared objects installed in hardware capability
directories (see below) are preferred to other shared objects.
o In the default path /lib, and then /usr/lib. (On some 64-bit
archiectures, the default paths for 64-bit shared objects are
/lib64, and then /usr/lib64.) If the binary was linked with the
-z nodeflib linker option, this step is skipped.
with readelf readelf -d libfftw3_mpi.so you can check if your lib contains such a attribute in the dynamic section.
with export LD_DEBUG=libs you can debug the search path used to find your libs
with chrpath -r<new_path> <executable> the rpath can be changed
I see two possible reasons for this.
First, libfftw3_mpi.so may be linked with /usr/lib64/ as RPATH. In that case, providing LD_LIBRARY_PATH will have no effect. To check if it is your case, run readelf -d libfftw3_mpi.so | grep RPATH and see if it has /usr/lib64/ as a library path. If it does, use chrpath utility to change or remove it.
Alternatively, you may be running a system that does not support LD_LIBRARY_PATH at all (like HP-UX).
I'm using eclipse cdt to compile and run C++ application.
My_main_program needs specifically libjpeg.so.62.
My Ubuntu system previously have libjpeg.so.9 at /usr/local/lib/. I happened to compiled and run using libjpeg.so.9 before run-time compatibility errors was raised.
Then I deleted all libjpeg.* and installed libjpeg.la, libjpeg.so, libjpeg.so.62 and libjpeg.so.62.0.0 from source. Then I run ldconfig.
I can build the project. The problem is the dynamic linker keeps searching for libjpeg.so.9 and throwing
'error while loading shared libraries: libjpeg.so.9: cannot open shared object file: No such file or directory'
at run-time.
This problem is killing me.
I have checked that the symlink of libjpeg.so is correct.
Please help!
I can build the project. The problem is the dynamic linker keeps searching for libjpeg.so.9 and throwing
'error while loading shared libraries: libjpeg.so.9: ... No such file ...
You need to understand a couple of things:
A shared library may have SONAME dynamic tag (visible with readelf -d foo.so | grep SONAME).
If an executable is linked against such a library, the SONAME is recorded as a NEEDED dynamic tag (in the executable), regardless of what the library itself is called. That is, you can name the library foo.so, foo.so.1234, or anything else. IF the library has SONAME of libbar.so.7, then the executable will require libbar.so.7, no matter what [1].
On to your problem. Your executable fails to load libjpeg.so.9, therefor we conclude that it is being linked (at build time) with a shared library which has SONAME: libjpeg.so.9.
I deleted all libjpeg.* and installed libjpeg.so.62
You must not have deleted the libjpeg.so that is used at executable build time (which is somewhere other than /usr/local/lib). That library still has SONAME: libjpeg.so.9, and is causing you grief.
You can find out which libraries are being used at link time by passing -Wl,-t flag on the link line.
[1] Not strictly true: if the executable doesn't need any symbols from foo.so, and if --as-needed linker option is in effect, then NEEDED: libbar.so.7 will not be recorded after all.
Update:
I have also check ldd executable and it returns libjpeg.so.62
This means that the executable that you run ldd on is correct, but the executable that actually run is not, and they must be different executables.
Update 2:
You're right. ldd executable shows both libjpeg.so.62 and libjpeg.so.9 are included
Actually, no, I wasn't. But I will be right this time.
What's happening is that your executable correctly records NEEDED: libjpeg.so.62 (you can verify this with the following command: readelf -d /path/to/exe | grep 'NEEDED.*libjpeg').
But you also have some other shared library (one of the ones listed in ldd output), which has not been rebuilt, and still has a dependency on libjpeg.so.9.
You can find that library by running readelf -d /path/to/libXXX.so | grep 'NEEDED.*libjpeg\.so\.9' on all libraries listed in ldd output.
Once you find it, you'll have to rebuild it so it also depends on libjpeg.so.62.
When I link executable elf file dynamically it needs libc.so.6 shared library.
When I link executable elf file statically it doesn't need libc.so.6 shared library (it's not surprise).
Does it mean, that to assemble executable file with --static, linker includes libc.so.6 in it?
If not - what file does linker include? where can I search it?
As far as I know, linker includes static libraries in statically assembled file.
If you link as static, the linker will link all the needed object (.o) files from the static libraries (.a). For example, the following command lists the object files which are included in the libc6 library:
ar t /usr/lib/libc.a
(the exact path to libc.a of course differs from system to system)
So answer to your question is no, it will not link the whole libc6 library, but only the needed object files. And also, it doesn't do anything with libc.so.6, since this is only for dynamic linking. It works with the libc.a - static version of the library.
According to #janneb comment, the smallest unit to be linked is "section", so it might not even need to link the whole object files.
The linker is the ld command. If you use that command, it does what you ask. Notice that GNU ld can accept scripts
However, most people are using the gcc command. This is a compiler from the Gnu Compiler Collection suite. Actually, the gcc command is just a driver program, which will run cc1 (the proper C compiler), as, ld and collect2 (which deals with initializations, etc. then invoke the linker).
To understand what exact command[s] gcc is running, pass it the -v program flag.
When you pass -static to gcc it will probably link with e.g. /usr/lib/x86_64-linux-gnu/libc.a or some other static form of the GNU Libc library.