Managing secondary dependencies of shared libraries - shared-libraries

Question

I have a problem with the seconday dependences of the shared libraries.
Suppose we have the following dependency tree:
libA.so
├─libB.so
└─libC.so
└─libD.so
That is, a shared library A depends on shared libraries B and C, and shared library C depends itself on shared library D. A good example for libC is GSL shared library which depends on CBLAS.
To make a self-sufficient easy-to-use package and to avoid problems with different versions of installed shared libraries, I package the libraries libB, libC and libD with libA, and as explained, e.g., in Drepper’s article, “How to write shared libraries” (2011), I add the link flag -Wl,-rpath,\$ORIGIN --enable-new-dtags to set the RUN_PATH of libA.so, so that the dynamic loader can find the dependencies of libA in the corresponding directories, without needing to set LD_LIBRARY_PATH (which has its downsides).
The problem is that setting the RUN_PATH in libA does not help with the secondary dependencies, like libD.so.
Turning on the dynamic loader debug messages (via setting LD_DEBUG) reveals that the loader attempts to find libD.so only in the standard library locations, and not in the location of libA (contrary to what it does for finding libB or libC).
Is there a way to overcome this problem?
Indeed, I can link the libraries C and D statically, if I have the source code or properly compiled static libraries.
But is there a better way?
Solution:
As explained by Employed Russian the solution is to set RPATH instead of RUNPATH.
With recent versions of GCC, one should use the following link flag:
-Wl,--disable-new-dtags,-rpath,\$ORIGIN
RPATH is searched for transitive dependencies; that is, paths in RPATH will be considered for everything that is dynamically loaded, even dependencies of dependencies.
In contrast, the ld dynamic linker does not search RUNPATH locations for transitive dependencies (unlike RPATH).
See also:
Wikipedia: https://en.wikipedia.org/wiki/Rpath#The_role_of_GNU_ld
SO question: How to set RPATH and RUNPATH with GCC/LD?

Answer 1

By setting DT_RUNPATH you are telling the loader that each of your binaries is linked with all of its dependencies.
But that isn't true for libC.so -- it (apparently) doesn't have DT_RUNPATH of its own.
I can link the libraries C and D statically, ... But is there a better way?
Yes: link libC.so with correct DT_RUNPATH (if libC.so and libD.so are in the same directory, then -Wl,-rpath,\$ORIGIN will work for libC.so as well).
Update:
The problem is that I may not have the source or the properly compiled object files of libC
In that cause you should use RPATH instead of RUNPATH. Unlike the latter, the former applies to the object itself and to all dependencies of that object.
In other words, using --enable-new-dtags is wrong in this case -- you want the opposite.
In this case, there is no solution (besides static linking); correct?
The other solution (besides using RPATH) is to set LD_LIBRARY_PATH in the environment.
Update 2:
Difference between RPATH and RUNPATH
The difference is explained in the ld.so man page:
If a shared object dependency does not contain a slash, then it
is searched for in the following order:
o Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH
attribute does not exist. Use of DT_RPATH is deprecated.
...
o Using the directories specified in the DT_RUNPATH dynamic
section attribute of the binary if present. Such directories
are searched only to find those objects required by DT_NEEDED
(direct dependencies) entries and do not apply to those
objects' children, which must themselves have their own
DT_RUNPATH entries. This is unlike DT_RPATH, which is applied
to searches for all children in the dependency tree.