Background:
I wrote a program that uses the OpenBLAS libraries for a very heterogeneous compute cluster. OpenBLAS uses different libraries for different architectures.
So on one machine 'ls -l /usr/lib64/libopenblas.so' results in
lrwxrwxrwx 1 root root 31 Mar 6 15:13 /usr/lib64/libopenblas.so -> libopenblas_barcelona-r0.2.6.so*
On another, the result is
lrwxrwxrwx 1 root root 33 Mar 4 09:43 /usr/lib64/libopenblas.so -> libopenblas_sandybridge-r0.2.6.so*
There may be others, but these are the two I have checked. Both implement the same API, just use different optimizations.
The problem:
When I compile my own shared object file using these I use
gcc ... -lopenblas and it compiles just fine, and it runs on machines with similar architecture (i.e., those with 'libopenblas_barcelona-r0.2.6.so'), but on other architectures it fails to run.
ldd shows that it links against libopenblas_barcelona-r0.2.6.so rather than libopenblas.so.
Is there any way I can tell gcc to link against the symlink, rather than the result of that symlink, so that it will be "right" regardless of the architecture of the machine it's running on?
Related
I run two versions of subversion on my system, using different libraries. This worked until yesterday, with the old version picking up the system libraries in /usr/lib64, and the new version picking up the new libraries in ~/local/lib, but this has now broken.
The old version was installed with the distro, and I built the new one from source a few months ago. I built the new one in ~/local, with the new binary in ~/local/bin/svn, and the new libraries in ~/local/lib. I also created a new file in /etc/ld.so.conf.d, which included the single line /home/me/local/lib.
This worked fine until I tried to install Qt5 yesterday: I could run both the old Subversion (1.7.14), as /usr/bin/svn, and the new one (1.12.2), as ~/local/bin/svn.
If I now try to run the old svn it tells me that there's a missing symbol in one of the Subversion libraries. If I run
$ ldd /usr/bin/svn
it turns out that the old svn now tries to load the new libraries in ~/local/lib, so it doesn't work. Running ldconfig makes no difference. The old and new libraries are completely different, but somehow have the same version number:
$ ll /usr/lib64/libsvn_client-1.so.*
lrwxrwxrwx. 1 root root 24 Jan 3 11:37 /usr/lib64/libsvn_client-1.so.0 -> libsvn_client-1.so.0.0.0
-rwxr-xr-x. 1 root root 379656 Nov 20 2015 /usr/lib64/libsvn_client-1.so.0.0.0
$ ll /home/me/local/lib/libsvn_client-1.so*
lrwxrwxrwx. 1 me me 24 Oct 8 16:32 /home/me/local/lib/libsvn_client-1.so -> libsvn_client-1.so.0.0.0
lrwxrwxrwx. 1 me me 24 Oct 8 16:32 /home/me/local/lib/libsvn_client-1.so.0 -> libsvn_client-1.so.0.0.0
-rwxr-xr-x. 1 me me 2986816 Oct 8 16:32 /home/me/local/lib/libsvn_client-1.so.0.0.0
There's obviously something I don't understand about shared libs here. Presumably the dynamic loader selects the required library from the minor version numbers, but they're all 0 here. So how did this ever work? And surely svn doesn't ship with all-zero version numbers? Is it possible that the Qt5 install has caused a problem? Is there some way to fix this short of deleting one of the versions?
In a Linux machine with kernel debug packages installed, I could see that two copies of kernel modules are there in two locations as mentioned below:
/lib/modules/<$KERNELVERSION>/kernel/
/usr/lib/debug/lib/modules/<$KERNELVERSION>/kernel/
I do have a doubt that which module will be executed and what is the need for two modules.
/lib/modules/<$KERNELVERSION>/kernel/ - modules, that will be loaded with kernel ( they are without debug symbols )
Example:
ll /lib/modules/4.15.0-20-generic/kernel/fs/xfs/xfs.ko
-rw-r--r-- 1 root root 1883966 Apr 24 2018 /lib/modules/4.15.0-20-generic/kernel/fs/xfs/xfs.ko
/usr/lib/debug/lib/modules/<$KERNELVERSION>/kernel/ - modules with debug symbols
Example:
ll /usr/lib/debug/lib/modules/4.15.0-20-generic/kernel/fs/xfs/xfs.ko
-rw-r--r-- 1 root root 40247182 Apr 24 2018 /usr/lib/debug/lib/modules/4.15.0-20-generic/kernel/fs/xfs/xfs.ko
As you can see, it's 1.8Mb vs 40Mb. If you compare outputs of readelf -S <module>, then you will notice additional sections like debug_aranges, debug_info, debug_ranges, etc. in debug module
Recently I ran into a problem with the Python function ctypes.util.find_library. The function is used to locate shared libraries by name; for example, CuPy uses it to locate cuDNN. In my case, I had several versions of cuDNN installed, and it picked up the latest (as per the documentation). However, the contents of the directory look like this:
$ l /usr/local/cuda-8.0/lib64 | grep -i cudnn
lrwxrwxrwx 1 root root 13 Oct 3 08:21 libcudnn.so -> libcudnn.so.6*
lrwxrwxrwx 1 1000 users 17 Jul 27 2016 libcudnn.so.5 -> libcudnn.so.5.1.5*
-rwxrwxr-x 1 1000 users 79337624 Jul 27 2016 libcudnn.so.5.1.5*
lrwxrwxrwx 1 root root 18 Oct 3 08:21 libcudnn.so.6 -> libcudnn.so.6.0.21*
-rwxr-xr-x 1 1000 users 154322864 Apr 12 2017 libcudnn.so.6.0.21*
lrwxrwxrwx 1 root root 17 Oct 2 10:32 libcudnn.so.7 -> libcudnn.so.7.0.3*
-rwxrwxr-x 1 1000 1000 217188104 Sep 16 05:09 libcudnn.so.7.0.3*
-rw-r--r-- 1 1000 users 143843808 Apr 12 2017 libcudnn_static.a
Even though the latest version is 7.0.3, judging from the symbolic link hierarchy, I would have expected version 6.0.21 to be picked up. My questions are:
Which version would the gcc (or clang) toolchain have picked up during compile-time?
Which version would a C/C++ executable have picked up during run-time?
Is there any kind of information source (an article, a man page, a book, ...) out there that contains explicitly the answers to the first two questions? I tried googling it, but nothing definitive came up.
Traditionally you'd build with a command like gcc -lcudnn. This would find libcudnn.so which points to libcudnn.so.6 which points to libcudnn.so.6.0.21. So libcudnn.so.6.0.21 would be linked at build time.
Traditionally a shared library will contain a "SONAME" which indicates the ABI compatible version to be loaded at runtime. I'm pretty sure that would be libcudnn.so.6 in this case. So building against libcudnn.so.6.0.21 would give you a runtime dependency on libcudnn.so.6 (you can verify this using ldd myprog | grep libcudnn.so).
Probably, but that's not an on-topic question for Stack Overflow ("recommending an off-site resource").
Installing liboost-dev on Debian Squeeze gives me several libraries like /usr/lib/libboost_thread.so.1.42.0, but no libboost_thread.so. Now I can't link using the -l flag of gcc / ld because the names don't end in .so.
I notice that /usr/lib has plenty of other libraries of the form libfoo.so.N without a libfoo.so, so this isn't peculiar to Boost. I ended up adding libboost_thread.so.1 and libboost_thread.so symlinks links by hand. (The man page for ldconfig suggests it will add the links, but it didn't do anything).
Everything works fine, but it feels dirty. What should I have done?
use some more specific linker option I haven't found yet (at the cost of making my makefiles depend on a specific version number).
just add the symlinks by hand (at the risk of subverting package management).
some other Debian 'right way' to do it.
You installed the run-time package libboost-thread1.42.0 but the development package libboost-thread-dev (or even the catch-all package libboost-all-dev.
Once you have the corresponding -dev package, linking will work. That is a general feature of most Linux distribution---you almost never want to mess with the symlinks by hand.
Edit: In response to your comment:
edd#max:~$ ls -l /usr/lib/libboost_thread.*
-rw-r--r-- 1 root root 176324 2010-10-21 00:56 /usr/lib/libboost_thread.a
lrwxrwxrwx 1 root root 25 2011-05-14 10:17 /usr/lib/libboost_thread.so -> \
libboost_thread.so.1.42.0
-rw-r--r-- 1 root root 88824 2010-10-21 00:56 /usr/lib/libboost_thread.so.1.42.0
edd#max:~$ dpkg -S /usr/lib/libboost_thread.so
libboost-thread1.42-dev: /usr/lib/libboost_thread.so
edd#max:~$
Clearly the package management system created the links, and owns them.
Dirk's answer is correct about the general principle but there seems to be an extra trap for the unwary with the boost packaging.
Normally the headers and the library symlink are in the same package, so you get the library symlink without thinking about it. However with boost "libboost<version>-dev" provides the headers but "libboost-<lib><version>-dev" provides the shared library symlink. So if you only install the former you get stuff compiling but not linking.
When I do ls -l in /usr/lib I see lots of libs with "sameName.so.*.*" extension.
What is the significance of these extensions?
Why softlinks are created? what are their use?
One example will help a lot in understanding.
This is a trick used to version shared object files. It's a way of avoiding the dreaded DLL hell which came about because of lazy linking.
The advantage of lazy linking (or late binding) is that components of your executable can be changed without actually re linking those executables. This allows for bug fixes in third party components without having to ship a new executable, among other things.
The disadvantage is exactly the same as the advantage. Your executable can find that assumptions it made about the underlying libraries have been changed and this is likely to cause all sorts of issues.
Versioning of shared objects is one way to avoid this. Another would be to not share objects at all but that also has pros and cons which I won't get into here.
By way of example, let's say you have version 1 of xyz.so. You have a file and a symbolic link to that file:
pax> ls -al xyz*
-rw-r--r-- 1 pax paxgroup 12345 Nov 18 2009 xyz.so.1
lrwxrwxrwx 1 pax paxgroup 0 Nov 18 2009 xyz.so -> xyz.so.1
Now, when you create an executable file exe1, linking it with xyz.so, it will follow the symbolic link so that it stores xyz.so.1 in the executable as the thing it needs to load at runtime.
That way, when you upgrade the shared library thus:
pax> ls -al xyz*
-rw-r--r-- 1 pax paxgroup 12345 Nov 18 2009 xyz.so.1
-rw-r--r-- 1 pax paxgroup 67890 Nov 18 2009 xyz.so.2
lrwxrwxrwx 1 pax paxgroup 0 Nov 18 2009 xyz.so -> xyz.so.2
your original executable exe1 will still load version 1 of the shared object.
However, any executables you create now (such as exe2) will be linked with version 2 of the shared object.
The actual implementation details may vary somewhat (I'm basing my answer on earlier UNIXes and Linux appears to do versioning a little more intelligently than just following symbolic links). IBM developerWorks has a nice article on how it's done here.
When you create a shared object, you give it both a real name and an soname. These are used to install the shared object (which creates both the object and a link to it).
So you can end up with the situation:
pax> ls -al xyz*
-rw-r--r-- 1 pax paxgroup 12345 Nov 18 2009 xyz.so.1.5
lrwxrwxrwx 1 pax paxgroup 0 Nov 18 2009 xyz.so.1 -> xyz.so.1.5
lrwxrwxrwx 1 pax paxgroup 0 Nov 18 2009 xyz.so -> xyz.so.1
with xyz.so.1.5 possessing the SONAME of xyz.so.1.
When the linker links in xyz.so, it follows the links all the way to xyz.so.1.5 and uses its SONAME of xyz.so.1 to store in the executable. Then, when you run the executable, it tries to load xyz.so.1 which will point to a specific xyz.so.1.N (not necessarily version 1.5).
So you could install xyz.so.1.6 and update the xyz.so.1 link to point to it instead and already-linked executables would use that instead.
One advantage of this multi-layer method is that you can have multiple potentially incompatible libraries of the same name (xyz.so.1.*, xyz.so.2.*) but, within each major version, you can freely upgrade them since they're supposed to be compatible.
When you link new executables:
Those linking with xyz.so will get the latest minor version of the latest major version.
Others linking with xyz.so.1 will get the latest minor version of a specific major version.
Still others linking with xyz.so.1.2 will get a specific minor version of a specific major version.
It's a versioning scheme for shared libraries. Every library should have 3 names:
Real name: the actual library name, libfoo.so.1.2.3
"SONAME": the name recorded in the executable, and the name dynamic linker looks for, libfoo.so.1.2. This name is actually written inside the library binary itself, and will be recorded in the executable at link time. It is usually a symlink to library's real name (usually latest version).
Linker name: the name you give to the linker when building your program. Usually links to the latest SONAME.
Example
Say you have libfoo version 1 installed: libfoo.so -> libfoo.so.1.0 -> libfoo.so.1.0.0. You build your program bar with -lfoo. it now links to libfoo and will load libfoo.so.1.0 at runtime due to SONAME. Then you upgrade to a patched but binary-compatible libfoo.so.1.0.1 by replacing real binary. bar still links to libfoo.so.1.0 and doesn't need to be rebuilt.
Now imagine you want to build a new program baz that takes advantage of incompatible changes in libfoo v1.1. You install new version and your system now have two versions installed in parallel:
libfoo.so.1.0 -> libfoo.so.1.0.1
libfoo.so -> libfoo.so.1.1 -> libfoo.so.1.1.0
Note linker name was updated to the latest version (this is the version corresponding to the headers you installed in /usr/include).
You build baz, and it links to libfoo.so and loads libfoo.so.1.1 at runtime. Not that bar still runs against libfoo.so.1.0 and doesn't need to be updated.