I had symbolic link to the target libs:
~/opt/OpenBLAS/lib $ ls -al
total 0
drwxrwxr-x 2 user user 59 Jul 9 13:03 .
drwxrwxr-x 5 user user 147 Jul 9 12:48 ..
lrwxrwxrwx 1 user user 64 Jul 9 13:03 libopenblas.a -> ../openBLAS_v0.2.9df/lib/libopenblas_df029_sandybridgep-r0.2.9.a
lrwxrwxrwx 1 user user 65 Jul 9 13:03 libopenblas.so -> ../openBLAS_v0.2.9df/lib/libopenblas_df029_sandybridgep-r0.2.9.so
$ echo $LD_LIBRARY_PATH
/home/user/opt/OpenBLAS/lib
Then the gcc has following args:
-L/home/user/opt/OpenBLAS/lib -lopenblas
however, after compiling, run the command it always throws out error:
error while loading shared libraries: libopenblas_df029.so.0: cannot open shared object file: No such file or directory
If I create a symbolic link libopenblas_df029.so.0 under the /home/user/opt/OpenBLAS/lib, it will then work.
Can anyone explain to me why this is happening and how can I change the behaviour?
Does this mean the libopenblas contain some suffix and the OS always append this suffix when trying to find lib file?
I think, your library "libopenblas_df029_sandybridgep-r0.2.9.so" is compiled with "-soname" switch
something like:
gcc -shared -Wl,-soname,libopenblas_df029.so.0 source.c -o libopenblas_df029_sandybridgep-r0.2.9.so
When you link against such library, your executable tries to find library by name "libopenblas_df029.so.0" ( i.e. whatever name you specified in -soname switch )
The best way to find if this is true is to run following command and look for "SONAME"
readelf -d <shared_object> | head -10
Like #Icarus3 said, the linked executable contains the name of the library as it wants to be called (the SONAME).
There should be a symbolic link from that name to the actual shared library. If it is missing, the library is not properly installed -- usually it is a sign that you need to run ldconfig to update the symbolic links and the dynamic linker cache.
Related
I'm unsure about some small obscure details that I worry will have large effects. On my Raspbian Debian 11 running on a Raspberry Pi, sudo apt update && sudo apt full-upgrade only updates my libxml2 library to version 2.9.10, no further. However, I need version 2.9.14 for the security patches contained within. With help from this question (thank you Esther!), I decided to compile version 2.9.14 from source. Everything went well, and the library was placed into /usr/local/lib. I then updated ldconfig by following this answer. However, although that should have made Debian use the new 2.9.14 version, apt-cache policy libxml2 still shows:
libxml2:
Installed: 2.9.10+dfsg-6.7+deb11u2
Candidate: 2.9.10+dfsg-6.7+deb11u2
Version table:
*** 2.9.10+dfsg-6.7+deb11u2 500
500 http://raspbian.raspberrypi.org/raspbian stable/main armhf Packages
100 /var/lib/dpkg/status
I think I know why this is. If I was installing a never-before-seen library, everything might have worked properly. However, since I now have a second libxml2 library without removing the 1st, any time the system needs to use libxml2, the search first reaches /usr/lib/arm-linux-gnueabihf where the old libxml2 is, so the system finds the old version, is satisfied, and so stops searching before finding the new version.
For context before I continue:
(link to below but in color: https://i.stack.imgur.com/OJLJW.png)
pi#fuelightcontrol:~ $ cd /usr/lib/arm-linux-gnueabihf/
pi#fuelightcontrol:/usr/lib/arm-linux-gnueabihf $ ls -l | grep libxml2
lrwxrwxrwx 1 root root 17 May 15 14:58 libxml2.so.2 -> libxml2.so.2.9.10
-rw-r--r-- 1 root root 1510312 May 15 14:58 libxml2.so.2.9.10
pi#fuelightcontrol:/usr/lib/arm-linux-gnueabihf $ cd /usr/local/lib
pi#fuelightcontrol:/usr/local/lib $ ls -l
total 12120
drwxr-xr-x 3 root root 4096 Jun 14 18:17 cmake
-rw-r--r-- 1 root root 7145994 Jun 14 18:17 libxml2.a
-rwxr-xr-x 1 root root 944 Jun 14 18:17 libxml2.la
lrwxrwxrwx 1 root root 17 Jun 14 18:17 libxml2.so -> libxml2.so.2.9.14
lrwxrwxrwx 1 root root 17 Jun 14 18:17 libxml2.so.2 -> libxml2.so.2.9.14
-rwxr-xr-x 1 root root 5242072 Jun 14 18:17 libxml2.so.2.9.14
drwxr-xr-x 2 root root 4096 Jun 14 18:17 pkgconfig
drwxr-xr-x 3 root root 4096 Jun 13 21:43 python3.9
-rw-r--r-- 1 root root 205 Jun 14 18:17 xml2Conf.sh
pi#fuelightcontrol:/usr/local/lib $
The question is, what would be the best way to go about fixing the problem of the old version still being used by apt-cache policy libxml2 and other programs? I could:
Just delete /usr/lib/arm-linux-gnueabihf/libxml2.so.2.9.10 (the old one) and its symbolic link, so the system keeps searching past that point and eventually finds /usr/local/lib/libxml2.so.2.9.14 (the new one). However, something feels... off about having my libraries scattered around in different directories. My gut tells me to keep them in one place. Also, see paragraph below the next list item.
I could delete /usr/lib/arm-linux-gnueabihf/libxml2.so.2.9.10 (the old one) and its symbolic link, then move the new version into /usr/lib/arm-linux-gnueabihf to replace the old version. However, there's more libxml2 related files and 1 more symbolic link in /usr/local/lib that are not present in /usr/lib/arm-linux-gnueabihf. Do I need to move those too, or should I just move libxml2.so.2.9.14 and one (both?) of the symbolic links? If only 1 link, which?
Should I delete the files left behind after I move the required ones over? Also, see paragraph below.
What concerns me about deleting anything is if some other script comes looking for libxml2.2.9.10, can't find it, and fails. I don't know how to tell the rest of the programs that libxml2's filename is different now. I suppose both options 1 and 2 might work, but is one option a cleaner, smarter idea? I'm trying to save myself some work in the future.
Sorry this is such a small silly question. Thank you for your help!
Edit: After making backups of both directories, I tried option 1 first, then option 2. Neither changed the output of apt-cache policy libxml2 - it still says I have libxml2 2.9.10 installed, even though I deleted /usr/lib/arm-linux-gnueabihf/libxml2.so.2.9.10 and its symbolic link, rebooted, and ran sudo apt update
Here's how I updated ldconfig (same as the second link), to clear up loose ends. The link to /usr/local/lib was done for me already, which was nice.
Link to screenshot of below but in color: https://i.stack.imgur.com/7w6XR.png
pi#fuelightcontrol:/etc $ ls -l ld.so.conf
ld.so.conf ld.so.conf.d/
pi#fuelightcontrol:/etc $ cat ld.so.conf
include /etc/ld.so.conf.d/*.conf
pi#fuelightcontrol:/etc $ ls -l ld.so.conf.d
total 16
-rw-r--r-- 1 root root 12 Dec 1 2021 00-vmcs.conf
-rw-r--r-- 1 root root 109 May 14 2019 arm-linux-gnueabihf.conf
-rw-r--r-- 1 root root 41 Jun 25 2018 fakeroot-arm-linux-gnueabihf.conf
-rw-r--r-- 1 root root 44 Jun 14 19:08 libc.conf
pi#fuelightcontrol:/etc $ cat ld.so.conf.d/libc.conf
# libc default configuration
/usr/local/lib
pi#fuelightcontrol:/etc $ sudo ldconfig
pi#fuelightcontrol:/etc $ sudo ldconfig /usr/local/lib
pi#fuelightcontrol:/etc $ sudo ldconfig -n /usr/local/lib
pi#fuelightcontrol:/etc $ cat ld.so.conf.d/00-vmcs.conf
/opt/vc/lib
pi#fuelightcontrol:/etc $ cat ld.so.conf.d/arm-linux-gnueabihf.conf clear
# Multiarch support
/usr/local/lib/arm-linux-gnueabihf
/lib/arm-linux-gnueabihf
/usr/lib/arm-linux-gnueabihf
cat: clear: No such file or directory
pi#fuelightcontrol:/etc $ cat ld.so.conf.d/fakeroot-arm-linux-gnueabihf.conf
/usr/lib/arm-linux-gnueabihf/libfakeroot
pi#fuelightcontrol:/etc $
I am trying to run a simple tcl script with tclsh8.6 that I compiled from source using a musl toolchain on x86_64 Debian.
The script, hello.tcl, looks like this:
puts hello
and when I try to run it I get the following error:
$ /usr/local/x86_64-linux-musl/bin/tclsh8.6 hello.tcl
-bash: /usr/local/x86_64-linux-musl/bin/tclsh8.6: No such file or directory
The necessary tcl libraries and include files are all installed to the prefix /usr/local/x86_64-linux-musl. The binary also exists:
$ ls -l /usr/local/x86_64-linux-musl/bin/tclsh8.6
-rwxr-xr-x 1 root root 8328 Nov 15 12:18 /usr/local/x86_64-linux-musl/bin/tclsh8.6
$ ldd /usr/local/x86_64-linux-musl/bin/tclsh8.6
linux-vdso.so.1 (0x00007ffd04718000)
libtcl8.6.so => /usr/local/x86_64-linux-musl/lib/libtcl8.6.so (0x00007f005981e000)
libc.so => /usr/local/x86_64-linux-musl/lib/libc.so (0x00007f0059587000)
I would like to keep things musl based, so I am trying to avoid a potential solution I found elsewhere, which is to reinstall tcl or tcl-dev with apt-get.
What is causing this error? Any guidance/help is greatly appreciated. Thanks :)
EDIT
As requested by some of the comments here is some additional information.
The output of the file command:
$ file /usr/local/x86_64-linux-musl/bin/tclsh8.6
/usr/local/x86_64-linux-musl/bin/tclsh8.6: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, not stripped
The output of the strace command:
strace /usr/local/x86_64-linux-musl/bin/tclsh8.6 hello.tcl
execve("/usr/local/x86_64-linux-musl/bin/tclsh8.6", ["/usr/local/x86_64-linux-musl/bin"..., "hello.tcl"], 0x7fffb8c5b278 /* 19 vars */) = -1 ENOENT (No such file or directory)
strace: exec: No such file or directory
+++ exited with 1 +++
After seeing this output, I checked to see if the loader/interpreter existed at /lib and it didn't. To fix this, I ran:
$ sudo ln -s /usr/local/x86_64-linux-musl/lib/ld-musl-x86_64.so.1 /lib/ld-musl-x86_64.so.1
That gave me a different error when I ran tclsh8.6:
$ /usr/local/x86_64-linux-musl/bin/tclsh8.6 hello.tcl
-bash: /usr/local/x86_64-linux-musl/bin/tclsh8.6: Permission denied
Running it with sudo gives the same thing:
$ sudo /usr/local/x86_64-linux-musl/bin/tclsh8.6 hello.tcl
sudo: unable to execute /usr/local/x86_64-linux-musl/bin/tclsh8.6: Permission denied
The permissions on the interpreter and script are as follows:
$ ls -l /usr/local/x86_64-linux-musl/bin/tclsh8.6
-rwxr-xr-x 1 root root 8328 Nov 15 13:34 /usr/local/x86_64-linux-musl/bin/tclsh8.6
$ ls -l hello.tcl
-rwxr-xr-x 1 user user 11 Nov 15 12:44 hello.tcl
Hopefully this new information helps a bit more.
EDIT 2
More permission information :)
$ ls -l /lib/ld-musl-x86_64.so.1
lrwxrwxrwx 1 root root 52 Nov 16 10:04 /lib/ld-musl-x86_64.so.1 -> /usr/local/x86_64-linux-musl/lib/ld-musl-x86_64.so.1
$ ls -l /usr/local/x86_64-linux-musl/lib/ld-musl-x86_64.so.1
lrwxrwxrwx 1 root root 12 Nov 15 12:02 /usr/local/x86_64-linux-musl/lib/ld-musl-x86_64.so.1 -> /lib/libc.so
$ $ ls -l /lib/libc.so
-rw-r--r-- 1 root root 246 Nov 14 15:15 /lib/libc.so
And here is the contents of /lib/libc.so:
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( //lib/libc.so.6 //lib/libc_nonshared.a AS_NEEDED ( //lib/ld-linux-x86-64.so.2 ) )
It is a linker script, which if you make executable fails and says the library is corrupted.
With the help of #Shawn I was able to figure out my problem.
tclsh8.6 was looking for the program interpreter at /lib/ld-musl-x86_64.so.1 which didn't exist - I guess when I was installing musl-cross-make it didn't set this up. To fix this problem, I simply symlinked from where ld-musl-x86_64.so.1 was located to lib:
$ sudo ln -s /usr/local/x86_64-linux-musl/lib/ld-musl-x86_64.so.1 /lib/ld-musl-x86_64.so.1
However, I still was getting an error, but this time it was a permissions error:
$ /usr/local/x86_64-linux-musl/bin/tclsh8.6 hello.tcl
-bash: /usr/local/x86_64-linux-musl/bin/tclsh8.6: Permission denied
It turns out that ld-musl-x86_64.so.1 ended up being a symlink that resolved to /lib/libc.so which is a linker script and not the executable DSO that we want. I changed this symlink accordingly:
sudo ln -fs /usr/local/x86_64-linux-musl/lib/libc.so /usr/local/x86_64-linux-musl/lib/ld-musl-x86_64.so.1
After that, I could execute my script and everything worked like a charm! :)
I'm using Fedora 25.
I have a binary that needs multiple libraries. The binary can't find libRblas.so:
$ ldd XPore-Engine | less | grep not
libvtkRenderingAnnotation.so.1 => /usr/lib64/vtk/libvtkRenderingAnnotation.so.1 (0x00007fac12563000)
libRblas.so => not found
libRblas.so => not found
libRblas.so => not found
The library path is properly configured with a .conf file:
$ cat /etc/ld.so.conf.d/R-x86_64.conf
/usr/lib64/R/lib
$ ll /usr/lib64/R/lib
lrwxrwxrwx. 1 root root 11 dic 16 20:46 libopenblas.so.0 -> libRblas.so
lrwxrwxrwx. 1 root root 27 oct 31 21:16 libRblas.so -> /usr/lib64/libopenblas.so.0
-rwxr-xr-x. 1 root root 1989312 oct 31 21:16 libRlapack.so
-rwxr-xr-x. 1 root root 178856 oct 31 21:16 libRrefblas.so
-rwxr-xr-x. 1 root root 2911536 oct 31 21:16 libR.so
And I load the configuration with ldconfig:
$ ldconfig -v | grep libRblas
libopenblas.so.0 -> libRblas.so
However, after executing ldd again it returns the same output saying that libRblas.so wasn't found.
How can I fix this?
I've found a workaround provided by Tom in the Read Hat Bugzilla bug-tracking system at https://bugzilla.redhat.com/show_bug.cgi?id=1404662.
Yeah, so it looks like while R is perfectly happy using libRblas.so as a > symlink to libopenblas.so.0, externally, nothing else is.
The speedup from using openblas is significant, so the fix is to build a copy of openblas that has the libRblas.so filename and soname, and use that instead of the symlink. I have a new build of openblas going which adds this, then I'll do a new round of R builds that depend on it.
As a temporary workaround, you can run (as root):
rm -f /usr/lib64/R/lib/libRblas.so
mv /usr/lib64/R/lib/libRrefblas.so /usr/lib64/R/lib/libRblas.so
That will restore the unoptimized libRblas.so that R provides.
Oh, and run /sbin/ldconfig (as root) after moving libRrefblas.so so that the ldcache is updated.
I messed up the symbolic links of my libblas.so.3
I get the error message:
sudo update-alternatives --list libblas.so.3
update-alternatives: error: cannot stat file '/usr/lib/libblas/libblas.so.3': Too many levels of symbolic links
When I do:
ls -l /usr/lib/libblas/libblas.so.3:
lrwxrwxrwx 1 root root 30 Nov 23 15:15 /usr/lib/libblas/libblas.so.3 -> /etc/alternatives/libblas.so.3
Then, again:
ls -l /etc/alternatives/libblas.so.3
lrwxrwxrwx 1 root root 29 Nov 25 14:36 /etc/alternatives/libblas.so.3 -> /usr/lib/libblas/libblas.so.3
Any help to get ot of this situation would be very much appreciated. I do not know if it is enough information. If not, let me know and I try to provide more. Thanks.
I guess the problem is that /usr/lib/libblas/libblas.so.3 links back to etc/.., however it should point to the actual file. How can I do that?
I had to remove the link in the alternatives directory:
sudo rm /etc/alternatives/libblas.so.3
Then, I recreated the link, but pointnig to a real library file. For that I chose the corresponding one from the atlas package:
sudo ln -s /usr/lib/atlas/atlas-base/libblas.so.3 /etc/alternatives/libblas.so.3
I build the ffmpeg with librtmp. My librtmp is at /opt/librtmp/lib. When I execute the ffmpeg, it said:
./ffmpeg: error while loading shared libraries: librtmp.so.0: cannot open shared object file: No such file or directory
I use ldd command it displays not found:
[qty#testing bin]# ldd ffmpeg
linux-vdso.so.1 => (0x00007fff15576000)
librtmp.so.0 => not found
libz.so.1 => /lib64/libz.so.1 (0x00002b9a71e10000)
libm.so.6 => /lib64/libm.so.6 (0x00002b9a72025000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b9a722a8000)
libc.so.6 => /lib64/libc.so.6 (0x00002b9a724c3000)
/lib64/ld-linux-x86-64.so.2 (0x00002b9a71bf2000)
I know my so at:
[qty#testing bin]# ls -alh /opt/librtmp/lib/
total 300K
drwxr-xr-x 3 root root 4.0K Sep 25 17:10 .
drwxr-xr-x 7 root root 4.0K Sep 25 17:10 ..
-rw-r--r-- 1 root root 158K Sep 25 17:10 librtmp.a
lrwxrwxrwx 1 root root 12 Sep 25 17:10 librtmp.so -> librtmp.so.0
-rwxr-xr-x 1 root root 118K Sep 25 17:10 librtmp.so.0
drwxr-xr-x 2 root root 4.0K Sep 25 17:10 pkgconfig
I found several ways to fix the problem
modify /etc/ld.so.conf, but it required a supper user
set LD_LIBRARY_PATH variable, but it is not conventient to users
pass rpath to gcc, like this
configure args for my ffmpeg
PKG_CONFIG_PATH="/opt/librtmp/lib/pkgconfig" ./configure --disable-doc \
--disable-ffserver --disable-avdevice \
--disable-postproc --disable-avfilter --disable-bsfs \
--disable-filters \
--disable-asm \
--disable-bzlib \
--enable-librtmp \
--prefix=/opt/ffmpeg \
--extra-ldflags="-Wl,-rpath,/opt/librtmp/lib"
Assume there are no source code to re-compile? How do add the shared library search path to a executable file ?
I realize that OP has probably moved on but this is the kind of thing that NixOS does regularly and they have released a tool for this very problem. Also this was a problem I had before even hearing of NixOS.
Here's an example usage of their tool patchelf
...
Likewise, you can change the RPATH, the linker search path embedded into executables and dynamic libraries:
patchelf --set-rpath /opt/my-libs/lib:/foo/lib program
This causes the dynamic linker to search in /opt/my-libs/lib and /foo/lib for the shared libraries needed by program....
From https://nixos.org/patchelf.html
You could use addrpath to add an RPATH to your elf file.
The RPATH will work like LD_LIBRARY_PATH, that is, telling the dynamic loader to search for the shared libraries in that path. RPATH will be permanently in your ELF file.
nixos
this might be nixos specific but provides an interesting insight on ldd/patchelf:
https://lastlog.de/blog/posts/playing_FTL_on_NIXOS.html
ubuntu
on ubuntu/fedora you would use: LD_LIBRARY_PATH with a starter script ./ftl, again, see my above posting about FTL and how it is deployed.
My fix to this problem is to install librtmp into /usr/local/lib and run 'sudo ldconfig' after installing it.
Ffmpeg can then be configured simply by adding --enable-librtmp.
For me this works fine: No system modifications necessary!