I have an excutable and a shared library in same directory, the executable running well, however, after moving the shared library to other pos, I can not run the executable showing can not find libxxx.so
I want to know why it happens?
You can move the library to some standard library path. It is defined in /etc/ld.so.conf. The loader will only find libraries defined in there. In the other hand, you can also use LD_LIBRARY_PATH env variable to put your shared library path. It makes sure the path is searched first
Related
So I am working on a project that is intended to run on a remote server. I develop the program on a local pc, compile it, then upload it to the remote server. Both the local pc and the remote server are run on CentOS 7.7.
The program is developed using the CLion IDE, configured with CMake. The program depends a few shared libraries, which are supposed to link to the executable according to what I wrote in CMake. At my local PC, I can compile and run the program perfectly. However, after I scp the whole directory of the project to the remote server, the executable fails to run. It cannot find any of the ".so" files, according to what ldd says.
This is my CMakeList.txt, with every path being relative path, instead of absolute path.
cmake_minimum_required(VERSION 3.15)
project(YS_Test)
set(CMAKE_CXX_STANDARD 11)
set(SOURCE_PATH_ src)
file(GLOB SOURCE_FILES_ ${SOURCE_PATH_}/*.*)
set(PROJECT_LIBS_ libTapQuoteAPI.so libTapTradeAPI.so libTapDataCollectAPI.so)
include_directories(api/include)
link_directories(api/lib/linux)
add_executable(YS_Test ${SOURCE_FILES_})
target_link_libraries(YS_Test ${PROJECT_LIBS_})
Please do not tell me to set LD_LIBRARY_PATH to fix my issue. The program worked fine on my local pc without LD_LIBRARY_PATH, so I expect it to run on the remote server without LD_LIBRARY_PATH. I would like to know what is really going on here, instead of a work around. Thanks!
If I understand your problem correctly, you want to ship your compiled YS_Test program along with some dependencies and have it run on a remote server. By default an executable will only look in the directories configured in /etc/ld.so, which will not include the deploy path.
Note: Typically you do not deploy your entire build directory but only the compiled artifacts and dependencies. For this answer I will assume you deploy the binary and its dependencies to the same directory.
You have two options:
Require users of your program to set LD_LIBRARY_PATH, either by themselves or by a wrapper script. This variable will instruct the dynamic linker to look in the specified directories as well. Even if you do not like this solution, it is by far the most common approach.
Add -Wl,-rpath='$ORIGIN' to your linker options. This will add a DT_RUNPATH attribute to the executable's dynamic section. As you are using CMake you can also set this using BUILD_RPATH and/or INSTALL_RPATH target properties.
The ld.so manpage describes this attribute as follows:
If a shared object dependency does not contain a slash, then it is
searched for in the following order:
...
Using the directories specified in the DT_RUNPATH dynamic section
attribute of the binary if present.
The $ORIGIN part expands to the directory containing the program or shared
object.
If you really insist on shipping your build directory (eg during development), you can take a look at the CMake BUILD_RPATH_USE_ORIGIN property (and its usual global counterpart CMAKE_BUILD_RPATH_USE_ORIGIN), this will embed relative paths into binaries instead of absolute paths.
As you don't want a workaround (#Botje has given you two already), I will try an explanation instead. In your development machine, if you use this command:
ldd YS_Test
You will see all the shared libraries used by your program, with their corresponding paths. The libTapQuoteAPI.so libTapTradeAPI.so libTapDataCollectAPI.so are found at your 'api/lib/linux' directory, but resolved with full absolute paths. If you do the same at your server, some shared objects can't be resolved because they aren't at the same location.
If you use one of these commands (not sure which are available in Centos):
chrpath --list YS_Test
or
patchelf --print-rpath YS_Test
You will see the RPATH or RUNPATH tags embedded in your program. This is the path used by the Linux linker to locate dependencies that are outside the standard ld locations. You may find extended explanations on Internet about this, like this one or the Wikipedia article.
Breaking my promise, I give you a third workaround: use patchelf or chrpath at your server after scp to change the embedded RPATH tag, pointing it relative to $ORIGIN (which represents the program location).
Assume that a program needs a shared library(bar.so) under the folder /usr/lib/foo. I understand if the bar.so was directly under /usr/lib, it would be automatically found. But as in my case the library could not be found automatically, because -I think so- it is under the folder /usr/lib/foo. However there are tons of other folders under /usr/lib and the corresponding programs using those libraries are working seamlessly.
So, how does this process work and how can I fix my issue?
Thanks.
Individual programs can control where they search for their libraries.
Also the search path can be controlled using the LD_LIBRARY_PATH env var:
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
Try appending /usr/lib/foo to your LD_LIBRARY_PATH env var.
I am trying to make a Qt5 part of my source tree, so I haven't installed it on my machine, just copied it from source control. I am having a problem when I try to run uic.exe:
stiopa#stiopa-VirtualBox:~/ct/LinuxLibs/Qt/bin > ./uic
./uic: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
I am still getting the same error even when I copy the libQt5Core library to bin directory. How is uic looking for shared libraries? Is there any environment variable I need to set to fix it?
This is yet another case of not putting the dependent shared libraries in a defined location that is supported by the program.
If you're planning on doing the 'copy the files to the same directory as the executable', the fast solution is to reference the directory in the library load path; e.g. if the binary is in $HOME/foo, you do:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}${LD_LIBRARY_PATH:+:}$HOME/foo
This adds or makes $HOME/foo the run-time-linker's load path. As a result, any programs you run will look in this directory for libraries, as well as the default set for the OS (defined by the ld.so configuration), as well as the paths that are defined within the application itself (the rpath).
If you're going to follow this route, what you can do is to move the binary to target.bin, create a target bash script, which invokes the bin file automatically; e.g.
#!/bin/bash -p
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}${LD_LIBRARY_PATH:+:}$(dirname $0)
exec $0.bin "$*"
A secondary mechanism which will permit you to change the search location for a binary; without requiring an environment variable insert is to modify the binary so that it searches in different locations than it usually does; this takes advantage of some features in the run-time linker (which looks for libraries).
There is a program called chrpath, which can be added by various package managers, which allows you to edit the rpath directly. In this case; you can change the additional search path of the binary using:
chrpath -r '$ORIGIN' foo
This means that the program will look in the same directory as the binary for .so files, thus allowing it to run.
On a Linux system, I am trying to run a Fortran program that makes use of some shared libraries (netCDF libs, if that makes a difference). Before I run, I set LD_LIBRARY_PATH so that it points to the location of my libraries. Then I run the executable and I quickly get the error
../my_program: error while loading shared libraries: libnetcdff.so.5: cannot open shared object file: No such file or directory
Now, I double check the value of LD_LIBRARY_PATH, and then cd to it and find plain as day
$ ls *ff*
$ libnetcdff.a libnetcdff.la libnetcdff.so libnetcdff.so.0 libnetcdff.so.0.0.0
So the libnetcdff is absolutely present.
Could anyone point me to the problem?
The one thing that has occurred to me is that the executable seems to want to find libnetcdff.so.5, while the library that is present is actually libnetcdff.so.0. Is that the problem? If so, is there a way to convince the executable to not insist on "5"? Alternatively, would a link from libnetcdff.so.0 to libnetcdff.so.5 solve the problem? (I don't have permissions in the directory, BTW, which is why I haven't tried that yet.)
Environment info: CentOS machine, code compiled with gfortran. And yes, when I compiled, my -L flags were pointing to the same directory that LD_LIBRARY_PATH points to.
Thanks.
A library link should work. Since you mention that you do not have root/sudo access, what you can do is link in a file you do have access in:
ln -s /path/to/libnetcdff.so.0 /path/you/have/access/to/libnetcdff.so.5
And then add in the /path/you/have/access/to/ in your LD_LIBRARY_PATH.
I have some shared/dynamic libraries installed in a sandbox directory. I'm building some applications which link agains the libraries. I'm running into what appears to be a difference between OSX and Linux in this regard and I'm not sure what the (best) solution is.
On OSX the location of library itself is recorded into the library, so that if your applications links against it, the executable knows where to look for the library at runtime. This works like expected with my sandbox, because the executable looks there instead of system wide install paths.
On Linux I can't get this to work. Apparently the library location is not present in the library itself. As I understand it you have to add the folders which contain libraries to /etc/ld.so.conf and regenerate the ld cache by running ldconfig.
This doesn't seem to do the trick for me because my libraries are located inside a users home directory. It looks like ldconfig doesn't like that, which makes sense actually.
How can I solve this? I don't want to move the libraries out of my sandbox.
On Linux, run your program with the environment variable LD_LIBRARY_PATH set to your sandbox dir.
(I remember having used a flag -R to include library paths in the binary, but either it has been removed from gcc or it was only available on BSD systems.)
On Linux you should set LD_RUN_PATH to your sandbox dir. This is better than setting LD_LIBRARY_PATH because you're telling the linker where the library is at link time, rather than telling the shared library loader at run time.
See: Link