The other day I came across a linux command that let me see where a program is expecting to find its libraries. It is very useful to solve library dependency problems for not so popular or proprietary software. I used ldd, it was very informative, but missed one crucial piece of information for me:
ldd -v ./my_executable
gave good information for libraries that my_executable can link to.
But for those it can not link/find, ldd only gave information like:
<a_library_name.so.version> => not found
What I want is, instead of "not found", I want to see
not found at /path/to/<a_library_name.so.version>.
on linux you can use the LD_DEBUG. This link is helpful.
Probably you need the strace command
Take a look here http://www.thegeekstuff.com/2011/11/strace-examples/
The information about library paths is stored in /etc/ld.so.conf:
/usr/local/lib64
/usr/local/lib
include /etc/ld.so.conf.d/*.conf
# /lib64, /lib, /usr/lib64 and /usr/lib gets added
# automatically by ldconfig after parsing this file.
# So, they do not need to be listed.
See man ldconfig for more information.
Related
So I got several shared libraries that I am trying to permanently install on my Ubuntu system but I am having some difficulty with it.
I want to install the libraries and the headers in a separate folder under /usr/local/lib and /usr/local/include (for example a folder named agony) so it would be clean and removing them would just require that I delete those folders. so it looks something like this:
/usr/local/lib/agony/libbtiGPIO.so
/usr/local/lib/agony/libbtiDSP.so
...
/usr/local/include/agony/GPIO.h
/usr/local/include/agony/DSP.h
...
And I added a file here /etc/ld.so.conf.d/agony.conf which include a line describing the path to the library folder:
$ cat /etc/ld.so.conf.d/agony.conf
/usr/local/lib/agony
and I perform sudo ldconfig to update the library database.
So to double check if the library is found I do ldconfig -p | grep bti* and
I see the following result:
$ ldconfig -p | grep bti
...
libbtiGPIO.so (libc6,x86-64) => /usr/local/lib/agony/libbtiGPIO.so
libbtiDSP.so (libc6,x86-64) => /usr/local/lib/agony/libbtiDSP.so
...
At this point I should be able to use the libraries without specifying the library path. But When I attempt to compile an application without providing the library path (-L) it fails. However, when I supply gcc with the library path ex:
gcc source.c -L /usr/local/lib/agony output -lbtiGPIO -lbtiDSP
it works!!
I don't want to use LD_LIBRARY_PATH environment variable because this library is going to be used everywhere on the system and I don't want other compilers to worry about providing LD_LIBRARY_PATH.
What am I doing wrong here?
At this point I should be able to use the libraries without specifying the library path
Here lies the confusion.
You have built your shared library libbtiGPIO.so (just sticking with that one),
placed it in /usr/local/lib/agony, and updated the ldconfig database accordingly.
The effect of that is when you run a program that has been linked with libbtiGPIO
then the dynamic linker (/lib/x86_64-linux-gnu/ld-2.21.so, or similar) will know where to look
to load that library into the process and you will not need to tell it by setting an LD_LIBRARY_PATH in the environment.
However, you haven't done anything that affects the list of default library
search directories that are hardwired into your build of gcc, that it passes to
the linker (/usr/bin/ld) when you link a program with libbtiGPIO in the first place.
That list of default search directories is what you will find if your do a verbose
build of your program - gcc -v ... - and then pick out the value of LIBRARY_PATH
from the output, e.g.
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/5/:\
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/:\
/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib/:\
/lib/x86_64-linux-gnu/:\
/lib/../lib/:\
/usr/lib/x86_64-linux-gnu/:\
/usr/lib/../lib/:\
/usr/lib/gcc/x86_64-linux-gnu/5/../../../:\
/lib/:\
/usr/lib
/usr/local/lib/agony is not one of those and to make it one of those you
would have to build gcc from source yourself. Hence, in order to link your
program with libbtiGPIO you still need to tell ld where to find it with
-L/usr/local/lib/agony -lbtiGPIO.
man, you misunderstand the procedure of complier and link.
First, libbtiGPIO.so is a shared link library not a static link library. it is important to know those difference .
Then you need to know something else. changing ld.so.conf.d/*.conf and run sudo ldconfig, it affects the procedure of link. in other words, if you don't add agony.conf and sudo ldconfig, you will receive a error when you run ./a.out rather than gcc source.c -L ...., the gcc command can run successfully even thougth you don't ldconfig.
Finally,if you don't pollute the LD_LIBRARY_PATH environment variable, you have to add -L ... options in your gcc command. What'more, if you don't want to input too many words in your shell frequently, you can learn to use Makefile.
Can any body explain to me what does the whole sentence mean?
I know this is to set Macro BLAS_LIBS as another string.
But I'm not sure what's the "-lblas" mean and I don't know how to use it.
Similar as the following code. "-llapack"
export LAPACK_LIBS="-L$LAPACKHOME/lib -llapack"
How can the program find out the BLAS and LAPACK libraries just by "-lblas" and "-llapack" ?
Thanks for advance.
I'm not sure why you say "just by -llapack" because that's not what is happening here. Specifically, the -L option just before it specifies a directory path to add to the library resolution path. This works roughly like PATH in the shell.
For example, with the command line fragment gcc -Lfoodir -Lbardir -lfoo -lbar, you basically instruct the linker to search the directories foodir and bardir for the library files libfoo.a and libbar.a.
The -l option is described in GCC: Options for Linking and -L and friends in the following section GCC: Options for Directory Search.
This build arrangement -- configure the build to show where the required files are before compiling -- is common for libraries, where if a user has already downloaded and compiled a required library for some other project, they don't need to rebuild it; they can just point the compiler to wherever they already have the stuff needed for this project.
Building your own libraries is becoming increasingly unnecessary anyway, as prepackaged binaries of most common libraries are available for most systems these days. But of course, if you are on an unusual platform, or have specialized needs which dictate recompilation with different options than any available prebuilt binary, you will still need to understand how to do this.
The problem is I use dlopen to load a library (the .so is written by me, it's not a system library), but I got the error shown in the title.
I have included dlfcn.h
in compiler, I used the -ldl command
What I want to load is just the source code folder, I tried to add -L., but it did not work.
The most brutal and effective way to find out where your code goes wrong is the following command which will activate the debugging mode for shared libraries and is documented here:
export LD_DEBUG=libs
Then, you will be surprised that so much information pops up. Don't worry, these information tells you which shared libraries the command you just typed needs and where to locate these needed libraries. For example, if you type reset, the screen will be reseted and then information about the shared libraries reset command needs will be printed.
Then, execute your "problematic" executable to see what's going wrong.
PS.1 : According to your accepted mythagal's solution :
Specify the full path to the file in dlopen
dlopen("/full/path/to/libfile.so");
It seemed that even though you use absolute or relative path in the dlopen function, the directory not found error will still show up. I am using CentOS, and my Debian is also having this problem. So I think the first solution mythagal provide is wrong. You can verify that in the "debugging" mode I mentioned above.
PS.2: If you "install" or "compile" a shared library rather than install it through package manager, you MUST run sudo ldconfig /path/where/not/found/shared/library/reside to notify the system of the newly added shared library. For example :
cp /etc/ld.so.cache ~/ld.so.cache.backup
#cp -r /etc/ld.so.conf.d ~/ld.so.conf.d.backup #sometimes this backup is unnecessary.
#cp /etc/ld.so.conf ~/ld.so.conf.backup #sometimes this backup is unnecessary.
sudo ldconfig /PATH/WHERE/NOT/FOUND/SHARED/LIBRARY/RESIDE
###I am omitting the cp commands to roll back.
###For example, sudo cp -f ld.so.cache /etc/ld.so.cache
To understand what's going on here, please carefully read all the contents in the link above.
PS.3 : You can always use the command export LD_DEBUG=help,export LD_DEBUG=libs to figure out how -rpath or LD_LIBRARY_PATH solve your problem. You will love this debugging mode.
PS.4: A less brutal way to figure out what's going wrong:
ldd ./YOURproblematicEXECUTABLE
This command can tell you whether your shared library to be opened is located or not. Besides, there are so many ways to fix your problem and each way has its limitation and application. So I strongly suggested you read the link I provide you above and understand how to choose the way to solve your problem. After reading that, if you actually feel like being very "OK", you can also read this Better understanding Linux secondary dependencies solving with examples for deeper understanding.
If the library you want to dlopen is not in the standard search path you have a number of options:
Specify the full path to the file in dlopen
dlopen("/full/path/to/libfile.so");
Add the path to the library via LD_LIBRARY_PATH
LD_LIBRARY_PATH=/path/to/library/ ./executable
use the ld -rpath option to add a library path to the application.
g++ -link stuff- -Wl,-rpath=/path/to/library/
Note that options 1 & 3 hardcode the library path into your application. -rpath does have an option to specify a relative path, i.e.
-Wl,-rpath=$ORIGIN/../lib/
Will embed a relative path into the application.
the dlopen's declaration look like,
void *dlopen(const char *filename, int flag);
if you set the para 'filename' as shared library's name , you should add you current path into the 'LD_LIBRARY_PATH'.for instance,
1, dlopen("libtest.so" , RTLD_LAZY)
2, in shell , export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
For my case, the solution was very simple:
The path is expected to be absolute unless clearly specified as relative
So I replaced
dlopen("mylib.so", RTLD_NOW)
with
dlopen("./mylib.so", RTLD_NOW)
And the problem solved.
I would recommed dlerror to get the reason
void* handle = dlopen(SO_FILE, RTLD_NOW | RTLD_LOCAL | RTLD_DEEPBIND);
if(handle == NULL)
{
printf(LOG_ERROR, "Error: %s\n", dlerror());
assert(0);
}
this will report detailed reason for the error
I experienced a (for me) strange behaviour today: Using QMake with the PkgConfig-options etc. I was able to link the opencv libraries, but then I switched to CMake using PkgConfig. Once I tried to build my software, the linker complained that it was not able to find the library libcvaux, which pkg-config returns asked to deliver the libraries for opencv (pkg-config --libs opencv).
In /usr/lib I found a libcvaux.so.{version}, but no "plain" entry libcvaux.so. So what I did was to create a symlink, and now it works.
Now I wonder why it worked before. Is there something to pass ld an option saying "use the newest version, and you get the version by looking at the numbers behind the so suffix"? Or is it more some kind of bug that the maintainers of the opencv package forgot to add this symlink? Because e.g. libcv or libhighgui have such symbolic links.
Thank you!
From the ldconfig manpage:
ldconfig checks the header and file
names of the libraries it encounters
when determining which versions should
have their links updated.
Maybe an earlier ldconfig run deleted the link.
I'm trying to compile/link a very old piece of software on a linux system and I can't for some reason link with a shared library that's installed on my system.
I get the following error from the linker:
/usr/bin/ld: cannot find -lXaw
However, the lib itself is installed. If I run
ldconfig -v | grep libXaw
I get (among other things) this hit:
libXaw.so.7 -> libXaw7.so.7.0.0
The library and the links to it are in /usr/lib btw. So nothing special.
So the library is there and ldconfig finds it. What could ld cause ld from not finding the library during link-time? As you may have already guessed I'm quite new to the shared library stuff.
Any ideas?
The linker may be looking, literally, for "libXaw.so". Is that in /usr/lib? If not, you could try adding it as another soft link from libXaw7.so.7.0.0.
The reason for the symlink btw is to select the default version to link against in the case of multiple versions, keep in mind the name of the library is integrated into the binary. (which you can see with ldd).
Are the -L library directories being overridden, and it's not looking in /usr/lib?
To link it, you need the .a file, NOT the .so file, which is the runtime library. The shared object is only useful to a program already linked against the non-shared parts of a library. This is typically distributed in a ".a" file.