Given ar libraries and possibly object files, what is the best way to find all of the unresolved external symbols? One possibility is to run the linker and then capture the errors, but sometimes it stops after a certain number of symbols. Is there a better way?
In your mingw-w64 installation's bin directory, along with
the C and C++ compilers, linker and make tool, you should other
programs that make up the GNU binutils.
Several of these (nm, objdump, readelf) can parse the symbol tables of object files or shared or
static libraries. The simplest to use is probably nm. Assuming
that the bin directory is in your PATH, open a command prompt
in the directory containing the libraries or object files your are
interested in and run:
nm -u libfoo.a
or:
nm -u foo.obj
to list the undefined symbols in libfoo.a or in foo.obj.
If these files contain C++ symbols that you want to see demangled then
add -C to the nm options.
These tools all recognize that a static library libfoo.a is just
an archive of object files so nm ... libfoo.a gives you just the
same results as if libfoo.a was replaced with a list of the object
files within it.
Related
A cleaned up version of my compile command looks like gcc -o semantic_seg -Wl,-rpath,... -l:libnvrtc-5e8a26c9.so.10.1 ... and I have a dynamic library file named exactly libnvrtc-5e8a26c9.so.10.1 in the directory specified by the -rpath flag. The command succeeds.
When I go to run my program, it says semantic_seg: error while loading shared libraries: libnvrtc.so.10.1: cannot open shared object file: No such file or directory and when I do ldd it shows libnvrtc.so.10.1 => not found.
So the problem looks like the name of the shared library in the executable is not the same as the filename. Could that be true? Did the 5e8a26c9 part of the name somehow get stripped off?
Update: Creating a symbolic link libnvrtc-5e8a26c9.so.10.1 -> libnvrtc.so.10.1 allows the executable to run. But I'm still not sure the mechanism that causes this name modification to happen. It seems a bit magic.
Could that be true?
This is often true.
On ELF systems, the linker uses SONAME of the library (if it has one) and not its filename to record a runtime dependency.
Running readelf -d libnvrtc-5e8a26c9.so.10.1 | grep SONAME will likely show that in fact that library does have SONAME, and the value of SONAME is libnvrtc.so.10.1.
This used to be very convenient for external library versioning.
Update:
it allows you to link against a library which will be different than the one which will used at run time, but why would I ever want that?
Like I said, it's useful for external library versioning and ABI evolution.
Suppose you ship version 1.0 of libfoo.so. You provide libfoo.so as a symlink to libfoo.so.1, and you use libfoo.so.1 as SONAME.
Any program that links with gcc main.c -lfoo (note: no funny -l:libfoo.so.1 syntax required) will record libfoo.so.1 as its external dependency, and will use that name at runtime.
Time passes, and you are ready to ship version 2, which is not ABI-compatible.
You don't want to cause all your end-users to change their link line, and you don't want to break any existing binaries.
With the SONAME, this is trivial: new package will include libfoo.so.2 with SONAME of libfoo.so.2, and a symlink libfoo.so now pointing to libfoo.so.2.
Voila: both requirements are achieved: existing binaries continue to use libfoo.so.1, newly-linked binaries use libfoo.so.2, no Makefile changes required.
Edit: I resolved this issue, the solution is below.
I am building a code in a shared computing cluster dedicated for scientific computing, thus I can only control files in my home folder. Although I am using fftw as an example, I would like to understand the specific reason, why my attempt to setup LD_LIBRARY_PATH does not work.
I build the fftw and fftw_mpi libraries in my home folder like this
./configure --prefix=$HOME/install/fftw --enable-mpi --enable-shared
make install
It builds fine, but in install/fftw/lib, I find that the freshly built libfftw3_mpi.so links to wrong version of fftw library.
$ ldd libfftw3_mpi.so |grep fftw
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007f7df0979000)
If I now try to set the LD_LIBRARY_PATH correctly pointing to this directory, it still prefers the wrong library:
$ export LD_LIBRARY_PATH=$HOME/install/fftw/lib
$ ldd libfftw3_mpi.so |grep fftw
libfftw3.so.3 => /usr/lib64/libfftw3.so.3 (0x00007f32b4794000)
Only if I explicitly use LD_PRELOAD, I can override this behavior. I don't think LD_PRELOAD is a proper solution though.
$ export LD_PRELOAD=$HOME/install/fftw/lib/libfftw3.so.3
$ ldd libfftw3_mpi.so |grep fftw
$HOME/install/fftw/lib/libfftw3.so.3 (0x00007f5ca3d14000)
Here is what I would have expecting, a small test done in Ubuntu desktop, where I installed fftw to /usr/lib first, and then override this search path with LD_LIBRARY_PATH.
$ export LD_LIBRARY_PATH=
$ ldd q0test_mpi |grep fftw3
libfftw3.so.3 => /usr/lib/x86_64-linux-gnu/libfftw3.so.3
$ export LD_LIBRARY_PATH=$HOME/install/fftw-3.3.4/lib
$ ldd q0test_mpi |grep fftw3
libfftw3.so.3 => $HOME/install/fftw-3.3.4/lib/libfftw3.so.3
In short: Why is libfft3_mpi library still finding the wrong dynamic fftw3 library? Where is this searchpath hard coded in a such way that it is prioritized over LD_LIBARY_PATH? Why is this is not the case in another computer?
I am using intel compilers 13.1.2, mkl 11.0.4.183 and openmpi 1.6.2 if this matters.
Edit: Thanks for all the answers. With help of those, we were able to isolate the problem to RPATH, and from there, the cluster support was able to figure out the problem. I accepted the first answer, but both answers were good.
The reason, why this was so hard to figure out, is that we did not know that the compilers were actually wrapper scripts, adding things to compiler command line. Here a part of a reply from the support:
[The] compilation goes through our compiler wrapper. We do RPATH-ing
by default as it helps most users in correctly running their jobs
without loading LD-LIBRARY_PATH etc. However we exclude certain
library paths from default RPATH which includes /lib, /lib64 /proj
/home etc. Earlier the /usr/lib64 was not excluded by mistake
(mostly). Now we have added that path in the exclusion list.
From http://man7.org/linux/man-pages/man8/ld.so.8.html
When resolving shared object dependencies, the dynamic linker first
inspects each dependency string to see if it contains a slash (this
can occur if a shared object pathname containing slashes was
specified at link time). If a slash is found, then the dependency
string is interpreted as a (relative or absolute) pathname, and the
shared object is loaded using that pathname.
If a shared object dependency does not contain a slash, then it is
searched for in the following order:
o (ELF only) Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH
attribute does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH. Except if the
executable is a set-user-ID/set-group-ID binary, in which case it
is ignored.
o (ELF only) Using the directories specified in the DT_RUNPATH
dynamic section attribute of the binary if present.
o From the cache file /etc/ld.so.cache, which contains a compiled
list of candidate shared objects previously found in the augmented
library path. If, however, the binary was linked with the -z
nodeflib linker option, shared objects in the default paths are
skipped. Shared objects installed in hardware capability
directories (see below) are preferred to other shared objects.
o In the default path /lib, and then /usr/lib. (On some 64-bit
archiectures, the default paths for 64-bit shared objects are
/lib64, and then /usr/lib64.) If the binary was linked with the
-z nodeflib linker option, this step is skipped.
with readelf readelf -d libfftw3_mpi.so you can check if your lib contains such a attribute in the dynamic section.
with export LD_DEBUG=libs you can debug the search path used to find your libs
with chrpath -r<new_path> <executable> the rpath can be changed
I see two possible reasons for this.
First, libfftw3_mpi.so may be linked with /usr/lib64/ as RPATH. In that case, providing LD_LIBRARY_PATH will have no effect. To check if it is your case, run readelf -d libfftw3_mpi.so | grep RPATH and see if it has /usr/lib64/ as a library path. If it does, use chrpath utility to change or remove it.
Alternatively, you may be running a system that does not support LD_LIBRARY_PATH at all (like HP-UX).
For .so I use nm -gC, for .a I use just nm. What should I use for .la?
.la files are only auxiliary files used by libtool, so they contain no executable code and no symbols.
I'm building a library that needs to be dynamically linked to my project. The output is a .so file, so I think I'm on the right track. I'm concerned by the way it's being linked at compile time - by specifying the location of its makefile and depending on a bunch of macros, which I've never encountered before.
Can I assume that since I'm building a .so library (rather than a .a) that I'm in fact dynamically linking? Or is it possible for .so libs to be statically linked, in which case I need to rip apart the make/config files to better understand what's going on?
Thanks,
Andrew
I'm not familiar with internal structure of executables and shared objects, so I could only give some practical hints.
Assuming you use gcc, it should have -shared option when linking object files into library - this way ld (called by gcc) makes shared object instead of executable binary.
gcc -shared -o libabc.so *.o ...
When you link some application with this libabc.so it should link without errors and after that with ldd command you should be able to see libabc.so among its dependencies.
$ ldd app
...
libabc.so => ...............
How can I link a shared library function statically in gcc?
Refer to:
http://www.linuxquestions.org/questions/linux-newbie-8/forcing-static-linking-of-shared-libraries-696714/
You need the static version of the library to link it.
A shared library is actually an executable in a special format
with entry points specified (and some sticky addressing issues
included). It does not have all the information needed to
link statically.
You can't statically link a shared library (or dynamically link a static one).
The flag -static will force the linker to use static libraries (.a) instead of shared (.so) ones. But static libraries aren't always installed by default, so you may have to install the static library yourself.
Another possible approach is to use statifier or Ermine. Both tools take as input a dynamically linked executable and as output create a self-contained executable with all shared libraries embedded.
If you want to link, say, libapplejuice statically, but not, say, liborangejuice, you can link like this:
gcc object1.o object2.o -Wl,-Bstatic -lapplejuice -Wl,-Bdynamic -lorangejuice -o binary
There's a caveat -- if liborangejuice uses libapplejuice, then libapplejuice will be dynamically linked too.
You'll have to link liborangejuice statically alongside with libapplejuice to get libapplejuice static.
And don't forget to keep -Wl,-Bdynamic else you'll end up linking everything static, including libc (which isn't a good thing to do).
Yeah, I know this is an 8 year-old question, but I was told that it was possible to statically link against a shared-object library and this was literally the top hit when I searched for more information about it.
To actually demonstrate that statically linking a shared-object library is not possible with ld (gcc's linker) -- as opposed to just a bunch of people insisting that it's not possible -- use the following gcc command:
gcc -o executablename objectname.o -Wl,-Bstatic -l:libnamespec.so
(Of course you'll have to compile objectname.o from sourcename.c, and you should probably make up your own shared-object library as well. If you do, use -Wl,--library-path,. so that ld can find your library in the local directory.)
The actual error you receive is:
/usr/bin/ld: attempted static link of dynamic object `libnamespec.so'
collect2: error: ld returned 1 exit status
Hope that helps.
If you have the .a file of your shared library (.so) you can simply include it with its full path as if it was an object file, like this:
This generates main.o by just compiling:
gcc -c main.c
This links that object file with the corresponding static library and creates the executable (named "main"):
gcc main.o mylibrary.a -o main
Or in a single command:
gcc main.c mylibrary.a -o main
It could also be an absolute or relative path:
gcc main.c /usr/local/mylibs/mylibrary.a -o main
A bit late but ... I found a link that I saved a couple of years ago and I thought it might be useful for you guys:
CDE: Automatically create portable Linux applications
http://www.pgbovine.net/cde.html
Just download the program
Execute the binary passing as a argument the name of the binary you want make portable, for example: nmap
./cde_2011-08-15_64bit nmap
The program will read all of libs linked to nmap and its dependencias and it will save all of them in a folder called cde-package/ (in the same directory that you are).
Finally, you can compress the folder and deploy the portable binary in whatever system.
Remember, to launch the portable program you have to exec the binary located in cde-package/nmap.cde
Best regards
In gcc, this isn't supported. In fact, this isn't supported in any existing compiler/linker i'm aware of.