I've always been confused about how the linker works, and it's a difficult subject to search for.
To demonstrate my question and to provide a framework for an answer, I'll put down what I know (or think I know) so far. I may be very wrong. :)
First, each .cpp file is built into an intermediate file (.o for Posix/ELF and .obj for Win/PE I believe). This intermediate file contains all the symbols defined by the .cpp it was built from and has instructions for what external links it needs to be properly resolved. As an extension to this, Posix systems let you combine the .o files into a .a file (Which doesn't seem to do anything more than combine? What command does this?). Is .lib the Win/PE equivalent of the Posix .a file?
Next, the intermediate files are linked together, external dependencies resolved, and you have your executable. Am I missing any steps?
Thanks!
Here's a few pieces of the puzzle:
ar(1) is used to create .a files. They are similar to tar(1) or zip files (possibly
with a index to look up an object file by symbol name)
The linker copies together the sections of object files (text, data, bss). For GNU ld, the precise copying of sections can be controlled with a linker script (e.g. copy all sections from .o files containing "text" in their names into a single text section)
The linker also does relocations: patching instructions (jump and data load) with the respective target addresses, once the value of a symbol is known. In some cases, this can't be done at link time, so the linker copies/adjusts the relocation records from the .o files into the final executable.
the windows .lib serves two purposes: a static library (.lib) is similar to .a libraries. An import library (.lib) does not contain the actual code, but only symbol lists. The linker can resolve symbols from the import library, but then knows it needs to put a reference to the corresponding .dll into the executable. On Unix/ELF, the .so file has both the code and the symbol table.
Related
I have a libA.so that depends on libB.so and having trouble finding it even though it's in the same directory.
ldd libA.so
linux-vdso.so.1 (0x00007fff50bdb000)
libB.so => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4aeb902000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4aebadb000)
I'm wondering if there is a way to make libA.so always look for libB.so in the same directory as this will be the case for my application? I know updating LD_LIBRARY_PATH is an option as well but wanted to reduce the amount of work required.
The .dynamic section of an ELF file (.so libraries on Linux use ELF format) contains information to help the library find its dependencies. .dynamic entries with type DT_NEEDED contain the names of other .so files for the dynamic linker to find, but they do not contain any information on where to find those files. For that, as you mentioned, you can use LD_LIBRARY_PATH, but the ELF format also provides a way to specify it in the file itself.
A .dynamic entry with type DT_RUNPATH gives the dynamic linker a path to a directory where the dynamic linker should look for DT_NEEDED files. DT_RUNPATH allows a special variable, $ORIGIN, which refers to the file's current directory. This allows you to use relative paths, without requiring the user to invoke an executable from a specific working directory.
You use the -rpath linker flag to specify a DT_RUNPATH entry. In order to pass the literal string $ORIGIN, however, you must wrap it in single quotes to prevent your shell from interpreting it as an environment variable.
Assuming you are using gcc, you should use add this argument to the link step:
-Wl,-rpath,'$ORIGIN'
From 'man 8 ld.so':
If a shared object dependency does not contain a slash, then it
is searched for in the following order:
o Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH
attribute does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH, unless the
executable is being run in secure-execution mode (see below),
in which case this variable is ignored.
o Using the directories specified in the DT_RUNPATH dynamic
section attribute of the binary if present. Such directories
are searched only to find those objects required by DT_NEEDED
(direct dependencies) entries and do not apply to those
objects' children, which must themselves have their own
DT_RUNPATH entries. This is unlike DT_RPATH, which is applied
to searches for all children in the dependency tree.
o From the cache file /etc/ld.so.cache, which contains a
compiled list of candidate shared objects previously found in
the augmented library path. If, however, the binary was
linked with the -z nodeflib linker option, shared objects in
the default paths are skipped. Shared objects installed in
hardware capability directories (see below) are preferred to
other shared objects.
o In the default path /lib, and then /usr/lib. (On some 64-bit
architectures, the default paths for 64-bit shared objects are
/lib64, and then /usr/lib64.) If the binary was linked with
the -z nodeflib linker option, this step is skipped.
The key here is to use the DT_RUNPATH, which can be embedded into the binary you are creating. You can link it so that it points to the same directory, as you wanted.
See this post about how to do this: https://stackoverflow.com/a/67131878
On the top of the answers provided, it is also possible to change DT_RUNPATH using patchelf:
patchelf --set-rpath . <your-binary>
Obviously be careful to understand security implications of allowing to load libraries from local directory.
patchelf allows to change DT_RUNPATH without recompiling or re-linking so may be convenient if you want to do this and do not have source or do not want to deal with re-compiling (which may be actually painful process on distros like Alpine).
I want to build statically program with GCC/G++ without shared dependencies. but i don't know how to do that.
With below command in Netbeans IDE i can build with shared dependency, but in some OS can not find this library(i don't want to install on new system)
-Wl,--dynamic-linker=/usr/lib/libexpat.so.1
To statically link a program, you need a static library, which is a library with a filename finished in .a.
Linker, by default, if using default search paths (as you do with /usr/lib), will select the .so library version and will do a dynamic link of it, so if you want to specify that you want some static library, you'll need to specify the full path name of it, instead of using the -l option. So,
gcc -o your_program mod_a.o mod_b.o ... /usr/lib/libexpat.a
is better than
gcc -o your_program mod_a.o mod_b.o ... -lexpat
(the latter will select the file /usr/lib/libexpat.so instead, which should be a link to /usr/lib/libexpat.so.1, which is normally the soname of the library, and is also a symbolic link of /usr/lib/libexpat.so.1.xx.xx)
NOTE
In the examples, I'm trying to call the linker through the compiler, as the default c runtime and libraries are automatically selected by the compiler when calling this way. If you prefer to call directly the linker, the procedure doesn't change, but then you have to add the C runtime module and the standard c library yourself.
NOTE 2
If you want to statically link everything, then you have to use static versions of all the libraries you are going to use (they are normally installed in the same directory as the dynamic ones, so you have to specify the full pathname of all in the command line) To cope with this in a permanent development system, you can make symbolic links to them all from another path and then specify that directory as the search path for your projects that must be statically linked.
If you allways want some library to be statically linked, just erase the .so link (not the .so.X and the .so.X.YY links, they are not tried by the compiler) in /usr/lib, and the .a file will be selected by default by the compiler. Of course, if you want this made for every library, you can erase all the .so links, but you'll end with larger executables (much larger) than the original dynamically linked versions.
I'm trying to link a C++ binary, but I get undefined symbol errors. My binary shouldn't need those symbols, and I'd like to understand the dependency chain causing the linker (GNU ld or GNU gold) think that they are needed. There is libfoo.a containing hundreds of .o files. My program is calling function in libfoo.a. I'd like to get a dependency graph containing all .o files in libfoo.a which the linker thinks are needed to link my program.
I need it because I suspect that there is a mistake somewhere in libfoo.a, calling functions which are not really needed. I can modify the source code of libfoo.a (and thus remove the unneeded calls), and for that I need to understand where the unneeded calls are. The dependency graph could help me find it.
Please note that there is no resulting executable, because of the undefined symbols.
Please note that my ultimate goal is not to build this particular binary, but to make sure that unneeded functions are not called in libfoo.a.
I've looked at man ld, but I couldn't find any command-line flag that could give me more verbose output.
Example error from the linker:
libfoo++.a(foo1.o):foo1.cc:function foo1f: error: undefined reference to 'bar'
How do I figure out what caused foo1.o to be linked to the executable? (It's OK for me that bar is undefined, because I don't need it. My problem is that foo1.o is needed, but it shouldn't be, and I'd like to remove the call which caused foo1.o to be linked in.)
I'd like to get a dependency graph containing all .o files in libfoo.a which the linker thinks are needed to link my program.
The linker map, printed with -M (or --print-map) flag contains exactly that info. If you are using compiler driver (e.g. gcc) to perform the link (you should), then add -Wl,-M to the link line.
I am working on a very complex Linux driver. I am moving some files around trying to "cut the fat" a little bit. I have 4 "undefined!" symbols at the linking stage. I have searched my source code and I really don't know where these symbols are getting included.
I thought that if I did an objdump -DS on my compiled .ko and searched for the symbols, I might be able to tell from the surrounding decompiled source where they were coming from. But, when I search for the symbols, they aren't even found!
Is there a way to tell WHERE or HOW undefined symbols are getting included?
Ok, I figured out a good way to do this.
I used nm -u to list the undefined symbols from my object files. (the .o files)
nm - list symbols from object files
Doing that one by one on each .o and searching for the symbols I knew were undefined showed me what file they were coming from. Then I had a starting place for finding the symbol in my actual .c file, or a header file included from that .c file.
Now it's not a needle in a haystack, but rather a needle in a chunk of hay that broke off of the haystack.
If I have two libraries with the same library name but stored in different directories (and they may contain different code) and I list both directories in the LDFLAGS variable in a makefile, how does the linker know where to look first and which library to use?
LDFLAGS+= \
-L${INSTALL_DIR}/lib\
-L${EVO_INSTALL_DIR}/lib\
Will it look in the INSTALL_DIR path first or in the EVO_INSTALL_DIR path?
INSTALL_DIR. It will look in the order they are listed.
By the way, it's your linker (probably the same program as your compiler) that's making this choice, not the Makefile. Make (which is reading your Makefile) only runs the build tools.