Sometimes when I do nm on a .so file (for example, libstdc++.so.6), it says no symbols, and I need to use nm --dynamic. But for some other .so files, I can see the symbols without --dynamic.
The doc says:
Display the dynamic symbols rather than the normal symbols. This is only meaningful for dynamic objects, such as certain types of shared libraries.
But it is confusing... what "types" of shared libraries need --dynamic? How is this determined? During the compilation of the library? I thought all shared libraries are dynamic (I mean, can be dynamically loaded during run time), but seems that my understanding is not quite right.
It could very well possible that if your symbol is not exported from your shared library, it would end up in the normal symbol table instead of the dynamic symbol table.
There are many types of symbols in ELF files.
symbols part of the Normal Symbol table. This is the output from mere nm libabc.so or objdump --syms libabc.so. These symbols are only used during static linking.
symbols part of the Dynamic Symbol table. This is the output from nm --dynamic libabc.so or objdump --dynamic-syms libabc.so. Dynamic symbol table is the one used by the run-time linker/loader which binds the symbols between the ELF file which refers them and the ELF file which defines them. It is also used by the static linker, while linking a shared library with an application which requires it. This is the component which helps in showing all the undefined symbol errors during linking.
Hidden symbols - these are the symbols which have been marked using _attribute_ ((visibility("hidden"))). These symbols are not exported outside and can only be used within the library. The visibility is checked during the linking step, and hence is enforced only for shared libraries. The default visibility is public, i.e. the symbols are exported unless otherwise specified. The behavior can be modified using the -fvisibility=default|internal|hidden|protected.
Set the default ELF image symbol visibility to the specified
option—all symbols will be marked with this unless overridden within
the code. Using this feature can very substantially improve linking
and load times of shared object libraries, produce more optimized
code, provide near-perfect API export and prevent symbol clashes. It
is strongly recommended that you use this in any shared objects you
distribute. Despite the nomenclature, default always means public ie;
available to be linked against from outside the shared object.
protected and internal are pretty useless in real-world usage so the
only other commonly used option will be hidden. The default if
-fvisibility isn't specified is default, i.e., make every symbol public—this causes the same behavior as previous versions of GCC.
An overview of these techniques, their benefits and how to use them is
at http://gcc.gnu.org/wiki/Visibility.
To answer your question when would you use the --dynamic option of nm, it would be when you want to list all the symbols exported by your shared library, and the only ones that are available to the ELF images which reference them.
You need to use --dynamic or -D option on a shared library if it is stripped and thus only contains a dynamic symbol table.
You may want to use this option for other shared libraries to explicitly display the dynamic symbol table as this is the table that is consulted by the dynamic linker.
The file utility indicates whether a shared library is stripped or not. Example:
$ file /usr/lib64/libcrypt-nss-2.26.so
[..] ELF 64-bit LSB shared object, x86-64 [..], not stripped
$ file /usr/lib64/libxml2.so.2.9.7
[..] ELF 64-bit LSB shared object, x86-64 [..], stripped
Example for how the different symbol tables may contain different symbols:
$ nm -D /usr/lib64/libcrypt-nss-2.26.so | wc -l
39
$ nm /usr/lib64/libcrypt-nss-2.26.so | wc -l
112
Related
For instance I have a program that reports undefined reference to 'XRenderFindVisualFormat'
Can I use something like a combination of find/grep/file ?
You can use nm (or nm -D) on a library do get the symbols defined or used there. See nm(1) and also objdump(1).
You can use ldd on some executable to understand which shared dynamic libraries it is linking. See ldd(1).
You can also search the web for XRenderFindVisualFormat
Remember that order of libraries and program options to gcc is significant.
What is the search order for symbol lookup when resolving dynamic relocations?
When resolving symbols for a shared library does the loader first search in the 'main executable' (to let the main executable override definitions...) or what?
Per my understanding, each executable object has its own "lookup scope":
The main executable is usually the first object in the "global" lookup scope. This means that symbols defined in the main executable would override those in dependent shared libraries. Shared objects that are added using the LD_PRELOAD facility are added to the global lookup scope, right after the main executable.
However, if the shared object being loaded uses the DF_SYMBOLIC flag, then symbol references that originate within that object will look for definitions within the object before searching in the global lookup scope.
Shared objects opened using dlopen() may have their own dependencies. If the RTLD_GLOBAL flag was not set during the call to dlopen(), these dependencies are added to the lookup scope for that object, but do not affect the global lookup scope. If the RTLD_GLOBAL flag was passed to dlopen(), then the shared object (and its dependencies) will be added to the "global" lookup scope, changing the behavior of subsequent symbol lookups.
Ulrich Drepper's guide "How to Write Shared Libraries" is recommended reading on this topic.
When resolving symbols for a shared library does the loader first search in the 'main executable' (to let the main executable override definitions...) or what?
Yes, exactly. The dynamic loader has a linked list of loaded ELF objects (the head of the list is _r_dynamic.r_map) and searches dynamic symbol tables of objects in that list linearly, until it finds the symbol definition it is looking for.
The head of the list always points to the main executable. If a given symbol is exported from the main executable, then it (almost) always "wins" (overrides other definitions).
However, note that -Bsymbolic linker flag changes the picture a bit.
I have an executable in linux - exe
This executable has some functions in it, that are used throughout the code:
sendMsg
debugPrint
I then want to dynamically load a .so that provides extra functionality to my executable.
In this shared library I include the headers for sendMsg and debugPrint.
I load this shared library with dlopen() and create an API with dlsym().
However, at dlopen() I use RTLD_NOW to resolve all symbols at load time.
It fails stating that it cannot find sendMsg symbol.
This symbol must be in the executable as the sendMsg.c is compiled in there.
However, my executable is stripped by the make process. As such, it would make sense that dlopen cannot find the symbol.
How can i solve this situation?
I could build the shared functions into a static library and link that static library into both exe and the .so. This would increase code size :(
I could remove the stripping of the exe so the symbols can be found
Do some compile time linking magic that I don't know about so the .so knows where the symbols are in exe
man ld:
-E
--export-dynamic
--no-export-dynamic
When creating a dynamically linked executable, using the -E option or the --export-dynamic option causes the linker to add all symbols to the dynamic symbol table. The
dynamic symbol table is the set of symbols which are visible from dynamic objects at run time.
If you do not use either of these options (or use the --no-export-dynamic option to restore the default behavior), the dynamic symbol table will normally contain only those
symbols which are referenced by some dynamic object mentioned in the link.
If you use "dlopen" to load a dynamic object which needs to refer back to the symbols defined by the program, rather than some other dynamic object, then you will probably
need to use this option when linking the program itself.
You can also use the dynamic list to control what symbols should be added to the dynamic symbol table if the output format supports it. See the description of
--dynamic-list.
Note that this option is specific to ELF targeted ports. PE targets support a similar function to export all symbols from a DLL or EXE; see the description of
--export-all-symbols below.
You can also pass the -rdynamic option to gcc/g++ (as noted int the comment). Depending on how you setup your make script, this will be convenient
Can anyone please suggest some way we can restrict exporting of our symbols to global symbol table?
Thanks in advance
Hi,
Thanks for replying...
Actually I have an executable which is statically linked to a third party library say "ver1.a" and also uses a third party ".so" file which is again linked with same library but different version say "ver2.a". Problem is implementation of both these versions is different. At the beginning, when executable is loaded, symbols from "ver1.a" will get exported to global symbol table. Now whenever ".so" is loaded it will try to refer to symbols from ver2.a, it will end up referring to symbols from "ver1.a" which were previously loaded.Thus crashing our binary.
we thought of a solution that we wont be exporting the symbols for executable to Global symbol table, thus when ".so" gets loaded and will try to use symbols from ver2.a it wont find it in global symbol table and it will use its own symbols i.e symbols from ver2.a
I cant find any way by which i can restrict exporting of symbols to global symbol table. I tried with --version-script and retain-symbol-file, but it didn't work. For -fvisibility=hidden option, its giving an error that " -f option may only be used with -shared". So I guess, this too like "--version-script" works only for shared libraries not for executable binaries.
code is in c++, OS-Linux, gcc version-3.2. It may not be possible to recompile any of the third party libraries and ".so"s. So option of recompiling "so' file with bsymbolic flag is ruled out.
Any help would be appreciated.
Pull in the 3rd party library with dlopen.
You might be able to avoid that by creating your own shared lib that hides all the third party symbols and only exposes your own API to them, but if all else fails dlopen gives you complete control.
I had, what sounds like, a similar issue/question: Segfault on C++ Plugin Library with Duplicate Symbols
If you can rebuild the 3rd party library, you could try adding the linker flag -Bsymbolic (the flag to gcc/g++ would be -Wl,-Bsymbolic). That might solve your issue. It all depends on the organization of your code and stuff, as there are caveats to using it:
http://www.technovelty.org/code/c/bsymbolic.html
http://software.intel.com/en-us/articles/performance-tools-for-software-developers-bsymbolic-can-cause-dangerous-side-effects/
If you can't rebuild it, according to the first caveat link:
In fact, the only thing the -Bsymbolic
flag does when building a shared
library is add a flag in the dynamic
section of the binary called
DT_SYMBOLIC.
So maybe there's a way to add the DT_SYMBOLIC flag to the dynamic section post-linking?
The simplest solution is to rename the symbols (by changing source code) in your executable so they don't conflict with the shared library in the first place.
The next simplest thing is to localize the "problem" symbols with 'objcopy -L problem_symbol'.
Finally, if you don't link directly with the third party library (but dlopen it instead, as bmargulies suggests), and none of your other shared libraries use of define the "problem" symbol, and you don't link with -rdynamic or one of its equivalents, then the symbol should not be exported to the dynamic symbol table of the executable, and thus you shouldn't have a conflict.
Note: 'nm a.out' will still, show the symbol as globally defined, but that doesn't matter for dynamic linking. You want to look at the dynamic symbol table of a.out with 'nm -D a.out'.
I'm writing some code that uses dynamic shared libraries as plugins.
My command line for building the shared libraries looks like:
cc -shared -fPIC -o module.so -g -Wall module.c
Within the module, I can call functions that are in any other shared library that has been loaded within the main executable.
However I cannot access (exported) functions that are in the executable itself (I get undefined symbol errors).
My call to dlopen looks like this:
void *handle = dlopen(plugin, RTLD_NOW);
Can anyone please advise how my module can call back to my executable, without having to put all of the executable's utility functions into yet another shared library?
Correct solution is to add -rdynamic to the link command of the main executable. This will add appropriate option to ld (which, when using GNU ld, happens to be --export-dynamic).
Adding --export-dynamic directly is technically incorrect: it's a linker option, and so should be added as -Wl,--export-dynamic, or -Wl,-E. This is also less portable than -rdynamic (other linkers have an equivalent, but the option itself is different).
I've found the answer myself.
I had to add the --export-dynamic flags to the link options for the main executable.
When creating a dynamically linked
executable, add all symbols to the
dynamic symbol table. The dynamic
symbol table is the set of symbols
which are visible from dynamic objects
at run time.
If you do not use this option, the
dynamic symbol table will normally
contain only those symbols which are
referenced by some dynamic object
mentioned in the link.
If you use "dlopen" to load a dynamic
object which needs to refer back to
the symbols defined by the program,
rather than some other dynamic object,
then you will probably need to use
this option when linking the program
itself.
When I encountered the same problem, I just used the following solution. Before loading any plugin, just load the program itself, bringing its symbols to dynamic tables:
dlopen(NULL,RTLD_NOW|RTLD_GLOBAL);
I think the solution is better. The reason is that, it also solves the same problem if you
a) your program (or a trird-party module) is linked (not in runtime) against the shared library, which symbols need to be in dynamic table;
b) can not recompile that module with -rdynamic flag.