Short:
I'm looking for something that will list all unresolved dependencies in an SO, taking into account the SOs that are in it's dependencies.
Long:
I'm converting a lot of static-compiled code to Shared Objects in Linux- is there an easy way to determine what other SOs my recently compiled SO is dependent on besides trial & error while trying to load it?
I'm sure there is a better way, but I haven't been able to find it yet.
I've found "ldd", but that only lists what the SO says it's dependent on.
I've also used "nm" to figure out once an SO fails to load to verify what other SO contains it.
I don't have code for you, but I can give pointers:
It's just a graph problem. You should use objdump -T to dump the dynamic symbol table for a given binary or shared object. You'll see many lines of output, and the flags can be a little confusing, but the important part if that symbols will either be *UND* or they'll have a segment name (.text etc).
Any time you see *UND*, that means that it's an undefined symbol which has to be resolved. Defined symbols are the targets of resolution.
With that, and a little Python, you should be able to find what you need.
"ldd -r foo.so" should print the set of symbols which foo.so needs but which aren't provided by its direct dependencies.
Alternatively, link foo.so like this:
gcc -shared -o foo.so foo.o bar.o -ldep1 -ldep2 -Wl,--no-undefined
This should fail (to link) if foo.o or bar.o uses something not provided by libdep1 or libdep2 or libc.
Related
For instance I have a program that reports undefined reference to 'XRenderFindVisualFormat'
Can I use something like a combination of find/grep/file ?
You can use nm (or nm -D) on a library do get the symbols defined or used there. See nm(1) and also objdump(1).
You can use ldd on some executable to understand which shared dynamic libraries it is linking. See ldd(1).
You can also search the web for XRenderFindVisualFormat
Remember that order of libraries and program options to gcc is significant.
I have a static lib (my_static_lib) which I link to an executable binary file. Some of the symbols, but not all, are used in my binary.
A second library, dynamically loaded(my_shared_lib), is expecting to receive some symbols from my_static_lib through symbol injection from the binary. But those symbols are not used by my_binary, so they are stripped off the final bin file.
So, at runtime, my_shared_lib complains that it cannot find __my_stripped_symbols__ and crashes.
Is there a way to force the linker to keep __my_stripped_symbols__? I would prefer something that can be cleanly written in a Makefile.am (autotools)
(-binary file makefile)
-L$(top_builddir)/static_lib -lmy_static_lib --magic-flag-to-keep-stripped-symbol
I do not want to link my_static_lib with my_shared_lib because it will generate strange conflicts in other parts of a rather complex group of executables/shared libraries.
When you link my_static_lib to your application, you want to use the --whole-archive option. It's documented in the ld options docs.
If you're linking with gcc, it looks something like this:
-L$(top_builddir)/static_lib -Wl,-whole-archive -lmy_static_lib -Wl,-no-whole-archive
That will make sure the entire library is kept, and not just the specific functions that your executable uses.
You also need to make sure that the symbols get exported. If the symbols from your static library aren't being exported already, you make need a combination of -fvisibility=hidden and use __attribute__ ((visibility("default"))) to mark up the ones you want exported. You can read a little more about it in the gcc docs
I have a plug-in in the form of a shared library (bar.so) that links into a larger program (foo). Both foo and bar.so depend on the same third party library (baz) but they need to keep their implementations of baz completely separate. So when I link foo (using the supplied object files and archives) I need it to ignore any use of baz in bar.so and vice versa.
Right now if I link foo with --trace-symbol=baz_fun where baz_fun is one of the offending symbols I get the following output:
bar.so: definition of baz_fun
foo/src.a(baz.o): reference to baz_fun
I believe this is telling me that foo is referencing baz_fun from bar.so (and execution of foo confirms this).
Solutions that I have tried:
Using objcopy to "localize" the symbols of interest: objcopy --localize-symbols=local.syms bar.so where local.syms contains all of the symbols of interest. I think I might just be confused here and maybe "local" doesn't mean what I think it means. Regardless, I get the same output from the link above. I should note that if I run the nm tool on bar.so prior to using objcopy all of the symbols in question have the T flag (upper-case indicating global) and after objcopy they have a t indicating they are local now. So it appears I am using objcopy correctly.
Compiling with -fvisibility=hidden however due to some other constraints I need to use GCC 3.3 which doesn't appear to support that feature. I might be able to upgrade to a newer version of GCC but would like confirmation that compiling with this flag will help me before heading down that road.
Other things to note:
I do not have access to the source code of either foo or baz
I would prefer to keep all of my plug-in in one shared object (bar.so). baz is actually a licensing library so I don't want it separated
Use dlopen to load your plugin with RTLD_DEEPBIND flag.
(edit)
Please note that RTLD_DEEPBIND is Linux-specific and need glibc 2.3.4 or newer.
Can anyone please suggest some way we can restrict exporting of our symbols to global symbol table?
Thanks in advance
Hi,
Thanks for replying...
Actually I have an executable which is statically linked to a third party library say "ver1.a" and also uses a third party ".so" file which is again linked with same library but different version say "ver2.a". Problem is implementation of both these versions is different. At the beginning, when executable is loaded, symbols from "ver1.a" will get exported to global symbol table. Now whenever ".so" is loaded it will try to refer to symbols from ver2.a, it will end up referring to symbols from "ver1.a" which were previously loaded.Thus crashing our binary.
we thought of a solution that we wont be exporting the symbols for executable to Global symbol table, thus when ".so" gets loaded and will try to use symbols from ver2.a it wont find it in global symbol table and it will use its own symbols i.e symbols from ver2.a
I cant find any way by which i can restrict exporting of symbols to global symbol table. I tried with --version-script and retain-symbol-file, but it didn't work. For -fvisibility=hidden option, its giving an error that " -f option may only be used with -shared". So I guess, this too like "--version-script" works only for shared libraries not for executable binaries.
code is in c++, OS-Linux, gcc version-3.2. It may not be possible to recompile any of the third party libraries and ".so"s. So option of recompiling "so' file with bsymbolic flag is ruled out.
Any help would be appreciated.
Pull in the 3rd party library with dlopen.
You might be able to avoid that by creating your own shared lib that hides all the third party symbols and only exposes your own API to them, but if all else fails dlopen gives you complete control.
I had, what sounds like, a similar issue/question: Segfault on C++ Plugin Library with Duplicate Symbols
If you can rebuild the 3rd party library, you could try adding the linker flag -Bsymbolic (the flag to gcc/g++ would be -Wl,-Bsymbolic). That might solve your issue. It all depends on the organization of your code and stuff, as there are caveats to using it:
http://www.technovelty.org/code/c/bsymbolic.html
http://software.intel.com/en-us/articles/performance-tools-for-software-developers-bsymbolic-can-cause-dangerous-side-effects/
If you can't rebuild it, according to the first caveat link:
In fact, the only thing the -Bsymbolic
flag does when building a shared
library is add a flag in the dynamic
section of the binary called
DT_SYMBOLIC.
So maybe there's a way to add the DT_SYMBOLIC flag to the dynamic section post-linking?
The simplest solution is to rename the symbols (by changing source code) in your executable so they don't conflict with the shared library in the first place.
The next simplest thing is to localize the "problem" symbols with 'objcopy -L problem_symbol'.
Finally, if you don't link directly with the third party library (but dlopen it instead, as bmargulies suggests), and none of your other shared libraries use of define the "problem" symbol, and you don't link with -rdynamic or one of its equivalents, then the symbol should not be exported to the dynamic symbol table of the executable, and thus you shouldn't have a conflict.
Note: 'nm a.out' will still, show the symbol as globally defined, but that doesn't matter for dynamic linking. You want to look at the dynamic symbol table of a.out with 'nm -D a.out'.
I'm writing some code that uses dynamic shared libraries as plugins.
My command line for building the shared libraries looks like:
cc -shared -fPIC -o module.so -g -Wall module.c
Within the module, I can call functions that are in any other shared library that has been loaded within the main executable.
However I cannot access (exported) functions that are in the executable itself (I get undefined symbol errors).
My call to dlopen looks like this:
void *handle = dlopen(plugin, RTLD_NOW);
Can anyone please advise how my module can call back to my executable, without having to put all of the executable's utility functions into yet another shared library?
Correct solution is to add -rdynamic to the link command of the main executable. This will add appropriate option to ld (which, when using GNU ld, happens to be --export-dynamic).
Adding --export-dynamic directly is technically incorrect: it's a linker option, and so should be added as -Wl,--export-dynamic, or -Wl,-E. This is also less portable than -rdynamic (other linkers have an equivalent, but the option itself is different).
I've found the answer myself.
I had to add the --export-dynamic flags to the link options for the main executable.
When creating a dynamically linked
executable, add all symbols to the
dynamic symbol table. The dynamic
symbol table is the set of symbols
which are visible from dynamic objects
at run time.
If you do not use this option, the
dynamic symbol table will normally
contain only those symbols which are
referenced by some dynamic object
mentioned in the link.
If you use "dlopen" to load a dynamic
object which needs to refer back to
the symbols defined by the program,
rather than some other dynamic object,
then you will probably need to use
this option when linking the program
itself.
When I encountered the same problem, I just used the following solution. Before loading any plugin, just load the program itself, bringing its symbols to dynamic tables:
dlopen(NULL,RTLD_NOW|RTLD_GLOBAL);
I think the solution is better. The reason is that, it also solves the same problem if you
a) your program (or a trird-party module) is linked (not in runtime) against the shared library, which symbols need to be in dynamic table;
b) can not recompile that module with -rdynamic flag.