I am building a shared object (.so), and I am linking it against a third party static library. In an ideal world, that third party library would have been compiled with -fvisibility=hidden - but it hasn't been.
Is there a way to avoid including the symbols defined by that static library in the global symbols exported by the dynamic library?
(I am using gcc on the linux platform if it matters)
You can control symbols that get exported via -Wl,--version-script=<version-script>. Version script would look like
LIBMYLIB_1.0 {
global:
libmylib_foo; libmylib_bar;
local:
*;
};
(library symbols will get filtered in local section).
Note that you should still use hidden visibility wherever possible (even if you enable version scripts) as it allows compiler to optimize better.
Related
On linux I am trying to create a shared library, libbar.so, that embeds a commercial static library (licensing is fine). The commercial library has 4 versions: libfoo-seq.a, libfoo-mt.a, libfoo-seq.so, and libfoo-mt.so (they all provide the same symbols, just the code is sequential/multi-threaded, and the lib is static/shared). Of these four I want my code always to use the sequential foo library, so when I create libbar.so I link together my object files and libfoo-seq.a.
The problem is that the users of my library may have already pulled in libfoo-mt.so by the time they pull in my libbar.so, thus all symbols from libfoo are already present by the time libbar.so is read in, so my calls to the functions in foo are resolved to the multithreaded version.
I wonder how can I resolve this issue? What kind of magic flags do I need to use when I compile to create my object files and when I link my object files with libfoo-seq.a to create libbar.so?
You can hide libfoo's symbols in libbar via version script:
$ cat libbar.map
{
global: libbar_*;
local: libfoo_*;
};
$ gcc ... -o libbar.so -Wl,--version-script=libbar.map
I have a problem with an embedded linux C++ application I've written that consists of an executable and a dynamically linked library. The executable calls a function that is one of the entry points in the library, but that function misbehaves. I've investigated using gdb, and find that the library function, which is supposed to make a call to another function xyz() within the library, actually calls a function of the same name xyz()within the executable.
I'm very surprised this can happen, so maybe I'm doing something stupid. Isn't the library linked within itself without reference to the executable? If the executable wrongly made a call to abc() in the library instead of abc() in the executable that would make slightly more sense, because it is at least linked with the library, although in that case would the linker spot the dual definition? Or prioritise the local function?
I could just rename my functions so none of them have matching names, but I'd like to understand what is going on. I don't have much experience in this area, or with the gcc tools. Firstly, is what I think is happening in the above scenario even possible?
Both the executable and the library make calls to another library.
The link command for the library I'm using is:
powerpc-unknown-linux-gnuspe-g++-4.9.3 aaa.o bbb.o [etc] -shared -o libmylibary.so -L ../otherlibpath -Wl,-rpath-link,../otherlibpath -lotherlibname
That is way how the dynamic linker works. The symbols in executable have higher priority then symbols in dynamic libraries. Dynamic library designer must be aware about it. She must do measures to avoid unwanted symbol mismatch. Most libraries use:
In case of C++ use namespaces. All symbols exported from library should be in a library namespace.
In case of C use a name prefix or suffix for all exported symbol. For example OpenSSL library uses the prefix SSL_ and the public functions have names like SSL_set_mode() so the unwanted symbol collision is avoided.
Do not export symbols from the library that are supposed to be private. If the symbol is not exported from the library then the dynamic linker use the local symbol in the library. #pragma visibility is your friend. See https://gcc.gnu.org/wiki/Visibility
If the library with duplicate symbols is a 3rd party library and its author does not follow the recommendations above then you have to rename your function or perhaps ask the author for a library update.
EDIT
Export/do not export may be controlled by #pragma visibility directive (gcc specific extension):
void exported_function1(int);
void exported_function2(int);
#pragma GCC visibility push(hidden)
void private_function1(int);
void private_function2(int);
#pragma GCC visibility pop
Detail at the link above.
When linking against libraries using the -l option (say -lfoo), gcc will prefer a shared object to a static library if both are found (will prefer libfoo.so to libfoo.a). Is there a way to make gcc prefer the static library, if both are found?
The issue I'm trying to solve is the following: I'm creating a plugin for an application (the flight simulator called X-Plane), with the following constraints:
the plugin is to be in the form of a 32 bit shared object, even when running on a 64 bit system
the running environment does not provide a convenient way to load shared objects which are not in the 'normal' locations, say /usr/lib or /usr/lib32:
one cannot expect the user to set LD_PRELOAD or LD_LIBRARY_PATH to find shared objects shipped with my plugin
the X-Plane running environment would not add my plugins directory to ``LD_LIBRARY_PATH, before dynamically loading the plugin shared object, which would allow me to ship all my required shared objects alongside my plugin shared object
one cannot expect 64 bit users to install 32 bit shared objects that are non-trivial (say, are not included in the ia32-libs package on ubuntu)
to solve the above constraints, a possible solution is to link the generated shared object against static, 32 bit versions of all non-trivial libraries used. but, when installing such libraries, usually both static and dynamic versions are installed, and thus gcc will always link against the shared object instead of the static library.
of course, moving / removing / deleting the shared objects in question, and just leaving the static libraries in say /usr/lib32, is a work-around, but it is not a nice one
note:
yes, I did read up on how to link shared objects & libraries, and I'm not trying to creatae a 'totally statically linked shared object'
yes, I tried -Wl,-static -lfoo -Wl,-Bdynamic, but didn't bring the expected results
yes, I tried -l:libfoo.a as well, but this didn't bring the expected results either
You can specify the full path to the static libs without the -l flag to link with those.
gcc ... source.c ... /usr/lib32/libmysuperlib.a ...
Just add the .a file to the link line without -l as if it were a .o file.
It's dated, but may work: http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html
(almost end of the page)
"As noted earlier, it is also possible to link directly with individual library files by specifying the full path to the library on the command line."
In every platform there are various versions of a given library: multi-threaded, debug, dynamic, etc..
Correct me if I am wrong here, but in Linux an object can link to any version of a library just fine, regardless of how its compiled. For example, there is no need to use any special flags at compile time to specify whether the link will eventually be to a dynamic or a static version of the run-time libraries (clarification: I am not talking about creating dynamic/static libraries, I am talking about linking to them - so -fPIC doesn't apply). Same goes for debug or optimized version of libraries.
Why in MSVC (Windows in general with other compilers. true?) I need to recompile the code every time in order to link to different versions of libraries? I am talking the /MD, /MT, /MTd, /MDd, etc flags. Is the code actually using different system headers each time. If so, why?
I would really appreciate any pointers to solid documentation that discusses these library matters in Windows for a C/C++ programmer..
thanks!
The compiler setting does very little other than simple change some macro definitions. Its microsoft's c-runtime header files that change their behaviour based on the runtime selected.
First, the header files use a # pragma directive to embed in the object file a directive specifying which .lib file to include, choosing one of: msvcrt.lib, msvcrtd.lib, libcmt.lib and mibcmtd.lib
The directives look like this
#ifdef <release dll runtime>
#pragma comment(lib,"msvcrt.lib")
#endif
Next, it also modifies a macro definition used on all c-rt functions that adds the __declspec(dllimport) directive if a dll runtime was selected. the effect of this directive is to change the imported symbol from, say, '_strcmp' to '__imp__strcmp'.
The dll import libraries (msvcrt.lib and msvcrtd.lib) export their symbols (to the linker) as __imp_<function name>, which means that, in the Visual C++ world, once you have compiled code to link against the dll runtimes you cannot change your mind - they will NOT link against a static runtime.
Of course, the reverse is not the case - dll import libraries actually export their public symbols both ways: with and without the __imp_ prefix.
Which means that code built against a static runtime CAN be later co-erced into linking with the dll or static runtimes.
If you are building a static library for other consumers, you should ensure that your compiler settings include:
One of the static library settings, so that consumers of your .lib can choose themselves which c-runtime to use, and
Set the 'Omit Default Library Name' (/Zl)flag. This tells the compiler to ignore the #pragma comment(lib,... directives, so the obj files and resulting lib does NOT have any kind of implicit runtime dependency. If you don't do this, users of your lib who choose a different runtime setting will see confusing messages about duplicate symbols in libc.lib and msvcrt.lib which they will have to bypass by using the ignore default libraries flag.
These using these compiler options have two effects. The automatically #define a macro that may be used by header files (and your own code) to do different things. This effects only a small part of the C runtime, and you can check the headers to see if it's happening in your case.
The other thing is that the C++ compiler embeds a comment in your object file that tells the linker to automatically include a particular flavor of the MSVC runtime, whether you specify that library at link time or not.
This is convenient for small programs, where you simply type at a command prompt cl myprogram.cpp to compile and link, producing myprogram.exe.
You can defeat automatic linking of the commented-in flavor of the c-runtime by passing /nodefaultlib to the linker. And then specify a different flavor of the c-runtime instead. This will work if you are careful not to depend on the #defines for _MT and
_DLL (keep in mind that the standard C headers might be looking at these also).
I don't recommend this, but if you have a reason to need to do this, it can be made to work in most cases.
If you want to know what parts of the C header files behave differently, you should just search for _MT and _DLL in the headers and see.
All of the options use the same header files, however they all imply different #define which affect the header files. So they need to be recompiled.
The switches also link to the appropriate library, but the recompile is not because of the linking.
See here for a list of what is defined when you use each.
Can anyone please suggest some way we can restrict exporting of our symbols to global symbol table?
Thanks in advance
Hi,
Thanks for replying...
Actually I have an executable which is statically linked to a third party library say "ver1.a" and also uses a third party ".so" file which is again linked with same library but different version say "ver2.a". Problem is implementation of both these versions is different. At the beginning, when executable is loaded, symbols from "ver1.a" will get exported to global symbol table. Now whenever ".so" is loaded it will try to refer to symbols from ver2.a, it will end up referring to symbols from "ver1.a" which were previously loaded.Thus crashing our binary.
we thought of a solution that we wont be exporting the symbols for executable to Global symbol table, thus when ".so" gets loaded and will try to use symbols from ver2.a it wont find it in global symbol table and it will use its own symbols i.e symbols from ver2.a
I cant find any way by which i can restrict exporting of symbols to global symbol table. I tried with --version-script and retain-symbol-file, but it didn't work. For -fvisibility=hidden option, its giving an error that " -f option may only be used with -shared". So I guess, this too like "--version-script" works only for shared libraries not for executable binaries.
code is in c++, OS-Linux, gcc version-3.2. It may not be possible to recompile any of the third party libraries and ".so"s. So option of recompiling "so' file with bsymbolic flag is ruled out.
Any help would be appreciated.
Pull in the 3rd party library with dlopen.
You might be able to avoid that by creating your own shared lib that hides all the third party symbols and only exposes your own API to them, but if all else fails dlopen gives you complete control.
I had, what sounds like, a similar issue/question: Segfault on C++ Plugin Library with Duplicate Symbols
If you can rebuild the 3rd party library, you could try adding the linker flag -Bsymbolic (the flag to gcc/g++ would be -Wl,-Bsymbolic). That might solve your issue. It all depends on the organization of your code and stuff, as there are caveats to using it:
http://www.technovelty.org/code/c/bsymbolic.html
http://software.intel.com/en-us/articles/performance-tools-for-software-developers-bsymbolic-can-cause-dangerous-side-effects/
If you can't rebuild it, according to the first caveat link:
In fact, the only thing the -Bsymbolic
flag does when building a shared
library is add a flag in the dynamic
section of the binary called
DT_SYMBOLIC.
So maybe there's a way to add the DT_SYMBOLIC flag to the dynamic section post-linking?
The simplest solution is to rename the symbols (by changing source code) in your executable so they don't conflict with the shared library in the first place.
The next simplest thing is to localize the "problem" symbols with 'objcopy -L problem_symbol'.
Finally, if you don't link directly with the third party library (but dlopen it instead, as bmargulies suggests), and none of your other shared libraries use of define the "problem" symbol, and you don't link with -rdynamic or one of its equivalents, then the symbol should not be exported to the dynamic symbol table of the executable, and thus you shouldn't have a conflict.
Note: 'nm a.out' will still, show the symbol as globally defined, but that doesn't matter for dynamic linking. You want to look at the dynamic symbol table of a.out with 'nm -D a.out'.