On linux I am trying to create a shared library, libbar.so, that embeds a commercial static library (licensing is fine). The commercial library has 4 versions: libfoo-seq.a, libfoo-mt.a, libfoo-seq.so, and libfoo-mt.so (they all provide the same symbols, just the code is sequential/multi-threaded, and the lib is static/shared). Of these four I want my code always to use the sequential foo library, so when I create libbar.so I link together my object files and libfoo-seq.a.
The problem is that the users of my library may have already pulled in libfoo-mt.so by the time they pull in my libbar.so, thus all symbols from libfoo are already present by the time libbar.so is read in, so my calls to the functions in foo are resolved to the multithreaded version.
I wonder how can I resolve this issue? What kind of magic flags do I need to use when I compile to create my object files and when I link my object files with libfoo-seq.a to create libbar.so?
You can hide libfoo's symbols in libbar via version script:
$ cat libbar.map
{
global: libbar_*;
local: libfoo_*;
};
$ gcc ... -o libbar.so -Wl,--version-script=libbar.map
Related
I am building a shared object (.so), and I am linking it against a third party static library. In an ideal world, that third party library would have been compiled with -fvisibility=hidden - but it hasn't been.
Is there a way to avoid including the symbols defined by that static library in the global symbols exported by the dynamic library?
(I am using gcc on the linux platform if it matters)
You can control symbols that get exported via -Wl,--version-script=<version-script>. Version script would look like
LIBMYLIB_1.0 {
global:
libmylib_foo; libmylib_bar;
local:
*;
};
(library symbols will get filtered in local section).
Note that you should still use hidden visibility wherever possible (even if you enable version scripts) as it allows compiler to optimize better.
In our build system, we generate multiple .so files (foo.so, bar.so, ...) that are loaded during runtime by the main executable (biz). So the .so files are linked separately.
We also have our own util.a static library, that has some utility functions and global data.
The problem comes when some of the .so want to use util.a data/function, but we can't link each .so to util.a. It's because of the data section: global data must be unique in the program address space. If more than one .so is linked to util.a and has a copy of the data, the program behavior will be very surprising but hard to debug.
We can't link executable (biz) to util.a either. The linker will not put everything to the target, since biz doesn't reference the functions on behalf of .so.
Of course, unless linking util.a with -Wl,-whole-archive. But is there a better way to do this?
Solution 1: consider making util.a a dynamic library util.so.
Solution 2: don't let the linker export any symbols exported by util.a. When using gcc you can achieve this for example by using __attribute__((visibility("hidden"))):
int __attribute__((visibility("hidden"))) helperfunc(void *p);
You can use objdump to check which symbols are exported.
To answer myself's question, the eventual solution was like:
http://lists.gnu.org/archive/html/qemu-devel/2014-09/msg00099.html
TL;DR: Search for all the interesting symbols (that you want to pull from archives) inside the .so objects with nm (1), and inject into the compiling command line with -Wl,-u,$SYMBOL. Note that the -Wl,-u,$SYMBOL arguments need to come before archive names in the command line, so the linker knows that it needs to link them.
I am a cmake newbie (on Debian/Sid/Linux/x86-64)
I forked libonion on https://github.com/bstarynk/onion to enable customization of malloc with Boehm's garbage collector; see this mail thread.
I added two files there onion/src/low_util.c and onion_src/low_util.h (which is #include-d successfully in several other patched files.
It is compiled but not linked.
set(SOURCES onion.c codecs.c dict.c low_util.c request.c response.c handler.c
log.c sessions.c sessions_mem.c shortcuts.c block.c mime.c url.c ${POLLER_C}
listen_point.c request_parser.c http.c ${HTTPS_C} websocket.c ${RANDOM_C} ${SQLITE3_C})
later:
SET(INCLUDES_ONION block.h codecs.h dict.h handler.h http.h https.h listen_point.h low_util.h log.h mime.h onion.h poller.h request.h response.h server.h sessions.h shortcuts.h types.h types_internal.h url.h websocket.h ${SQLITE3_H})
MESSAGE(STATUS "Found include files ${INCLUDES_ONION}")
but when I build, my file low_util.c got compiled but not linked.
Linking C executable otemplate
CMakeFiles/opack.dir/__/__/src/onion/dict.c.o: In function `onion_dict_new':
dict.c:(.text+0x1bc): undefined reference to `onionlow_calloc'
CMakeFiles/opack.dir/__/__/src/onion/dict.c.o: In function `onion_dict_node_data_free':
dict.c:(.text+0x2ec): undefined reference to `onionlow_free'
CMakeFiles/opack.dir/__/__/src/onion/dict.c.o: In function `onion_dict_node_add':
Notice that libonion is a library (in C, providing HTTP service) and that I just want to add a low_util.c file (wrapping malloc & pthread_create etc... to make Boehm's GC happy: it is calling GC_malloc and GC_pthread_create ....) with its low_util.h header. Surprisingly, they get compiled, but do not seems to be linked. And I am not familiar with cmake and I am not familiar with how D.Moreno (the main author of libonion) has organized his cmake files.
Any clues?
Apply the following patch to make it link. The two executables which are being linked with the symbols generated from the .c file you added are missing and are added in the patch.
http://pastebin.com/mDMRiUQu
Based on what you posted, its hard to tell what could be wrong. The cake source code above says that a variable ${SOURCES} is equivalent to onion.c codecs.c dict.c low_util.c ... ${SQLITE3_C}, and a variable ${INCLUDE_ONION} is equivalent to block.h codecs.h dict.h ... ${SQLITE3_H}. You did not provide any targets or the files included in those targets.
A brief list of things that may help:
where do you define the top level library or executable? If your making a library, you will need the command add_library(). If you are making an executable, you will need the add_executable() command.
Use the command target_link_libraries() to resolve dependencies. Rather than placing all of the source files in a single library, group similar together in a single target (a target is defined by the add_* commands), and use this command to link the targets after compilation.
Use the find_package() to get any libraries which are defined on your system but not in you project. Then, link to that library using the target_link_libraries() command.
In this case, if the onion_dict_* functions are defined within the same library, your not including those files in library. When you use add_library or add_executable, ensure you add those files to the list. If the functions are within your project but not in the same library, use the target_link_libraries() command to link to the library which contains the correct files. If those commands are defined in an external library, then first find the library using find_package(), and then link to the library using target_link_libraries().
I have a plug-in in the form of a shared library (bar.so) that links into a larger program (foo). Both foo and bar.so depend on the same third party library (baz) but they need to keep their implementations of baz completely separate. So when I link foo (using the supplied object files and archives) I need it to ignore any use of baz in bar.so and vice versa.
Right now if I link foo with --trace-symbol=baz_fun where baz_fun is one of the offending symbols I get the following output:
bar.so: definition of baz_fun
foo/src.a(baz.o): reference to baz_fun
I believe this is telling me that foo is referencing baz_fun from bar.so (and execution of foo confirms this).
Solutions that I have tried:
Using objcopy to "localize" the symbols of interest: objcopy --localize-symbols=local.syms bar.so where local.syms contains all of the symbols of interest. I think I might just be confused here and maybe "local" doesn't mean what I think it means. Regardless, I get the same output from the link above. I should note that if I run the nm tool on bar.so prior to using objcopy all of the symbols in question have the T flag (upper-case indicating global) and after objcopy they have a t indicating they are local now. So it appears I am using objcopy correctly.
Compiling with -fvisibility=hidden however due to some other constraints I need to use GCC 3.3 which doesn't appear to support that feature. I might be able to upgrade to a newer version of GCC but would like confirmation that compiling with this flag will help me before heading down that road.
Other things to note:
I do not have access to the source code of either foo or baz
I would prefer to keep all of my plug-in in one shared object (bar.so). baz is actually a licensing library so I don't want it separated
Use dlopen to load your plugin with RTLD_DEEPBIND flag.
(edit)
Please note that RTLD_DEEPBIND is Linux-specific and need glibc 2.3.4 or newer.
When linking against libraries using the -l option (say -lfoo), gcc will prefer a shared object to a static library if both are found (will prefer libfoo.so to libfoo.a). Is there a way to make gcc prefer the static library, if both are found?
The issue I'm trying to solve is the following: I'm creating a plugin for an application (the flight simulator called X-Plane), with the following constraints:
the plugin is to be in the form of a 32 bit shared object, even when running on a 64 bit system
the running environment does not provide a convenient way to load shared objects which are not in the 'normal' locations, say /usr/lib or /usr/lib32:
one cannot expect the user to set LD_PRELOAD or LD_LIBRARY_PATH to find shared objects shipped with my plugin
the X-Plane running environment would not add my plugins directory to ``LD_LIBRARY_PATH, before dynamically loading the plugin shared object, which would allow me to ship all my required shared objects alongside my plugin shared object
one cannot expect 64 bit users to install 32 bit shared objects that are non-trivial (say, are not included in the ia32-libs package on ubuntu)
to solve the above constraints, a possible solution is to link the generated shared object against static, 32 bit versions of all non-trivial libraries used. but, when installing such libraries, usually both static and dynamic versions are installed, and thus gcc will always link against the shared object instead of the static library.
of course, moving / removing / deleting the shared objects in question, and just leaving the static libraries in say /usr/lib32, is a work-around, but it is not a nice one
note:
yes, I did read up on how to link shared objects & libraries, and I'm not trying to creatae a 'totally statically linked shared object'
yes, I tried -Wl,-static -lfoo -Wl,-Bdynamic, but didn't bring the expected results
yes, I tried -l:libfoo.a as well, but this didn't bring the expected results either
You can specify the full path to the static libs without the -l flag to link with those.
gcc ... source.c ... /usr/lib32/libmysuperlib.a ...
Just add the .a file to the link line without -l as if it were a .o file.
It's dated, but may work: http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html
(almost end of the page)
"As noted earlier, it is also possible to link directly with individual library files by specifying the full path to the library on the command line."