I have two libs: libmy1.so and libmy2.so. libmy1.so is small and need the support of libmy2.so, which have a very large size. However, libmy1.so only need several function from libmy2.so. Is there any way I can use to link there function into libmy1.so, so that libmy1.so can be used without libmy2.so?
If you create a static version of my2 you can link them and libmy1.so will contain what it needs from libmy2.a removing the need to include both libs.
Take a look at this page for some pointers: Examples of .so and .a libraries
You may also use dlopen/dlsym for the functions that require libmy2.so.
Unfortunately it is not possible to check for the presence of libmy2.so on start-up. So when the function that requires libmy2.so is called the first time libmy1.so must load libmy2.so.
If libmy2.so is not present or cannot be loaded you'll have to do some error reaction.
Related
Say I have a binary server, and when it's compiled, it's linked from server.c, static_lib.a, and dynamically with dynamic_lib.so.
When server is executed and it loads dynamic_lib.so dynamically, but on the code path, dynamic_lib.so actually expects some symbols from static_lib.a. What I'm seeing is that, dynamic_lib.so pulls in static_lib.so so essentially I have two static_lib in memory.
Let's assume there's no way we can change dynamic_lib.so, because it's a 3rd-party library.
My question is, is it possible to make dynamic_lib.so or ld itself search the current binary first, or even not search for it in ld's path, just use the binary's symbol, or abort.
I tried to find some related docs about it, but it's not easy for noobs about linkers like me :-)
You can not change library to not load static_lib.so but you can trick it to use static_lib.a instead.
By default ld does not export any symbols from executables but you can change this via -rdynamic. This option is quite crude as it exports all static symbols so for finer-grained control you can use -Wl,--dynamic-list (see example use in Clang sources).
In our build system, we generate multiple .so files (foo.so, bar.so, ...) that are loaded during runtime by the main executable (biz). So the .so files are linked separately.
We also have our own util.a static library, that has some utility functions and global data.
The problem comes when some of the .so want to use util.a data/function, but we can't link each .so to util.a. It's because of the data section: global data must be unique in the program address space. If more than one .so is linked to util.a and has a copy of the data, the program behavior will be very surprising but hard to debug.
We can't link executable (biz) to util.a either. The linker will not put everything to the target, since biz doesn't reference the functions on behalf of .so.
Of course, unless linking util.a with -Wl,-whole-archive. But is there a better way to do this?
Solution 1: consider making util.a a dynamic library util.so.
Solution 2: don't let the linker export any symbols exported by util.a. When using gcc you can achieve this for example by using __attribute__((visibility("hidden"))):
int __attribute__((visibility("hidden"))) helperfunc(void *p);
You can use objdump to check which symbols are exported.
To answer myself's question, the eventual solution was like:
http://lists.gnu.org/archive/html/qemu-devel/2014-09/msg00099.html
TL;DR: Search for all the interesting symbols (that you want to pull from archives) inside the .so objects with nm (1), and inject into the compiling command line with -Wl,-u,$SYMBOL. Note that the -Wl,-u,$SYMBOL arguments need to come before archive names in the command line, so the linker knows that it needs to link them.
I've written a Tcl script that uses the TclMagick extension together with GraphicsMagick.
For GraphicsMagick, I've both the Windows DLLs and the Linux SO files. I want to be able to make two Starkit/Starpack applications bundled with those libraries: one for Windows (with the DLLs) and one for Linux (with the SO files).
Is this reasonable? Can it be done?
EDIT
I cannot seem to use DLLs with dependencies under Windows. In my case, I want to use the TclMagick extension, but it needs the GraphicsMagick's DLLs and the starkit cannot find those. What should I do in this situation?
Yes. In the lib/tclmagick subdirectory of $starkit::topdir, you'll place the dynamic library and an appropriate pkgIndex.tcl file that loads the library. Use a Makefile or some other build script to use the correct dynamic library file, and generate the pkgIndex, depending the target platform.
The directory hierarchy:
appname.vfs/
main.tcl
lib/
app-appname/
appname.tcl
pkgIndex.tcl
tclmagick/
pkgIndex.tcl
tclMagick.so
package require tclmagick will work as you expect, for some capitalization of "tclmagick"
You can do it, but you might need some extra windows trickery to get things to work properly.
Windows has quite a few options to load dependent libraries, this page explains the basics:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx
There are is one part that can help you:
If a DLL with the same module name is already loaded in memory, the system checks only for redirection and a manifest before resolving to the loaded DLL, no matter which directory it is in. The system does not search for the DLL.
So, to get the dependencies right, you could get the dependent libraries loaded into memory first (sadly you cannot use load for this, but could use something from twapi, e.g. twapi::load_libary (see http://wiki.tcl.tk/9886) to get the library loaded from some temporary location).
Sadly most OS's do not provide an easy or portable way to load a dynamic library directly from memory, so you need to copy the dependent libs to a temporary location first (you can do it with appropriate hacks or by using something like an installable filesystem on windows/FUSE on Linux).
In most cases the easiest route is to just link all dependencies statically into the extension instead.
I have built a linux shared object which I inject into a 3rd party program to intercept some dynamic function calls using LD_PRELOAD.
The 3rd party program uses a SO "libabc.so" located at some path. My injected SO uses another SO, also called "libabc.so" located at another path (essentially identical but slight code differences).
My problem is now, that calls to a function "def" which appear in both libabc.so are always resolved by the first. (Presumably because it is loaded first?!) How can I get them to be resolved with the second libabc.so?
Many thanks!
Unless something changed since I used to do this, you will need to dlopen() the library you want to pass calls on to and call the function manually, something like;
handle = dlopen("/path/to/libabc.so", RTLD_LAZY);
otherDef = dlsym(handle, "def");
orderDef(parameter);
There is a complete example how to do this very thing at LinuxJournal.
If you only want to use one libabc.so version, you can always use LD_PRELOAD to load it along with your own shared object before anything else.
If you want to use multiple versions, you have a few alternatives:
Use dlopen() in your shared object to load that library. Since you have created a function injection object you should be familiar with this procedure. This is the more generic and powerful way - you could even mix & match functions from different library versions.
Use a different DT_SONAME for the library version your shared object links against. Unfortunately this requires (slightly) changing the build system of that library and recompiling.
Link your shared object statically against the library in question. Not always possible but it does not require modifying the library in question. The main issue with this alternative is that any change in the library should be followed by a relinking of your shared object for the changes to be pulled in.
Warning: you may need to use a custom linker script or specific linker options to avoid symbol conflicts.
When linking against libraries using the -l option (say -lfoo), gcc will prefer a shared object to a static library if both are found (will prefer libfoo.so to libfoo.a). Is there a way to make gcc prefer the static library, if both are found?
The issue I'm trying to solve is the following: I'm creating a plugin for an application (the flight simulator called X-Plane), with the following constraints:
the plugin is to be in the form of a 32 bit shared object, even when running on a 64 bit system
the running environment does not provide a convenient way to load shared objects which are not in the 'normal' locations, say /usr/lib or /usr/lib32:
one cannot expect the user to set LD_PRELOAD or LD_LIBRARY_PATH to find shared objects shipped with my plugin
the X-Plane running environment would not add my plugins directory to ``LD_LIBRARY_PATH, before dynamically loading the plugin shared object, which would allow me to ship all my required shared objects alongside my plugin shared object
one cannot expect 64 bit users to install 32 bit shared objects that are non-trivial (say, are not included in the ia32-libs package on ubuntu)
to solve the above constraints, a possible solution is to link the generated shared object against static, 32 bit versions of all non-trivial libraries used. but, when installing such libraries, usually both static and dynamic versions are installed, and thus gcc will always link against the shared object instead of the static library.
of course, moving / removing / deleting the shared objects in question, and just leaving the static libraries in say /usr/lib32, is a work-around, but it is not a nice one
note:
yes, I did read up on how to link shared objects & libraries, and I'm not trying to creatae a 'totally statically linked shared object'
yes, I tried -Wl,-static -lfoo -Wl,-Bdynamic, but didn't bring the expected results
yes, I tried -l:libfoo.a as well, but this didn't bring the expected results either
You can specify the full path to the static libs without the -l flag to link with those.
gcc ... source.c ... /usr/lib32/libmysuperlib.a ...
Just add the .a file to the link line without -l as if it were a .o file.
It's dated, but may work: http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html
(almost end of the page)
"As noted earlier, it is also possible to link directly with individual library files by specifying the full path to the library on the command line."