In Windows, many .dlls come with a static .lib counterpart. My understanding is that the .lib counterpart basically contains LoadProcAddress calls so that the programmer doesn't have to do it him/herself. Essentially, a time saver. When I switched to Linux, I was assuming the situation was the same, replacing .dll with .so and .lib with .a, but I have come to a situation that is showing me this is wrong and I can't figure out what is going on:
I am using a library that comes as a .a/.so pair. I was linking against the .a, but when I executed ldd on the binary that was produced, it contained no reference to the corresponding .so file. So then, I tried linking against the .so file and to my surprise, this worked. In addition, the .so file showed up when I executed ldd against the resulting binary.
So, I am really confused as to what is going on. In Windows, I would never think to link against a .dll file. Also, in Windows, if a .dll file was accompanied with a .lib and I linked against the .lib at compile-time, then I would expect to have a dependency on the corresponding .dll at runtime. Both these things are not true in this case.
Yes, I have read the basic tutorials about shared objects in Linux, but everything I read seems to indicate that my initial assumption was correct. By the way, I should mention that I am using Code::Blocks as an IDE, which I know complicates things, but I am 99% sure that when I tell it to link against the .so file, it is not simply swapping out the .a file because the resulting binary is smaller. (Plus the whole business about ldd...)
Anyway, thanks in advance.
I was linking against the .a, but when I executed ldd on the binary that was produced, it contained no reference to the corresponding .so file.
This is expected. When you link statically, the static library's code is integrated into the resulting binary. There are no more references to or dependencies on the static library.
So then, I tried linking against the .so file and to my surprise, this worked.
What do you mean, that the static linking did not work? There's no reason that it shouldn't...
.lib are used in Windows to dynamically link. You don't have them in Linux, you link with .so directly.
The .a file is the statically built library, you use it to link statically.
To add to already correct answer by tharibo - in some situations (e.g. delayed shared library load) it might be desirable to do it the Windows way i.e. by linking against a static stub instead of .so. Such stubs can be written by hand, generated by project-specific scripts or by generic Implib.so tool.
Related
I'm creating an desktop app on Golang with Muon UI (using Ultralight instead of Chromium) and cross-build my app for Linux and Windows. For now the app work fine but it required Ultralight libraries (*.dll for Windows and *.so for Linux). But I wanna distribution my app as single executable file. How I can create two executable files? First file for Linux, it's should include main executable file for Linux and only *.so libraries. And second file should include main executable file for Windows and only *.dll libraries. How I can to do this?
Are there any CLI utils for this? (for using in gitlab CI inside Docker for example) Or maybe I can to do this via Golang (for example using embed package. Can I embedded libraries into exe file, that it is can run)?
Or can I use cgo for link dynamic libs as static into binary file?
The honest answer would be: "With great difficulty, lots of pain, blood and tears."
The somewhat longer answer is, that a precompiled DLL/.so may contain slightly more than a mere static library. It it possible to "convert" a DLL/.so into a static library? Somewhat. It boils down to dumping its contents into object files, reverting all the relocation entries, possibly dealing with versioned symbols and weak symbols. No, there are no kitchen sink utilities out there, doing all that for you on an executable binary level.
If you can limit yourself to Linux, you may want to look into Flatpak. What this does is wrapping everything up into a sort of "self extracting archive", which upon launch will transparently and invisibly unpack itself into an in-situ temporary mount point (which you won't see from the rest of the system).
Now, one option would be to build all the dependencies of your program yourself, and arranging for those builds to be created as static libraries. In that case you're no longer dealing with DLLs. However some libraries do not want to be built for static linking, so your mileage may vary there.
Truth to be told: Why is distributing multiple files any issue at all? On Linux/*BSD you must ship separate icon and .desktop files anyway, so that stuff shows up in the Desktop application menus. Yes, it'd be nice if instead of dealing with XDG desktop entry files we had the option to place all of that information into a special – let's call it .xdgdata – readonly section, with some well known symbol names, so that we could have truly single file distributable executables.
My honest suggestion: Don't sweat about it. Just ship the whole bunch of files and don't worry too much about "how this looks".
I have a shared library that consists of quite a few .c modules, some of which are themselves linked into the shared library from other static .a libraries. Most of these are referenced internally within the library, but some are not. I'm finding that the linker does not include those modules in the shared library unless there is at least one call to a function in the module from within the shared library. I've been working around this problem by adding calls in a dummy ForceLinkages() function in a module that I know will be included.
That's okay, but it's surprising, since I'm using a .version file to define a public API to this library. I would've thought including functions in those unreferenced .c modules in the .version file would constitute a reference to the modules and force them to be included in the library.
This library was originally developed on AIX, which uses a .exp file to define the public API. And there, I've never had the issue of unreferenced modules not getting included. I.e., referencing the modules in the .exp file was enough to get the linker to pull them in. Is there a way to get the linux linker to work like that. If not, I can continue to use my silly ForceLinkages() function to get the job done...
That's okay, but it's surprising, since I'm using a .version file to define a public API to this library.
The .version file (assuming it's a linker version script) does not define the library API. If only determines which functions are exported from the library (and with which version label) and which are hidden.
I would've thought including functions in those unreferenced .c modules in the .version file would constitute a reference to the modules and force them to be included in the library.
The version script is applied after the linker has decided which objects are going to be part of the library, and which are going to be discarded, and has no effect on the decisions taken earlier.
This is all working as designed.
You need to either use --whole-archive --no-whole-archive (this has a danger of linking in code you don't need and bloating your binaries), or keep adding references as you've done before.
I'm writing a shared library that itself depends on boost and pcl libraries.
When generating .pc file for my library should I add all these libraries also to the .pc file as dependencies?
It's been a long time since I last time studied these things and I'm a bit confused how this worked again on Linux. When my test app links to my lib I have to add all these pcl and boost libs again to the build even though the lib already has been linked against these libs.
But when I look at the deps of libQtGui.so, for example, it has tens of all kinds of libs it links to, but I don't need to make my app link to those libs...only -lQtGui is enough.
I have just used CMake and link_libraries to add boost and pcl libs.
When generating .pc file for my library should I add all these libraries also to the .pc file as dependencies?
It depends on API of your library:
if public (i.e. installable) headers of your lib use boost/pcl (i.e. have #inclue <boost/...>) (in other words you used PUBLIC (or INTERFACE) named keywords when link your library against boost/pcl in CMake+target_link_libraries) -- then yes you need to add 'em;
otherwise, it depends on what exactly you have at the end -- i.e. does your DSO has DT_NEEDED entries for boost/pcl libs (most likely) or not (you can check it w/ ldd <your-lib>.so). For the last case, you also need to add your dependencies to the *.pc files.
Also, in case of binary dependency from boost/pcl (dunno if the latter has any DSO or not) please make sure you specify exact location of the linked libs -- cuz a user may have multiple (co-existed) boost installations (potentially incompatible) or can do upgrade (later) to other (binary incompatible) version (and you can't really do smth w/ it)… It is important to be linked to the same (or at least binary compatible, which is kinda hard to guarantee for boost) library as you did…
I have just used CMake and link_libraries to add boost and pcl libs.
Please read smth about "Modern CMake" and stop using link_libraries :-) -- use target_link_libraries instead…
After much struggling with attempting to link a C++ project to a 3rd party static library (.lib), I think I have solved the issue by verifying that I compile my project using the same compiler in which the .lib file was compiled (MSVC11).
Until this point, I assumed all .obj files were equivalent, thus I could take a .lib file (containing various .objs), and use that with any other project I might want to develop in the future. However, this was an incorrect assumption.
So I'm curious as to why (in the context of MSVC) .obj files differ from one version of the compiler to the next. Assuming you're targeting an x86 application, shouldn't the obj files be comprised of the same types of instructions regardless of whether or not you compiled using MSVC11/12/14?
TLDR; Why can't I link a project to an .obj that was created using a different MSVC compiler?
That's because it could be linked to another version of Visual C++ runtime libraries, which is incompatible with the version you are using.
This problem could be even with DLLs if you try to expose C++ objects from it.
I think a major design flaw in Linux is the shared object hell when it comes to distributing programs in binary instead of source code form.
Here is my specific problem: I want to publish a Linux program in ELF binary form that should run on as many distributions as possible so my mandatory dependencies are as low as it gets: The only libraries required under any circumstances are libpthread, libX11, librt and libm (and glibc of course). I'm linking dynamically against these libraries when I build my program using gcc.
Optionally, however, my program should also support ALSA (sound interface), the Xcursor, Xfixes, and Xxf86vm extensions as well as GTK. But these should only be used if they are available on the user's system, otherwise my program should still run but with limited functionality. For example, if GTK isn't there, my program will fall back to terminal mode. Because my program should still be able to run without ALSA, Xcursor, Xfixes, etc. I cannot link dynamically against these libraries because then the program won't start at all if one of the libraries isn't there.
So I need to manually check if the libraries are present and then open them one by one using dlopen() and import the necessary function symbols using dlsym(). This, however, leads to all kinds of problems:
1) Library naming conventions:
Shared objects often aren't simply called "libXcursor.so" but have some kind of version extension like "libXcursor.so.1" or even really funny things like "libXcursor.so.0.2000". These extensions seem to differ from system to system. So which one should I choose when calling dlopen()? Using a hardcoded name here seems like a very bad idea because the names differ from system to system. So the only workaround that comes to my mind is to scan the whole library path and look for filenames starting with a "libXcursor.so" prefix and then do some custom version matching. But how do I know that they are really compatible?
2) Library search paths: Where should I look for the *.so files after all? This is also different from system to system. There are some default paths like /usr/lib and /lib but *.so files could also be in lots of other paths. So I'd have to open /etc/ld.so.conf and parse this to find out all library search paths. That's not a trivial thing to do because /etc/ld.so.conf files can also use some kind of include directive which means that I have to parse even more .conf files, do some checks against possible infinite loops caused by circular include directives etc. Is there really no easier way to find out the search paths for *.so?
So, my actual question is this: Isn't there a more convenient, less hackish way of achieving what I want to do? Is it really so complicated to create a Linux program that has some optional dependencies like ALSA, GTK, libXcursor... but should also work without it! Is there some kind of standard for doing what I want to do? Or am I doomed to do it the hackish way?
Thanks for your comments/solutions!
I think a major design flaw in Linux is the shared object hell when it comes to distributing programs in binary instead of source code form.
This isn't a design flaw as far as creators of the system are concerned; it's an advantage -- it encourages you to distribute programs in source form. Oh, you wanted to sell your software? Sorry, that's not the use case Linux is optimized for.
Library naming conventions: Shared objects often aren't simply called "libXcursor.so" but have some kind of version extension like "libXcursor.so.1" or even really funny things like "libXcursor.so.0.2000".
Yes, this is called external library versioning. Read about it here. As should be clear from that description, if you compiled your binaries using headers on a system that would normally give you libXcursor.so.1 as a runtime reference, then the only shared library you are compatible with is libXcursor.so.1, and trying to dlopen libXcursor.so.0.2000 will lead to unpredictable crashes.
Any system that provides libXcursor.so but not libXcursor.so.1 is either a broken installation, or is also incompatible with your binaries.
Library search paths: Where should I look for the *.so files after all?
You shouldn't be trying to dlopen any of these libraries using their full path. Just call dlopen("libXcursor.so.1", RTLD_GLOBAL);, and the runtime loader will search for the library in system-appropriate locations.