I have a large body of code, compiled with /MT (i.e. expecting to statically link against the CRT). I need to combine this with a static third-party library, which has been built with /MD (i.e. expecting to link the CRT dynamically).
Is it theoretically possible to link the two into one executable without recompiling either?
If I link with /nodefaultlib:msvcrt, I end up with a small number of undefined references to things like __imp__wgetenv. I'm tempted to try implementing those functions in my own code, forwarding to wgetenv, etc. Is that worth trying, or will I run straight into the next problem?
Unfortunately I'm Forbidden from taking the easy option of packing the thirdparty code into a separate DLL :-/
No. /MT and /MD are mutually exclusive.
All modules passed to a given invocation of the linker must have been compiled with the same run-time library compiler option (/MD, /MT, /LD).
Source
I found such solution in OpenSSL sources: All obj files of the library are compiled with combination: /MT /Zl. As author described, such combination allows to build static library with ability to compile with applications either dynamic CRT (/MD) or static CRT (/MT).
I faced similar situation where in I had two libraries one was built with MT and another one with MD. I had to build an executable which uses functionalities from both the libraries. The library built as MD was third party thus I couldn't rebuilt it and library built as MT has many dependencies and to built all of them as MD is a big pain. I was getting error from the third party config header file which made it mandatory to built the executable as MD. I was looking for the easy way of packaging third party dll as a separate dll as mentioned in question. However, I couldn't find enough explanation online on this easy way. Hence my two cents below.
The following is the way I circumvent it
I built another .dll which acted as an interface. This interface basically wrapped all api calls that was made to third party dll. The header file for this interface did not include any header file from third party dll rather all those header files were included in the interface.cpp file. Interface as you expect was built as MD.
Now In my main.cpp file I included this interface header file to make all the calls to third party dll through the interface.
Extra care has to be taken in passing arguments to the interface. Basic variables like int,bool etc can be passed as value. However any class or structure needs to be passed as const reference to avoid heap corruption. This is applicable to even string.
Happy to share more details if it is not clear!
Related
Let's consider Debian or Ubuntu distro where one can install some library package, say libfoobar, and a corresponding libfoobar-dev. The first one contains shared library object. The later usually contains headers, static library variant and cmake exported targets (say, FoobarTargets.cmake) which allow CMake stuff like find_package to work flawlessly. To do so FoobarTargets.cmake have to contain both targets: for static and for shared variant.
From the other side, I've read many articles stating that I (as a libfoobar's author) should refrain from declaring add_library(foobar SHARED ....) and add_library(foobar_static STATIC ...) in CMakeLists.txt in favour of building and installing library using BUILD_SHARED_LIBS=ON and BUILD_SHARED_LIBS=OFF. But this approach will export only one variant of the library into FoobarTargets.cmake because the later build will overwrite the first one.
So, the question is: how to do it the right way? So that package maintainer would not need to patch library's CMakeLists.txt to properly export both variants from the one side. And to adhere CMake's sense of a true way, removing duplicating targets which differ only by static/shared from the other side?
I wrote an entire blog post about this. A working example is available on GitHub.
The basic idea is that you need to write your FoobarConfig.cmake file in such a way that it loads one of FoobarSharedTargets.cmake or FoobarStaticTargets.cmake in a principled, user-controllable way, that's also tolerant to only one or the other being present. I advocate for the following strategy:
If the find_package call lists exactly one of static or shared among the required components, then load the corresponding set of targets.
If the variable Foobar_SHARED_LIBS is defined, then load the corresponding set of targets.
Otherwise, honor the setting of BUILD_SHARED_LIBS for consistency with FetchContent users.
In all cases, your users will link to Foobar::Foobar.
Ultimately, you cannot have both static and shared imported in the same subdirectory while also providing a consistent build-tree (FetchContent) and install-tree (find-package) interface. But this is not a big deal since usually consumers want only one or the other, and it's totally illegal to link both to a single target.
I have a shared library that consists of quite a few .c modules, some of which are themselves linked into the shared library from other static .a libraries. Most of these are referenced internally within the library, but some are not. I'm finding that the linker does not include those modules in the shared library unless there is at least one call to a function in the module from within the shared library. I've been working around this problem by adding calls in a dummy ForceLinkages() function in a module that I know will be included.
That's okay, but it's surprising, since I'm using a .version file to define a public API to this library. I would've thought including functions in those unreferenced .c modules in the .version file would constitute a reference to the modules and force them to be included in the library.
This library was originally developed on AIX, which uses a .exp file to define the public API. And there, I've never had the issue of unreferenced modules not getting included. I.e., referencing the modules in the .exp file was enough to get the linker to pull them in. Is there a way to get the linux linker to work like that. If not, I can continue to use my silly ForceLinkages() function to get the job done...
That's okay, but it's surprising, since I'm using a .version file to define a public API to this library.
The .version file (assuming it's a linker version script) does not define the library API. If only determines which functions are exported from the library (and with which version label) and which are hidden.
I would've thought including functions in those unreferenced .c modules in the .version file would constitute a reference to the modules and force them to be included in the library.
The version script is applied after the linker has decided which objects are going to be part of the library, and which are going to be discarded, and has no effect on the decisions taken earlier.
This is all working as designed.
You need to either use --whole-archive --no-whole-archive (this has a danger of linking in code you don't need and bloating your binaries), or keep adding references as you've done before.
After much struggling with attempting to link a C++ project to a 3rd party static library (.lib), I think I have solved the issue by verifying that I compile my project using the same compiler in which the .lib file was compiled (MSVC11).
Until this point, I assumed all .obj files were equivalent, thus I could take a .lib file (containing various .objs), and use that with any other project I might want to develop in the future. However, this was an incorrect assumption.
So I'm curious as to why (in the context of MSVC) .obj files differ from one version of the compiler to the next. Assuming you're targeting an x86 application, shouldn't the obj files be comprised of the same types of instructions regardless of whether or not you compiled using MSVC11/12/14?
TLDR; Why can't I link a project to an .obj that was created using a different MSVC compiler?
That's because it could be linked to another version of Visual C++ runtime libraries, which is incompatible with the version you are using.
This problem could be even with DLLs if you try to expose C++ objects from it.
I have a process A which uses two shared libraries: libA.so and libB.so. Because the two libraries were written by different people. Unfortunately libA.so uses version 1.0 of the 3rd party library libD.so. While libB.so uses version 2.0 of the library in static form libD.a. I know that if libA.so and libA.so use libD.so, some errors might happen because of the Global Symbol Interpose. But does this situation have the same problem too?
I know the link flag -Bsymbolic could be used on libA.so or libB.so to force the symbol resolving symbols with the library first. In order to make process A run correctly, both of the two libraries must be linked with this flag, am I right? However, I don't have the source code of libA.so. So I cannot re-link the libA.so again.
To be more general, if one process uses two 3rd party libraries, which contains another same 3rd party library. Will the same thing happen? Is there anything I can do to solve this problem?
This may or may not help you, but given the lack of information I'm hoping it at least sparks an idea or leads you to something similar.
This is an application that allows you to alter your shell settings on a per directory basis:
https://github.com/zimbatm/direnv
It sounds like you actually have an issue that would require you to recompile one of your libraries from source though. That's not ideal, but if there is no build using a compatible thirdparty version you might seek a completely different library to accomplish the original task.
I want to make a quick fix to one of the project's .so libraries. Is it safe to just recompile the .so and replace the original? Or I have to rebuild and reinstall the whole project? Or it depends?
It depends. Shared library needs to be binary-compatible with your executable.
For example,
if you changed the behaviour of one of library's internal functions, you probably don't need to recompile.
If you changed the size of a struct (e.g. by adding a member) that's known by the application, you will need to recompile, otherwise the library and the application will think the struct is smaller than it is, and will crash when the library tries to read an extra uninitialized member that the application didn't write to.
If you change the type or the position of arguments of any functions visible from the applications, you do need to recompile, because the library will try to read more arguments off the stack than the application has put on it (this is the case with C, in C++ argument types are the part of function signature, so the app will refuse run, rather than crashing).
The rule of thumb (for production releases) is that, if you are not consciously aware that you are maintaining binary compatibility, or not sure what binary compatibility is, you should recompile.
That's certainly the intent of using dynamic libraries: if something in the library needs updating, then you just update the library, and programs that use it don't need to be changed. If the signature of the function you're changing doesn't change, and it accomplishes the same thing, then this will in general be fine.
There are of course always edge cases where a program depends on some undocumented side-effect of a function, and then changing that function's implementation might change the side-effect and break the program; but c'est la vie.
If you have not changed the ABI of the shared library, you can just rebuild and replace the library.
It depends yes.
However, I assume you have the exact same source and compiler that built the other stuff and now if you only change in a .cpp file something, it is fine.
Other things e.g. changing an interface (between the shared lib and the rest of the system) in a header file is not fine.
If you don't change your library binary interface, it's ok to recompile and redeploy only the shared library.
Good references:
How To Write Shared Libraries
The Little Manual of API Design