To repeat: I'm looking for ABI compatibility between libraries of the same Visual-C++ version!
We want to mix and match some internal C++ DLLs from different teams - built at different times with different project files. Because of long build times, we exactly want to avoid large monolithic builds where each team re-compiles the source code of another team's library.
When consuming C++ DLLs with C++ interfaces it is rather clear that you only can do this if all DLLs are compiled with the same compiler / Visual Studio version.
What is not readily apparent to me is what, exactly needs to be the same to get ABI compatibility.
Obviously debug (_DEBUG) and release (NDEBUG) cannot be mixed -- but that's also apparent from the fact that these link to different versions of the shared runtime.
Do you need the exact same compiler version, or is it sufficient that the resulting DLL links to the same shared C++ runtime -- that is, basically to the same redistributable? (I think static doesn't fly when passing full C++ objects around)
Is there a documented list of compiler (and linker) options that need to be the same for two C++ DLLs of the same vc++ version to be compatible?
For example, is the same /O switch necessary - does the optimization level affect ABI compatibility´? (I'm pretty sure not.)
Or do both version have to use the same /EH switch?
Or /volatile:ms|iso ... ?
Essentially, I'd like to come up with a set of (meta-)data to associate with a Visual-C++ DLL that describes it's ABI compatibility.
If differences exist, my focus is on VS2015 only at the moment.
Have been thinking this through the last days, and what I did do was to try to see if some use-cases exists where devs have already needed to categorize their C++ build to make sure binaries are compatible.
One such place is the Native Packages from nuget. So I looked at one package there, specifically the cpprestsdk:
The binaries in the downloadable package as split like this:
native\v120\windesktop\msvcstl\dyn\rt-dyn\x64\Release\
^ ^ ^ ^ ^
VS version | not sure | uses cpp-runtime dynamically
| lib itself dynamic (as opposed to static)
or WinXP or WinApp(WinRT?)
I pulled this out from this example, because I couldn't find any other docs. I also know that the boost binaries build directory is separated in a similar way.
So, to get to a list of meta data to identify the ABI compatibility, I can preliminarily list the following:
VC version (that is, the version of the C and CPP runtime libraries used)
one point here is that e.g. vc140 should be enough nowadays - given how the CRT is linked in, all possible bugfixes to the versioned CRT components must be ABI compatible anyway, so it shouldn't matter which version a given precompiled library was built with.
pure native | managed (/CLI) | WinRT
how the CRT is consumed (statically / dynamically)
bitness / platform (Win32, x64, ARM, etc.)
Release or Debug version (i.e. which version of the CRT we link to)
plus: _ITERATOR_DEBUG_LEVEL ... if everyone goes with the defaults, fine, if a project does not, it must declare so
Additionally my best guess as to the following items:
/O must not matter - we constantly mix&match binaries with different optimization settings - specifically, this is even working for object files within the same binary
/volatile - since this is a code-gen thing, I have a hard time imagining how this could break an ABI
/EH - except for the option to disable all exception, in which case you obviously can't call anything that throws, I'm pretty confident this is save from an ABI perspective: There are possible pitfalls here, but I think they can't really be categorized into ABI compat. (Maybe some complex callback chains could be said to be ABI incompatible, not sure)
Others:
Default calling convention (/G..) : I think this would break at link time, when mangled export symbols and header declarations don't match up.
/Zc:wchar_t - will break at link time (It's actually ABI compatible, but the symbols won't macth.)
Enable RTTI (/GR) - not too sure 'bout this one - I never have worked with this disabled.
Related
I'm looking for the best way to develop and package different variants of a library with different compile settings but for the same ABI and then selecting the best fit at runtime. In more concrete terms, I'd like a NEON and non-NEON armeabi-v7a build.
The native library has a public C interface that third parties link to. They seem to need to link to one of the variants to prevent link errors, but I'd like to load the alternative variant at runtime if it's a better fit for the device, and have the runtime loader do the correct relocations.
From what I see so far it seems I need to give both variants the same file name, so need to put them in different folders. Subfolders under the abi folder don't seem to get copied by the package installation process so that approach doesn't work. The best suggestion I've seen so far is to manually copy one variant from the res folder to a known device path and to call System.loadLibrary() with a full path. Reference: https://groups.google.com/forum/#!topic/android-ndk/zu_dmcmUlMo
Is this still the best/recommended approach?
How will this interact with the binary translation done on non-arm devices? (Although I can supply an x86 build, some third parties may leave it out of their apk).
I'm assuming cpufeatures on a device using binary translation will not report the cpu family as ARM, so my proposed solution would be to build a standard armeabi-v7a library in the normal way (which I guess will get binary translated), and ship a NEON-supporting library in res/raw. Then at runtime if cpufeatures reports an ARM CPU with NEON support then copy out that library and call loadLibrary with the full path. Can anyone see any problems with that approach?
If you explicitly want to have two different builds of a lib, then yes, it's probably the best compromise.
First off - do note that many libraries that can use NEON can be built with those parts runtime-enabled so that you can have a normal ARMv7 build which doesn't strictly require NEON but can enable those codepaths at runtime if detected - e.g. libav/FFmpeg do that, and the same goes for many other similar libraries. This allows you to have one single ARMv7 binary that fully utilizes NEON where applicable, while still works on the few ARMv7 devices without NEON.
If you're trying to use compiler autovectorization, or if this is a library where the NEON routines aren't easily confined to restricted parts that are enabled at runtime (or hoping to gain extra performance by building the whole library with NEON enabled), your approach sounds sane.
Keep in mind that you want to have at least one native library that is packaged "normally" (which you seem to have, but which has been an issue in e.g. https://stackoverflow.com/a/29329413/3115956). On installation, the installer picks the best match of the bundled architectures and only extracts the libs from that one, and runs the process in that mode. On devices with multiple ABIs (32 and 64 bit), this is essential since if the process is started in a different mode it's too late to switch mode once you try to load a library in a different form.
On an x86 device that emulates ARM binaries, at least the cpufeatures library will return ARM if the process is running in ARM mode. If you use system properties to find the primary and secondary ABIs, you won't know which of them the current process is using though.
EDIT: x86 devices with binary translation actually seem to be able to load an armeabi library even if the same process already has loaded some bundled x86 libraries as well. So apparently this translation is done on a per library basis, not like 32 vs 64 bit, where a certain mode is chosen for the process at startup, which excludes loading any libraries of the other variant.
Let me get right to the point:
Main application:
C# (4.0), AnyCPU.
Library:
Wrapper for native .dll written in C++/CLI. Compiled in two versions; x86 and x64, both signed with the same .snk key (using this workaround)
Limitations:
In the end a single distribution package is required for x86 and x64 platforms.
Main application needs strong name due to references to other strongly named libs.
Rewriting the library using managed C# and P/Invoke is an absolute last way out.
The problem:
As long as the main application, at compile time, references the version (x86 or x64) of the library that is needed when run, this is all working fine.
Moving the same compiled output - and exchanging the library with the right platform version during installation - does not work since the signature of the library changes from that of the referenced one.
In a test application without any strong naming I can switch between them as needed.
The question:
Is there a way to enable switching between the x86 and x64 libraries within the set limitations, or is strong naming preventing any possible solution other than rewriting the lib?
Let me clarify that it is not a question about finding the correct .dll (as discussed here) but about being able to load the .dll once found.
#Damien_The_Unbeliever's comment got me thinking and he is right in that the strong names are the same, and it was not the actual issue.
I found another difference between the two versions of the library; the output name was set to nnn.dll and nnnx64.dll. Changing it so that both have the same output name magically made it all work.
Perhaps someone knows why such a setting matters, I certainly don't.
Does anyone know if LLVM binary compatibility is planned for visual studio combiled .obj and static .lib files?
Right now I can only link LLVM made .obj files with dynamic libs that loads a DLL at runtime (compiled from visual studio).
While there probably is very small chances that binary compatibility will happen between the two compilers, does anybody know why it is so difficult achieving this between compilers for one platform?
As Neil already said, the compatibility includes stuff like calling convention, name mangling, etc. Though these two are the smallest possible problems. LLVM already knows about all windows-specific calling conventions (stdcall, fastcall, thiscall), this is why you can call stuff from .dll's.
If we speak about C++ code then the main problem is C++ ABI: vtable layout, rtti implementation, etc. clang follows Itanium C++ ABI (which gcc use, for example, among others), VCPP - doesn't and all these are undocumented, unfortunately. There is some work going in clang in this direction, so stuff might start to work apparently. Note that most probably some parts will never be covered, e.g. seh-based exception handling on win32, because it's patented.
Linking with pure C code worked for ages, so, you might workaround these C++ ABI-related issues via C stubs / wrappers.
Apart from anything else, such as calling conventions, register usage etc, for C++ code to binary compatible the two compilers must use the same name-mangling scheme. These schemes are proprietory (so MS does not release the details if its scheme) and are in any case in a constant state of flux.
It seems that all my adult life I've been tormented by the VC++ linker complaining or balking because various libraries do not agree on which version of the Runtime library to use. I'm never in the mood to master that dismal subject. So I just try to mess with it until it works. The error messages are never useful. Neither is the Microsoft documentation on the subject - not to me at least.
Sometimes it does not find functions - because the name-mangling is not what was expected? Sometimes it refuses to mix-and-match. Other times it just says, "LINK : warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library" Using /NODEFAULTLIB does not work, but the warning seems to be benign. What the heck is "DEFAULTLIB" anyway? How does the linker decide? I've never seen a way to specify to the linker which runtime library to use, only how to tell the compiler which library to create function calls for.
There are "dependency walker" programs that can inspect object files to see what DLL's they depend on. I just ran one on a project I'm trying to build, and it's a real mess. There are system .libs and .dll's that want conflicting runtime versions. For example, COMCTL32.DLL wants MSVCRT.DLL, but I am linking with MSVCRTD.DLL. I am searching to see if there's a COMCTL32D.DLL, even as I type.
So I guess what I'm asking for is a tutorial on how to sort those things out. What do you do, and how do you do it?
Here's what I think I know. Please correct me if any of this is wrong.
The parameters are Debug/Release, Multi-threaded/Single-threaded, and static/DLL. Only six of the eight possible combinations are covered. There is no single-threaded DLL, either Debug or Release.
The settings only affect which runtime library gets linked in (and the calling convention to link with it). You do not, for example, have to use a DLL-based runtime if you are building a DLL, nor do you have to use a Debug version of runtime when building the Debug version of a program, although it seems to help when single-stepping past system calls.
Bonus question: How could anyone or any company create such a mess?
Your points (1) and (2) look correct to me. Another thing to note with (2) is that linking in the debug CRT also gives you access to things like enhanced heap checking, checked iterators, and other assorted sanity checks. You cannot redistribute the debug CRT with your application, however -- you must ship using the release build only. Not only is it required by the VC license, but you probably don't want to be shipping debug binaries anyway.
There is no such thing as COMCTL32D.DLL. DLLs that are part of Windows must load the CRT that they were linked against when Windows was built -- this is included with the OS as MSVCRT.DLL. This Windows CRT is completely independent from the Visual C++ CRT that is loaded by the modules that comprise your program (MSVCRT.DLL is the one that ships with Windows. The VC CRT will include a version number, for example MSVCRT80.DLL). Only the EXE and DLL files that make up your program are affected by the debug/release multithreaded/single-threaded settings.
The best practice here IMO is to pick a setting for your CRT and standardize upon it for every binary that you ship. I'd personally use the multithreaded DLL runtime. This is because Microsoft can (and does) issue security updates and bug fixes to the CRT that can be pushed out via Windows Update.
How to create a shared object that is statically linked with pthreads and libstdc++ on Linux/gcc?
Before I go to answering your question as it was described, I will note that it is not exactly clear what you are trying to achieve in the end, and there is probably a better solution to your problem.
That said - there are two main problems with trying to do what you described:
One is, that you will need to decompose libpthread and libstdc++ to the object files they are made with. This is because ELF binaries (used on Linux) have two levels of "run time" library loading - even when an executable is statically linked, the loader has to load the statically linked libraries within the binary on execution, and map the right memory addresses. This is done before the shared linkage of libraries that are dynamically loaded (shared objects) and mapped to shared memory. Thus, a shared object cannot be statically linked with such libraries, as at the time the object is loaded, all static linked libraries were loaded already. This is one difference between linking with a static library and a plain object file - a static library is not merely glued like any object file into the executable, but still contains separate tables which are referred to on loading. (I believe that this is in contrast to the much simpler static libraries in MS-DOS and classic Windows, .LIB files, but there may be more to those than I remember).
Of course you do not actually have to decompose libpthread and libstdc++, you can just use the object files generated when building them. Collecting them may be a bit difficult though (look for the objects referred to by the final Makefile rule of those libraries). And you would have to use ld directly and not gcc/g++ to link, to avoid linking with the dynamic versions as well.
The second problem is consequential. If you do the above, you will sure have such a shared object / dynamic library as you asked to build. However, it will not be very useful, as once you try to link a regular executable that uses those libpthread/libstdc++ (the latter being any C++ program) with this shared object, it will fail with symbol conflicts - the symbols of the static libpthread/libstdc++ objects you linked your shared object against will clash with the symbols from the standard libpthread/libstdc++ used by that executable, no matter if it is dynamically or statically linked with the standard libraries.
You could of course then try to either hide all symbols in the static objects from libstdc++/libpthread used by your shared library, make them private in some way, or rename them automatically on linkage so that there will be no conflict. However, even if you get that to work, you will find some undesireable results in runtime, since both libstdc++/libpthread keep quite a bit of state in global variables and structures, which you would now have duplicate and each unaware of the other. This will lead to inconsistencies between these global data and the underlying operating system state such as file descriptors and memory bounds (and perhaps some values from the standard C library such as errno for libstdc++, and signal handlers and timers for libpthread.
To avoid over-broad interpretation, I will add a remark: at times there can be sensible grounds for wanting to statically link against even such basic libraries as libstdc++ and even libc, and even though it is becoming a bit more difficult with recent systems and versions of those libraries (due to a bit of coupling with the loader and special linker tricks used), it is definitely possible - I did it a few times, and know of other cases in which it is still done. However, in that case you need to link a whole executable statically. Static linkage with standard libraries combined with dynamic linkage with other objects is not normally feasible.
Edit: One issue which I forgot to mention but is important to take into account is C++ specific. C++ was unfortunately not designed to work well with the classic model of object linkage and loading (used on Unix and other systems). This makes shared libraries in C++ not really portable as they should be, because a lot of things such as type information and templates are not cleanly separated between objects (often being taken, together with a lot of actual library code at compile time from the headers). libstdc++ for that reason is tightly coupled with GCC, and code compiled with one version of g++ will in general only work with the libstdc++ from with this (or a very similar) version of g++. As you will surely notice if you ever try to build a program with GCC 4 with any non-trivial library on your system that was built with GCC 3, this is not just libstdc++. If your reason for wanting to do that is trying to ensure that your shared object is always linked with the specific versions of libstdc++ and libpthread that it was built against, this would not help because a program that uses a different/incompatible libstdc++ would also be built with an incompatible C++ compiler or version of g++, and would thus fail to link with your shared object anyway, aside from the actual libstdc++ conflicts.
If you wonder "why wasn't this done simpler?", a general rumination worth pondering: For C++ to work nicely with dynamic/shared libraries (meaning compatibility across compilers, and the ability to replace a dynamic library with another version with a compatible interface without rebuilding everything that uses it), not just compiler standartization is needed, but at the level of the operating system's loader, the structure and interface of object and library files and the work of the linker would need to be significantly extended beyond the relatively simple Unix classics used on common operating systems (Microsoft Windows, Mach based systems and NeXTStep relatives such as Mac OS, VMS relatives and some mainframe systems also included) for natively built code today. The linker and dynamic loader would need to be aware of such things as templates and typing, having to some extent functionality of a small compiler to actually adapt the library's code to the type given to it - and (personal subjective observation here) it seems that higher-level intermediate intermediate code (together with higher-level languages and just-in-time compilation) is catching ground faster and likely to be standardized sooner than such extensions to the native object formats and linkers.
You mentioned in a separate comment that you are trying to port a C++ library to an embedded device. (I am adding a new answer here instead of editing my original answer here because I think other StackOverflow users interested in this original question may still be interested in that answer in its context)
Obviously, depending on how stripped down your embedded system is (I have not much embedded Linux experience, so I am not sure what is most likely), you may of course be able to just install the shared libstdc++ on it and dynamically link everything as you would do otherwise.
If dynamically linking with libstdc++ would not be good for you or not work on your system (there are so many different levels of embedded systems that one cannot know), and you need to link against a static libstdc++, then as I said, your only real option is static linking the executable using the library with it and libstdc++. You mentioned porting a library to the embedded device, but if this is for the purpose of using it in some code you write or build on the device and you do not mind a static libstdc++, then linking everything statically (aside from perhaps libc) is probably OK.
If the size of libstdc++ is a problem, and you find that your library is actually only using a small part of its interfaces, then I would nonetheless suggest first trying to determine the actual space you would save by linking against only the parts you need. It may be significant or not, I never looked that deep into libstdc++ and I suspect that it has a lot of internal dependencies, so while you surely do not need some of the interfaces, you may or may not still depend on a big part of its internals - I do not know and did not try, but it may surprise you. You can get an idea by just linking a binary using the library against a static build of it and libstdc++ (not forgetting to strip the binary, of course), and comparing the size of the resulting executable that with the total size of a (stripped) executable dynamically linked together with the full (stripped) shared objects of the library and libstdc++.
If you find that the size difference is significant, but do not want to statically link everything, you try to reduce the size of libstdc++ by rebuilding it without some parts you know that you do not need (there are configure-time options for some parts of it, and you can also try to remove some independent objects at the final creation of libstdc++.so. There are some tools to optimize the size of libraries - search the web (I recall one from a company named MontaVista but do not see it on their web site now, there are some others too).
Other than the straightforward above, some ideas and suggestions to think of:
You mentioned that you use uClibc, which I never fiddled with myself (my experience with embedded programming is a lot more primitive, mostly involving assembly programming for the embedded processor and cross-compiling with minimal embedded libraries). I assume you checked this, and I know that uClibc is intended to be a lightweight but rather full standard C library, but do not forget that C++ code is hardly independent on the C library, and g++ and libstdc++ depend on quite some delicate things (I remember problems with libc on some proprietary Unix versions), so I would not just assume that g++ or the GNU libstdc++ actually works with uClibc without trying - I don't recall seeing it mentioned in the uClibc pages.
Also, if this is an embedded system, think of its performance, compute power, overall complexity, and timing/simplicity/solidity requirements. Take into consideration the complexity involved, and think whether using C++ and threads is appropriate in your embedded system, and if nothing else in the system uses those, whether it is worth introducing for that library. It may be, not knowing the library or system I cannot tell (again, embedded systems being such a wide range nowadays).
And in this case also, just a quick link I stumbled upon looking for uClibc -- if you are working on an embedded system, using uClibc, and want to use C++ code on it -- take a look at uClibc++. I do not know how much of the standard C++ stuff you need and it already supports, and it seems to be an ongoing project, so not clear if it is in a state good enough for you already, but assuming that your work is also under development still, it might be a good alternative to GCC's libstdc++ for your embedded work.
I think this guy explains quite well why that wouldn't make sense. C++ code that uses your shared object but a different libstdc++ would link alright, but wouldn't work.