It seems that all my adult life I've been tormented by the VC++ linker complaining or balking because various libraries do not agree on which version of the Runtime library to use. I'm never in the mood to master that dismal subject. So I just try to mess with it until it works. The error messages are never useful. Neither is the Microsoft documentation on the subject - not to me at least.
Sometimes it does not find functions - because the name-mangling is not what was expected? Sometimes it refuses to mix-and-match. Other times it just says, "LINK : warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library" Using /NODEFAULTLIB does not work, but the warning seems to be benign. What the heck is "DEFAULTLIB" anyway? How does the linker decide? I've never seen a way to specify to the linker which runtime library to use, only how to tell the compiler which library to create function calls for.
There are "dependency walker" programs that can inspect object files to see what DLL's they depend on. I just ran one on a project I'm trying to build, and it's a real mess. There are system .libs and .dll's that want conflicting runtime versions. For example, COMCTL32.DLL wants MSVCRT.DLL, but I am linking with MSVCRTD.DLL. I am searching to see if there's a COMCTL32D.DLL, even as I type.
So I guess what I'm asking for is a tutorial on how to sort those things out. What do you do, and how do you do it?
Here's what I think I know. Please correct me if any of this is wrong.
The parameters are Debug/Release, Multi-threaded/Single-threaded, and static/DLL. Only six of the eight possible combinations are covered. There is no single-threaded DLL, either Debug or Release.
The settings only affect which runtime library gets linked in (and the calling convention to link with it). You do not, for example, have to use a DLL-based runtime if you are building a DLL, nor do you have to use a Debug version of runtime when building the Debug version of a program, although it seems to help when single-stepping past system calls.
Bonus question: How could anyone or any company create such a mess?
Your points (1) and (2) look correct to me. Another thing to note with (2) is that linking in the debug CRT also gives you access to things like enhanced heap checking, checked iterators, and other assorted sanity checks. You cannot redistribute the debug CRT with your application, however -- you must ship using the release build only. Not only is it required by the VC license, but you probably don't want to be shipping debug binaries anyway.
There is no such thing as COMCTL32D.DLL. DLLs that are part of Windows must load the CRT that they were linked against when Windows was built -- this is included with the OS as MSVCRT.DLL. This Windows CRT is completely independent from the Visual C++ CRT that is loaded by the modules that comprise your program (MSVCRT.DLL is the one that ships with Windows. The VC CRT will include a version number, for example MSVCRT80.DLL). Only the EXE and DLL files that make up your program are affected by the debug/release multithreaded/single-threaded settings.
The best practice here IMO is to pick a setting for your CRT and standardize upon it for every binary that you ship. I'd personally use the multithreaded DLL runtime. This is because Microsoft can (and does) issue security updates and bug fixes to the CRT that can be pushed out via Windows Update.
Related
To repeat: I'm looking for ABI compatibility between libraries of the same Visual-C++ version!
We want to mix and match some internal C++ DLLs from different teams - built at different times with different project files. Because of long build times, we exactly want to avoid large monolithic builds where each team re-compiles the source code of another team's library.
When consuming C++ DLLs with C++ interfaces it is rather clear that you only can do this if all DLLs are compiled with the same compiler / Visual Studio version.
What is not readily apparent to me is what, exactly needs to be the same to get ABI compatibility.
Obviously debug (_DEBUG) and release (NDEBUG) cannot be mixed -- but that's also apparent from the fact that these link to different versions of the shared runtime.
Do you need the exact same compiler version, or is it sufficient that the resulting DLL links to the same shared C++ runtime -- that is, basically to the same redistributable? (I think static doesn't fly when passing full C++ objects around)
Is there a documented list of compiler (and linker) options that need to be the same for two C++ DLLs of the same vc++ version to be compatible?
For example, is the same /O switch necessary - does the optimization level affect ABI compatibility´? (I'm pretty sure not.)
Or do both version have to use the same /EH switch?
Or /volatile:ms|iso ... ?
Essentially, I'd like to come up with a set of (meta-)data to associate with a Visual-C++ DLL that describes it's ABI compatibility.
If differences exist, my focus is on VS2015 only at the moment.
Have been thinking this through the last days, and what I did do was to try to see if some use-cases exists where devs have already needed to categorize their C++ build to make sure binaries are compatible.
One such place is the Native Packages from nuget. So I looked at one package there, specifically the cpprestsdk:
The binaries in the downloadable package as split like this:
native\v120\windesktop\msvcstl\dyn\rt-dyn\x64\Release\
^ ^ ^ ^ ^
VS version | not sure | uses cpp-runtime dynamically
| lib itself dynamic (as opposed to static)
or WinXP or WinApp(WinRT?)
I pulled this out from this example, because I couldn't find any other docs. I also know that the boost binaries build directory is separated in a similar way.
So, to get to a list of meta data to identify the ABI compatibility, I can preliminarily list the following:
VC version (that is, the version of the C and CPP runtime libraries used)
one point here is that e.g. vc140 should be enough nowadays - given how the CRT is linked in, all possible bugfixes to the versioned CRT components must be ABI compatible anyway, so it shouldn't matter which version a given precompiled library was built with.
pure native | managed (/CLI) | WinRT
how the CRT is consumed (statically / dynamically)
bitness / platform (Win32, x64, ARM, etc.)
Release or Debug version (i.e. which version of the CRT we link to)
plus: _ITERATOR_DEBUG_LEVEL ... if everyone goes with the defaults, fine, if a project does not, it must declare so
Additionally my best guess as to the following items:
/O must not matter - we constantly mix&match binaries with different optimization settings - specifically, this is even working for object files within the same binary
/volatile - since this is a code-gen thing, I have a hard time imagining how this could break an ABI
/EH - except for the option to disable all exception, in which case you obviously can't call anything that throws, I'm pretty confident this is save from an ABI perspective: There are possible pitfalls here, but I think they can't really be categorized into ABI compat. (Maybe some complex callback chains could be said to be ABI incompatible, not sure)
Others:
Default calling convention (/G..) : I think this would break at link time, when mangled export symbols and header declarations don't match up.
/Zc:wchar_t - will break at link time (It's actually ABI compatible, but the symbols won't macth.)
Enable RTTI (/GR) - not too sure 'bout this one - I never have worked with this disabled.
Why do ucrtbase.dll and vcruntime140.dll overlap in some of the functions they export according to Dependency Walker?
Disclaimer: This is currently purely of academic interest to me.
I'm currently trying to understand the layout of the Microsoft Visual-C++ CRT related DLL files. Find Info un the UCRT and the files in general here:
Introducing the Universal CRT
CRT Library Features
In short, you have these (toplevel) DLL dependencies at runtime for a normal C++ App:
ucrtbase.dll - "compiler independent" stuff
vcruntime<ver>.dll - "compiler dependent" stuff
msvcp<ver>.dll - C++ standard library
What can be highlighted from this info is:
From the blog entry:
... split the CRT into two logical parts: The VCRuntime, which contained
the compiler support functionality required for things like process
startup and exception handling ...
and from the MSDN page:
The vcruntime library contains Visual C++ CRT implementation-specific
code, such as exception handling and debugging support, runtime checks
and type information, implementation details and certain extended
library functions. This library is specific to the version of the
compiler used.
While browsing the DLLs with Dependency Walker, I noticed that both the ucrt and the vcruntime export the function _CxxThrowException. This function is an old acquaintance if you've ever been looking at vc++ stack traces:
Builds the exception record and calls the runtime environment to start processing the exception.
I am quite surprised to find this exported from the ucrtbase.dll as - as both quotes above indicate - I'd have thought this machinery to firmly belong to the compiler specific side of things.
While writing this up, I've noticed some other overlaps: A very few of the standard C library functions (memcpy, ..., strstr, ...) are also exported from vcruntime140.dll although I'd have expected them to only live in ucrtbase.
So what is going on here and what can I learn from this?
The Universal CRT (ucrtbase.dll) contains a private copy of the VCRuntime, for use by Windows operating system components. This private copy of the VCRuntime is an internal implementation detail of the operating system and may change at any time (i.e., there is no application compatibility guarantee whatsoever.
Do not under any circumstances use these exports from the Universal CRT. (No library in the Windows SDK provides linkable symbols for these exports, so it's impossible to accidentally use them.)
In my open-source project Artha I use libnotify for showing passive desktop notifications to the user.
Instead of statically linking libnotify, a lookup at runtime is made for the shared object (.so) file via dlload, if available on the target machine, Artha exposes the notification feature in it's GUI. On app. start, a call to dlload with filename param as libnotify.so.1 is made and if it returns a non-null pointer, then the feature is exposed.
A recurring problem with this model is that every time the version number of the library is bumped, Artha's code needs to be updated, currently libnotify.so.4 is the latest to entail such an occurance.
Is there a linux system call (irrespective of the distro the app. is running on), which can tell me if a particular library's shared object is available at runtime? I know that there exists the bruteforce option of enumerating the library by going from 1 to say 10, I find the solution ugly and inelegant.
Also, if this can be addressed via autoconf, then that solution is welcome too I.e. at build time, based on the target machine, the configure.h generated should've the right .so name that can be passed to dlload.
P.S.: I think good distros follow the style of creating links to libnotify.so.x so that a programmer can just do dlload("libnotify.so", RTLD_LAZY) and the right version numbered .so is loaded; unfortunately not all distros follow this, including Ubuntu.
The answer is: you don't.
dlopen() is not designed to deal with things like that, and trying to load whichever soversion you find on the system just because it happens to have the symbols you need is not a good way to do it.
Different sonames have different ABIs, and different ABIs means that you may be calling the same exact symbol name that is expecting a different set (or different size) of parameters, which will cause crashes or misbehaviour that are extremely difficult do debug.
You should have a read on how shared object versions work and what an ABI is.
The libfoo.so link is there for the link editor (ld) and is usually installed with the -devel packages for that reason; it might also very well not be a link but rather a text file with a linker script, often times on purpose to avoid exactly what you're trying to do.
Let me get right to the point:
Main application:
C# (4.0), AnyCPU.
Library:
Wrapper for native .dll written in C++/CLI. Compiled in two versions; x86 and x64, both signed with the same .snk key (using this workaround)
Limitations:
In the end a single distribution package is required for x86 and x64 platforms.
Main application needs strong name due to references to other strongly named libs.
Rewriting the library using managed C# and P/Invoke is an absolute last way out.
The problem:
As long as the main application, at compile time, references the version (x86 or x64) of the library that is needed when run, this is all working fine.
Moving the same compiled output - and exchanging the library with the right platform version during installation - does not work since the signature of the library changes from that of the referenced one.
In a test application without any strong naming I can switch between them as needed.
The question:
Is there a way to enable switching between the x86 and x64 libraries within the set limitations, or is strong naming preventing any possible solution other than rewriting the lib?
Let me clarify that it is not a question about finding the correct .dll (as discussed here) but about being able to load the .dll once found.
#Damien_The_Unbeliever's comment got me thinking and he is right in that the strong names are the same, and it was not the actual issue.
I found another difference between the two versions of the library; the output name was set to nnn.dll and nnnx64.dll. Changing it so that both have the same output name magically made it all work.
Perhaps someone knows why such a setting matters, I certainly don't.
I'm trying to hook up a real-time crash reporting service like airbrake, bugsense or TestFlight's SDK but I'm wondering if the crash reports that are generated from crashes are any good when compiling your MonoTouch project using the LLVM compiler.
When you're configuring an iPhone build if you go to the proj settings > iPhone Build > Advanced tab it says "Experimental, not compatible with debug mode". This is why I'm questioning the stacktrace from the crash reports.
There are several points to consider here:
a) enabling debug on your builds:
tells the compilers to emit debugging symbols (e.g. the .mdb files) which includes a lot of information (variables names, scopes, lines numbers...);
add extra debugging code to your application (e.g. to connect the application, on the device, to the debugger, on your Mac);
tells the compiler (e.g. AOT) to disable some optimizations (that would make debugging harder);
This result in larger, slower applications that contains a lot of data you don't want people to access (e.g. if you fear reverse engineering). For releases it's a no win situation for everyone.
b) using the LLVM compiler won't work with debug mode. It's generally not an issue since, when debugging, you'll likely want the build process to be as fast as possible (and LLVM is slower to build). A problematic case is if your bug shows up only on LLVM builds.
c) The availability of managed stack traces do not requires debug symbols. They are built from the metadata available in your .dll and .exe files. But, when debugging symbols are available, the stack trace will include the line numbers and filenames for each stack frame.
d) I never used the tools you mentioned, but I do believe them to be useful :-) You might wish to ask specific questions about them (wrt MonoTouch). Otherwise I think it's worth testing to see if the level of details differ (and if the extra details are of any help to you). IMO I doubt it will bring you more than the actual 'cost' of shipping 'debug' builds.
first create a "crash me" feature in your application;
then compare reported results from non-LLVM "release" and "debug" builds;
next compare the non-LLVM "release" and LLVM "release" builds;
It be nice to post your experience of the above: here, monotouch mailing-list and/or a blog entry :-)