We are currently switching the W32 build-process of a cross-platform (linux, osx, w32) project from VisualStudio to mingw.
One of the problems we are facing is, that our projects creates a dynamic library (foo.dll), against which 3rd party projects can link. For this to work on W32/MSVC, an import library is required (foo.lib).
Now, following the documentation it is pretty easy to create a .def file which holds all the information required for importing the library:
gcc -shared -o foo.dll foo-*.o -Wl,--output-def,foo.def
In order to use the foo.def file, the docs tell me to use the Microsoft LIB tool to build a foo.lib from it:
lib /machine:i386 /def:testdll.def
This obviously requires me to have (a subset of) MSVC installed on the build computer.
However, we'd like to cross-compile the entire thing on our linux systems (probably even on some CI), which makes the installation of MSVC rather tedious.
So I wonder, whether there's a native MinGW way to convert the foo.def file into a foo.lib import library?
(We are aware that in the end, only MSVC users will require the import library and that they will have the lib tool ready at hand. However, since we've always shipped the foo.lib file, switching to foo.def would break 3rd parties build systems - something we would like to avoid).
To produce an import library that is similar to the one generated by Microsoft's link.exe, you can use llvm-dlltool (part of the LLVM compiler project):
llvm-dlltool -m i386:x86-64 -d foo.def -l foo.lib
Substitute i386:x86-64 for i386 if you would like to create a 32-bit library. For more details see this answer to How to generate an import library (LIB-file) from a DLL?.
Note that some MinGW projects generate a .dll.a file (as produced by binutils dlltool). While this could be renamed to .lib and function as import library, I found that it would result in broken binaries if a MSVC projects links to multiple .dll.a libraries. So stick to llvm-dlltool instead for improved compatibility.
we'd like to cross-compile the entire thing on our linux systems
I'm not aware of any MS LIB clone portable to Linux, however POLIB from Pelles C distribution is free, small, self-contained and compatible with MS tool. It has no dependencies other than kernel32.dll, so, I believe, it will run under Wine too.
Related
I'm writing a shared library that itself depends on boost and pcl libraries.
When generating .pc file for my library should I add all these libraries also to the .pc file as dependencies?
It's been a long time since I last time studied these things and I'm a bit confused how this worked again on Linux. When my test app links to my lib I have to add all these pcl and boost libs again to the build even though the lib already has been linked against these libs.
But when I look at the deps of libQtGui.so, for example, it has tens of all kinds of libs it links to, but I don't need to make my app link to those libs...only -lQtGui is enough.
I have just used CMake and link_libraries to add boost and pcl libs.
When generating .pc file for my library should I add all these libraries also to the .pc file as dependencies?
It depends on API of your library:
if public (i.e. installable) headers of your lib use boost/pcl (i.e. have #inclue <boost/...>) (in other words you used PUBLIC (or INTERFACE) named keywords when link your library against boost/pcl in CMake+target_link_libraries) -- then yes you need to add 'em;
otherwise, it depends on what exactly you have at the end -- i.e. does your DSO has DT_NEEDED entries for boost/pcl libs (most likely) or not (you can check it w/ ldd <your-lib>.so). For the last case, you also need to add your dependencies to the *.pc files.
Also, in case of binary dependency from boost/pcl (dunno if the latter has any DSO or not) please make sure you specify exact location of the linked libs -- cuz a user may have multiple (co-existed) boost installations (potentially incompatible) or can do upgrade (later) to other (binary incompatible) version (and you can't really do smth w/ it)… It is important to be linked to the same (or at least binary compatible, which is kinda hard to guarantee for boost) library as you did…
I have just used CMake and link_libraries to add boost and pcl libs.
Please read smth about "Modern CMake" and stop using link_libraries :-) -- use target_link_libraries instead…
I have my project currently compiling under gcc. It uses Boost, ZeroMQ as static .a libraries and some .so libraries like SDL. I want to go clang all the way but not right now. I wonder if it is possible to compile code that uses .a and .so libraries that were compiled under gcc with clang?
Yes, you usually can use clang with GCC compiled libraries (and vice versa, use gcc with CLANG compiled libraries), because in fact it is not compilation but linking which is relevant. You might be unlucky and get unpleasant suprises.
You could in principle have some dependencies on the version of libstdc++ used to link the relevant libraries (if they are coded in C++). Actually, that usually does not matter much.
In C++, name mangling might in theory be an issue (there might be some corner cases, even incompatibilities between two different versions of g++). Again, in practice it is usually not an issue.
So usually you can mix CLANG (even different but close versions of it) with GCC but you may have unpleasant surprises. What should be expected from any C++ compiler (be it CLANG or GCC) is just to be able to compile and link an entire software (and all libraries) together using the same compiler and version (and that includes the same C++ standard library implementation). This is why upgrading a compiler in a distribution is a lot of work: the distribution makers have to ensure that all the packages compile well (and they do get surprises!).
Beware that the version of libstdc++ does matter. Both Clang & GCC communities work hard to make its ABI compatible for compiler upgrades, but there are subtle corner cases. Read the documentation of your particular and specific C++ standard library implementation. These corner cases could explain mysterious crashes when using a good C++ library binary (compiled with GCC 5) in your code compiled with GCC 8. The bug is not in the library, but the ABI evolved incompatibly.
At least for Crypto++ library this does not work (verified :-( ). So for c++ code it is less likely to work, while pure c code would probably link OK.
EDIT: The problem started appearing with Mac OS X 10.9 Mavericks and Xcode-5, which switched the default C++ library for clang from libstdc++ to libc++. It did not exist on Mac OS X 10.8 and earlier.
The solution appears to be: if you need to compile C++ code with clang, and link it to a gcc-compiled library, use "clang++ -stdlib=libstdc++". The linking is successful, and the resulting binary runs correctly.
CAVEAT: It does not seem to work the other way: even though you can build a library compiled with "clang++ -stdlib=libstdc++" and link gcc-compiled code with it, this code will crash with SEGV. So far I found the only way to link with a clang-compiled library is compiling your code with clang, not gcc.
EDIT2:
GCC-12 seems to include -stdlib= flag. Compiling with g++ -stdlib=libc++ creates Clang++-compatible object files. Very nice.
I do have an additional data point to contribute, on the topic of "unpleasant surprises" mixing code from different versions of different compilers. Therein, I link Victor Shoup's C++-based NTL number theory library with a small piece of driver code that just prints out a large factorial computed by the NTL code, a number with a decimal representation that might span multiple lines if sufficiently large.
I have built and installed SageMath (and its version of NTL) on my system running OS X 10.11.6, and also have a current installation of MacPorts. In /usr/bin I find for gcc --version
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
My MacPorts gcc gives
gcc (MacPorts gcc9 9.1.0_2) 9.1.0
Now, the SageMath build system requires that MacPorts be moved out of the way, so I assume SageMath builds NTL using Apple's development toolset. The SageMath build log is full of invocations of gcc. SageMath actually builds gcc from source if the system on which the makefile is run has too old a version of Apple's developer tools.
My driver code computes big factorials and uses methods of the NTL class ZZ; I initially had tested this by linking to an NTL static library I built myself, and I changed it to link to the SageMath version because I find it pleasing not to duplicate libraries. Now I understand a bit more about the pitfalls which may arise in this process.
The old makefile invoked g++ to make the executable, but this failed at linking phase with the message:
Undefined symbols for architecture x86_64:
"NTL::operator<<(std::basic_ostream<char, std::char_traits<char> >&, NTL::ZZ const&)",
referenced from:
prn_factorial(int, NTL::ZZ&) in print.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
I had to think about this and run experiments for about 15 minutes before deciding on my own to change the makefile to invoke clang++ which in my current path invokes the MacPorts version
clang version 7.0.1 (tags/RELEASE_701/final)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-7.0/bin
This time, the makefile successfully linked and built my executable. I conclude that this represents one of those edge cases with "unpleasant surprises". Probably I should conclude that working with details of C++ is not for me; big software systems like SageMath are developed just so hobbyists don't really have to muck around with details like these.
I have a code that I have written in Fortran during my PhD, and now I am collaborating with some researcher that uses Linux, and they need my model, that is basically a single executable file. In the future I will probably make it open source, but up to know they just want the executable, also because they are not programmers and they have never compiled a program in their life. So the question is: is it possible to compile it on my linux machine and then send it to them in order to use it in another linux machine?Or does the linux version and distribution matter?
thank you very much
A.
If you do not use many libraries you can do that. One option is statically linking the executable (-static or similar compiler option). You need to have the static versions of all needed libraries for that. The have .a suffix. They are often not installed by default in Linux distributions and often they are not supplied in the repositories at all.
In my distrbution (OpenSuSE) they are in packages like glibc-devel-static, lapack-devel-static and similar.
The other option would be to compile the executable on a compatible distribution the users will have (GLIBC version is important) and supply all .so dynamically linked libraries they will need with your executable.
All of this assumes you use the same platform, like i586 or amd64 or arm like wallyk comments. I mostly assumed you are on a PC. You can force most compilers to produce a 32-bit or 64-bit executable by -m32 or -m64 option. You need the right version of the development libraries for that.
I work for a company that recently purchased a piece of hardware accompanied by an SDK. Unfortunately, all the SDK libraries were compiled with a Microsoft Visual C++ compiler (I don't know which one) and cannot be used by MinGW (I develop in Code::Blocks using the MinGW C++ compiler).
I've attempted to convert the libraries using reimp (from the MinGW utilities collection), which has worked in the past with static libraries from other vendors, but in this case the converted libraries result in "undefined reference" errors when linked.
The def files generated by reimp for each library during the conversion process don't look very good (they're filled with lines like ??0nameOfFunction##QEAA#AEBV0##Z, while the def files generated during a successful conversion contain lines similar to nameOfFunction#32), so it seems that the vendor's libraries are simply of a type that reimp can't convert.
Are there any other options that would allow me to use these libraries with MinGW? If not, is it reasonable to request that the vendor recompile their libraries with g++ (i.e., is it something they could feasibly do given that the libraries were originally developed using MSVC)?
Any comments or suggestions are appreciated!
I have an executable on Linux that loads libfoo.so.1 (that's a SONAME) as one of its dependencies (via another shared library). It also links to another system library, which, in turn, links to a system version, libfoo.so.2. As a result, both libfoo.so.1 and libfoo.so.2 are loaded during execution, and code that was supposed to call functions from library with version 1 ends up calling (binary-incompatible) functions from a newer system library with version 2, because some symbols stay the same. The result is usually stack smashing and a subsequent segfault.
Now, the library which links against the older version is a closed-source third-party library, and I can't control what version of libfoo it compiles against. Assuming that, the only other option left is rebuilding a bunch of system libraries that currently link with libfoo.so.2 to link with libfoo.so.1.
Is there any way to avoid replacing system libraries wiith local copies that link to older libfoo? Can I load both libraries and have the code calling correct version of symbols? So I need some special symbol-level versioning?
You may be able to do some version script tricks:
http://sunsite.ualberta.ca/Documentation/Gnu/binutils-2.9.1/html_node/ld_26.html
This may require that you write a wrapper around your lib that pulls in libfoo.so.1 that exports some symbols explicitly and masks all others as local. For example:
MYSYMS {
global:
foo1;
foo2;
local:
*;
};
and use this when you link that wrapper like:
gcc -shared -Wl,--version-script,mysyms.map -o mylib wrapper.o -lfoo -L/path/to/foo.so.1
This should make libfoo.so.1's symbols local to the wrapper and not available to the main exe.
I can only come up with a work-around. Which would be to statically link a version of the "system library" that you are using. For your static build, you could make it link against the same old version as the third-party library. Given that it does not rely on the newer version...
Perhaps it is also possible to avoid these problems with not linking to the third-party library the ordinary way. Instead, your program could load it at execution time. Perhaps then it could be shadowed against the rest. But I don't know much about that.