I'm writing a Rust wrapper for a C library that conditionally requires the UniformTypeIdentifiers framework depending on the values of __MAC_OS_X_VERSION_MIN_ALLOWED and __MAC_OS_X_VERSION_MAX_ALLOWED (which are the minimum and maximum macOS versions targeted). The C library does conditional compilation that uses the UniformTypeIdentifiers library (which was introduced recently) because the old way of doing things is now deprecated.
Is there a way to obtain these two values (the minimum and maximum macOS versions targeted) in build.rs so that I can decide whether to emit the cargo:rustc-link-lib=framework=UniformTypeIdentifiers line that links the library? I can't simply emit it all the time, because on older Macs this will lead to a build error, since the library would be unavailable.
Related
By forking and playing with some code, I noticed that Cargo can download and bundle multiple versions of the same crate in the same project (native-tls 0.1.5 and 0.2.1, for example). I have wasted so much time by looking at the documentation of the wrong version.
I have looked for some information about this behaviour but I was not able to find anything. Is this documented somewhere?
Is there an easy way to determine/detect the version used by the code you're working on (current edited file)? Or can we tell Cargo to show some warnings/prevent the build if two versions the same crate are required?
It was a conscious decision when designing Rust to allow multiple versions of the same crate.
You have probably heard of Dependency Hell before, which occurs when 2 (or more) dependencies A and B have a common dependency C but each require a version which is incompatible with the other.
Rust was designed to ensure that this would not be an issue.
In general, cargo will attempt to find a common version which satisfies all requirements. As long as crate authors use SemVer correctly, and the requirements give enough leeway, a single version of the dependency can be computed and used successfully.
Sometimes, however, multiple versions of the same dependency are necessary, such as in your case since 0.1.x and 0.2.x are considered two different major versions. In this case, Rust has two features to allow the two versions to be used within the same binary:
a unique hash per version is appended to each symbol.
the type system considers the same type Foo from two versions of C to be different types.
There is a limitation, of course. If a function of A returns an instance of C::Foo and you attempt to pass it to a function of B, the compiler will refuse it (it considers the two types to be different). This is an intractable problem1.
Anytime the dependency on C is internal, or the use of C is isolated, it otherwise works automatically. As your experience shows, it is so seamless that the user may not even realize it is happening.
1 See the dtolnay trick that crate authors can use to allow some types to be interchangeable.
Cargo can indeed link multiple versions of some crate, but only one of those versions can be a direct dependency. The others are indirect references.
The direct reference is always the version referenced by Cargo.toml and on top-level of Cargo.lock (while the indirect references are in the dependencies subsections).
I am not sure how much it is documented, unfortunately.
Currently I need to resort in a sparse solver for a project. However I use an old version of Eigen3 on Ubuntu 12.04 (during the thesis I avoid unnecessary updates/upgrades), which means that all the information that I find online cannot be used at the moment because of my outdated version, while the few unsupported tools of my version are very hard to use (weird compilation errors - e.g. with unsupported/Eigen/SparseExtra)
I think that I should upgrade to the last stable version, however it is very critical that I will be able to replicate the numbers of all the experiments that I got with the current outdated version. Is Eigen safe when it comes to backwards compatibility?
Eigen is also a dependency for PCL that I'm using, so I'm not sure if this complicates things. Everything is installed with apt-get. Linking to a new version of Eigen locally for experimentation is not possible, because PCL complains and expects to find Eigen installed globally (i.e. in /usr/local/include).
Eigen is source (API) and binary (ABI) backward compatible (of course, except for unsupported/*). However, the numerical results might be slightly different due to different rounding errors, but that's already the case when, e.g., enabling/disabling SSE or OpenMP.
Since Eigen is header only, it is very easy to try the newest version.
I follow a version scheme for a library with a version number of three parts and a so version of two parts. example-1.0.0 and libexample.so.1.0.
The last number in the version string is updated when I make changes without breaking the ABI. The second number is updated when I add new symbols and the major version number is used for incompatible changes.
The so version is updated when symbols are added even if it does not break compatibility with other programs. This means that programs need to be recompiled because the so version has changed even if the library still is ABI compatible with older versions.
Should I avoid updating the so version when I add new symbols?
This means that programs need to be recompiled because the so version has changed even if the library still is ABI compatible with older versions.
That means you are not doing it correctly. You should only change SONAME when doing ABI-incompatible change. It is customary to use example.1 as the SONAME. Documentation.
P.S. If you only care about Linux, you likely should stop doing external versioning altogether, and instead use symbol versioning to provide a single libexample.so.1 that provides multiple ABI-incompatible symbols to both old and new client binaries.
Background
I inherited and maintain a Linux shared library that is very closely coupled with specific hardware; let's call it libfoo.so.0.0.0. This library has been around for some time and "just worked". This library has now become a dependency for several higher-layer applications.
Now, unfortunately, new hardware designs have forced me to create symbols with wider types, thereby resulting in libfoo.so.0.1.0. There have been only additions; no deletions or other API changes. The original, narrow versions of the updated symbols still exist in their original form.
Additionally, I have an application (say, myapp) that depends on libfoo. It was originally written to support the 0.0.0 version of the library but has now been reworked to support the new 0.1.0 APIs.
For backwards compatibility reasons, I would like to be able to build myapp for either the old or new library via a compile flag. The kernel that a given build of myapp will be loaded on will always have exactly one version of the library, known at compile time.
The Question
It is very likely that libfoo will be updated again in the future.
When building myapp, is it possible to specify a minimum version of libfoo to link against based on a build flag?
I know it is possible to specify the library name directly on the build CLI. Will this cause myapp to require exactly that version or will later versions of the lib with the same major revision still be able to link against it (ex. libfoo.so.0.2.0)? I am really hoping to not have to update every dependent app's build each time a new minor version is released.
Is there a more intelligent way of accomplishing this in an application-agnostic way?
References
How do you link to a specific version of a shared library in GCC
You are describing external library versioning, where the app is built against libfoo.so.0, libfoo.so.1, etc. Documentation here.
Using external library versioning requires that exactly the same version of libfoo.so.x be present at runtime.
This is generally not the right technique on Linux, which, through the magic of symbol versioning, allows a single libfoo.so.Y to provide multiple incompatible definitions of the same symbol, and thus allows a single library serve both the old and the new applications simultaneously.
In addition, if you are simply always adding new symbols, and are not modifying existing symbols in incompatible way, then there is no reason to increment the external version. Keep libfoo.so at version 0, provide a int foo_version_X_Y; global variable in it (as well as all previous versions: foo_version_1_0, foo_version_1_1, etc.), and have an application binary read the variable that it requres. If an application requires a new symbol foo_version_1_2 and is run with an old library that only provides foo_version_1_1, then the application will fail to start with an obvious error.
What is the best way to determine a pre-compiled binary's dependencies (specifically in regards to glibc and libstdc++ symbols & versions) and then ensure that a target system has these installed?
I have a limitation in that I cannot provide source code to compile on each machine (employer restriction) so the defacto response of "compile on each machine to ensure compatibility" is not suitable. I also don't wish to provide statically compiled binaries -> seems very much a case of using a hammer to open an egg.
I have considered a number of approaches which loosely center around determining the symbols/libraries my executable/library requires through use of commands such as
ldd -v </path/executable>
or
objdump -x </path/executable> | grep UND
and then somehow running a command on the target system to check if such symbols, libraries and versions are provided (not entirely certain how I do this step?).
This would then be followed by some pattern or symbol matching to ensure the correct versions, or greater, are present.
That said, I feel like this will already have been largely done for me and I'm suffering from ... "a knowledge gap ?" of how it is currently implemented.
Any thoughts/suggestions on how to proceed?
I should add that this is for the purposes of installing my software on a wide variety of linux distributions - in particular customised clusters - which may not obey distribution guidelines or standardised packaging methods. The objective being a seamless install.
I wish to accomplish binary compatibility at install time, not at a subsequent runtime, which may occur by a user with insufficient privileges to install dependencies.
Also, as I don't have source code access to all the third party libraries I use and install (specialised maths/engineering libraries) then an in-code solution does not work so well. I suppose I could write a binary that tests whether certain symbols (&versions) are present, but this binary itself would have compatibility issues to run.
I think my solution has to be to compile against older libraries (as mentioned) and install this as well as using the LSB checker (looks promising).
GNU libraries (glibc and libstdc++) support a mechanism called symbol versioning. First of all, these libraries export special symbols used by dynamic linker to resolve the appropriate symbol version (CXXABI_* and GLIBCXX_* in libstdc++, GLIBC_* in glibc). A simple script to the tune of:
nm -D libc.so.6 | grep " A "
will return a list of version symbols which can then be further shell processed to establish the maximum supported libc interface version (same works for libstdc++). From a C code, one has the option to do the same using dlvsym() (first dlopen() the library, then check whether certain minimal version of the symbols you need can be looked up using dlvsym()).
Other options for obtaining a glibc version in runtime include gnu_get_libc_version() and confstr() library calls.
However, the proper use of versioning interface is to write code which explicitly links to a specific glibc/libstdc++ library version. For example, code linking to GLIBC_2.10 interface version is expected to work with any glibc version newer than 2.10 (all versions up to 2.18 and beyond). While it is possible to enable versioning on a per symbol basis (using a ".symver" assembler/linker directive) the more reasonable approach is to set up a chroot environment using older (minimal supported) version of the toolchain and compile the project against it (it will seamlessly run with whatever newer version encountered).
Use Linux Application Checker tool ([1], [2], [3]) to check binary compatibility of your application with various Linux distributions. You can also check compatibility with your custom distribution by this tool.