I currently have one main applications where parts of which can be "live-patched". Eg. some functions with predefined name and signature can be updated during runtime. Currently, the repatching is performed by this two steps:
use std::process::Command to call rustc to compile a cdylib from the source. Each output file has a new name in order to make sure that dlopen does not use the cached old file
use libloading to load and run the newly patched function
This is obviously not ideal for a couple of reasons. My question would be if there is a better way to achieve this? Eg do the compilation from within rust.
Additional info:
The patch files do not require any external crates
The patch files need to be aware of some common lib modules, which do not get live patched
Related
Let's consider Debian or Ubuntu distro where one can install some library package, say libfoobar, and a corresponding libfoobar-dev. The first one contains shared library object. The later usually contains headers, static library variant and cmake exported targets (say, FoobarTargets.cmake) which allow CMake stuff like find_package to work flawlessly. To do so FoobarTargets.cmake have to contain both targets: for static and for shared variant.
From the other side, I've read many articles stating that I (as a libfoobar's author) should refrain from declaring add_library(foobar SHARED ....) and add_library(foobar_static STATIC ...) in CMakeLists.txt in favour of building and installing library using BUILD_SHARED_LIBS=ON and BUILD_SHARED_LIBS=OFF. But this approach will export only one variant of the library into FoobarTargets.cmake because the later build will overwrite the first one.
So, the question is: how to do it the right way? So that package maintainer would not need to patch library's CMakeLists.txt to properly export both variants from the one side. And to adhere CMake's sense of a true way, removing duplicating targets which differ only by static/shared from the other side?
I wrote an entire blog post about this. A working example is available on GitHub.
The basic idea is that you need to write your FoobarConfig.cmake file in such a way that it loads one of FoobarSharedTargets.cmake or FoobarStaticTargets.cmake in a principled, user-controllable way, that's also tolerant to only one or the other being present. I advocate for the following strategy:
If the find_package call lists exactly one of static or shared among the required components, then load the corresponding set of targets.
If the variable Foobar_SHARED_LIBS is defined, then load the corresponding set of targets.
Otherwise, honor the setting of BUILD_SHARED_LIBS for consistency with FetchContent users.
In all cases, your users will link to Foobar::Foobar.
Ultimately, you cannot have both static and shared imported in the same subdirectory while also providing a consistent build-tree (FetchContent) and install-tree (find-package) interface. But this is not a big deal since usually consumers want only one or the other, and it's totally illegal to link both to a single target.
I have a shared library that consists of quite a few .c modules, some of which are themselves linked into the shared library from other static .a libraries. Most of these are referenced internally within the library, but some are not. I'm finding that the linker does not include those modules in the shared library unless there is at least one call to a function in the module from within the shared library. I've been working around this problem by adding calls in a dummy ForceLinkages() function in a module that I know will be included.
That's okay, but it's surprising, since I'm using a .version file to define a public API to this library. I would've thought including functions in those unreferenced .c modules in the .version file would constitute a reference to the modules and force them to be included in the library.
This library was originally developed on AIX, which uses a .exp file to define the public API. And there, I've never had the issue of unreferenced modules not getting included. I.e., referencing the modules in the .exp file was enough to get the linker to pull them in. Is there a way to get the linux linker to work like that. If not, I can continue to use my silly ForceLinkages() function to get the job done...
That's okay, but it's surprising, since I'm using a .version file to define a public API to this library.
The .version file (assuming it's a linker version script) does not define the library API. If only determines which functions are exported from the library (and with which version label) and which are hidden.
I would've thought including functions in those unreferenced .c modules in the .version file would constitute a reference to the modules and force them to be included in the library.
The version script is applied after the linker has decided which objects are going to be part of the library, and which are going to be discarded, and has no effect on the decisions taken earlier.
This is all working as designed.
You need to either use --whole-archive --no-whole-archive (this has a danger of linking in code you don't need and bloating your binaries), or keep adding references as you've done before.
I am trying to build a shared Haskell library that is used by a C project afterwards. I am on a linux platform so my question is from that context.
Suppose I have a haskell package foo with a library named foo, say version 0.1 which exports some functions via ffi.
I can easily generate a shared library (.so) that I can then link with, but my issue is that the generated library is named libHSfoo-0.1-$COMPONENT_ID.so which makes it quite cumbersome to link with since $COMPONENT_ID is unpredictable as far as I can tell.
The $COMPONENT_ID comes, to the best of my knowledge from the following Cabal structure and it looks like I could write cabal hooks to at least copy the generated shared library, or create a symbolic link to it from a fixed location.
I am wondering whether there is a better way to specify the component-id to get an easily predictable name of the shared library without post-processing?
It seems like I can achieve this if in the configure hook I set the configArgs to just the library component, and the configCID to my desired name of the library, but that seems like a fragile solution and I am thinking there is a better way for this.
The name of the library also affects linking when there are other Haskell packages dependent on this one, which would make it even more convenient to specify/override the name.
I am using stack to drive cabal, if that is relevant.
In order to reduce the executable size of a Rust program (called runtime in my code), I am trying to compress it and then include it in a second program (called szl) that decompresses it and executes it.
I have done that by using a Cargo build script in szl that opens the output binary from runtime, compresses it, and then generates a file that is ready for use by include_bytes!.
The issue with this approach is the dependencies are not handled properly. For example, Cargo may try to build szl before runtime (and fail), and when the source code of runtime is modified, szl is not rebuilt.
Is there a way to tell Cargo that szl depends on the binary from runtime (and transitively on the source code of runtime), or should I use another approach such as an external Makefile?
While not exactly your use case, you might get it to work with the links manifest key. It would allow you to express a dependency between the two programs and you can pass more information with DEP_FOO_KEY variables.
Before you go to such drastic measures, it might be worth it to try other known strategies for reducing rust binary size (such as calling strip, remove debug symbols, LTO, panic=abort) etc.
Background
I inherited and maintain a Linux shared library that is very closely coupled with specific hardware; let's call it libfoo.so.0.0.0. This library has been around for some time and "just worked". This library has now become a dependency for several higher-layer applications.
Now, unfortunately, new hardware designs have forced me to create symbols with wider types, thereby resulting in libfoo.so.0.1.0. There have been only additions; no deletions or other API changes. The original, narrow versions of the updated symbols still exist in their original form.
Additionally, I have an application (say, myapp) that depends on libfoo. It was originally written to support the 0.0.0 version of the library but has now been reworked to support the new 0.1.0 APIs.
For backwards compatibility reasons, I would like to be able to build myapp for either the old or new library via a compile flag. The kernel that a given build of myapp will be loaded on will always have exactly one version of the library, known at compile time.
The Question
It is very likely that libfoo will be updated again in the future.
When building myapp, is it possible to specify a minimum version of libfoo to link against based on a build flag?
I know it is possible to specify the library name directly on the build CLI. Will this cause myapp to require exactly that version or will later versions of the lib with the same major revision still be able to link against it (ex. libfoo.so.0.2.0)? I am really hoping to not have to update every dependent app's build each time a new minor version is released.
Is there a more intelligent way of accomplishing this in an application-agnostic way?
References
How do you link to a specific version of a shared library in GCC
You are describing external library versioning, where the app is built against libfoo.so.0, libfoo.so.1, etc. Documentation here.
Using external library versioning requires that exactly the same version of libfoo.so.x be present at runtime.
This is generally not the right technique on Linux, which, through the magic of symbol versioning, allows a single libfoo.so.Y to provide multiple incompatible definitions of the same symbol, and thus allows a single library serve both the old and the new applications simultaneously.
In addition, if you are simply always adding new symbols, and are not modifying existing symbols in incompatible way, then there is no reason to increment the external version. Keep libfoo.so at version 0, provide a int foo_version_X_Y; global variable in it (as well as all previous versions: foo_version_1_0, foo_version_1_1, etc.), and have an application binary read the variable that it requres. If an application requires a new symbol foo_version_1_2 and is run with an old library that only provides foo_version_1_1, then the application will fail to start with an obvious error.