How to export lib incorporated with C dynllib? - rust

Say, I have a lib package with a C shared library in a sub-directory. This rust library package compiles without errors, by means of tailored build.rs which sub-calls make in the sub-directory and then dynamicly links against the *.so product of make.
The problem arises when I try to link a binary package against this rust lib has the C shared lib within itself.
error: linking with `cc` failed: exit status: 1
= note: /usr/bin/ld: cannot find -lcuda_wrapper
collect2: error: ld returned 1 exit status
The dependency graph can be shown as the following one:
C-shared-lib < rust-lib < rust-bin
And cargo build fails to build rust-bin for not finding that C-shared-lib.
providing LD_LIBRARY_PATH for cargo build neither by command line nor build.rs within rust-bin facilitates the problem and lifts the error.
How to propagate the directory wherein resides libcuda_wrapper.so?

You can link your C library dynamically or statically.
In order to do that you can use add one of the following lines to your build.rs (note that you linkely won't need a lib prefix):
println!("cargo:rustc-link-lib=static=cuda_wrapper");
println!("cargo:rustc-link-lib=dylib=cuda_wrapper");
In order to define a search path, add this line to build.rs
println!("cargo:rustc-link-search=native={}", path);
You would likely want to add it to your rust-lib build.rs. Altenatively, you can create a build.rs for every executable which depends on rust-bin and add the same set of lines here.
It is a good idea to ensure that your C library build output is located under OUT_DIR env variable (you can get it using env::var("OUT_DIR").unwrap() in build.rs). The way cargo clean will work as expected as well.
More details here:
https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib
https://doc.rust-lang.org/rustc/command-line-arguments.html#option-l-link-lib
https://doc.rust-lang.org/cargo/reference/build-script-examples.html#building-a-native-library
https://doc.rust-lang.org/cargo/reference/environment-variables.html

Related

Why Rust build.rs is unable to find C headers?

Basically I am trying to cargo build a crate which has a build.rs file.
This crate is located inside a bigger project and it's supposed to be a lib crate.
And inside this build.rs file, I am trying to compile a .C file which includes a couple of headers.
Fun fact: I got this build.rs file and crate structure from another little demo crate, and in that demo crate I had no problem to compile this exact C file with these headers.
FULL ERROR HERE:
Here is a link to github: https://github.com/mihaidogaru2537/FirecrackerPlayground/tree/dpdk_component/firecracker/src/dpdk_component
There is the crate I am talking about and you can also see the bigger project in which it resides. In the README file you can see the full error.
Either I do cargo build from the root of the big project or from the root of this problematic crate, the error is the same.
"cargo:warning=/usr/include/asm-generic/errno.h:5:10: fatal error: asm-generic/errno-base.h: No such file or directory
cargo:warning= 5 | #include <asm-generic/errno-base.h>"
The missing file might change depending on the .flag("-I/path/..") calls I am doing inside the build.rs
As you can see, right now it's unable to find errno-base.h, but I am including the path to asm-generic.
Here is the code of the build.rs file from the crate where the compilation of this C file works, as you can see, I did not have to add any include flags before calling compile.
fn main() {
// Tell cargo to tell rustc to link the system bzip2
// shared library.
// println!("cargo:rustc-link-lib=rte_ring");
// println!("cargo:rustc-link-lib=rte_mempool");
// Tell cargo to invalidate the built crate whenever the wrapper changes
// println!("cargo:rerun-if-changed=wrapper.h");
let _src = ["src/static-functions.c"];
println!("cargo:rerun-if-changed=build.rs");
let mut builder = cc::Build::new();
let build = builder
.file("src/static-functions.c")
.flag("-Wno-unused-parameter");
build.compile("foo");
}
Additional info:
The problematic crate is pretty small, see the link above. There is the build.rs file, C file and header file is inside the include directory.
One thing that I suspect, is that the target of the bigger project:
TARGET = Some("x86_64-unknown-linux-musl")
might affect the way the C file is compiled.
In the project where the compilation is working, I am not using that linux-musl stuff.
I am a total noob when it comes to Rust, but I do have a decent understanding of how C/C++ works.
I am running the project on Ubuntu 20.04
Those missing headers are a result of importing DPDK headers, I have DPDK libraries installed on the machine in question.
Let me know if you have any questions, sorry for the long read and thank you.
I somehow managed to fix it by adjusting the cargo build command to use x86_64-unknown-linux-gnu as target instead of x86_64-unknown-linux-musl (By default cargo build was doing the musl target somehow)
So if you are trying to build a rust app which is using DPDK libraries and you are getting missing headers, make sure to try the following:
cargo build --target=x86_64-unknown-linux-gnu
Well if you have to use musl and there is no alternative, I don't have an answer. But to me this was enough.
If someone has an explanation why musl is not working in this scenario, please let us know.
Reddit Rust community helped me as well, check this link out if you are interested:
https://www.reddit.com/r/rust/comments/mo3i08/unable_to_compile_c_file_inside_buildrs_headers/
So Why build.rs was unable to find .C headers?
ANSWER
Because I was using x86_64-unknown-linux-musl as target.

How do I pass the linker the -rlink argument?

I'm trying to cross compile a rust project with cross. My project depends on libcurl. The linker can find the libcurl .so but not the libraries which it depends on:
I'm running:
cross build -vv --target=arm-unknown-linux-gnueabihf
which gives me:
= note: /usr/arm-linux-gnueabihf/bin/../lib/gcc/arm-linux-gnueabihf/8.3.0/../../../../arm-linux-gnueabihf/bin/ld: warning: libnghttp2.so.14, needed by /usr/lib/arm-linux-gnueabihf/libcurl.so, not found (try using -rpath or -rpath-link)
I've already confirmed that the library its looking for exists:
root#80a13d68ad84:/# find /usr/lib -name "libnghttp2.so.14"
/usr/lib/x86_64-linux-gnu/libnghttp2.so.14
/usr/lib/arm-linux-gnueabihf/libnghttp2.so.14
I've read about rust build scripts here, where I can add paths to the library search path, but that has not resolved the issue, this is what my build.rs looks like:
fn main() {
println!("cargo:rustc-link-search=native=/usr/lib/arm-linux-gnueabihf");
}
The error suggests using -rpath or -rpath-link, so how do I configure the rpath in rust? Or is there a different way to resolve this issue?
You might be able to specify linker arguments by using an environment variable like
RUSTFLAGS="-C link-arg=--rpath=..."
Cargo allows placing the same kind of flag in the configuration file (see here) but I'm not sure if cross has the same support for something like that.

Linking failed in rust bindgen

So I have a header library (a directory of .h files) that I need to make use of in my rust code. To get this working I have been looking into various crates to help me achieve the this and it seems like bindgen is the most promising option.
i followed the documentation for bindgen and have created a wrapper.h and a build.rs file.
in the build.rs file i have whitelisted all the needed functions, and the link parameter mentioned in the documentation specified as println! ("cargo:rustc-link-lib=C/complete path to my header files");.
when doing a cargo build this however fails with this error:
error: linking with link.exe failed: exit code: 1181
which I have not been able to find a solution for.
Any help or suggestions are greatly appreciated.
rustc-link-lib directive is only for .a/.so/.dll and cannot work with either .h or .rs files. It's completely inappropriate here.
If you make your build.rs script write to a directory specified by std::env::var("OUT_DIR") path, then your main library code can include it dynamically with:
include!(concat!(env!("OUT_DIR"), "/temp_file_built_by_build_dot_rs.rs"));
Alternatively, don't bother with build.rs at all. Use command-line bindgen and include generated .rs file as a module in your project (just as a regular source code file).

Cabal can't find foreign library when building on NixOS

I am trying to build an internal Haskell project on NixOS using cabal2nix. It wraps (and thus depends on) a foreign library which on Ubuntu one would build by wgetting the source, then running make && make install && ldconfig. Thus when cabal goes to build the program, it is apparently able to find the appropriate header files (which are in /usr/local/include/ta-lib or /usr/include/ta-lib).
On Nix, the process as I understand is to setup a .nix file to specify how to get and build the source, and then Nix sets up the isolated build environments. When I do this, the foreign library is fetched and built appropriately.
When Nix runs the configure step, it looks alright:
configureFlags: --verbose --prefix=/nix/store/fwpw03bd0c2m5yb7v2wc7g6f0qj912ra-talib-0.1.0.0 --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/tmp/nix-build-talib-0.1.0.0.drv-0/package.conf.d --ghc-option=-optl=-Wl,-rpath=/nix/store/fwpw03bd0c2m5yb7v2wc7g6f0qj912ra-talib-0.1.0.0/lib/ghc-7.10.2/talib-0.1.0.0 --enable-split-objs --disable-library-profiling --disable-executable-profiling --enable-shared --enable-library-vanilla --enable-executable-dynamic --enable-tests --extra-include-dirs=/nix/store/gvglncjgd5yif9bc03qalmp2mrjp524n-ta-lib-0.4.0/include --extra-lib-dirs=/nix/store/gvglncjgd5yif9bc03qalmp2mrjp524n-ta-lib-0.4.0/lib
With --extra-include-dirs and --extra-lib-dirs set to the correct paths in the Nix store. However, when it goes to build it complains with,
Setup: Missing dependency on a foreign library:
* Missing C library: ta_lib
Unfortunately I don't understand how cabal is determining whether the foreign library is present. I read here (Haskell how to resolve cabal error: Missing dependencies on foreign libraries?) that cabal will try to build and link a C program that consists of for each header it finds. So, somehow it is not finding the correct library.
What is wrong? Does this have to do with the step in Ubuntu of running ldconfig?
The problem is that ta_lib depends on the system math library m, but that library isn't linked by default. You can check that by creating a stub C program
echo "int main() { return 0; }" >test.c
and trying to link that with ta_lib:
$ nix-shell -p ta_lib --run "gcc test.c -lta_lib"
/nix/store/ghinzmxfm2s41nz8y873jlywwmcbw38l-ta-lib-0.4.0/lib/libta_lib.so: undefined reference to `sinh'
/nix/store/ghinzmxfm2s41nz8y873jlywwmcbw38l-ta-lib-0.4.0/lib/libta_lib.so: undefined reference to `sincos'
[...]
collect2: error: ld returned 1 exit status
Now, when Cabal tries to determine whether the library is available, it will attempt to link it to a stub test program, but that attempt will fail because of all those undefined symbol. Hence, Cabal complains that the library cannot be linked (even though its paths are configured and set-up correctly).
To remedy that issue, add the m library to the extra-libraries attribute in your project's Cabal file, like so:
extra-libraries: ta_lib, m
That should make the Cabal configure phase succeed.

Getting "undefined reference to" when using the lib for ARM, but not when compiling it

For one of my Qt Embedded projects I'm using a external Qt lib called SMTPEmail. This lib needs to be compiled before being included into a project, something that I managed to do successfully both for Qt 4.8 ARM and for Desktop.
The problem I'm getting is that when I include the headers into my project and include the library in the .pro, the linker gives me
(path_to_libs)/libSMTPEmail.so: undefined reference to `QSslSocket::connectToHostEncrypted(QString const&, unsigned short, QFlags<QIODevice::OpenModeFlag>)'
(path_to_libs)/libSMTPEmail.so: undefined reference to `QSslSocket::QSslSocket(QObject*)'
collect2: ld returned 1 exit status
make: *** [re8k_interface-tgt] Error 1
but only for compiling for ARM. IOW compiling the lib for both ARM and Desktop goes OK, compiling the project for Desktop using the lib goes OK but compiling it for ARM using the lib goes wrong.
Following this forum thread I suspected this could be due to missing the link to the library file of openssl (the project points to different lib folders when compiling for different environments). So I searched for all "openssl" related files inside the compiler for ARM (arm-arago-linux-gnueabi) and included in the same folder where the .so is located; same error. I then suspected the lib itself had other dependencies which were not in the path_to_libs, so I did a readelf -d libSMTPEmail.so and later in the .so.1 and readelf did return some lib dependencies that were not inside the same folder of the library. I then copied all such dependencies to the folder and got no success either.
So what could be happening? All dependencies known by me were put in place and I still get the error only for the situation where the lib is included by another project compiled for ARM.
You need to point your QMake where your libs and header file is in your .pro file;
So find where your library is assume /usr/local/include then ;
INCLUDEPATH += /usr/local/include
Add which libs you will use;
LIBS += -lSMTPEmail
You can check my answer here;
Two things stand out for me in your question:
1.
undefined reference to
This error message means that there was an error in the linking step of compilation. This occurs when you include a header to a function/class/variable but don't have the definition included in your own sources, or you do not link in a static library that does.
Searching for dependencies in libraries that are already compiled (.dll or .so) is too late, the compiler is looking for a static link, not a dynamic link.
2.
compiling the lib for both ARM and Desktop goes OK, compiling the
project for Desktop using the lib goes OK but compiling it for ARM
using the lib goes wrong.
This suggests that you are using conditional compilation in your .pro file that does a "both" compilation, a "desktop only" compilation and a "ARM only" compilation. If this is correct, you need to examine your compilation instructions for your "ARM only" compilation.
The error message itself refers to you using two functions from the QSslStock class. These are part of the QtNetwork module so you should have the following in your .pro file in order for the necessary links to be formed.
Qt += network

Resources