Error when cross-compiling Rust: Relocations in generic ELF (EM: 62) - rust

I am trying to compile a rust project (amp) for an OpenWrt router running mipsel.
I've already been able to cross-compile a few programs for this target.
However when I try to compile this project I come across this error:
mipsel-openwrt-linux-musl-ld: /tmp/amp/target/mipsel-unknown-linux-musl/release/deps/libbacktrace_sys-75cc20f8a52b27e7.rlib(fileline.o): Relocations in generic ELF (EM: 62)
The error suggests the library has been compiled for x86 instead of mipsel.
I tried putting all dependencies for amp in an empty cargo crate and it compiles fine. However the files found in target/mipsel-unknown-linux-musl/release/deps don't match that of those in amp.
I tried removing that particular backtrace_sys from the linker command, and then it fails on miniz-sys and if I remove that it fails on onig_sys. So it seems the issue lies at the compilation of the syntect crate which depends on the previously mentioned libraries. But I can cross-compile syntect and scribe (which requires syntect) just fine.
My flags and environment:
cargo +nightly test --lib --release --target mipsel-unknown-linux-musl -Z build-std
set -x PATH $TOOLCHAIN/bin $PATH
set -x AR_mipsel_unknown_linux_msl mipsel-openwrt-linux-musl-ar
set -x CC_mipsel_unknown_linux_msl mipsel-openwrt-linux-musl-gcc
set -x CXX_mipsel_unknown_linux_msl mipsel-openwrt-linux-musl-g++
set -x CARGO_TARGET_MIPSEL_UNKNOWN_LINUX_MUSL_LINKER mipsel-openwrt-linux-musl-ld

Related

Cross-compiling rust with cargo tries to link with host libs

I'm trying to cross-compile a crate from a FreeBSD 13.1 x86_64 host to a FreeBSD 13.1 aarch64 target. I have already succeeded in cross-compiling my crate from FreeBSD 12.1 x86_64 to FreeBSD 12.1 aarch64.
To do so, I have the following environment:
cargo version
cargo 1.67.0-nightly (f6e737b1e 2022-12-02)
rustc --version
rustc 1.67.0-nightly (01fbc5ae7 2022-12-07)
export AARCH64_SYSROOT=/path/to/aarch64/root
export PKG_CONFIG_SYSROOT_DIR=$AARCH64_SYSROOT
export RUSTFLAGS="-C link-args=-fuse-ld=lld -C link-args=-L$AARCH64_SYSROOT/usr/lib -C link-args=-L$AARCH64_SYSROOT/lib -C link-args=-L$AARCH64_SYSROOT/usr/local/lib -C link-args=--sysroot=$AARCH64_SYSROOT"
The command I run to build is the following, and I get this error on FreeBSD 13.1:
cargo build -Z build-std --release --target aarch64-unknown-freebsd
ld.lld: error: /usr/local/lib/libssl.so is incompatible with $AARCH64_SYSROOT/usr/lib/Scrt1.o
ld.lld: error: /usr/local/lib/libcrypto.so is incompatible with $AARCH64_SYSROOT/usr/lib/Scrt1.o
ld.lld: error: /usr/local/lib/libcurl.so is incompatible with $AARCH64_SYSROOT/usr/lib/Scrt1.o
cc: error: linker command failed with exit code 1 (use -v to see invocation)
It looks like the linker is trying to link with my host libraries even though I specify the link paths in RUSTFLAGS.
I ran cargo with the --verbose flags on both machines and saw that -L native=/usr/local/lib appears on FreeBSD 13.1 but -L native=$AARCH64_SYSROOT/usr/local/lib is shown on FreeBSD 12.1 so I think this is a pretty big clue but I might be completly wrong.
Why does my host libs path appear on FreeBSD13 but not on FreeBSD12 ? How can I remove this default flags when running my cargo build command ?
I tried invoking rustc directly with the command shown by cargo build --verbose (removing the -L native=/usr/local/lib option) which seems to work. However this is not practical as my crate has many dependencies and building properly without using cargo would be a mess...

Cross compiling with Cargo from 64 Bit Linux to 32 Bit windows in Rust

I'm trying to cross compile a program from 64 bit Linux to 32 bit Windows.
I am already able to cross compile to x86_64-pc-windows-gnu target, but unable to cross-compile to i686-pc-windows-gnu or i686-pc-windows-msvc.
I have w64-mingw32-gcc installed. When I try to cross compile to 32 bit (i686) I get the following error:
error: linking with `i686-w64-mingw32-gcc` failed: exit status: 1
|
= note: "i686-w64-mingw32-gcc"
= note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
I've tried
rustflags = "-C panic=abort"
inside my .cargo/config.toml, but still always getting the same error.
The weird part is, cross compiling using crossy (cross build --target i686-pc-windows-gnu --release) works using the same host machine. Unfortunately, there are other complications with the compile-time requirements and cross being done inside a docker container that prevents me from being able to use it.

How do I link libgcc statically in rust

I am trying to compile a Rust program with statically linked libgcc.
The program is meant to run in initramfs in a restricted environment and is compiled for different platforms (arm, x86_64).
Currently my only solution is to compile against musl which produces a statically linked binary.
Unfortunately this adds a bit of complexity to the build process - I have not found a way to cross-compile for arm-musl on x86 and I have extra installation requirements (musl-gcc).
I have tried to add this:
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "target-feature=+crt-static", "-C", "link-args=-static-libgcc"]
...to .cargo/config, but as far as I understand +crt-static only works on Windows, and -static-libgcc showed no effect either.

How to compile a static musl binary of a Rust project with native dependencies?

I have a project with dependencies on Hyper and Diesel, and because of that, on native libraries OpenSSL and libpq. The project builds on nightly Rust because it uses compiler plugins.
My current attempt is to build on a Docker container. I have the MUSL libc and the libraries make'd and installed with prefix /usr/local/musl. I run cargo with the following command: (Not sure if some of the options are redundant, I'm not too well-versed with the compiler chain, and not even sure if they end up to the linker, but I have to try, right.)
LDFLAGS="-static -L/usr/local/musl/lib" \
LD_LIBRARY_PATH=/usr/local/musl/lib:$LD_LIBRARY_PATH \
CFLAGS="-I/usr/local/musl/include" \
PKG_CONFIG_PATH=/usr/local/musl/lib/pkgconfig \
cargo build --release --target=x86_64-unknown-linux-musl
When I ldd the resulting file, it reveals this:
$ ldd server
linux-vdso.so.1 (0x00007fffb878e000)
libpq.so.5 => /usr/local/musl/lib/libpq.so.5 (0x00007f4d730e7000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f4d72e82000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f4d72a85000)
libc.so => /usr/local/musl/lib/libc.so (0x00007f4d727f6000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4d725f2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4d72246000)
/lib/ld64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x000055e2124a2000)
There's all that dynamically linked stuff, and some even with the "x86_64-linux-gnu" chain! What went wrong?
I can make statically linked, simple pure-Rust projects without problems. ldd says that they are statically linked, and they run without problems, unlike the executable I have problems with.
When I used --verbose with Cargo, I got the following rustc command that actually builds the executable: http://pastebin.com/ywv0zNBK (Oops, that one had a custom outdir and -Z print-link-args, added by me)
Adding the print-link-args flag, I got the following linker command: http://pastebin.com/Aw43qd7h
How do I get cargo or rustc to believe that I want a static binary?
The problem was that for each crate providing a native dependency – say OpenSSL – there is the build.rs build script that is in charge of communicating the build and linking options to Cargo and to rustc. (For example: they print out something like cargo:rustc-link-lib=static=ssl which Cargo then reads and acts accordingly.)
So just setting the "standard" GCC environmental variables is hardly going to have any effect. You must check each and every build.rs separately to know how to coerce that exact crate to convey cargo its options. For OpenSSL, its env vars like OPENSSL_DIR, OPENSSL_STATIC etc.
Another hurdle is that if you use compiler plugins, they might be compiled with the target triplet too (at least docker_codegen). On the other hand, they are linked dynamically during the compiling process. This mean that not only must static libraries be linked correctly, you must also have dynamic libraries of the target host variety, like Musl libc.so in place, and correctly set (LD_LIBRARY_PATH etc.).
I made a thoroughly commented Dockerfile that builds my project statically with some native dependencies. It might be of help for others too.
https://gitlab.com/rust_musl_docker/image
If you want to statically link a Rust program without native dependencies, that is much easier:
$ rustup target add x86_64-unknown-linux-musl
$ cargo build --release --target=x86_64-unknown-linux-musl
I had the same problem with ldd and GCC. The musl target was generated in a different directory; not in target/release/... but in target/x86_64-unknown-linux-musl/release/....

Adding codegen flags to a Cargo build

On Macintosh, to allow some symbols to go unlinked, it is necessary to pass -C link-args='-Wl,-undefined,dynamic_lookup' to the Rust compiler. One needs to do this when building Postgres plugins, because some of the Postgres intrinsics are only compiled into the Postgres server, and not available for linking from shared libs.
At present, the project's process is as follows:
Build is run with cargo build -v.
Failing call to rustc is copied and -C link-args='-Wl,-undefined,dynamic_lookup' added to it.
Success!
This seems like a hard sell for automation. What options are available for adding codegen flags to Rust builds through cargo?
cargo provides rustc command which allows one to pass arbitrary compiler flags. The following should do it:
% cargo rustc -- -C link-args='-Wl,-undefined,dynamic_lookup'

Resources