How to build rdkafka for x86_64-unknown-linux-musl - rust

I have this in a Cargo.toml
rdkafka = { version = "0.29.0", features = [ "ssl", "cmake-build"] }
I tried to compile to x86_64-unknown-linux-musl using 2 options. Both failed.
cargo build --target x86_64-unknown-linux-musl --release
OUTPUT:
linking with cc failed: exit status: 1
rdkafka_ssl.c:(.text.rd_kafka_ssl_ctx_term+0x2b): undefined reference to `ENGINE_free'
collect2: error: ld returned 1 exit status
cross build --target=x86_64-unknown-linux-musl --release
OUTPUT:
rdkafka_ssl.c:(.text.rd_kafka_ssl_ctx_term+0x2b): undefined reference to `ENGINE_free'
collect2: error: ld returned 1 exit status
They both give the same error.
Does anyone know how to fix this? Or can someone try this on their machine.. I'd really appreciate it.
I expected the build to pass.
POST
I have added github repo and recreated the issue. It uses Dockerfile
https://github.com/0xDjole/rust-rdkafka-musl
To test just docker build -t muslkafka .
Here is the screenshot of the issue

I am able to build a rust crate depending on rdkafka on x86_64-unknown-linux-musl by disabling the cmake-build feature flag and using the following Dockerfile:
FROM clux/muslrust:1.63.0 AS chef
RUN cargo install cargo-chef --locked
RUN ln -s /usr/bin/musl-gcc /usr/bin/musl-g++
WORKDIR /usr/src/foo
FROM chef AS planner
COPY . .
RUN cargo chef prepare
FROM chef AS builder
COPY --from=planner /usr/src/foo/recipe.json recipe.json
RUN cargo chef cook --release
COPY . .
RUN cargo build --release \
&& strip --strip-debug target/*/release/foo \
&& cp target/*/release/foo /usr/local/bin/
FROM alpine as final
COPY --from=builder /usr/local/bin/foo /usr/local/bin
ENTRYPOINT ["/usr/local/bin/foo"]
(I'm somewhat surprised to see that this does seem to work with the ssl feature and produces a statically linked binary, I hadn't tested this before and openssl is notorious for being "difficult". I can't test right now whether said binary also actually works, so… please test.)

Related

Cross compile shared library for armv5te-unknown-linux-gnueabi Rust [Mindstorm Ev3dev]

Parameters:
source = x86_x64 windows 10 or x86_x64 linux (ubuntu wsl)
target = armv5te linux
target_type = cdylib
target_glibc = 2.24
language = rust
build_tool = cargo
compiler = rustc
(The target is a Lego Mindstorm running a linux image from Ev3dev)
Cargo Configuration:
[package]
name = "ev3"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
jni = "0.19"
ev3dev-lang-rust = { version = "0.12.1", features=["screen"]}
jni_proc_macro= {path= "./jni_proc_macro"}
[lib]
crate-type= ["cdylib"]
[workspace]
members= ["jni_proc_macro"]
Build Configuration:
[build]
target = "armv5te-unknown-linux-gnueabi"
[target.armv5te-unknown-linux-gnueabi]
linker = "rust-lld"
Build Error:
error: linking with `rust-lld` failed: exit code: 1
|
= note: {...}
= note: rust-lld: error: unable to find library -lgcc_s
rust-lld: error: unable to find library -lutil
rust-lld: error: unable to find library -lrt
rust-lld: error: unable to find library -lpthread
rust-lld: error: unable to find library -lm
rust-lld: error: unable to find library -ldl
rust-lld: error: unable to find library -lc
error: could not compile `ev3` due to previous error
As the error suggests the linker is missing libraries. I found no clear solution where I can download and or provide these dependencies.
My question is, A is there a diffrent way to build this successfully or B how do I solve these dependencies.
The result needs to be a shared library (.so) for linux and armv5te
Requirements
wsl or linux installed
cargo and rustc installed
(everthing is done in wsl/linux)
Prep/Build
Install cross on cargo
cargo install cross --git https://github.com/cross-rs/cross
Install docker
Clone the cross repository
Navigate into the docker folder
Create a new file with the name "Dockerfile.armv5te-unknown-linux-gnueabi-cross"
Paste this in the new file:
FROM ubuntu:16.04
ARG DEBIAN_FRONTEND=noninteractive
COPY common.sh lib.sh /
RUN /common.sh
COPY cmake.sh /
RUN /cmake.sh
COPY xargo.sh /
RUN /xargo.sh
RUN apt-get update && apt-get install --assume-yes --no-install-recommends \
g++-arm-linux-gnueabi \
crossbuild-essential-armel \
libc6-dev-armel-cross
COPY deny-debian-packages.sh /
RUN TARGET_ARCH=armel /deny-debian-packages.sh \
binutils \
binutils-arm-linux-gnueabi
# Qemu is disabled since we've changed the scripts to require newer Python versions.
#COPY qemu.sh /
#RUN /qemu.sh arm
COPY qemu-runner base-runner.sh /
ENV CROSS_TOOLCHAIN_PREFIX=arm-linux-gnueabi-
ENV CROSS_SYSROOT=/usr/arm-linux-gnueabi
ENV CARGO_TARGET_ARMV5TE_UNKNOWN_LINUX_GNUEABI_LINKER="$CROSS_TOOLCHAIN_PREFIX"gcc \
CARGO_TARGET_ARMV5TE_UNKNOWN_LINUX_GNUEABI_RUNNER="/qemu-runner arm" \
AR_armv5te_unknown_linux_gnueabi="$CROSS_TOOLCHAIN_PREFIX"ar \
CC_armv5te_unknown_linux_gnueabi="$CROSS_TOOLCHAIN_PREFIX"gcc \
CXX_armv5te_unknown_linux_gnueabi="$CROSS_TOOLCHAIN_PREFIX"g++ \
BINDGEN_EXTRA_CLANG_ARGS_armv5te_unknown_linux_gnueabi="--sysroot=$CROSS_SYSROOT" \
QEMU_LD_PREFIX="$CROSS_SYSROOT" \
RUST_TEST_THREADS=1 \
PKG_CONFIG_PATH="/usr/lib/arm-linux-gnueabi/pkgconfig/:${PKG_CONFIG_PATH}"
Make sure the project uses "LF" newlines. if not this fixes it.
Compile the custom cross/docker build using the following command in the root of the cloned repository:
cargo build-docker-image armv5te-unknown-linux-gnueabi-cross
This will create a new docker image that will be used to compile the rust code.
Then navigate to your target project folder and run:
export CROSS_TARGET_ARMV5TE_UNKNOWN_LINUX_GNUEABI_IMAGE=ghcr.io/cross-rs/armv5te-unknown-linux-gnueabi-cross:local
(Do not close this terminal)
Now add the following to the Cargo.toml file:
[package.metadata.cross.build]
default-target = "armv5te-unknown-linux-gnueabi"
now you can run:
cross build
Many cargo options like "--release" can be used (for more info have a look at cross in the credits)
Credits
MeetTitan(Stackoverflow) who recomended me to use cross
Cross project(GitHub) which powers the whole solution
Custom cross version discussion(Cross Github)
Alexhuszagh(Cross Github) who showed me how to build a custom cross version
Emilgardis(Cross Github) who explained the newline bug

How to build with wasm-pack for wasm64 target?

I've tried so far to
[build]
target = "wasm64-unknown-unknown"
In .cargo/config.toml,
and #![cfg(target_arch = "wasm64")]
in my lib.ts
Still, the target directory that I have as a result of wasm-pack build -t web is wasm32-unknown-unknown.
Is there any way to build wasm64-unknown-unknown with wasm-pack?
It works for cargo build, but as I understand only wasm-pack would generate appropriate bindings for JS.

Mozilla syncstorage-rs installation / make fails under RHEL8

I'm currently trying to install Mozilla syncstorage-rs on a RockyLinux machine, and step by step I could reduce the amount of errors, but now I got stuck. I found in Makefile to execute the following command in case of SSL trouble, so I did so cargo build --features grpcio/openssl
and this is fine, this finally works:
cargo build --features grpcio/openssl
Compiling syncstorage v0.10.2 (/var/www/html/syncstorage-rs)
Finished dev [unoptimized + debuginfo] target(s) in 35.59s
But when I try to now run either make run or make test or anything else, I just get:
= note: /usr/bin/ld: /var/www/html/syncstorage-rs/target/debug/deps/libopenssl-dc6f50ea1194640f.rlib(openssl-dc6f50ea1194640f.openssl.9bacb62b-cgu.1.rcgu.o): undefined reference to symbol 'OPENSSL_cipher_name##OPENSSL_1_1_1'
//usr/lib64/libssl.so.1.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
= help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
error: could not compile `syncstorage` due to previous error
make: *** [Makefile:45: test] Error 101
What am I missing? I thought after building it could be run easily.
So the answer is to use cargo clean before continuing. I don't know why, but there seemed to be some old artifacts even if I hadn't built before. However, after executing cargo clear command, the build openssl runs completely (many lines more than before) and after that, there is one more little step to do:
python3 -m venv venv # creates a virtual environment named "venv"
source venv/bin/activate # enters the virtual environment
python3 -m pip install -r requirements.txt # installs the packages
and finally make run works.
For more details, see https://github.com/mozilla-services/syncstorage-rs/issues/1241
See https://www.youtube.com/watch?v=jdtoyIW4Lec
for a step by step instruction.

How do I run a project's example using Cargo?

I'm trying to run the example code from this project. Following the instructions on the Cargo docs, I did the following:
git clone https://github.com/basiliscos/rust-procol-ftp-client
cd rust-procol-ftp-client
cargo run
cargo test
cargo test should also have compiled the example according to the Rust docs.
Although cargo test executes successfully, when I change into the target/debug directory, I don't find an executable for ftp-get (which is the example code). The target/debug/examples directory is also empty.
What is the best way to go about running this example?
You can run a specific example with:
cargo run --example name_of_example
where name_of_example is the base filename (without .rs)
or to run it in release mode:
cargo run --release --example name_of_example
To pass arguments to the example:
cargo run --example name_of_example -- arguments go here
cargo run will automatically build (or rebuild) the program first if it's out of date.
Try the following:
cd rust-procol-ftp-client
cargo build --examples
./target/debug/examples/ftp-get

failure to build rust-libc using cargo when rustc is musl-enabled

I successfully created a musl configured rustc by following this link
My attempt to build a project (which builds fine using non-musl configured rust) failed when I used cargo rustc -- --target=x86_64-unknown-linux-musl
'error: could not find crate `libc` with expected target triple x86_64-unknown-linux-musl'
Then, I tried to create rust-libc library using the code from crate. To be more accurate, I used the command provided by cargo to build rust-libc, I've only added --target=x86_64-unknown-linux-musl to the command. This time it failed reporting:
'error: could not find native static library `c`, perhaps an -L flag is missing?`'
I have two questions:
Is it mandatory to build musl configured cargo to be able to use cargo build --target=x86_64-unknown-linux-musl?
How can I address this:
'error: could not find native static library `c`, perhaps an -L flag is missing?'
This worked for me to build libc:
rustc --target=x86_64-unknown-linux-musl /address-of-libc/lib.rs --crate-name libc --crate-type lib -L /address-of-musldist/musldist/lib/ --out-dir=/your-chosen-address/target --cfg feature=\"default\" --cfg feature=\"cargo-build\" --emit=dep-info,link

Resources