How to cross compile Rust code from Intel Mac to M1 Mac by Zig? - rust

These days I'm curious about the cross-compile in Rust.As I know, it's very simple to do the same thing in Go or Zig.But in Rust, it's very difficult.I write Rust code in my Intel MacBook Pro, want to cross compile to get the executable file which could be run in M1 MacBook Pro. I take some tries and get some errors in linker.
Here is what I have tried.
I install Rust tool and Zig following the official instruction.
$ rustc --version
rustc 1.65.0 (897e37553 2022-11-02)
$
$ zig version
0.10.0-dev.4560+828735ac0
And my rust project structure is just like this:
hello-world -----
|--- .cargo--
|-- config.toml
|--- src-----
|-- main.rs
Cargo.toml
zig
And there are some files look like:
// main.rs
fn main() {
println!("hello world")
}
// config.toml
[target.aarch64-apple-darwin]
rustflags = ["-C", "linker-flavor=ld", "-C", "link-self-contained=no", "-C", "link-arg=-L/usr/local/opt/libiconv/lib"]
linker = "./zig"
// Cargo.toml
[package]
name = "hello-world"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
#[profile.release]
#lto=true
// zig
#!/usr/bin/env zsh
zig cc -target aarch64-macos-none $#
And I install libiconv by brew:
$ brew install libiconv
it puts libiconv in directory /usr/local/opt/libiconv/lib
$ ls /usr/local/opt/libiconv/lib
libcharset.1.dylib libcharset.a libcharset.dylib libiconv.2.dylib libiconv.a libiconv.dylib
I install target aarch64-apple-darwin by rustup and could see:
$ rustup show
Default host: x86_64-apple-darwin
rustup home: /Users/<my name>/.rustup
installed targets for active toolchain
--------------------------------------
aarch64-apple-darwin
x86_64-apple-darwin
x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl
active toolchain
----------------
stable-x86_64-apple-darwin (default)
rustc 1.65.0 (897e37553 2022-11-02)
then I start to build in hello-world directory:
$ CC=./zig cargo build --target aarch64-apple-darwin
...
...
note: error(link): mismatched cpu architecture: expected aarch64, found x86_64
error: MismatchedCpuArchitecture
I find some solutions from stackoverflow, it said I can use macOS sdk to cross-compile.
But I think there are more than one method to finish cross-compile, and really want to know how to solve it using zig.
By the way,I just know the basic steps of assembly, not in details, I don't have experience in linking.
Could anyone give me some suggestions ?
Any answer will be appreciated.

This might not be enough to solve your problem fully, but you need at the very least to pass the cross-compilation flags to Zig, and you also need to use the cc subcommand.
Note also that Zig uses macos instead of darwin.
So you need to set CC="./zig cc -target aarch64-macos", that said, I'm not sure if Rust supports spaces in the CC command now (at least a while ago it did not). If it doesn't you will need to wrap everything in a bash script (zcc.sh):
#!/bin/sh
ZIG_LOCAL_CACHE_DIR="$HOME/tmp" zig cc -target aarch64-macos $#
And then CC=./zcc.sh
Note also that the same will need to be done for zig c++ if you also depend on C++ code. The env variable to set in that case is CXX.
Here you can find somebody doing this to build wasmtime:
https://actually.fyi/posts/zig-makes-rust-cross-compilation-just-work/
Finally, you can probably save yourself all this trouble simply by using cargo-zigbuild:
https://github.com/rust-cross/cargo-zigbuild

Related

How to fix "error: could not find native static library `c`, perhaps an -L flag is missing?" or how do I compile rust binaries for openwrt?

I'm wanting to compile binaries for openwrt 19.07.7 on mips with limited space.
OpenWrt 19.07.7, r11306-c4a6851c72
-----------------------------------------------------
root#OpenWrt:~# df /
Filesystem 1K-blocks Used Available Use% Mounted on
overlayfs:/overlay 3392 1276 2116 38% /
root#OpenWrt:~# ls -s
662 crosshello 3 helloworld
root#OpenWrt:~# ldd --version
musl libc (mips-sf)
Version 1.1.24
Above you can see the size of crosshello (cross compiled using rust) and helloworld (cross compiled using gcc). There is limited space on the target as you can see.
I followed https://github.com/japaric/rust-cross and it works. However I had to reduce the size of the binary to get it to fit. Following https://github.com/johnthagen/min-sized-rust I got the binary (the "Cross compiling with cargo" hello world with clap from the first link) to fit. I haven't compiled my own program yet. I got stuck a the "Optimize libstd with build-std" section.
I'm getting the above error:
$ cargo +nightly build -Z build-std=std,panic_abort --target=mips-unknown-linux-musl --release
Compiling libc v0.2.106
...
error: could not find native static library `c`, perhaps an -L flag is missing?
error: could not compile `libc` due to previous error
warning: build failed, waiting for other jobs to finish...
error: build failed
How do I fix that? I don't want to static link against the system libc, I want it dynamically linked like other binaries on that platform. But not sure how to do that. If I use the standard libstd the binaries are dynamically linked just fine.
Alternatively I could try nostd but my code would need significant modification and I'm not sure that would fix this problem anyway. Alternatively I could just go ahead compile my actual project with libstd and it might fit. However my project is doing data logging, so the binary is competing for space with data I want to store, so I want to minimize space as much as possible.
I've also seen reference to cargo-bloat so I'll probably check that out to find what all the space is being used for in the binary.
I'm open to different strategies to achieve what I want if anyone has ideas?
On my dev system:
$ cat .cargo/config
[target.mips-unknown-linux-musl]
ar = "/home/alex/projects/openwrt-sdk-19.07.7-ath79-generic_gcc-7.5.0_musl.Linux-x86_64/staging_dir/toolchain-mips_24kc_gcc-7.5.0_musl/bin/mips-openwrt-linux-ar"
linker = "/home/alex/projects/openwrt-sdk-19.07.7-ath79-generic_gcc-7.5.0_musl.Linux-x86_64/staging_dir/toolchain-mips_24kc_gcc-7.5.0_musl/bin/mips-openwrt-linux-gcc"
$ cat Cargo.toml
#cargo-features = ["strip"]
[package]
name = "crosshello"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
clap = "3.0.0-beta.5"
[profile.release]
#strip = true
opt-level = "z"
lto = true
codegen-units = 1
panic = "abort"
(The binaries are stripped, currently I'm stripping them manually.)

pkg-config error during Rust cross-compilation

I am getting this error when I try to cross-compile some Rust; does anyone know what I should be doing?
This happens when I run cargo build --target aarch64, I get:
Compiling glib-sys v0.10.1
error: failed to run custom build command for `glib-sys v0.10.1`
...
pkg-config has not been configured to support cross-compilation.
Install a sysroot for the target platform and configure it via
PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
cross-compiling wrapper for pkg-config and set it via
PKG_CONFIG environment variable.
I might have asked this before a year or so ago, but anyway I cannot find any answer. I have tried adding various lines to my Cargo.toml but nothing seems to help.
The relevant part of Cargo.toml has:
[target.'cfg(target_os = "android")'.dependencies]
cairo = "0.0.4"
freetype = "0.7.0"
glib = "0.14.2"
openssl = "0.10.36"
openssl-sys = {version = "0.9.66", features = ["vendored"]}
There is probably a one-line answer to this; can anyone help me please.
You could try adding feature vendored for the openssl crate.
openssl = { version = "0.10.35", features = ["vendored"] }
reference
I'm also new to rust so there may be some mistakes in my answer.
So the warning suggests that the pkg-config is not correctly set, which suggests that this is not a rust issue, it's a pkg-config issue.
If you check the doc of pkg-config, you might see there's an environment variable called PKG_CONFIG_SYSROOT_DIR which "is useful for cross compilation". What you'll need to do is to run cargo with this environment set, e.g. PKG_CONFIG_SYSROOT_DIR=/usr/x86_64-w64-mingw32/ cargo build, where /usr/x86_64-w64-mingw32 contains all the file necessary for cross build for windows (as you can see from the name).
I'm not sure how to cross compile for aarch64, and I'm not sure how to add environment variable for cargo build, but I hope my experience above can give you some hint.

How do I cross-compile openssL libraries on Mac OS by rust?

When I was compiling a Linux binary on MacOS, I encountered this problem:
It looks like the rust library "openssl-sys v0.9.61" is bound to the C library of MacOS. can't link to linux.
cargo build --release --target=x86_64-unknown-linux-musl
Compiling openssl-sys v0.9.61
error: failed to run custom build command for `openssl-sys v0.9.61`
Caused by:
process didn't exit successfully: `~/test/../bin/release/build/openssl-sys-96148dcd52905249/build-script-main` (exit status: 101)
--- stdout
cargo:rustc-cfg=const_fn
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR
X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR unset
cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
OPENSSL_LIB_DIR unset
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR
X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_INCLUDE_DIR unset
cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
OPENSSL_INCLUDE_DIR unset
cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR
X86_64_UNKNOWN_LINUX_MUSL_OPENSSL_DIR unset
cargo:rerun-if-env-changed=OPENSSL_DIR
OPENSSL_DIR unset
cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64-unknown-linux-musl
cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS_x86_64_unknown_linux_musl
cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_ALLOW_CROSS
cargo:rerun-if-env-changed=PKG_CONFIG_ALLOW_CROSS
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-musl
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_musl
cargo:rerun-if-env-changed=TARGET_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-musl
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_musl
cargo:rerun-if-env-changed=TARGET_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
run pkg_config fail: "pkg-config has not been configured to support cross-compilation.\n\n Install a sysroot for the target platform and configure it via\n PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a\n cross-compiling wrapper for pkg-config and set it via\n PKG_CONFIG environment variable."
--- stderr
thread 'main' panicked at '
Could not find directory of OpenSSL installation, and this `-sys` crate cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it, you can set the `OPENSSL_DIR` environment variable for the
compilation process.
Make sure you also have the development packages of openssl installed.
For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.
If you're in a situation where you think the directory *should* be found
automatically, please open a bug at https://github.com/sfackler/rust-openssl
and include information about your system as well as this message.
$HOST = x86_64-apple-darwin
$TARGET = x86_64-unknown-linux-musl
openssl-sys = 0.9.61
', ~/.cargo/registry/src/openssl-sys-0.9.61/build/find_normal.rs:174:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I found that some people also encountered this problem:
https://www.andrew-thorburn.com/cross-compiling-a-simple-rust-web-app/
It's true that you need to determine which rust dependencies use openssl during the cross compile link phase. In my project, reqwest and mysql dependencies apparently use them when cross-compiling. I had a similar issue when cross-compiling from my mac to x86_64 or aarch64 (arm) linux targets on my mac m1. The linker was failing when linking to openssl.. The trick is to add openssl crate to your project. There is good information here: https://docs.rs/openssl/latest/openssl/
here are the steps with other helpful links also:
1.rustup add target (this is AMD64 linux)
rustup target add x86_64-unknown-linux-musl
for (arm7 linux)
rustup target add aarch64-unknown-linux-musl
1(a) add the xplatform linker
brew install FiloSottile/musl-cross/musl-cross
for x86 and arm7 support:
brew install FiloSottile/musl-cross/musl-cross --with-aarch64 --with-x86_64
1(b) add openssl crate
https://docs.rs/openssl/latest/openssl/
may need to install openssl (see directions)
brew install openssl#1.1
[dependencies]
openssl = { version = "0.10", features = ["vendored"] }
2. modify ~/.cargo/config.toml
add link targets
[target.x86_64-unknown-linux-musl]
linker = "x86_64-linux-musl-gcc"
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
[target.x86_64-unknown-linux-gnu] - notneeded
linker = "x86_64-unknown-linux-gnu-gcc"
3. export TARGET_CC
x86_64:
export TARGET_CC=x86_64-linux-musl-gcc
for arm7:
export TARGET_CC=aarch64-linux-musl-gcc
4. compiling openssl
include the openssl crate:
openssl = { version = "0.10", features = ["vendored"] }
5. build for target
cargo build --target x86_64-unknown-linux-musl
cargo build --target aarch64-unknown-linux-musl
The first step is to determine which dependency is using openssl, which you can do with crate tree:
crate tree --target=x86_64-unknown-linux-musl -i openssl-sys
In my case this identified the rusoto_core crate as the one depending on it. Some crates can be compiled with feature flags to use alternate TLS libraries, making them easier to use than OpenSSL.
For example, Rusoto enables rustls like this in the Cargo.toml:
rusoto_core = {version = "0.42.0", default_features = false, features=["rustls"]}
rusoto_s3 = {version = "0.42.0", default_features = false, features=["rustls"]}
rusoto_sqs = {version = "0.42.0", default_features = false, features=["rustls"]}
In the Rusoto case, there is documentation for cross-compiling for use with AWS Lambda: https://rusoto.org/lambdas.html
Alternately, you may be able to provide cross-compiled OpenSSL to the compiler, but from what I understand this is more difficult and poorly documented at this time.

Why does clap fail to compile when added to Cargo.toml?

Summary
I'm fairly new to Rust and decided to use it to port an existing project into it. I intended to use clap to handle CLI options, but I keep getting errors.
What do I need to do for clap to install correctly so that it's usable in my project as a dependency (e.g. extern crate clap; [...] use clap::App; [...]?
I haven't had problems with other crates (so far), so I'm not sure what's so different here or if there's a problem with the crate itself.
I've already seen a few questions (e.g. this one), which simply suggests that the dependency be added into the .toml file or don't seem to provide a solution to what I'm seeing.
I'm in Ubuntu Linux, if that makes a difference.
What I Tried
Adding clap = "2.33.0" to my Cargo.toml file (see https://crates.io/crates/clap) causes VSCode (through RLS) to log the following:
{
"resource": "[...]/Projects/takeout/Cargo.toml",
"owner": "rust",
"severity": 8,
"message": "Could not compile `clap`.\nprocess didn't exit successfully: `[...]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rls --crate-name clap [...]/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.0/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata -C debuginfo=2 --cfg 'feature=\"ansi_term\"' --cfg 'feature=\"atty\"' --cfg 'feature=\"color\"' --cfg 'feature=\"default\"' --cfg 'feature=\"strsim\"' --cfg 'feature=\"suggestions\"' --cfg 'feature=\"vec_map\"' -C metadata=630980a214d5fd10 -C extra-filename=-630980a214d5fd10 --out-dir [...]/Projects/takeout/target/rls/debug/deps -L dependency=[...]/Projects/takeout/target/rls/debug/deps --extern ansi_term=[...]/Projects/takeout/target/rls/debug/deps/libansi_term-1510a9addefc0253.rmeta --extern atty=[...]/Projects/takeout/target/rls/debug/deps/libatty-7c4847fd9fc1e3d9.rmeta --extern bitflags=[...]/Projects/takeout/target/rls/debug/deps/libbitflags-8369a9aec15a5abb.rmeta --extern strsim=[...]/Projects/takeout/target/rls/debug/deps/libstrsim-301d1cf239e9cd24.rmeta --extern textwrap=[...]/Projects/takeout/target/rls/debug/deps/libtextwrap-a799d71e2d028df4.rmeta --extern unicode_width=[...]/Projects/takeout/target/rls/debug/deps/libunicode_width-58e38dd9d658dcfb.rmeta --extern vec_map=[...]/Projects/takeout/target/rls/debug/deps/libvec_map-4f8e59c92e9953d8.rmeta --cap-lints allow --error-format=json --sysroot [...]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu` (exit code: 101)",
"startLineNumber": 1,
"startColumn": 1,
"endLineNumber": 10000,
"endColumn": 1
}
According to the README in the clap repo itself, just adding it should work:
For full usage, add clap as a dependency in your Cargo.toml to use from crates.io:
[dependencies]
clap = "~2.33"
But it doesn't.
I've tried it with and without the ~ prefix as well as clap = {version = "2.33", features = ["yaml"]}, which is also shown in the repo, but no luck. (Yes, I'm trying to load the CLI options from a .yaml file.)
Trying cargo install clap --version 2.33.0 from the shell simply returns an error message saying: error: specified package has no binaries.
Aiming directly at the Git repo also produces the same error message:
cargo install --git https://github.com/clap-rs/clap.git --tag v2.31.2 --features yaml 101 ↵
Updating git repository `https://github.com/clap-rs/clap.git`
Installing clap v2.31.2 (https://github.com/clap-rs/clap.git?tag=v2.31.2#07c15d28)
error: specified package has no binaries
Note that there's no v2.33.0 tag in the Git repo (at the time of this writing).
Bonus if you know how to get VSCode to stop marking everything as an error:
cargo install
There's misunderstanding about the cargo install command. You can learn more about it here.
This command manages Cargo’s local set of installed binary crates. Only packages which have executable [[bin]] or [[example]] targets can be installed, and all executables are installed into the installation root’s bin folder.
It's not your case. The only thing you have to do is to list clap in the dependencies section (Cargo.toml). That's all. No need to use cargo install at all. cargo build, cargo run, ... commands will download & compile & statically link all dependencies.
An example
Folder structure:
.
├── Cargo.toml
└── src
   ├── cli.yaml
   └── main.rs
Current directory:
$ pwd
/Users/robertvojta/Projects/stackoverflow/clap-yaml
Cargo.toml content:
[package]
name = "clap-yaml"
version = "0.1.0"
authors = ["Zrzka"]
edition = "2018"
[dependencies]
clap = { version = "2.33.0", features = ["yaml"] }
src/cli.yaml content:
name: clap-yaml
version: "1.0"
author: Zrzka
about: Stackoverflow sample
args:
- lang:
short: l
long: lang
default_value: cz
takes_value: true
possible_values:
- cz
- en
src/main.rs content:
use clap::{App, load_yaml};
fn main() {
let yaml = load_yaml!("cli.yaml");
let matches = App::from_yaml(yaml).get_matches();
match matches.value_of("lang").unwrap() {
"cz" => println!("Ahoj"),
"en" => println!("Hello"),
_ => unreachable!("see possible_values in yaml, handled by clap"),
};
}
Run it with cargo:
$ cargo -q run -- --lang en
Hello
Run it directly:
$ cargo build
...
Finished dev [unoptimized + debuginfo] target(s) in 0.01s
$ target/debug/clap-yaml --lang cz
Ahoj
Visual Studio Code
I still have vscode complaining and underlining everything in red in the Cargo.toml file. Any suggestions to fix this completely? It seems close to a full resolution.
I can confirm that this problem do exist in Rust 1.34.0. I did install this version and I've got same symptoms:
could not compile clap
the whole Cargo.toml is underlined (error)
There're two ways how to fix this.
Update your Cargo.toml file dependencies section manually if you'd like to stick with Rust 1.34.0:
[dependencies]
bitflags = "=1.0.4"
clap = { version = "2.33.0", features = ["yaml"] }
Or update your Rust toolchain to >= 1.35.0.
I just tested both ways and it works.
Related issues:
Could not compile clap. process didn't exit successfully
Compiler panics on latest RLS when compiling crates that depend on bitflags 1.0.5

How to make Travis CI work with Rust 0.12.0 and Cargo?

I am trying to make Travis CI build and test the contents of my GitHub repository, which is compatible with Rust 0.12.0 and an older Cargo:
rustc 0.12.0 (ba4081a5a 2014-10-07 13:44:41 -0700)
cargo 0.0.1-pre-nightly (861c07f 2014-10-07 23:29:57 +0000)
I have specified Rust 0.12.0 in the .travis.yml:
language: rust
rust: 0.12.0
script:
- cargo build --verbose
- cargo test --verbose
- rustdoc --test README.md -L target
- cargo doc
The issue I seem to be hitting is that Travis will pick the latest Cargo nightly:
cargo 0.0.1-pre-nightly (fd5d7a9 2014-12-25 04:28:40 +0000)
But this newer version is incompatible with Rust 0.12.0, as it is now using --emit=dep-info where it used to use --dep-info:
--dep-info [FILENAME]
Output dependency info to <filename> after compiling,
in a format suitable for use by Makefiles
This gives rustc invocation errors, as the value for is --emit is invalid:
Running `rustc src/sqlite3.rs --crate-name sqlite3 --crate-type lib -g -C metadata=1c7080eec8c6f90d -C extra-filename=-1c7080eec8c6f90d --out-dir target/deps --emit=dep-info,link -L target/deps -L target/deps -Awarnings`
...
error: unknown emission type: `dep-info`
...
Could not compile `sqlite3`.
I have been looking at other repositories for how to get around this, but it seems they either do not use Cargo with 0.12.0 (which works with Travis) or hit the same issue, like here: https://travis-ci.org/eliovir/rust-ini
I have failed to find any repository that works with Travis, rust 0.12.0 and Cargo. If I can specify the version of Cargo somewhere, I would be able to get around this, but I have failed to find a way to do so in the .travis.yml file.
Of course, with Rust 1.0 coming up, I will just wait for it, if there is no obvious solution that I have overlooked :-)
I agree that tracking nightly is probably the best bet. If there are dependencies you rely on that aren't being updated, then maybe that's a sign that they wont be updated come 1.0 time, either!
All that being said, Travis does allow you to install things before your build. This is completely untested, but you might be able to do something like
before_script:
- wget https://static.rust-lang.org/dist/rust-0.12.0-x86_64-unknown-linux-gnu.tar.gz
- tar -xvf rust-0.12.0-x86_64-unknown-linux-gnu.tar.gz
- ./rust-0.12.0-x86_64-unknown-linux-gnu/install.sh --prefix /tmp/rust-0.12/
- export PATH=$PATH:$PWD/tmp/rust-0.12/bin
- export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/tmp/rust-0.12/lib
This would download Rust 0.12, unpack and install it somewhere writable (doesn't really matter where). Then you setup env vars to point out where Rust is. Do the same to download a compatible version of Cargo.
If Travis has a "bare" language pack, that would be the best. Otherwise you could try using the Rust buildpack, or maybe just anything else (to avoid dealing with multiple rustc versions).

Resources