I'm specifying the -Cpanic=abort and -Zbuild-std=panic_abort when compiling. Why does the linker still say it needs libunwind to compile a program?
I'm experimenting with various ways to cross-compile Rust programs as small as possible (using the min-sized-rust repo as a reference). Right now I'm trying to compile the powerpc64-unknown-linux-musl target and I'm stuck on trying to remove a dependency on libunwind.
Here's my setup:
# 1. Install the Rust std source code
rustup component add rust-src --toolchain nightly
# 2. Setup a simple rust repo
cargo init testing
cd testing
# 3. Download a musl toolchain
wget https://musl.cc/powerpc64-linux-musl-cross.tgz
tar xzf powerpc64-linux-musl-cross.tgz
# 4. Try to compile the project (options on the command line instead of in files for
# maximum obviousness).
# RUSTFLAGS:
# -Cpanic=abort - abort immediately on panic
# -Clink-self-contained=no - don't use rustc's builtin libraries and objects (this
# is needed because powerpc64-unknown-linux-musl is a tier 3 target)
# -Clink-arg=--sysroot and -Clink-arg=/path/to/sysroot - pass the option to the linker
# to specify the sysroot of cross-compilation toolchain
# Cargo options:
# --config target.<triple>.linker - specify the linker to use
# -Zbuild-std=std,panic_abort - build the standard library from source. Specify
# panic_abort to make the abort on panic work
RUSTFLAGS="-Cpanic=abort -Clink-self-contained=no -Clink-arg=--sysroot -Clink-arg=powerpc64-linux-musl-cross/powerpc64-linux-musl/" \
cargo +nightly build \
--config "target.powerpc64-unknown-linux-musl.linker=\"powerpc64-linux-musl-cross/bin/powerpc64-linux-musl-gcc\"" \
--target powerpc64-unknown-linux-musl -Zbuild-std=panic_abort,std --release
This fails with the following error:
error: linking with `/home/user/Projects/testing/powerpc64-linux-musl-cross/bin/powerpc64-linux-musl-gcc` failed: exit status: 1
<output snipped>
= note: /home/user/Projects/testing/powerpc64-linux-musl-cross/bin/../lib/gcc/powerpc64-linux-musl/11.2.1/../../../../powerpc64-linux-musl/bin/ld: cannot find -lunwind
From min-size-rust repository:
"Even if panic = "abort" is specified in Cargo.toml, rustc will still include panic strings and formatting code in final binary by default. An unstable panic_immediate_abort feature has been merged into the nightly rustc compiler to address this.
To use this, repeat the instructions above to use build-std, but also pass the following -Z build-std-features=panic_immediate_abort option."
Still, you will get "cannot find -lunwind", because the linker still uses libunwind, even though it's truly unneeded,why! I do not know, maybe it's a bug.(Maybe someone with fair knowledge about linkers can easily solve that.I tried a naive solution which is "cargo .... --verbose", copy , remove "libunwind" then relinking which failed)
I verified that is indeed the missing piece by build from source(--target=x86_64-unknown-linux-musl) AND using an old simple trick which is "touch libunwind.a" in the "self-contained" directory inside a target lib folder.(because the linker would still use it even though it's now truly unneeded, then I gave him a dummy libunwind.a)
In your case, I really tried to build it to your target until I got a headache, but couldn't and stopped, but here is possible solutions:
Giving that you're using "-Z build-std-features=panic_immediate_abort"
-If you can custom the linking process, then solve it (until what seems to be a bug is solved)
-Create a dummy(empty) libunwind.a where it should be in your toolchain
Related
I would like to write a simple program and compile it for atmega16. How do I add atmega16 as a compilation target? There are little to no docs about this...
Thanks
The only built-in AVR target is avr-unknown-gnu-atmega328. If you are using a different AVR microcontroller, you need a custom target.
The AVR-Rust Guidebook describes how to add custom targets for different AVR microcontrollers. The process boils down to:
Export the JSON specification for the built-in atmega328 target, to use as a template
rustc --print target-spec-json -Z unstable-options --target avr-unknown-gnu-atmega328 > avr-unknown-gnu-atmega16.json
Edit this JSON file. I found that to get it to compile for atmega16, I had to change the following:
"cpu": "atmega16",
"is-builtin": false,
"-mmcu=atmega16"
Note:
The CPU is specified as "atmega16" not "atmega16p" as you had attempted to use
I don't have any atmega16 MCU to hand so I have not been able to test the resulting binary. You may need to tweak other elements of the file.
You will also need to install avr-gcc, as Rust uses it to link the resulting binary. You can get that here.
You should then be able to build your project like this:
cargo build -Z build-std=core --target ./avr-unknown-gnu-atmega16.json --release
Note that --target is specified as the path to the JSON file. If in the same directory, prefix with ./.
I'm wanting to compile binaries for openwrt 19.07.7 on mips with limited space.
OpenWrt 19.07.7, r11306-c4a6851c72
-----------------------------------------------------
root#OpenWrt:~# df /
Filesystem 1K-blocks Used Available Use% Mounted on
overlayfs:/overlay 3392 1276 2116 38% /
root#OpenWrt:~# ls -s
662 crosshello 3 helloworld
root#OpenWrt:~# ldd --version
musl libc (mips-sf)
Version 1.1.24
Above you can see the size of crosshello (cross compiled using rust) and helloworld (cross compiled using gcc). There is limited space on the target as you can see.
I followed https://github.com/japaric/rust-cross and it works. However I had to reduce the size of the binary to get it to fit. Following https://github.com/johnthagen/min-sized-rust I got the binary (the "Cross compiling with cargo" hello world with clap from the first link) to fit. I haven't compiled my own program yet. I got stuck a the "Optimize libstd with build-std" section.
I'm getting the above error:
$ cargo +nightly build -Z build-std=std,panic_abort --target=mips-unknown-linux-musl --release
Compiling libc v0.2.106
...
error: could not find native static library `c`, perhaps an -L flag is missing?
error: could not compile `libc` due to previous error
warning: build failed, waiting for other jobs to finish...
error: build failed
How do I fix that? I don't want to static link against the system libc, I want it dynamically linked like other binaries on that platform. But not sure how to do that. If I use the standard libstd the binaries are dynamically linked just fine.
Alternatively I could try nostd but my code would need significant modification and I'm not sure that would fix this problem anyway. Alternatively I could just go ahead compile my actual project with libstd and it might fit. However my project is doing data logging, so the binary is competing for space with data I want to store, so I want to minimize space as much as possible.
I've also seen reference to cargo-bloat so I'll probably check that out to find what all the space is being used for in the binary.
I'm open to different strategies to achieve what I want if anyone has ideas?
On my dev system:
$ cat .cargo/config
[target.mips-unknown-linux-musl]
ar = "/home/alex/projects/openwrt-sdk-19.07.7-ath79-generic_gcc-7.5.0_musl.Linux-x86_64/staging_dir/toolchain-mips_24kc_gcc-7.5.0_musl/bin/mips-openwrt-linux-ar"
linker = "/home/alex/projects/openwrt-sdk-19.07.7-ath79-generic_gcc-7.5.0_musl.Linux-x86_64/staging_dir/toolchain-mips_24kc_gcc-7.5.0_musl/bin/mips-openwrt-linux-gcc"
$ cat Cargo.toml
#cargo-features = ["strip"]
[package]
name = "crosshello"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
clap = "3.0.0-beta.5"
[profile.release]
#strip = true
opt-level = "z"
lto = true
codegen-units = 1
panic = "abort"
(The binaries are stripped, currently I'm stripping them manually.)
I have a basic rust/cargo project with a single main file and some basic dependencies. The cargo build command works fine when the target is not specified (I am using windows so it builds to windows), but when I try to cross compile the program to linux using cargo build --target=x86_64-unknown-linux-gnu or cargo build --target=x86_64-unknown-linux-musl, the process fails with the following error: linker 'cc' not found.
Does anyone have an idea how to get around this? Is there a specific linker I need to install?
Thanks.
I've just figured it out.
It turns out you need to tell cargo to use the LLVM linker instead. You do this by creating a new directory called .cargo in your base directory, and then a new file called config.toml in this directory. Here you can add the lines:
[target.x86_64-unknown-linux-musl]
rustflags = ["-C", "linker-flavor=ld.lld"]
Then building with the command cargo build --target=x86_64-unknown-linux-musl should work!
As cargo check shows, it's often useful to check if your program is well-formed without actually generating code (an often pretty time-consuming task). I want to check a single (library) Rust file with rustc directly (I cannot use Cargo!).
cargo check apparently works by calling this:
rustc --emit=metadata -Z no-codegen
This only emits metadata, a .rmeta file. Cargo actually needs that to check crates dependent on the checked crate. In my case I really don't need the metadata file.
I tried the following:
rustc --crate-type=lib --emit=
rustc --crate-type=lib --emit=nothing
But both didn't work. I use --crate-type=lib because my file doesn't have a main function. I need a platform-independent solution (I don't just want to use it on my machine, but use it in a public script).
How do I make rustc not write a single file?
You can just skip the --emit flag.
The final command would then be: rustc -Z no-codegen rust.rs
To quote my own GitHub comment about this very question, there are a few options for stable Rust:
rustc --emit=mir -o /dev/null seems to work in 1.18 and newer, writing nothing. (--emit=mir is the only helpful --emit option—the others try to create silly files like /dev/null.foo0.rcgu.o, except --emit=dep-info, which does no checking.)
rustc -C extra-filename=-tmp -C linker=true (i.e. use /bin/true as a “linker”) seems to work in all versions, writing some intermediate files but cleaning them up.
rustc --out-dir=<new empty temporary directory> is less clever and therefore perhaps less likely to break?
Note that linker errors, if any, will not be found by the first two options (nor by the nightly-only -Zno-codegen option).
Suppose I execute cargo new one --bin and cargo new two --bin then add the same dependency to each project's Cargo.toml and build them.
Now there are two absolutely identical sets of libraries:
/one/target/debug/deps/ *.rlib
/two/target/debug/deps/ *.rlib
They are same files and waste storage space, but really the problem is that I have to compile these libraries again for every project. It takes a very much time. There is the same problem with cargo install.
Can I specify a place to store compiled libraries to avoid recompilation?
Several Cargo projects might share the libraries by using the same target dir.
.cargo/config
Place a ".cargo" folder in a project and create a "config" file there containing:
[build]
target-dir = "/path/to/your/shared/target/dir"
On Unix this might look like:
mkdir ~/shared_rust_target
mkdir .cargo
echo "[build]" > .cargo/config
echo "target-dir = \"$HOME/shared_rust_target\"" >> .cargo/config
CARGO_TARGET_DIR
Set the CARGO_TARGET_DIR environment variable.
On Unix this might look like:
export CARGO_TARGET_DIR = "$HOME/shared_rust_target"
See this commit for some extra target-dir documentation.
In particular, prior to Cargo 1.9 you shouldn't build several projects into the same target dir concurrently. (Here's more on how Cargo 1.9 supports concurrent builds).
target-dir is also mentioned in the Cargo docs.
Should work, according to this issue.
P.S. It is now also possible to achieve crate reuse with workspaces.
P.S. https://docs.rs/cargo-hakari/latest/cargo_hakari/ helps with keeping some of dependencies compatible between projects.
Even if there is a way to do it, you probably don't want to. Just because you happen to be using the same libraries doesn't mean that they were compiled the same. For example, Cargo supports the concept of features, compilation time configuration that changes how the crate was compiled.
Likewise, you may need to support multiple versions of Rust, such as nightly and stable. Or perhaps you need to cross-compile for a different architecture. Each of those will produce different code.
Cargo will cache the build products of a single project, so I've found the overhead not really noticeable, and I compile a lot of projects from people asking questions on Stack Overflow! :-)
I managed to piece together code from a few answers, but mainly this one.
This Dockerfile should not just cache Cargo dependancy downloads, but also their compilations and the crates.io index. All the other answers I could find only cached the downloads or the index, not both.
FROM arm64v8/rust as builder
# Capture dependencies
COPY Cargo.toml Cargo.lock /app/
# We create a dummy main.rs to build deps
WORKDIR /app
RUN mkdir src && echo "fn main() {}" > src/main.rs
# RUN rustup install nightly && rustup default nightly
# This step compiles only our dependencies and saves them in a layer. This is the most impactful time savings
# Note the use of --mount=type=cache. On subsequent runs, we'll have the crates already downloaded
RUN --mount=type=cache,target=/usr/local/cargo/registry cargo build --release && rm src/main.rs
# Copy our sources
COPY ./src /app/src
# A bit of magic here!
# * We're mounting that cache again to use during the build, otherwise it's not present and we'll have to download those again - bad!
# * Rust here is a bit fiddly, so we'll touch the files (even though we copied over them) to force a new build
RUN --mount=type=cache,target=/usr/local/cargo/registry \
set -e && \
# update timestamps to force a new build &&
touch /app/src/main.rs && \
cargo build --release
# Again, our final image is the same - a slim base and just our app
FROM debian:buster-slim as app
COPY --from=builder /app/target/release/app/app
CMD ["/app"]
Take note of the FROM arm64v8, if you are targeting x86, replace the builder and app FROMs with their respective x86 versions.