How to list a project's source files using the cargo crate? - rust

I'm trying to list the source files of a Rust project using the cargo crate. I can not just simply list all the .rs files present in a directory as I want to retrieve exactly the files that the compiler sees during the compilation, which may not be all the .rs files.
I'm conducting my experiments on the the Alacritty repository, which has a cargo workspace of 3 projects. Here is my code so far:
extern crate cargo;
use std::path::Path;
use cargo::core::Source;
fn main() {
let path = Path::new("/tmp/alacritty/Cargo.toml");
let config = cargo::util::config::Config::default().unwrap();
let ws = cargo::core::Workspace::new(&path, &config).unwrap();
for pkg in ws.members() {
println!("found package {}", pkg);
let config = ws.config();
let mut src = cargo::sources::PathSource::new(pkg.root(), pkg.package_id().source_id(), config);
src.update().unwrap();
let src_files = src.list_files(pkg).unwrap();
println!("found {} source files", src_files.len());
}
}
Here is the output:
found package alacritty v0.5.0-dev (/tmp/alacritty/alacritty)
found 0 source files
found package alacritty_terminal v0.5.0-dev (/tmp/alacritty/alacritty_terminal)
found 0 source files
found package font v0.1.0 (/tmp/alacritty/font)
found 0 source files
The members of the workspace are correctly found but I fail to retrieve the source files for each of these members. What am I missing?

Your code works!
If you run 'cargo vendor' in the alacritty tree, this should solve your issue. Study the 'cargo vendor' command Also, study the --offline switch for the cargo build command. I did not need to use this, but it is very helpful reading.
Basically, cargo vendor pulls in all the source.
I am not sure exactly why your code is not working. I had difficulty recreating this using the /tmp directory. I then used a normal directory combined with a call to 'cargo vendor', and it worked. Before cutting and pasting my code below, be sure to change '/Users/[username]' with your own path to your home directory.
Here is my procedure:
cd ~
git clone https://github.com/jwilm/alacritty
cargo vendor
This next part is probably not necessary:
mkdir /Users/[username]/alacritty/.cargo
Create a file at /Users/[username]/alacritty/.cargo/config
and, insert the following:
[source.crates-io]
replace-with = "vendored-sources"
[source.vendored-sources]
directory = "vendor"
Continuation of necessary part:
Modify the path statement to point to the newly created alacritty path:
let path = Path::new("/Users/[username]/alacritty/Cargo.toml");
Now, run your code
cargo run
Here is my output:
cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.27s
Running `target/debug/test3`
found package alacritty v0.5.0-dev (/Users/jmurray/alacritty/alacritty)
found 18 source files
found package alacritty_terminal v0.5.0-dev
(/Users/[username]/alacritty/alacritty_terminal)
found 172 source files
found package font v0.1.0 (/Users/jmurray/alacritty/font)
found 12 source files

Search for each element of the needle in the haystack in order. Each time you find a matching element, only continue the search in the remaining portion of the haystack. You can express this nicely by taking a new subslice of of the haystack each time you match an element.

Related

How to compile rust packages with different targets

Lets say I have nested packages like /foo/bar
Now I want to compile these packages with different targets.
So, How to compile /foo package with target wasm32-unknown-unknownand /foo/bar package with target x86_64-unknown-linux-gnu with one build command like cargo build --release?
You can specify the default target (that will be used absent an explicit --target option on the command line) in a configuration file. I therefore suggest that you create the following files:
/foo/.cargo/config.toml
[build]
target = "wasm32-unknown-unknown"
/foo/bar/.cargo/config.toml
[build]
target = "x86_64-unknown-linux-gnu"
To ensure that /foo/bar is built whenever /foo is built, you can use a build script:
/foo/build.rs
fn main() {
println!("cargo:rerun-if-changed=bar");
let profile = std::env::var("PROFILE").unwrap();
let status = std::process::Command::new("cargo")
.arg("build")
.arg(format!("--{}", profile))
.current_dir("bar")
.status()
.expect("failed to execute cargo");
assert!(status.success(), "failed to build bar");
}

Why Rust build.rs is unable to find C headers?

Basically I am trying to cargo build a crate which has a build.rs file.
This crate is located inside a bigger project and it's supposed to be a lib crate.
And inside this build.rs file, I am trying to compile a .C file which includes a couple of headers.
Fun fact: I got this build.rs file and crate structure from another little demo crate, and in that demo crate I had no problem to compile this exact C file with these headers.
FULL ERROR HERE:
Here is a link to github: https://github.com/mihaidogaru2537/FirecrackerPlayground/tree/dpdk_component/firecracker/src/dpdk_component
There is the crate I am talking about and you can also see the bigger project in which it resides. In the README file you can see the full error.
Either I do cargo build from the root of the big project or from the root of this problematic crate, the error is the same.
"cargo:warning=/usr/include/asm-generic/errno.h:5:10: fatal error: asm-generic/errno-base.h: No such file or directory
cargo:warning= 5 | #include <asm-generic/errno-base.h>"
The missing file might change depending on the .flag("-I/path/..") calls I am doing inside the build.rs
As you can see, right now it's unable to find errno-base.h, but I am including the path to asm-generic.
Here is the code of the build.rs file from the crate where the compilation of this C file works, as you can see, I did not have to add any include flags before calling compile.
fn main() {
// Tell cargo to tell rustc to link the system bzip2
// shared library.
// println!("cargo:rustc-link-lib=rte_ring");
// println!("cargo:rustc-link-lib=rte_mempool");
// Tell cargo to invalidate the built crate whenever the wrapper changes
// println!("cargo:rerun-if-changed=wrapper.h");
let _src = ["src/static-functions.c"];
println!("cargo:rerun-if-changed=build.rs");
let mut builder = cc::Build::new();
let build = builder
.file("src/static-functions.c")
.flag("-Wno-unused-parameter");
build.compile("foo");
}
Additional info:
The problematic crate is pretty small, see the link above. There is the build.rs file, C file and header file is inside the include directory.
One thing that I suspect, is that the target of the bigger project:
TARGET = Some("x86_64-unknown-linux-musl")
might affect the way the C file is compiled.
In the project where the compilation is working, I am not using that linux-musl stuff.
I am a total noob when it comes to Rust, but I do have a decent understanding of how C/C++ works.
I am running the project on Ubuntu 20.04
Those missing headers are a result of importing DPDK headers, I have DPDK libraries installed on the machine in question.
Let me know if you have any questions, sorry for the long read and thank you.
I somehow managed to fix it by adjusting the cargo build command to use x86_64-unknown-linux-gnu as target instead of x86_64-unknown-linux-musl (By default cargo build was doing the musl target somehow)
So if you are trying to build a rust app which is using DPDK libraries and you are getting missing headers, make sure to try the following:
cargo build --target=x86_64-unknown-linux-gnu
Well if you have to use musl and there is no alternative, I don't have an answer. But to me this was enough.
If someone has an explanation why musl is not working in this scenario, please let us know.
Reddit Rust community helped me as well, check this link out if you are interested:
https://www.reddit.com/r/rust/comments/mo3i08/unable_to_compile_c_file_inside_buildrs_headers/
So Why build.rs was unable to find .C headers?
ANSWER
Because I was using x86_64-unknown-linux-musl as target.

Is it possible to avoid recompiling a crate when I haven't made any changes to it?

I have a Rust crate which is a wrapper for a large C API and takes several minutes to compile. Running cargo build in the directory without making any changes always results in a recompile. It seems that Cargo should not be recompiling this crate unless I make a change, which I have not done.
I would like to compile the crate once and avoid re-compiling the crate unless I make a change. Is there any way for me to avoid constantly recompiling this scrate?
It seems that something is likely incorrect in my crate's build script. I will try to create a minimal reproducible example, but in the meantime I have provided the build script below:
use std::env;
use std::fs::copy;
use std::path::Path;
use std::process::Command;
fn main() {
let out_dir = env::var("OUT_DIR").unwrap();
let c_src_path = Path::new("parasail_c");
// configure the build
Command::new("cmake")
.arg(".")
.current_dir(&c_src_path)
.output()
.expect("Failed to configure parasail.");
// build the library
Command::new("make")
.current_dir(&c_src_path)
.output()
.expect("Failed to build parasail.");
// put the static library in the right directory so we can clean up
let target_file = format!("{}/libparasail.so", out_dir);
copy("parasail_c/libparasail.so", target_file)
.expect("Problem copying library to target directoy.");
let target_file = format!("{}/parasail.h", out_dir);
copy("parasail_c/parasail.h", target_file)
.expect("Problem copying header to target directoy.");
// clean up the temporary build files
Command::new("make")
.current_dir(&c_src_path)
.arg("clean")
.output()
.expect("Failed to clean up build files.");
// clean up the configuration files
Command::new("make")
.arg("distclean")
.current_dir(&c_src_path)
.output()
.expect("Failed to clean up configuration files.");
// let cargo know that it can find the file in the out directory
println!("cargo:rustc-link-search=native={}", out_dir);
println!("cargo:rustc-link-lib=dylib=parasail");
}
Here is the output from cargo build --verbose
cargo build --verbose
Compiling parasail-sys v0.1.0 (/home/fortier/testcode/rust/pairhmm/parasail-sys)
Fresh libc v0.2.51
Running `/home/fortier/testcode/rust/pairhmm/parasail-sys/target/debug/build/parasail-sys-f2d2d1f27a70b4d4/build-script-build`
Running `rustc --edition=2018 --crate-name parasail_sys src/lib.rs --color always --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=8879665b3d9bf7e1 -C extra-filename=-8879665b3d9bf7e1 --out-dir /home/fortier/testcode/rust/pairhmm/parasail-sys/target/debug/deps -C incremental=/home/fortier/testcode/rust/pairhmm/parasail-sys/target/debug/incremental -L dependency=/home/fortier/testcode/rust/pairhmm/parasail-sys/target/debug/deps --extern libc=/home/fortier/testcode/rust/pairhmm/parasail-sys/target/debug/deps/liblibc-bc949bf21f4fe772.rlib -L native=/home/fortier/testcode/rust/pairhmm/parasail-sys/target/debug/build/parasail-sys-2ac393455c1f3545/out -l dylib=parasail`
Finished dev [unoptimized + debuginfo] target(s) in 1m 58s
After further inspection I have discovered that the issue is somewhere in the C code that the sub-crate is wrapping. I replaced the current C code with an older version, while changing none of the Rust code and the issue has disappeared. I'll continue doing some further investigation to see exactly what was causing the problem and I'll update this post once I narrow it down.
Add to the build script println!("cargo:rerun-if-changed={}", &file); lines for each file and for each directory.

export shared library path via several level of crates denedicies

In build.rs of level1-sys crate I have:
let dst = cmake::Config::new(Path::new("somelib"))
.build()
.join("build");
println!("cargo:rustc-link-search=native={}", dst.display());
println!("cargo:rustc-link-lib=dylib=somelib");
then there is level2 create that deped on level1-sys,
and then there is level3binary crate.
If I run this level3 crates via cargo run all works fine,
but if I run it by hands it reports:
error while loading shared libraries: libsomelib.so: cannot open shared object file: No such file or directory
Is there way to find path to libsomelib.so from cargo?
I need this info for external script that should pack binary.
Obviously I can search in target subdirectory,
but I use debug/release/cross compilation and so on thing.
Plus even for concrete variant like target/release after several rebuild of level1-sys there are several libsomelib.so libraries, like:
target/release/build/level1-sys-46422ddc8585ba79/libsomelib.so
target/release/build/level1-sys-9daa760ee41fe7b8/libsomelib.so

capnpc::compile not writing files

I'm having difficulty working with the capnpc crate. I'm running Arch Linux and have installed capnp from the AUR and compiled capnpc-rust from the github project and put it in /usr/local/bin. I can manually compile the .capnp file easily with the command
capnp compile -orust --src-prefix=capnp capnp/message.capnp
I tried cloning the capnpc project and compiling the .capnp test file in the test directory and that didn't work either. I'm not getting any errors (whereas earlier I was getting "File not found") so it seems like capnpc is working, but I can't find files anywhere.
build.rs
extern crate capnpc;
fn main() {
::capnpc::compile("capnp", &["capnp/message.capnp"]).unwrap();
}
Cargo.toml
...
build = "build.rs"
[lib]
name = "rustp2p"
path = "src/lib.rs"
[build-dependencies]
capnpc = "*"
[dependencies]
capnp = "0.5.0"
Edit: .rs file builds out into /target/debug/build/.../out.
When you invoke capnpc::compile from a Cargo build script, the generated code goes in a subdirectory of target/ that can be found at main compile time through the OUT_DIR environment variable. This strategy is described in the Cargo documentation.
You shouldn't need to install the capnpc-rust binary in /usr/local/bin or anywhere else for this to work.
Your build.rs and Cargo.toml files look fine to me.
You might find it helpful to consult the addressbook example.

Resources