I'm coming to Rust from the JavaScript world, where you could have a .env file (git-ignored), and environmental values from this file could be made available at runtime via an npm package (say, dotenv).
I was able to use a dotenv crate to make this work similarly when I just cargo run the project, but after I build a release version, the value of the environment variable is lost and the app panics.
I understand that cargo doesn't evaluate env variables at build time, and using env!-like macros won't work either, at least to my best understanding.
What I'm asking is this: how do I store a variable in a file (that I can gitignore), make Rust pick up values from this file, and use them at build time so that these values are available to the release-built app?
I'm sure that there's a well-established way of doing this, but I struggle to figure it out.
There is the dotenv-codegen crate that provides a dotenv! macro that works like the env! macro except it will load from a .env file.
I'm not so sure this is well-established, but you can use a build script that will read the file and use println!("cargo:rustc-env=VAR=VALUE") to send the environment variables to Cargo, allowing you to retrieve them in the code with env!() or option_env!().
For example, to use a .env file, add dotenv to build-dependencies, and use it like so in build.rs:
fn main() {
let dotenv_path = dotenv::dotenv().expect("failed to find .env file");
println!("cargo:rerun-if-changed={}", dotenv_path.display());
// Warning: `dotenv_iter()` is deprecated! Roll your own or use a maintained fork such as `dotenvy`.
for env_var in dotenv::dotenv_iter().unwrap() {
let (key, value) = env_var.unwrap();
println!("cargo:rustc-env={key}={value}");
}
}
Related
In my library I have few examples -- (normally) each one is represented by a single x<N>.rs file living in examples directory.
One example uses a .proto file -- this file needs to be compiled during build (of said example) and it's generated output is used by example itself.
I've tried this in my Cargo.toml:
[[example]]
name = "x1"
path = "examples/x1/main.rs"
build = "examples/x1/build.rs"
but build key gets ignored when I run cargo build --example x1
Is it possible to have example-specific build.rs file?
If not -- what is the correct way to deal with this situation?
Edit: I ended up processing that .proto file in crate's build.rs (even though it is not required to build that crate) and using artefacts in the example like this:
pub mod my_proto {
include!(concat!(env!("OUT_DIR"), "/my_proto.rs"));
}
This is not possible. This issue explains why, but in a nutshell build scripts are used for whole crate. So you could move your example into separate crate.
My package has a binary target that uses include_bytes!(…) to bundle a copy of some precomputed values into the compiled binary. This is an optimization, but isn't strictly necessary: the program is capable of calculating these values at run time if the bundled data slice .is_empty().
The program needs to be able to build without this data. However, include_bytes!("data/computed.bin") causes a build error if the target file does not exist.
error: couldn't read src/data/computed.bin: No such file or directory (os error 2)
Currently, I have a Bash build script that uses touch data/computed.bin to ensure the file exists before building. However, I don't want to depend on platform-specific solutions like Bash; I want to be able to build this project on any supported platform using cargo build.
How can my Rust program include_bytes!(…) or include_str!(…) from a file if it exits, but gracefully fall back to an alternative value or behaviour if the file doesn't exist, while only using the standard Cargo build tools?
We can use a build script to ensure that the included file exists before out package tries to include it. However, build scripts can only write to the current build's unique output directory, so we can't just create the missing input files in the source directory directly.
error: failed to verify package tarball
Caused by:
Source directory was modified by build.rs during cargo publish. Build scripts should not modify anything outside of OUT_DIR.
Instead, our build script can create the file-to-include in the build directory, copying the source data if it exists, and we can update our package code to include this data from the build directory instead of from the source directory. The build path will be available in the OUT_DIR environment variable during the build, so we can access it from std::env::var("OUT_DIR") in our build script and from env!("OUT_DIR") in the rest of our package.
//! build.rs
use std::{fs, io};
fn main() {
let out_dir = std::env::var("OUT_DIR").unwrap();
fs::create_dir_all(&format!("{}/src/data", out_dir))
.expect("unable to create data directory");
let path = format!("src/data/computed.bin", name);
let out_path = format!("{}/{}", out_dir, path);
let mut out_file = fs::OpenOptions::new()
.append(true)
.create(true)
.open(&out_path)
.expect("unable to open/create data file");
if let Ok(mut source_file) = fs::File::open(&path) {
io::copy(&mut source_file, &mut out_file).expect("failed to copy data after opening");
}
}
//! src/foo.rs
fn precomputed_data() -> Option<&'static [u8]> {
let data = include_bytes!(concat!(env!("OUT_DIR"), "/src/data/computed.bin")).as_ref();
if !data.is_empty() {
Some(data)
} else {
None
}
}
While using a build script (like in this answer) would work, I am not a fan of that solution:
The build script copies the file – depending on the file size, that might be prohibitively expensive. Though one could probably solve this problem using hardlinks instead.
An empty file might be perfectly fine data – the solution would misdetect an empty file as missing. However, depending on the use case, an empty file might actually be perfectly valid.
It is very verbose – this turns a simple include_bytes! into a build script of approximately 20 lines and additionally a few more lines when including to handle the data.is_empty() case.
It is hard to grasp what is happening here for a casual reader – why is this script including something from $OUT_DIR? It would probably take a moment for the reader to get the idea a build script might be involved here.
It does not scale well – most of those problems would get even worse if there were multiple files that needed to be included optionally.
I therefore decided to write the procedural macro crate include_optional to solve this problem (currently only works on nightly Rust because it depends on some unstable features).
With this, the solution to this problem is a one liner:
use include_optional::include_bytes_optional;
fn precomputed_data() -> Option<&'static [u8]> {
include_bytes_optional!("./computed.bin")
}
There are also macros wrapping include_str! and include!.
The Rust language supports conditional compilation using attributes like #[cfg(test)].
Rust also supports build scripts using a build.rs file to run code as part of the build process to prepare for compilation.
I would like to use conditional compilation in Rust code to conditionally compile depending on whether we're compiling for a build script, similar to how that is possible for test builds.
Imagine the following:
#[cfg(build)]
fn main() {
// Part of build script
}
#[cfg(not(build))]
fn main() {
// Not part of build script, probably regular build
}
This does not work, because build is not a valid identifier here.
Is it possible to do this using a different attribute, or could some other trick be used to achieve something similar?
For some context on this issue:
My goal is to generate shell completion scripts through clap at compile time. I've quite a comprehensive App definition across multiple files in the application. I'd like to use this in build.rs by including these parts using the include!(...) macro (as suggested by clap), so I don't have to define App a second time. This pulls some dependencies with it, which I'd like to exclude when used by the build.rs file as they aren't needed in that case. This is what I'm trying to make available in my build.rs script.
You can just put the build code in build.rs (or presumably have build.rs declare a mod xyz to pull in another file).
I wonder if the question you are trying to ask is whether you can reference the same code from build.rs and main.rs, and if so can that code tell if it's being called by one or the other. It seems you could switch on an environment variable set when using build.rs (using something like option_env, but possibly a nicer way might be to enable a feature in the main code from within build.rs.
(Have a read of the documentation for build scripts if you haven't already.)
This is what works for me:
fn main() {
if std::env::var("PROFILE").unwrap() == "debug" {
// Here I do something that is needed for tests
// only, not for 'release'
}
}
How do you access a Cargo package's metadata (e.g. version) from the Rust code in the package? In my case, I am building a command line tool that I'd like to have a standard --version flag, and I'd like the implementation to read the version of the package from Cargo.toml so I don't have to maintain it in two places. I can imagine there are other reasons someone might want to access Cargo metadata from the program as well.
Cargo passes some metadata to the compiler through environment variables, a list of which can be found in the Cargo documentation pages.
The compiler environment is populated by fill_env in Cargo's code. This code has become more complex since earlier versions, and the entire list of variables is no longer obvious from it because it can be dynamic. However, at least the following variables are set there (from the list in the docs):
CARGO_MANIFEST_DIR
CARGO_PKG_AUTHORS
CARGO_PKG_DESCRIPTION
CARGO_PKG_HOMEPAGE
CARGO_PKG_NAME
CARGO_PKG_REPOSITORY
CARGO_PKG_VERSION
CARGO_PKG_VERSION_MAJOR
CARGO_PKG_VERSION_MINOR
CARGO_PKG_VERSION_PATCH
CARGO_PKG_VERSION_PRE
You can access environment variables using the env!() macro. To insert the version number of your program you can do this:
const VERSION: &str = env!("CARGO_PKG_VERSION");
// ...
println!("MyProgram v{}", VERSION);
If you want your program to compile even without Cargo, you can use option_env!():
const VERSION: Option<&str> = option_env!("CARGO_PKG_VERSION");
// ...
println!("MyProgram v{}", VERSION.unwrap_or("unknown"));
The built-crate helps with serializing a lot of Cargo's environment without all the boilerplate.
At build-time (as in build.rs), cargo_metadata may be useful. For example:
let path = std::env::var("CARGO_MANIFEST_DIR").unwrap();
let meta = MetadataCommand::new()
.manifest_path("./Cargo.toml")
.current_dir(&path)
.exec()
.unwrap();
let root = meta.root_package().unwrap();
let option = root.metadata["my"]["option"].as_str().unwrap();
let version = &root.version;
...
How do you access a Cargo package's metadata (e.g. version) from the Rust code in the package? In my case, I am building a command line tool that I'd like to have a standard --version flag, and I'd like the implementation to read the version of the package from Cargo.toml so I don't have to maintain it in two places. I can imagine there are other reasons someone might want to access Cargo metadata from the program as well.
Cargo passes some metadata to the compiler through environment variables, a list of which can be found in the Cargo documentation pages.
The compiler environment is populated by fill_env in Cargo's code. This code has become more complex since earlier versions, and the entire list of variables is no longer obvious from it because it can be dynamic. However, at least the following variables are set there (from the list in the docs):
CARGO_MANIFEST_DIR
CARGO_PKG_AUTHORS
CARGO_PKG_DESCRIPTION
CARGO_PKG_HOMEPAGE
CARGO_PKG_NAME
CARGO_PKG_REPOSITORY
CARGO_PKG_VERSION
CARGO_PKG_VERSION_MAJOR
CARGO_PKG_VERSION_MINOR
CARGO_PKG_VERSION_PATCH
CARGO_PKG_VERSION_PRE
You can access environment variables using the env!() macro. To insert the version number of your program you can do this:
const VERSION: &str = env!("CARGO_PKG_VERSION");
// ...
println!("MyProgram v{}", VERSION);
If you want your program to compile even without Cargo, you can use option_env!():
const VERSION: Option<&str> = option_env!("CARGO_PKG_VERSION");
// ...
println!("MyProgram v{}", VERSION.unwrap_or("unknown"));
The built-crate helps with serializing a lot of Cargo's environment without all the boilerplate.
At build-time (as in build.rs), cargo_metadata may be useful. For example:
let path = std::env::var("CARGO_MANIFEST_DIR").unwrap();
let meta = MetadataCommand::new()
.manifest_path("./Cargo.toml")
.current_dir(&path)
.exec()
.unwrap();
let root = meta.root_package().unwrap();
let option = root.metadata["my"]["option"].as_str().unwrap();
let version = &root.version;
...