I would like to initialise some constants at the very top of my main.rs file, like so:
const PRIVATE_KEY: Vec<u8> = std::fs::read("./jwtRS256.key").unwrap();
const PUBLIC_KEY: Vec<u8> = std::fs::read("./jwtRS256.pub.key").unwrap();
fn main () {
// do some stuff
}
However, I am getting a compiler error as follows:
error[E0015]: calls in constants are limited to constant functions, tuple structs and tuple variants
--> src/main.rs:16:30
|
16 | const PRIVATE_KEY: Vec<u8> = std::fs::read("./jwtRS256.key").unwrap();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I understand; makes sense. The compiler wants to be able to evaluate the values of these constants at compile time and it can't. What is the alternative, though? I need those constants available all through my code, including in other modules. So, I cannot just move them as regular variables into the main function. What do I do instead? (I am on Rust 1.47.)
THere are two possibilities and it's a bit unclear which one you want:
Embed the content of the files in the binary
include_bytes and include_str will read the files during compilation and embed their contents (according to the machine having compiled the binary) in the final program.
Runtime global
If you want to read the files when the program starts and make those contents available globally, you want lazy_static or once_cell instead, they allow running code to initialise a global at runtime.
const v static
static is probably what you want either way, it's much closer to globals in other languages.
const is more macro-ish, all references to the const's name will be replaced by the value, directly.
This is often what you want for e.g. simple numbers (that way they get compiled as immediates rather than loads from memory), but for more complex types or large amounts of data static is generally the better pick.
If these files are present and compile-time and your only goal is to include them in the compilation, then you can simply use include_bytes!("./jwtRS256.key"). However, if you want to read these files at runtime, then consider using lazy_static:
lazy_static! {
pub static ref PRIVATE_KEY: Vec<u8> = std::fs::read("./jwtRS256.key").unwrap();
}
This crate essentially allows you to lazily initialize static variables and use them anywhere.
Related
I'm trying to load JSON files that refer to structs implementing a trait. When the JSON files are loaded, the struct is grabbed from a hashmap. The problem is, I'll probably have to have a lot of structs put into that hashmap all over my code. I would like to have that done automatically. To me this seems to be doable with procedural macros, something like:
#[my_proc_macro(type=ImplementedType)]
struct MyStruct {}
impl ImplementedType for MyStruct {}
fn load_implementors() {
let implementors = HashMap::new();
load_implementors!(implementors, ImplementedType);
}
Is there a way to do this?
No
There is a core issue that makes it difficult to skip manually inserting into a structure. Consider this simplified example, where we simply want to print values that are provided separately in the code-base:
my_register!(alice);
my_register!(bob);
fn main() {
my_print(); // prints "alice" and "bob"
}
In typical Rust, there is no mechanism to link the my_print() call to the multiple invocations of my_register. There is no support for declaration merging, run-time/compile-time reflection, or run-before-main execution that you might find in other languages that might make this possible (unless of course there's something I'm missing).
But Also Yes
There are third party crates built around link-time or run-time tricks that can make this possible:
ctor allows you to define functions that are executed before main(). With it, you can have my_register!() create invididual functions for alice and bob that when executed will add themselves to some global structure which can then be accessed by my_print().
linkme allows you to define a slice that is made from elements defined separately, which are combined at compile time. The my_register!() simply needs to use this crate's attributes to add an element to the slice, which my_print() can easily access.
I understand skepticism of these methods since the declarative approach is often clearer to me, but sometimes they are necessary or the ergonomic benefits outweigh the "magic".
I'm learning Rust, and so far, there appears to be 3 ways to declare variables:
const A: u8 = 42;
static A: u8 = 42;
let A: u8 = 42;
I get that you can't have a mutable const, and the compiler will warn you if it's not all uppercase, and that when you use a const, the equivalent of the C pre-compiler will replace every place that A appears with a literal 42 (it won't have a consistent memory address).
From a practical standpoint, I don't see a difference between any of these in their immutable form. None of them can be mutated. They can all be used exactly the same. What's the difference?
- const defines values, which are replaced in the according code
- static defines global variables representing memory addresses, ie read-only memory, global atomic counters or locks (+initialization), C-abi interaction, etc
- let defines scoped variable bindings
You can not use let as global variable (static lifetime), so static fits that role. Note, that it also exists as lifetime annotation.
Reference explanation
What's the difference?
The question can be easily reduced to const vs. immutable static because that is the only case that is pretty similar (let is intended to introduce variables to the current scope).
const is supposed to be used when you don't mind the value to be inlined all over the place. Typically, that means small objects; commonly constant integers. Immutable static is used otherwise.
In practice, for most use cases (specially for private items), when considering optimizations done by LLVM, there is usually little difference.
You might need to take a closer look if codegen was particularly bad, if you want to export an object and related low-level concerns.
At the beginning of my program, I read data from a file:
let file = std::fs::File::open("data/games.json").unwrap();
let data: Games = serde_json::from_reader(file).unwrap();
I would like to know how it would be possible to do this at compile time for the following reasons:
Performance: no need to deserialize at runtime
Portability: the program can be run on any machine without the need to have the json file containing the data with it.
I might also be useful to mention that, the data can be read only which means the solution can store it as static.
This is straightforward, but leads to some potential issues. First, we need to deal with something: do we want to load the tree of objects from a file, or parse that at runtime?
99% of the time, parsing on boot into a static ref is enough for people, so I'm going to give you that solution; I will point you to the "other" version at the end, but that requires a lot more work and is domain-specific.
The macro (because it has to be a macro) you are looking for to be able to include a file at compile-time is in the standard library: std::include_str!. As the name suggests, it takes your file at compile-time and generates a &'static str from it for you to use. You are then free to do whatever you like with it (such as parsing it).
From there, it is a simple matter to then use lazy_static! to generate a static ref to our JSON Value (or whatever it may be that you decide to go for) for every part of the program to use. In your case, for instance, it could look like this:
const GAME_JSON: &str = include_str!("my/file.json");
#[derive(Serialize, Deserialize, Debug)]
struct Game {
name: String,
}
lazy_static! {
static ref GAMES: Vec<Game> = serde_json::from_str(&GAME_JSON).unwrap();
}
You need to be aware of two things when doing this:
This will massively bloat your file size, as the &str isn't compressed in any way. Consider gzip
You'll need to worry about the usual concerns around multiple, threaded access to the same static ref, but since it isn't mutable you only really need to worry about a portion of it
The other way requires dynamically generating your objects at compile-time using a procedural macro. As stated, I wouldn't recommend it unless you really have a really expensive startup cost when parsing that JSON; most people will not, and the last time I had this was when dealing with deeply-nested multi-GB JSON files.
The crates you want to look out for are proc_macro2 and syn for the code generation; the rest is very similar to how you would write a normal method.
When you are deserializing something at runtime, you're essentially building some representation in program memory from another representation on disk. But at compile-time, there's no notion of "program memory" yet - where will this data deserialize too?
However, what you're trying to achieve is, in fact, possible. The main idea is like following: to create something in program memory, you must write some code which will create the data. What if you're able to generate the code automatically, based on the serialized data? That's what uneval crate does (disclaimer: I'm the author, so you're encouraged to look through the source to see if you can do better).
To use this approach, you'll have to create build.rs with approximately the following content:
// somehow include the Games struct with its Serialize and Deserialize implementations
fn main() {
let games: Games = serde_json::from_str(include_str!("data/games.json")).unwrap();
uneval::to_out_dir(games, "games.rs");
}
And in you initialization code you'll have the following:
let data: Games = include!(concat!(env!("OUT_DIR"), "/games.rs"));
Note, however, that this might be fairly hard to do in ergonomic way, since the necessary struct definitions now must be shared between the build.rs and the crate itself, as I mentioned in the comment. It might be a little easier if you split your crate in two, keeping struct definitions (and only them) in one crate, and the logic which uses them - in another one. There's some other ways - with include! trickery, or by using the fact that the build script is an ordinary Rust binary and can include other modules as well, - but this will complicate things even more.
This question already has an answer here:
Rust Can't import Singleton From global space into another module in another file
(1 answer)
Closed 4 years ago.
I am trying to use the lazy_static crate to create a singleton and use it in a different module. Is that possible or even recommended? I'm still learning how Rust programs should be structured and have been making every file its own module.
I have the following in main.rs and I can access its values
lazy_static! {
static ref GAMEDATA: gamedata::data::GameDataS =
gamedata::data::load_data("./src/assets/data.json".to_string());
}
fn main() {
println!("data{}", GAMEDATA.width);
}
When trying to access GAMEDATA in a different module, I get
not found in this scope
for example in a module called game
pub struct Game {}
impl Game {
println!("data{}", GAMEDATA.width);
}
Is it possible to make a global variable across all modules? Is there some other way I should be thinking about it? Perhaps not using modules as often?
If your static variable is in another non-parent module, your problem seems to be a missing pub modifier before static. Also, as others pointed out, the code in which you use the variable (the impl block) is not valid Rust syntax.
Besides that, you will need to import the static variable with use (E.g. use GAMEDATA;), see Quan Brew's answer.
However, I want to discuss about the use of static and singleton pattern in Rust.
Static variables in Rust
In Rust we generally avoid static variables. In most cases they could be replaced with a proper constant via const. Because static variables may be shared between threads, having them with external mutability is unsafe in Rust. This is why you cannot have external mutability with lazy_static.
Although statics with external mutability do have their uses, they are kind of specific, should be justified and avoided otherwise. Interior mutability, as described in this section of the Rust Book is not even allowed to be shared among threads.
Singleton Pattern in Rust
I don't think it is a good idea to use static to have singleton pattern. This pattern is not common in Rust. We generally pass all mutable things as arguments.
Solutions if you need immutable data
Make it a constant with const.
If there is too much data, use static.
If you need non-constant initialization, you can keep lazy_static.
Solutions if you need to mutate data
Put your singleton into a Mutex or another lock. This will ensure correct concurrent accesses.
Make it thread local with thread_local macro + inner mutability with RefCell
Give up the "singleton pattern" idea and pass the structure via arguments (recommended).
You need use to import GAMEDATA into the current scope, as described in the modules section in the book.
Example code (playground):
#[macro_use]
extern crate lazy_static; // 1.1.0
lazy_static! {
static ref GAMEDATA: String = "hello".to_string();
}
mod foo {
use GAMEDATA;
pub fn bar() {
println!("{}", *GAMEDATA);
}
}
fn main() {
foo::bar();
}
However, the singleton pattern is not recommended in Rust. For beginners in the learning phase, you'd best avoid singletons. (see bzim's answer)
I've tried things like this:
const b: Box<i32> = Box::new(5);
Which gave me that function calls in constants are limited to struct and enum constructors.
I also tried
const b: Box<i32> = box 5;
Which gave me an error saying that I should use Box::new instead. Is there any way to do this? I need a box because it's in a struct and the struct requires a box.
Edit:
I know that Box::new is a function, and I can't use that in a const item. But is there another way to create a box that is allowed?
Not right now, not in the immediate future.
As #Paolo mentioned, the only way to initialize a const variable is to use a constant expression. Today, in stable, it is limited to a restricted set of operations (some integers manipulation, some casts, ...).
There is a RFC to extend the set of expressions available in constant expressions: const fn. It is about allowing functions (both free functions and methods) to be marked const, making them available in constant expressions.
The tracking issue is #24111, and const fn can be used on nightly with the #![feature(const_fn)] crate attribute...
... however, at the moment, const fn are mostly about integral manipulations too. There is no plan that I know of to extend to arbitrary (side-effect-less) expressions, and thus it would not work for Box::new.
At the moment, you are advised to use lazy_static!, it will not allow the item to be const (it will be initialized on first use).
As the const and static chapter of the book says:
Both const and static have requirements for giving them a value. They
may only be given a value that’s a constant expression. In other
words, you cannot use the result of a function call or anything
similarly complex or at runtime.
Box::new() is a function, so it falls in the scope of things you can't do to initialize a const.
You can look at lazy-static for a way to initialize a static variable at runtime.