I ran into an issue while trying to define and use a trait with methods that borrow self mutably.
Some context that might make it easier: I am working on a toy compiler, and the problem I was trying to solve was to define a trait for code nodes, which are either statements or expressions. The trait was meant to be used for traversing code mutably (for rewriting purposes). The abstraction I was trying to create was a "code node" that may have any number of children that are either statements or expressions. This is how it went:
// Actually these are enums with different payload types for different kinds of exprs/stmts,
// but this is not relevant.
struct Expression;
struct Statement;
trait CodeNode<'a>
where
Self::ExprIter: Iterator<Item = &'a mut Expression>,
Self::StmtIter: Iterator<Item = &'a mut Statement>,
{
type ExprIter;
type StmtIter;
fn child_exprs(&'a mut self) -> Self::ExprIter;
fn child_stmts(&'a mut self) -> Self::StmtIter;
}
This trait would be then implemented for quite a few types (I have a separate type for different kinds of statements and expressions).
The way I tried to use it was:
fn process<'a>(node: &'a mut impl CodeNode<'a>) {
for _stmt in node.child_stmts() {
// ...
}
for _expr in node.child_exprs() {
// ...
}
}
And this is where the problem lies. Rust compiler treats a call to node.child_stmts as a mutable borrow of node for the entire lifetime 'a, and so it does not allow a call to node.child_exprs later in the same function. Here is how the error looks:
error[E0499]: cannot borrow `*node` as mutable more than once at a time
--> src/main.rs:21:18
|
16 | fn process<'a>(node: &'a mut impl CodeNode<'a>) {
| -- lifetime `'a` defined here
17 | for _stmt in node.child_stmts() {
| ------------------
| |
| first mutable borrow occurs here
| argument requires that `*node` is borrowed for `'a`
...
21 | for _expr in node.child_exprs() {
| ^^^^ second mutable borrow occurs here
What I want to do is to somehow make compiler aware of the fact that node implements CodeNode<'a> for any lifetime parameter, and so it should use two separate lifetimes for two
calls, but I can't quite figure out a way to do it.
Any suggestions are welcome, I don't have a lot of experience with Rust, so maybe I am missing some more high-level solution to the original problem.
Your lifetime 'a is constrained by the CodeNode so both functions will be called with the same lifetime, but what you want are two lifetimes constrained by the two functions. So why not do something like this.
struct Expression;
struct Statement;
trait CodeNode
{
type ExprIter<'a> : Iterator<Item = &'a mut Expression>; //unstable
type StmtIter<'a> : Iterator<Item = &'a mut Statement>; //unstable
fn child_exprs<'a>(&'a mut self) -> Self::ExprIter<'a>;
fn child_stmts<'a>(&'a mut self) -> Self::StmtIter<'a>;
}
fn process(node: &mut impl CodeNode) {
for _stmt in node.child_stmts() {
// ...
}
for _expr in node.child_exprs() {
// ...
}
}
Unfortunately I had to use the unstable feature of generic associated types, but I believe this is what you want.
I also want to express that iterating over mutable references might not be a good idea and maybe you should change your program structure if that is possible.
EDIT:
#pretzelhammer proposed in the comments the following link which might be interesting: Solving the generalized streaming iterator problem without gats
Related
I am trying to have a deeper understanding of how rust works. I am trying to do some serializing and deserializing to save and load a struct with a generic type. I got it to work, but I don't understand the HRTB and why they made the code work.
Initially I have this
use serde::Deserialize;
use bincode;
use std::fs;
#[derive(Deserialize)]
pub struct Construct<T> {
data: Vec<T>
}
impl <'a, T: Deserialize<'a>> Construct<T> {
pub fn load() -> Self {
match fs::read("data.sav") {
Ok(d) => {
let c: Construct<T> = bincode::deserialize(&d).unwrap();
c
},
Err(e) => {
println!("{e}, passing empty Construct");
Self { data: Vec::new() }
}
}
}
}
whihc produces this error
error[E0597]: `d` does not live long enough
--> src/main.rs:14:49
|
10 | impl <'a, T: Deserialize<'a>> Construct<T> {
| -- lifetime `'a` defined here
...
14 | let c: Construct<T> = bincode::deserialize(&d).unwrap();
| ---------------------^^-
| | |
| | borrowed value does not live long enough
| argument requires that `d` is borrowed for `'a`
15 | c
16 | },
| - `d` dropped here while still borrowed
I have fixed the impl block to take a higher ranked trait bound. And it works just fine.
...
impl <T: for<'a> Deserialize<'a>> Construct<T> {
pub fn load() -> Self {
...
As I understand it Deserialize needs to make sure that the input reference lives as long as the out structure(https://serde.rs/lifetimes.html), and the difference between declaring the trait in the first example and using for<'a>. Is that the 1st example the lifetime is being provided by the caller and the for<'a> is getting the lifetime from the impl itself. (How does "for<>" syntax differ from a regular lifetime bound?)
Am I right in thinking that with the for<'a> syntax we are getting the lifetime from the implementation block and that gives us a longer lifetime than from calling the function? Is there another way to code this load function without using HRTBs?
Am I right in thinking that with the for<'a> syntax we are getting the lifetime from the implementation block
Yes, from the call bincode::deserialize(&d). Specifically, the lifetime of d.
and that gives us a longer lifetime than from calling the function
Nope, a shorter: instead of a caller-decided lifetime, that will always be longer than d's lifetime (because it is declared inside our function), we get a lifetime for only d.
Is there another way to code this load function without using HRTBs?
Yes, by bounding T to DeserializeOwned. But this just hides the HRTB: DeserializeOwned uses them behind the scene.
According to this question, it is not possible to borrow parts of a struct, only the entire struct. This makes sense, however I would not expect the following code (playground) to compile, which it does:
struct Struct (u32, u32);
fn foo(a: &mut u32, b: &u32) {
}
fn main() {
let mut s = Struct ( 1, 2 );
foo(&mut s.0, &s.1);
}
Why does this work?
Additionally, is there any way to get the compiler to make the same differentiation between borrowing members when some indirection is introduced via. a method, without using a RefCell or other run-time checking. See (playground):
struct Struct (u32, u32);
impl Struct {
fn a(&mut self) -> &mut u32 {
&mut self.0
}
fn b(&self) -> &u32 {
&self.1
}
}
fn foo(a: &mut u32, b: &u32) {
}
fn main() {
let mut s = Struct ( 1, 2 );
foo(s.a(), s.b());
}
At the moment this fails with:
error[E0502]: cannot borrow `s` as immutable because it is also borrowed as mutable
--> src/main.rs:18:16
|
18 | foo(s.a(), s.b());
| --- ----- ^^^^^ immutable borrow occurs here
| | |
| | mutable borrow occurs here
| mutable borrow later used by call
I realise this is a bit of an obtuse example, my real case involves borrowing a number of members so the borrowing methods' implementations are more complex. There are still no members which are borrowed by both functions however.
Rust very deliberately does not extend its inference powers across function boundaries. This helps in making code more forwards compatible as you only need to keep the signature consistent and not its internals.
For example, consider your own code. If you decided later that a() should actually return a reference to self.1 then it would break all the code that used a() and b() together like you did. With the current limitations, you can easily change what field the reference comes from and not have to worry about breaking anyone.
Unfortunately, this makes what you want to do impossible.
I suggest giving the problem a higher level look. Do a() and b() really belong together on the same struct? Would it perhaps be better to split the two fields into their own structs?
Ideally, when you take a mutable reference to a struct, you would be using all (or most) of the struct, and there would be no need for someone else to be using that struct.
Here, I pass some mutable references into a function to perform some action on those. Then, I drop those mutable references by turning them into immutable references. However, the Rust borrow checker still seems to think they are mutable. Here's the code:
//! src/lib.rs
fn append_1_to_all(strings: Vec<&mut String>) -> Vec<&mut String> {
strings.into_iter().map(|s| { s.push_str("1"); s }).collect()
}
fn get_shared_references(strings: Vec<&mut String>) -> Vec<&String> {
strings.into_iter().map(|c| &(*c)).collect()
}
#[test]
fn test() {
let strings = vec!["one".to_string(), "two".to_string(), "three".to_string()];
let strings_appended = append_1_to_all(strings.iter_mut().collect());
let strings_shared = get_shared_references(strings_appended);
assert_ne!(strings[0], *strings_shared[0]);
}
Compile error:
error[E0502]: cannot borrow `strings` as immutable because it is also borrowed as mutable
--> src/lib.rs:16:16
|
12 | let strings_appended = append_1_to_all(strings.iter_mut().collect());
| ------- mutable borrow occurs here
...
16 | assert_ne!(strings[0], *strings_shared[0]);
| ^^^^^^^ -------------- mutable borrow later used here
| |
| immutable borrow occurs here
How can I fix this error? I would think strings_shared shouldn't be related to the mutable borrow anymore.
The problem is not that your reference is actually mutable, but that it's keeping the mutable reference alive.
If you have a reference, type &'a ... or &'a mut ..., and you produce another reference using it, whether that is a reborrow &*some_reference or a field access &some_struct_reference.some_field or a method or anything else, then that is always considered to be a borrow from that original reference, requiring the original reference to live as long as the derived one.
Notice that in your function declaration,
fn get_shared_references(strings: Vec<&mut String>) -> Vec<&String> {...}
if we change it to write out the elided lifetimes, we get this:
fn get_shared_references<'a>(strings: Vec<&'a mut String>) -> Vec<&'a String> {...}
In order to make this work, we'd have to be writing some other lifetime for the result, &'b String instead of `&'a String. But what would that lifetime be, and how is it constrained? It's got to last no longer than the value it refers to does, but be longer than the mutable reference. Rust's lifetime system doesn't have a way to express that. A near miss is:
fn get_shared_references<'m, 'i: 'm>(strings: Vec<&'m mut String>) -> Vec<&'i String> {...}
But even if you could implement a function with this type, it wouldn't be useful to the caller, because they don't know anything about the lifetime 'i other than "at least as long as 'm".
We could imagine a system where mutable references carry two lifetimes — let's suppose the syntax &'m mut 'i T, where 'm is the lifetime for which the reference is mutable, and 'i is (no longer than) the lifetime of the value. In that case, we could possibly have
fn consumes_mut_ref_produces_ref<'m, 'i>(x: &'m mut 'i i32) -> &'i i32 {...}
But a whole new set of lifetime rules would have to be designed to make this work, and it might not even be possible to do consistently. It's certainly not something today's Rust supports.
My recommendation, in the absence of further details on what you're trying to do, would be to make this code to work on owned values. Owned values can be passed around in the way you're trying to do — because taking them as function arguments transfers ownership, so there's no lifetimes to worry about, and it's much more possible to say: "my function consumes this and produces that," with no obligatory relationship.
fn append_1_to_all(mut strings: Vec<String>) -> Vec<String> {
for s in strings.iter_mut() {
s.push_str("1")
}
strings
}
fn get_shared_references(strings: &Vec<String>) -> Vec<&String> {
// Or even better, just use strings.iter() instead of this function
strings.iter().collect()
}
#[test]
fn test() {
let strings = vec!["one".to_string(), "two".to_string(), "three".to_string()];
// .clone() here is necessarily only because strings is used in the assert below
let strings_appended = append_1_to_all(strings.clone());
let strings_shared = get_shared_references(&strings_appended);
assert_ne!(strings[0], *strings_shared[0]);
}
This whole lifetime thing in Rust is still dark magic for me. I have a general idea of how it works but whenever I have to define lifetimes myself I have a hard time figuring out what to do. Here's what I want to do:
I have a [&str, 100] that comes from a submodule and I want to write a very simple randomized iterator that uses data from this submodule. Here's roughly what I do:
use rand::distributions::{Distribution, Uniform};
use super::data:Data;
struct RandomData {
range: Uniform<usize>,
rng: rand::rngs::ThreadRng,
}
impl RandomData {
fn new () -> RandomData {
RandomData {
range:·Uniform::new(0, Data.len()),
rng: rand::thread_rng(),
}
}
}
impl Iterator for RandomData {
type Item = &str;
fn next(next(&mut self) -> Option<Self::Item> {
let index = self.range.sample(&mut self.rng);
Some(Data[index])
}
}
Now, obviously the compiler is asking for lifetimes here because of the &str and the easiest way would be to simply use a static lifetime &'static str. But I wondered how to do this right, so I tried the real deal.
I started with the following changes to the iterator implementation:
impl<'a> Iterator for RandomData {
type Item = &'a str;
fn next(next(&mut self) -> Option<Self::Item> { .. }
}
Now the compiler says: error[E0207]: the lifetime parameter 'a is not constrained by the impl trait, self type, or predicates and suggest to read more about this error E0207, which I did. I think the gist is, that the lifetime parameter needs to appear either in the Trait or implementing type. Both is not the case because I don't need it there and in this case the documentation suggests to use PhantomData. But it also only talks about types and I don't really get it to work.
If I try to do:
struct RandomData<'a> {
range: Uniform<usize>,
rng: rand::rngs::ThreadRng,
phantom: PhantomData<&'a str>
}
I get a whole new bunch of messages about anonymous lifetimes, so I added them, but then get stuck with:
error[E0106]: missing lifetime specifier --> src/epcs/random_epc.rs:12:22
|
12 | pub fn new () -> RandomEPC {
| ^^^^^^^^^ help: consider giving it a 'static lifetime: `RandomEPC + 'static`
|
= help: this function's return type contains a borrowed value, but there is no value for it to be borrowed from
and I'm not sure where to go from here.
Edit
Thanks phimuemue for the suggestion. I created a simplified example here:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=bddde9310da5cf838dafee83e05cd78a
Lifetime is Rust describe how long will the data live before being drop.
In your case, the item of your Iterator is a reference to the data hold by Data. Therefore its lifetime correspond to the lifetime of Data. If Data is of lifetime static (it will live during the whole life of the process) then the right way to do your impl is to output Item with lifetime `static.
The idea with Associated Types is that the trait should be implemented only once. You can't implement twice Iterator, once with String as Item and once with &'static str. (See the book). Therefore, in your case you should implement Iterator only once with &'static str and not try to implement it for every lifetime 'l with &'l str.
Can anyone tell what the problem is with the following code? The compiler is complaining about lifetimes, but the error message makes absolutely no sense. I've tried everything I could think of, but nothing seems to help.
use std::borrow::BorrowMut;
trait Trait<'a> {
fn accept(&mut self, &'a u8);
}
struct Impl<'a>{
myref: Option<&'a u8>,
}
impl<'a> Trait<'a> for Impl<'a> {
fn accept(&mut self, inp: &'a u8) { self.myref = Some(inp); }
}
fn new<'a>() -> Box<Trait<'a> + 'a> {
Box::new(Impl{myref: None})
}
fn user<'a>(obj: &mut Trait<'a>) {}
fn parent<'a>(x: &'a u8) {
let mut pool = new();
user(pool.borrow_mut());
}
The compiler error is
error: `pool` does not live long enough
--> src/wtf.rs:22:10
|
22 | user(pool.borrow_mut());
| ^^^^ does not live long enough
23 | }
| - borrowed value dropped before borrower
|
= note: values in a scope are dropped in the opposite order they are created
Which makes absolutely no sense. How is the borrower outliving anything? I'm not even using the borrowed value!
Ok, this does make sense, but it's hard to see due to lifetime elision. So, here's your code with all the lifetimes written out explicitly, and with irrelevant details culled:
use std::borrow::BorrowMut;
trait Trait<'a> {}
struct Impl<'a> {
myref: Option<&'a u8>,
}
impl<'a> Trait<'a> for Impl<'a> {}
fn new<'a>() -> Box<Trait<'a> + 'a> {
Box::new(Impl { myref: None })
}
fn user<'a, 'b>(obj: &'b mut (Trait<'a> + 'b)) {}
fn parent() {
/* 'i: */ let mut pool/*: Box<Trait<'x> + 'x>*/ = new();
/* 'j: */ let pool_ref/*: &'i mut Box<Trait<'x> + 'x>*/ = &mut pool;
/* BorrowMut<T>::borrow_mut<'d>(&'d mut Self) -> &'d mut T */
/* 'k: */ let pool_borrow/*: &'i mut (Trait<'x> + 'x)*/ = Box::borrow_mut(pool_ref);
user(pool_borrow);
}
Now, from the perspective of the last line of parent, we can work out the following equivalences by just reading the definition of user and substituting the lifetimes we have in parent:
'a = 'x
'b = 'i
'b = 'x
Furthermore, this lets us conclude that:
'x = 'i
This is the problem. Because of the way you've defined user, you've put yourself in a situation where the lifetime of the pool_ref borrow (which is equal to the lifetime of the pool storage location you're borrowing from) must be the same as the lifetime 'x being used in the thing being stored in pool.
It's a bit like the Box being able to have a pointer to itself before it exists, which doesn't make any sense.
Either way, the fix is simple. Change user to actually have the correct type:
fn user<'a, 'b>(obj: &'b mut (Trait<'a> + 'a)) {}
This matches the type produced by new. Alternately, just don't use borrow_mut:
user(&mut *pool)
This works because it is "re-borrowing". Calling borrow_mut translates the lifetimes more or less directly, but re-borrowing allows the compiler to narrow the borrows to shorter lifetimes. To put it another way, explicitly calling borrow_mut doesn't allow the compiler enough freedom to "fudge" the lifetimes to make them all line up, re-borrowing does.
As a quick aside:
I'm not even using the borrowed value!
Irrelevant. Rust does type- and lifetime-checking entirely locally. It never looks at the body of another function to see what it's doing; it goes on the interface alone. The compiler neither checks, nor cares, what you're doing inside a different function.
Note that there's more to the error message:
error: `pool` does not live long enough
--> src/main.rs:25:10
|>
25 |> user(pool.borrow_mut());
|> ^^^^
note: reference must be valid for the block at 23:25...
--> src/main.rs:23:26
|>
23 |> fn parent<'a>(x: &'a u8) {
|> ^
note: ...but borrowed value is only valid for the block suffix following statement 0 at 24:25
--> src/main.rs:24:26
|>
24 |> let mut pool = new();
|> ^
Let's look at user:
fn user<'a>(obj: &mut Trait<'a>) {}
This says that it will accept a mutable reference (with an unnamed lifetime) to a trait object parameterized with the lifetime 'a.
Turning to new, I'd say the method is highly suspicious:
fn new<'a>() -> Box<Trait<'a> + 'a> {
Box::new(Impl { myref: None })
}
This says that it will return a boxed trait object with whatever lifetime the caller specifies. That basically never makes sense.
All that said, I'm not clear why the code chooses to use borrow_mut. I would have written that more directly:
user(&mut *pool);
This dereferences the Box<Trait> to get a Trait, then takes a mutable reference, yielding &mut Trait, which compiles.
I cannot currently explain why BorrowMut differs in behavior.
I'm not sure why this error happens, but I can give solutions!
First, it seems that using borrow_mut unnecessarily restricts the lifetime of the returned reference. Using operators to create the reference solves the error.
fn parent() {
let mut pool = new();
user(&mut *pool);
}
However, if we don't do that, we can solve the error by adding a lifetime bound to the Trait object in user's obj argument.
fn user<'a>(obj: &mut (Trait<'a> + 'a)) {}