This code fails the dreaded borrow checker (playground):
struct Data {
a: i32,
b: i32,
c: i32,
}
impl Data {
fn reference_to_a(&mut self) -> &i32 {
self.c = 1;
&self.a
}
fn get_b(&self) -> i32 {
self.b
}
}
fn main() {
let mut dat = Data{ a: 1, b: 2, c: 3 };
let aref = dat.reference_to_a();
println!("{}", dat.get_b());
}
Since non-lexical lifetimes were implemented, this is required to trigger the error:
fn main() {
let mut dat = Data { a: 1, b: 2, c: 3 };
let aref = dat.reference_to_a();
let b = dat.get_b();
println!("{:?}, {}", aref, b);
}
Error:
error[E0502]: cannot borrow `dat` as immutable because it is also borrowed as mutable
--> <anon>:19:20
|
18 | let aref = dat.reference_to_a();
| --- mutable borrow occurs here
19 | println!("{}", dat.get_b());
| ^^^ immutable borrow occurs here
20 | }
| - mutable borrow ends here
Why is this? I would have thought that the mutable borrow of dat is converted into an immutable one when reference_to_a() returns, because that function only returns an immutable reference. Is the borrow checker just not clever enough yet? Is this planned? Is there a way around it?
Lifetimes are separate from whether a reference is mutable or not. Working through the code:
fn reference_to_a(&mut self) -> &i32
Although the lifetimes have been elided, this is equivalent to:
fn reference_to_a<'a>(&'a mut self) -> &'a i32
i.e. the input and output lifetimes are the same. That's the only way to assign lifetimes to a function like this (unless it returned an &'static reference to global data), since you can't make up the output lifetime from nothing.
That means that if you keep the return value alive by saving it in a variable, you're keeping the &mut self alive too.
Another way of thinking about it is that the &i32 is a sub-borrow of &mut self, so is only valid until that expires.
As #aSpex points out, this is covered in the nomicon.
Why is this an error: While a more precise explanation was already given by #Chris some 2.5 years ago, you can read fn reference_to_a(&mut self) -> &i32 as a declaration that:
“I want to exclusively borrow self, then return a shared/immutable reference which lives as long as the original exclusive borrow” (source)
Apparently it can even prevent me from shooting myself in the foot.
Is the borrow checker just not clever enough yet? Is this planned?
There's still no way to express "I want to exclusively borrow self for the duration of the call, and return a shared reference with a separate lifetime". It is mentioned in the nomicon as #aSpex pointed out, and is listed among the Things Rust doesn’t let you do as of late 2018.
I couldn't find specific plans to tackle this, as previously other borrow checker improvements were deemed higher priority. The idea about allowing separate read/write "lifetime roles" (Ref2<'r, 'w>) was mentioned in the NLL RFC, but no-one has made it into an RFC of its own, as far as I can see.
Is there a way around it? Not really, but depending on the reason you needed this in the first place, other ways of structuring the code may be appropriate:
You can return a copy/clone instead of the reference
Sometimes you can split a fn(&mut self) -> &T into two, one taking &mut self and another returning &T, as suggested by #Chris here
As is often the case in Rust, rearranging your structs to be "data-oriented" rather than "object-oriented" can help
You can return a shared reference from the method: fn(&mut self) -> (&Self, &T) (from this answer)
You can make the fn take a shared &self reference and use interior mutability (i.e. define the parts of Self that need to be mutated as Cell<T> or RefCell<T>). This may feel like cheating, but it's actually appropriate, e.g. when the reason you need mutability as an implementation detail of a logically-immutable method. After all we're making a method take a &mut self not because it mutates parts of self, but to make it known to the caller so that it's possible to reason about which values can change in a complex program.
Related
In attempting to refactor a Rust application that was working fine, I tried to separate out the contents of a loop into a new function. However, in this newly refactored out function, I needed to pass an argument that had to be mutable, and passed by reference. Suddenly the code that absolutely worked inline, broke just because of the mutable reference passing.
My question is: can someone please explain why this does not work with such a "simple" change? (i.e. refactoring out a new function of otherwise unchanged code)
I have a minimal demo of the problem, along with a couple of working comparisons below. Here is the error from that code:
error[E0499]: cannot borrow `str_to_int` as mutable more than once at a time
--> src/main.rs:30:22
|
30 | get_key(key, &mut str_to_int);
| ^^^^^^^^^^^^^^^ `str_to_int` was mutably borrowed here in the previous iteration of the loop
The sample code:
use std::collections::BTreeMap;
fn get_int (
key: u32,
values: &mut BTreeMap<u32, u32>,
) -> &u32 {
values.entry(key).or_insert_with(|| { 1 })
}
fn get_key<'a> (
key: &'a str,
values: &'a mut BTreeMap<&'a str, u32>,
) -> &'a u32 {
values.entry(key).or_insert_with(|| { 1 })
}
fn main() {
let mut int_to_int = BTreeMap::new();
for key in vec![1,2] {
get_int(key, &mut int_to_int);
}
let mut str_to_int_inline = BTreeMap::new();
for key in vec!["a","b"] {
str_to_int_inline.entry(key).or_insert_with(|| { 1 });
}
let mut str_to_int = BTreeMap::new();
for key in vec!["a","b"] {
get_key(key, &mut str_to_int);
}
}
Note that the first loop (int_to_int) is identical to the third loop (str_to_int) except for the data type of the key -- in that the key was not a reference, so no lifetime was required to be specified. And the second loop (str_to_int_inline) is identical to the third loop (str_to_int) except the behavior is inline instead of in a separate function.
There are many related questions and blogs on this topic, but they all seem more specifically focused on particular versions of this question, and I want to know the more generic explanation (to my current understanding). If the answer is already just to understand one of these links better, I will gladly mark this question as a duplicate.
Related questions:
How to fix ".. was mutably borrowed here in the previous iteration of the loop" in Rust?
https://users.rust-lang.org/t/mutable-borrow-starts-here-in-previous-iteration-of-loop/26145
https://github.com/rust-lang/rust/issues/47680#issuecomment-363131420
Why does linking lifetimes matter only with mutable references?
Something that I read also led me to https://github.com/rust-lang/polonius which also seemed like maybe it could make this work, in the future -- any thoughts?
Your problem is very simple: you are specifying the lifetimes for get_key() incorrectly.
The correct (and working) version is:
fn get_key<'a, 'b>(key: &'a str, values: &'b mut BTreeMap<&'a str, u32>) -> &'b u32 {
values.entry(key).or_insert_with(|| 1)
}
Maybe you can already guess what's going on.
Since you used 'a for both the HashMap itself and its keys, it means you were required to borrow the HashMap for as long as the keys' lifetime - 'static. This means two things:
You need a &'static mut HashMap, which you don't have.
You're borrowing the HashMap for 'static in the first iteration of the loop, then borrow it again in the next, while it is still borrowed (because it is borrowed for 'static). This error hides the first error, somewhat erroneously, and is the only error the compiler emits.
In general, using a lifetime twice with a mutable reference is almost always wrong (shared references are more tolerant, as they're covariant over their type).
This is my code:
use std::collections::HashMap;
struct Foo {
pub map : HashMap<i32, String>
}
impl Foo {
fn foo(&mut self, x: &String) -> i32 {
// I'm planning to use/modify "x" here and also modify "self"
42
}
fn bar(&mut self) -> i32 {
let x = self.map.get_mut(&1).unwrap();
self.foo(x)
}
}
I'm getting:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:13:9
|
12 | let x = self.map.get_mut(&1).unwrap();
| -------------------- first mutable borrow occurs here
13 | self.foo(x)
| ^^^^^^^^^-^
| | |
| | first borrow later used here
| second mutable borrow occurs here
What's going on?
Modifying self and x here breaks memory safety (at least in the general situation, which is what Rust must deal with). Consider the following implementation of foo which is allowed by your signature (fixing &String to &str):
fn foo(&mut self, x: &str) -> i32 {
self.map.clear();
println!("{}", x);
42
}
But you're calling this with x being a reference to something inside of self.map. So x could be destroyed by the time it's used. That's invalid, and Rust can't prove you won't do that, because you said you might. (Kevin Anderson provides a helpful comment below if you're coming from a GC language like C# where "reference" has a different meaning.)
How to fix this depends on what you're really trying to do, though one approach would be to clone the string so it cannot be destroyed:
fn bar(&mut self) -> i32 {
let x = self.map.get(&1).unwrap().clone(); // <== now you have a copy
self.foo(&x)
}
Note this got rid of the get_mut(). It's unclear what that was for. If you need an exclusive (mut) reference into the map, then you'll need to do that separately, and you can't do that directly while also holding an exclusive reference to self for the reasons above. Remember that mut means "exclusive access," not "mutable." A side effect of having exclusive access is that mutation is allowed.
If you really need something along these lines, you need to wrap your values (String) in Arc so that you can maintain reference counts and have shared ownership. But I would first try to redesign your algorithm to avoid this.
I'm trying to modify the borrow of a mutable value, here is a minimal example:
fn main() {
let mut w: [char; 5] = ['h', 'e', 'l', 'l', 'o'];
let mut wslice: &mut [char] = &mut w;
advance_slice(&mut wslice);
advance_slice(&mut wslice);
}
fn advance_slice(s: &mut &mut [char]) {
let new: &mut [char] = &mut s[1..];
*s = new;
}
the compiler gives me this error:
error[E0623]: lifetime mismatch
--> src/main.rs:10:10
|
8 | fn advance_slice(s: &mut &mut [char]) {
| ----------------
| |
| these two types are declared with different lifetimes...
9 | let new: &mut [char] = &mut s[1..];
10 | *s = new;
| ^^^ ...but data from `s` flows into `s` here
I've tried to give the same lifetime to both borrow, without success.
Also this works if I remove the mutability of w.
This error message is indeed unfortunate, and I don't think it provides a good explanation of what's going on here. The issue is a bit involved.
The error is local to the advance_slice() function, so that's all we need to look at. The type of the slice items is irrelevant, so let's take this function definition:
fn advance_slice_mut<T>(s: &mut &mut [T]) {
let new = &mut s[1..];
*s = new;
}
The first line creates a new slice object starting after the first item of the original slice.
Why is this even allowed? Don't we have two mutable references to the same data now? The original slice *s includes the new slice new, and both allow modifying data. The reason this is legal is that *s is implicitly reborrowed when creating the subslice, and *s cannot be used again for the lifetime of that borrow, so we still have only one active reference to the data in the subslice. The reborrow is scoped to the function advance_slice_mut(), and thus has a shorter lifetime than the original slice, which is the root cause of the error you get – you are effectively trying to assign a slice which only lives until the end of the function to a memory location that lives longer than the function call.
This kind of implicit reborrow happens everytime you call a function that takes an argument by mutable reference, including in the implicit call to index_mut() in &mut s[1..]. Mutable references can't be copied, since this would create two mutable references to the same memory, and the Rust language designers decided that an implicit reborrow is the more ergonomic solution than moving mutable references by default. The reborrow does not happen for shared references, though, since they can be freely copied. This means that &s[1..] will have the same lifetime as the original scope, since it is perfectly fine for two overlapping immutable slices to coexist. This explains why your function definition works fine for immutable slices.
So how do we fix this problem? I believe that what you intend to do is perfectly safe – after reassigning the new slice to the old one, the old one is gone, so we don't have two concurrent mutable references to the same memory. To create a slice with the same lifetime as the original slice in safe code, we need to move the original slice out of the reference we got. We can do this by replacing it with an empty slice:
pub fn advance_slice_mut<T>(s: &mut &mut [T]) {
let slice = std::mem::replace(s, &mut []);
*s = &mut slice[1..];
}
Alternatively, we can also resort to unsafe code:
use std::slice::from_raw_parts_mut;
pub fn advance_slice_mut<T>(s: &mut &mut [T]) {
unsafe {
assert!(!s.is_empty());
*s = from_raw_parts_mut(s.as_mut_ptr().add(1), s.len() - 1);
}
}
This code fails the dreaded borrow checker (playground):
struct Data {
a: i32,
b: i32,
c: i32,
}
impl Data {
fn reference_to_a(&mut self) -> &i32 {
self.c = 1;
&self.a
}
fn get_b(&self) -> i32 {
self.b
}
}
fn main() {
let mut dat = Data{ a: 1, b: 2, c: 3 };
let aref = dat.reference_to_a();
println!("{}", dat.get_b());
}
Since non-lexical lifetimes were implemented, this is required to trigger the error:
fn main() {
let mut dat = Data { a: 1, b: 2, c: 3 };
let aref = dat.reference_to_a();
let b = dat.get_b();
println!("{:?}, {}", aref, b);
}
Error:
error[E0502]: cannot borrow `dat` as immutable because it is also borrowed as mutable
--> <anon>:19:20
|
18 | let aref = dat.reference_to_a();
| --- mutable borrow occurs here
19 | println!("{}", dat.get_b());
| ^^^ immutable borrow occurs here
20 | }
| - mutable borrow ends here
Why is this? I would have thought that the mutable borrow of dat is converted into an immutable one when reference_to_a() returns, because that function only returns an immutable reference. Is the borrow checker just not clever enough yet? Is this planned? Is there a way around it?
Lifetimes are separate from whether a reference is mutable or not. Working through the code:
fn reference_to_a(&mut self) -> &i32
Although the lifetimes have been elided, this is equivalent to:
fn reference_to_a<'a>(&'a mut self) -> &'a i32
i.e. the input and output lifetimes are the same. That's the only way to assign lifetimes to a function like this (unless it returned an &'static reference to global data), since you can't make up the output lifetime from nothing.
That means that if you keep the return value alive by saving it in a variable, you're keeping the &mut self alive too.
Another way of thinking about it is that the &i32 is a sub-borrow of &mut self, so is only valid until that expires.
As #aSpex points out, this is covered in the nomicon.
Why is this an error: While a more precise explanation was already given by #Chris some 2.5 years ago, you can read fn reference_to_a(&mut self) -> &i32 as a declaration that:
“I want to exclusively borrow self, then return a shared/immutable reference which lives as long as the original exclusive borrow” (source)
Apparently it can even prevent me from shooting myself in the foot.
Is the borrow checker just not clever enough yet? Is this planned?
There's still no way to express "I want to exclusively borrow self for the duration of the call, and return a shared reference with a separate lifetime". It is mentioned in the nomicon as #aSpex pointed out, and is listed among the Things Rust doesn’t let you do as of late 2018.
I couldn't find specific plans to tackle this, as previously other borrow checker improvements were deemed higher priority. The idea about allowing separate read/write "lifetime roles" (Ref2<'r, 'w>) was mentioned in the NLL RFC, but no-one has made it into an RFC of its own, as far as I can see.
Is there a way around it? Not really, but depending on the reason you needed this in the first place, other ways of structuring the code may be appropriate:
You can return a copy/clone instead of the reference
Sometimes you can split a fn(&mut self) -> &T into two, one taking &mut self and another returning &T, as suggested by #Chris here
As is often the case in Rust, rearranging your structs to be "data-oriented" rather than "object-oriented" can help
You can return a shared reference from the method: fn(&mut self) -> (&Self, &T) (from this answer)
You can make the fn take a shared &self reference and use interior mutability (i.e. define the parts of Self that need to be mutated as Cell<T> or RefCell<T>). This may feel like cheating, but it's actually appropriate, e.g. when the reason you need mutability as an implementation detail of a logically-immutable method. After all we're making a method take a &mut self not because it mutates parts of self, but to make it known to the caller so that it's possible to reason about which values can change in a complex program.
First of all, I'm not asking what's the difference between &mut and ref mut per se.
I'm asking because I thought:
let ref mut a = MyStruct
is the same as
let a = &mut MyStruct
Consider returning a trait object from a function. You can return a Box<Trait> or a &Trait. If you want to have mutable access to its methods, is it possible to return &mut Trait?
Given this example:
trait Hello {
fn hello(&mut self);
}
struct English;
struct Spanish;
impl Hello for English {
fn hello(&mut self) {
println!("Hello!");
}
}
impl Hello for Spanish {
fn hello(&mut self) {
println!("Hola!");
}
}
The method receives a mutable reference for demonstration purposes.
This won't compile:
fn make_hello<'a>() -> &'a mut Hello {
&mut English
}
nor this:
fn make_hello<'a>() -> &'a mut Hello {
let b = &mut English;
b
}
But this will compile and work:
fn make_hello<'a>() -> &'a mut Hello {
let ref mut b = English;
b
}
My theory
This example will work out of the box with immutable references (not necessary to assign it to a variable, just return &English) but not with mutable references. I think this is due to the rule that there can be only one mutable reference or as many immutable as you want.
In the case of immutable references, you are creating an object and borrowing it as a return expression; its reference won't die because it's being borrowed.
In the case of mutable references, if you try to create an object and borrow it mutably as a return expression you have two mutable references (the created object and its mutable reference). Since you cannot have two mutable references to the same object it won't perform the second, hence the variable won't live long enough. I think that when you write let mut ref b = English and return b you are moving the mutable reference because it was captured by a pattern.
All of the above is a poor attempt to explain to myself why it works, but I don't have the fundamentals to prove it.
Why does this happen?
I've also cross-posted this question to Reddit.
This is a bug. My original analysis below completely ignored the fact that it was returning a mutable reference. The bits about promotion only make sense in the context of immutable values.
This is allowable due to a nuance of the rules governing temporaries (emphasis mine):
When using an rvalue in most lvalue contexts, a temporary unnamed lvalue is created and used instead, if not promoted to 'static.
The reference continues:
Promotion of an rvalue expression to a 'static slot occurs when the expression could be written in a constant, borrowed, and dereferencing that borrow where the expression was the originally written, without changing the runtime behavior. That is, the promoted expression can be evaluated at compile-time and the resulting value does not contain interior mutability or destructors (these properties are determined based on the value where possible, e.g. &None always has the type &'static Option<_>, as it contains nothing disallowed).
Your third case can be rewritten as this to "prove" that the 'static promotion is occurring:
fn make_hello_3<'a>() -> &'a mut Hello {
let ref mut b = English;
let c: &'static mut Hello = b;
c
}
As for why ref mut allows this and &mut doesn't, my best guess is that the 'static promotion is on a best-effort basis and &mut just isn't caught by whatever checks are present. You could probably look for or file an issue describing the situation.