The following minimal example does not compile:
fn main() {
let mut v = Vec::new();
foo(&mut v);
}
fn foo(v_ref: &mut Vec<u8>) {
for v in v_ref{
println!("{}", v);
}
for v in v_ref{
println!("{}", v);
}
}
The compiler suggests to modify the first for loop to this, which does compile.
for v in &mut *v_ref{
The reasoning given by the compiler is:
move occurs because v_ref has type &mut Vec<u8>, which does not implement the Copy trait
v_ref moved due to this implicit call to .into_iter()
Question 1: Why is this necessary? It's already a reference, shouldn't the for loop give the reference back after we're done with it?
Question 2: Why does the fix work? As I understand it, the new mutable reference should invalidate the old one.
Why is this necessary? It's already a reference, shouldn't the for loop give the reference back after we're done with it?
for loops desugar into (roughly):
{
let iterator = IntoIterator::into_iter(iter);
while let Some(v) = Iterator::next(&mut iterator) { ... }
}
Notice the IntoIterator::into_iter(). As explained in Do mutable references have move semantics?, mutable references, as opposed to shared references, are not Copy and thus once you move them you cannot use them anymore. Usually, the compiler inserts a reborrow, &mut *reference that creates a new reference from the existing one and allows the existing one to still be used (after the new is no longer used), but this only happens when it is known without inference that the type is a mutable reference, and here we need inference to determine that.
Why does the fix work? As I understand it, the new mutable reference should invalidate the old one.
You can think of references as a stack; each reference created from existing one pushes an item on the stack, and when we finished using it we pop it. &mut *reference creates a new reference, whose lifetime can be shorter than the original reference and allows the original reference to be used again after the end of the lifetime of the new reference.
Related
In attempting to refactor a Rust application that was working fine, I tried to separate out the contents of a loop into a new function. However, in this newly refactored out function, I needed to pass an argument that had to be mutable, and passed by reference. Suddenly the code that absolutely worked inline, broke just because of the mutable reference passing.
My question is: can someone please explain why this does not work with such a "simple" change? (i.e. refactoring out a new function of otherwise unchanged code)
I have a minimal demo of the problem, along with a couple of working comparisons below. Here is the error from that code:
error[E0499]: cannot borrow `str_to_int` as mutable more than once at a time
--> src/main.rs:30:22
|
30 | get_key(key, &mut str_to_int);
| ^^^^^^^^^^^^^^^ `str_to_int` was mutably borrowed here in the previous iteration of the loop
The sample code:
use std::collections::BTreeMap;
fn get_int (
key: u32,
values: &mut BTreeMap<u32, u32>,
) -> &u32 {
values.entry(key).or_insert_with(|| { 1 })
}
fn get_key<'a> (
key: &'a str,
values: &'a mut BTreeMap<&'a str, u32>,
) -> &'a u32 {
values.entry(key).or_insert_with(|| { 1 })
}
fn main() {
let mut int_to_int = BTreeMap::new();
for key in vec![1,2] {
get_int(key, &mut int_to_int);
}
let mut str_to_int_inline = BTreeMap::new();
for key in vec!["a","b"] {
str_to_int_inline.entry(key).or_insert_with(|| { 1 });
}
let mut str_to_int = BTreeMap::new();
for key in vec!["a","b"] {
get_key(key, &mut str_to_int);
}
}
Note that the first loop (int_to_int) is identical to the third loop (str_to_int) except for the data type of the key -- in that the key was not a reference, so no lifetime was required to be specified. And the second loop (str_to_int_inline) is identical to the third loop (str_to_int) except the behavior is inline instead of in a separate function.
There are many related questions and blogs on this topic, but they all seem more specifically focused on particular versions of this question, and I want to know the more generic explanation (to my current understanding). If the answer is already just to understand one of these links better, I will gladly mark this question as a duplicate.
Related questions:
How to fix ".. was mutably borrowed here in the previous iteration of the loop" in Rust?
https://users.rust-lang.org/t/mutable-borrow-starts-here-in-previous-iteration-of-loop/26145
https://github.com/rust-lang/rust/issues/47680#issuecomment-363131420
Why does linking lifetimes matter only with mutable references?
Something that I read also led me to https://github.com/rust-lang/polonius which also seemed like maybe it could make this work, in the future -- any thoughts?
Your problem is very simple: you are specifying the lifetimes for get_key() incorrectly.
The correct (and working) version is:
fn get_key<'a, 'b>(key: &'a str, values: &'b mut BTreeMap<&'a str, u32>) -> &'b u32 {
values.entry(key).or_insert_with(|| 1)
}
Maybe you can already guess what's going on.
Since you used 'a for both the HashMap itself and its keys, it means you were required to borrow the HashMap for as long as the keys' lifetime - 'static. This means two things:
You need a &'static mut HashMap, which you don't have.
You're borrowing the HashMap for 'static in the first iteration of the loop, then borrow it again in the next, while it is still borrowed (because it is borrowed for 'static). This error hides the first error, somewhat erroneously, and is the only error the compiler emits.
In general, using a lifetime twice with a mutable reference is almost always wrong (shared references are more tolerant, as they're covariant over their type).
This question already has an answer here:
Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in?
(1 answer)
Closed 3 years ago.
I'm trying to write a function that finds returns a mutable reference to an existing element in a Vec, or inserts it if it doesn't exist and returns a mutable reference to the new element.
I've tried a couple of times, but the borrow checker isn't convinced. I've simplified the code I was trying to write to the example below, which gives the same errors.
fn mut_find_or_insert<T: PartialEq>(vec: &mut Vec<T>, val: T) -> &mut T {
if let Some(u) = vec.iter_mut().find(|u| **u == val) {
u
} else {
vec.push(val);
vec.last_mut().unwrap()
}
}
Playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=cb12c38bcf3682b15a247d14aab48b6b
Rust gives me the following compiler error (full message through the playground link):
error[E0499]: cannot borrow `*vec` as mutable more than once at a time
This seems to be something that should be possible to implement in rust, however it's not clear to me how I reimplement this to avoid borrow checker errors.
The reason this doesn't work as written is because of a limitation in the current borrow checker. This is very similar to NLL case #3, in which the compiler borrows somewhat overzealously for an entire match statement when the borrow is only used in one of the branches. With the experimental "Polonius" borrow checker (available on the nightly compiler with the -Z polonius flag), your code is accepted as-is.
Working in the stable compiler, it's probably a good idea to redesign your data structures as Sébastien Renauld's answer also suggests, but if you need to make this work with a Vec, you can work around it by briefly using an index to end the borrow:
fn mut_find_or_insert<T: PartialEq>(vec: &mut Vec<T>, val: T) -> &mut T {
if let Some(i) = vec.iter().position(|each| *each == val) {
&mut vec[i]
} else {
vec.push(val);
vec.last_mut().unwrap()
}
}
This works because the result of calling position is not a reference, so the borrow of vec is not held during the if let.
This is similar to the following questions, which manage to find the same limitation using early return from a loop:
Double mutable borrow error in a loop happens even with NLL on
"Variable does not live long enough" when returning a Result containing a reference but it does live long enough
Vec is an unordered, not very structured type. It has no way to look up the exact position of an item within it; the closest the default functions get to is contains(), which only tells you if the item is contained.
Furthermore, due to the fact that a Vec is not a Set, the behavior of "find the item or append and return " is undefined - "find the item", if there is a duplicate, needs to be defined further.
To solve this problem without changing to the correct type (HashSet is the type you really want for this. Note the existence of get_or_insert(), which is literally what you are after. It pays to use the proper structure for the job, rather than to try to make everything fit a Vec), we're going to have to build it ourselves. Keeping to your signature, it looks like this (Playground):
trait VecSeekOrAppend<T:PartialEq>:Sized {
fn get_or_insert(&mut self, item: T) -> &mut T;
}
impl<T> VecSeekOrAppend<T> for Vec<T>
where T: PartialEq + Clone {
fn get_or_insert(&mut self, item: T) -> &mut T {
if !self.contains(&item) {
self.push(item.clone());
}
for i in self.iter_mut() {
if i == &mut item {
return i;
}
}
unreachable!();
}
}
The reason your initial version does not work is due to the returned lifetime requirement; all methods returning a reference from a Vec require a lifetime validity for the duration of use. By returning such a &mut reference, if you attempt to do it in one go, the mutation of the Vec<_> will happen while there is already a mutable borrow of it.
Splitting the cycle in two, and performing the insertion (without keeping a reference) to then find the eference, allows us to sidestep this problem. Another way to perform this is to store items by a serializable or hashable identifier (the exact way HashMap and HashSet work) in order to innately provide this layer of indirection.
There is a rust feature in the works to ease some of this pain (non-lexical lifetimes), but, as you can see from the github issue, it's not in the near future.
I understand you're not allowed to create two mutable references to an object at once in Rust. I don't entirely understand why the following code works:
fn main() {
let mut string = String::from("test");
let mutable_reference: &mut String = &mut string;
mutable_reference.push_str(" test");
// as I understand it, this creates a new mutable reference (2nd?)
test(&mut *mutable_reference);
println!("{}", mutable_reference);
}
fn test(s: &mut String) {
s.push_str(" test");
}
The rule
There shall only be one usable mutable reference to a particular value at any point in time.
This is NOT a spatial exclusion (there CAN be multiple references to the same piece) but a temporal exclusion.
The mechanism
In order to enforce this, &mut T is NOT Copy; therefore calling:
test(mutable_reference);
should move the reference into test.
Actually doing this would make it unusable later on and not be very ergonomic, so the Rust compiler inserts an automatic reborrowing, much like you did yourself:
test(&mut *mutable_reference);
You can force the move if you wanted to:
test({ let x = mutable_reference; x });
The effect
Re-borrowing is, in essence, just borrowing:
mutable_reference is borrowed for as long as the unnamed temporary mutable reference exists (or anything that borrows from it),
the unnamed temporary mutable reference is moved into test,
at the of expression, the unnamed temporary mutable reference is destroyed, and therefore the borrow of mutable_reference ends.
Is there more than one mutable pointer somewhere in memory referring to the same location? Yes.
Is there more than one mutable pointer in the code to the same location which is usable? No.
Re-borrowing a mutable pointer locks out the one you're re-borrowing from.
I analyzed the differences in MIR between test(mutable_reference) and test(&mut *mutable_reference). It appears that the latter only introduces an additional level of indirection:
When you are using an extra dereference in the function call, it doesn't create a mutable reference that holds; in other words, it doesn't cause any actual difference outside the function call.
I understand you're not allowed to create two mutable references to an object at once in Rust. I don't entirely understand why the following code works:
fn main() {
let mut string = String::from("test");
let mutable_reference: &mut String = &mut string;
mutable_reference.push_str(" test");
// as I understand it, this creates a new mutable reference (2nd?)
test(&mut *mutable_reference);
println!("{}", mutable_reference);
}
fn test(s: &mut String) {
s.push_str(" test");
}
The rule
There shall only be one usable mutable reference to a particular value at any point in time.
This is NOT a spatial exclusion (there CAN be multiple references to the same piece) but a temporal exclusion.
The mechanism
In order to enforce this, &mut T is NOT Copy; therefore calling:
test(mutable_reference);
should move the reference into test.
Actually doing this would make it unusable later on and not be very ergonomic, so the Rust compiler inserts an automatic reborrowing, much like you did yourself:
test(&mut *mutable_reference);
You can force the move if you wanted to:
test({ let x = mutable_reference; x });
The effect
Re-borrowing is, in essence, just borrowing:
mutable_reference is borrowed for as long as the unnamed temporary mutable reference exists (or anything that borrows from it),
the unnamed temporary mutable reference is moved into test,
at the of expression, the unnamed temporary mutable reference is destroyed, and therefore the borrow of mutable_reference ends.
Is there more than one mutable pointer somewhere in memory referring to the same location? Yes.
Is there more than one mutable pointer in the code to the same location which is usable? No.
Re-borrowing a mutable pointer locks out the one you're re-borrowing from.
I analyzed the differences in MIR between test(mutable_reference) and test(&mut *mutable_reference). It appears that the latter only introduces an additional level of indirection:
When you are using an extra dereference in the function call, it doesn't create a mutable reference that holds; in other words, it doesn't cause any actual difference outside the function call.
I suppose this question is about lifetimes in general, but I'm having difficulty with closures specifically because you can't write out their type.
This example is a bit contrived - I'm just starting to learn Rust, and this is something I've been hung up on.
This program won't compile:
fn main () {
let mut list: Vec<&Fn() -> i32> = Vec::new();
{
list.push(&|| 1);
}
}
Because:
src/main.rs:5:25: 5:24 error: borrowed value does not live long enough
src/main.rs:5 list.push(&|| 1);
^~~~
src/main.rs:2:50: 7:2 note: reference must be valid for the block suffix following statement 0 at 2:49...
src/main.rs:2 let mut list: Vec<&Fn() -> i32> = Vec::new();
src/main.rs:3
src/main.rs:4 {
src/main.rs:5 list.push(&move || 1);
src/main.rs:6 }
src/main.rs:7 }
src/main.rs:5:9: 5:26 note: ...but borrowed value is only valid for the statement at 5:8
src/main.rs:5 list.push(&|| 1);
^~~~~~~~~~~~~~~~~
src/main.rs:5:9: 5:26 help: consider using a `let` binding to increase its lifetime
src/main.rs:5 list.push(&|| 1);
^~~~~~~~~~~~~~~~~
What I gather from this error is that the closure's lifetime is limited to the
statement inside the block, but it needs to live for the entire body of main.
I know (or, I think) that passing the closure to push as a reference means that push is only borrowing the closure, and that ownership will be returned to the block. This code would work if I could just give the closure to push (i.e. if push took ownership of the closure), but since a closure isn't sized, I must pass it as a reference.
Is that right? How can I make this code work?
There are two things you are asking about:
specifying a typename for something that has no specifyable typename
letting a closure live longer than the block where it's defined.
The first issue is fixed by NOT specifying the typename, and letting rust's type inference do the work.
let mut list: Vec<_> = Vec::new();
The second issue is fixed by not trying to make the closure live longer, but by making it "by value" so you can move it. This enforces that your closure does not reference anything, but owns all the captured values.
for i in 0..10 {
list.push(move || i);
}
Now this gives us a new problem. If we add a different closure to the Vec, the types won't match. Therefore to achieve that, we need to box the closures.
fn main () {
let mut list: Vec<Box<Fn() -> i32>> = Vec::new();
for i in 0..10 {
list.push(Box::new(move|| i));
}
{
list.push(Box::new(move|| 42));
}
}
Borrows do not own the thing they point to. Your problem is that you're borrowing a temporary which is going to cease to exist right after it's borrowed because you haven't stored it anywhere. If it helps, consider that borrows don't borrow values, they borrow storage, and a temporary has only transient storage.
If you want a borrow to something to last for any given period, you must borrow from storage that will last at least that long. In this case, because you want to store the borrow in a Vec, this means that whatever storage you borrow from must outlive the Vec as well. Thus:
fn main () {
let closure;
let mut list: Vec<&Fn() -> i32> = Vec::new();
{
closure = || 1;
list.push(&closure);
}
}
Note that closure is defined before list is. In Rust, values are dropped in reverse lexical order at the end of their scope, so any variable defined after list will necessarily be dropped before it, thus leading to list containing invalid pointers.
If you want to push multiple closures, you will need a separate variable for each one.
To forestall a possible "my actual problem isn't this simple" addendum (:P): f you need to return list or in some way persist it beyond a single function call, note that there is no way to extend a borrow. In that case, what you need to do is change list to a vector of owned, boxed closures (i.e. Vec<Box<Fn() -> i32>>).