Mutable borrow issue while looping with reference pivot - rust

struct A {
next: Option<Box<A>>,
}
impl A {
fn grow(&mut self) {
self.next = Some(Box::new(A { next: None }));
}
}
fn main() {
let mut a = A{ next: Some(Box::new(A { next: None }))};
let mut p = &mut a;
// attempt to append to the list
loop {
match &mut p.next {
Some(n) => p = n,
None => {
p.grow();
break;
}
}
}
}
The code above is the simplified logic from a more complex data structure that is able to reproduce the borrow checker complaint:
error[E0499]: cannot borrow `*p` as mutable more than once at a time
--> t.rs:19:17
|
16 | match &mut p.next {
| ----------- first mutable borrow occurs here
...
19 | p.grow();
| ^
| |
| second mutable borrow occurs here
| first borrow later used here
error: aborting due to previous error
Why is p still thought to be mutably borrowed in the match case?
And, trying to move p.update() out of the loop doesn't help:
fn main() {
let mut a = A{ next: Some(Box::new(A { next: None }))};
let mut p = &mut a;
// attempt to append to the list
loop {
match &mut p.next {
Some(n) => p = n,
None => {
break;
}
}
}
p.grow();
}
I got the same error in this case. I know p = n is causing the problem because it compiles through without it, but why?

Here is my take on what's going on. To explain it, I simplified your code a little, without changing the idea:
fn main() {
let mut a = A{ next: Some(Box::new(A { next: None }))};
let mut p = &mut a;
let next = &mut p.next;
if let Some(n) = next {
p = n;
}
p.grow();
}
This code produces the same error as yours. What's more interesting, you cannot use p in any way at the end of main. Even println!("{:?}", p) produces the same error.
The compiler checks all the ways the code can execute and sees that at the point when p.grow() is called, p can either point to a, or to a.next.as_mut() (I mean inner value of a.next option struct). But p.grow() requires using one of this references.
If p == &mut a, and we call p.grow(), then the reference to a.next.as_mut() becomes invalid, thus error. If p == a.next.as_mut(), and we call p.grow(), we reference &mut a two times, as the first borrow cannot be dropped still.
If you don't assign p = n, there is no such issue, because there is only one valid reference, which the compiler can use when p.grow() is called.
In this example actually there 3 ways to prevent compiler error:
Remove p = n assignment.
Move p.grow() inside if let block, thus making sure that at the moment of the call p references the inner value.
Exiting main in case next is Some:
if let Some(n) = next {
p = n;
} else {
return;
}
p.grow();
The later works because compiler knows for sure that at the point of p.grow() call first borrow is not needed and can be dropped, and p can point only to the inner value.
The problem in the original question is same. Compiler must know for sure what p references at the point of the p.grow() call.
In the solution, provided by Lagerbaer, p can reference only one certain value when p.grow() is called (there is no reference to p.next at this time), thus it works:
let mut p = &mut a;
// attempt to append to the list
loop {
match p.next {
Some(ref mut n) => p = n,
None => {
p.grow();
break;
}
}
}

Why are you borrowing in the match expression? That's unnecessary. Just match on p.next. That in and of itself doesn't yet move. What moves is the type of pattern you're matching. So then instead of matching Some(n) we match Some(ref mut n).
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1859dc15cdc08ddec8cac3efb6d8fa31

Related

Why does the Rust compiler complain that I use a moved value when I've replaced it with a new value?

I am working on two singly linked lists, named longer and shorter. The length of the longer one is guaranteed to be no less than the shorter one.
I pair the lists element-wise and do something to each pair. If the longer list has more unpaired elements, process the rest of them:
struct List {
next: Option<Box<List>>,
}
fn drain_lists(mut shorter: Option<Box<List>>, mut longer: Option<Box<List>>) {
// Pair the elements in the two lists.
while let (Some(node1), Some(node2)) = (shorter, longer) {
// Actual work elided.
shorter = node1.next;
longer = node2.next;
}
// Process the rest in the longer list.
while let Some(node) = longer {
// Actual work elided.
longer = node.next;
}
}
However, the compiler complains on the second while loop that
error[E0382]: use of moved value
--> src/lib.rs:13:20
|
5 | fn drain_lists(mut shorter: Option<Box<List>>, mut longer: Option<Box<List>>) {
| ---------- move occurs because `longer` has type `std::option::Option<std::boxed::Box<List>>`, which does not implement the `Copy` trait
6 | // Pair the elements in the two lists.
7 | while let (Some(node1), Some(node2)) = (shorter, longer) {
| ------ value moved here
...
13 | while let Some(node) = longer {
| ^^^^ value used here after move
However, I do set a new value for shorter and longer at the end of the loop, so that I will never use a moved value of them.
How should I cater to the compiler?
I think that the problem is caused by the tuple temporary in the first loop. Creating a tuple moves its components into the new tuple, and that happens even when the subsequent pattern matching fails.
First, let me write a simpler version of your code. This compiles fine:
struct Foo(i32);
fn main() {
let mut longer = Foo(0);
while let Foo(x) = longer {
longer = Foo(x + 1);
}
println!("{:?}", longer.0);
}
But if I add a temporary to the while let then I'll trigger a compiler error similar to yours:
fn fwd<T>(t: T) -> T { t }
struct Foo(i32);
fn main() {
let mut longer = Foo(0);
while let Foo(x) = fwd(longer) {
longer = Foo(x + 1);
}
println!("{:?}", longer.0);
// Error: ^ borrow of moved value: `longer`
}
The solution is to add a local variable with the value to be destructured, instead of relying on a temporary. In your code:
struct List {
next: Option<Box<List>>
}
fn drain_lists(shorter: Option<Box<List>>,
longer: Option<Box<List>>) {
// Pair the elements in the two lists.
let mut twolists = (shorter, longer);
while let (Some(node1), Some(node2)) = twolists {
// Actual work elided.
twolists = (node1.next, node2.next);
}
// Process the rest in the longer list.
let (_, mut longer) = twolists;
while let Some(node) = longer {
// Actual work elided.
longer = node.next;
}
}
Other than getting rid of the tuple (shown by others), you can capture a mutable reference to the nodes:
while let (&mut Some(ref mut node1), &mut Some(ref mut node2)) = (&mut shorter, &mut longer) {
shorter = node1.next.take();
longer = node2.next.take();
}
The use of take() enables this to work: shorter = node1.next would complain of moving a field out of a reference, which is not allowed (it would leave the node in an undefined state). But takeing it is ok because it leaves None in the next field.
Looks like the destructuring on line 7 moves the value even when the block afterwards is not evaluated. (Edit: as #Sven Marnach pointed out in the comments, a temporary tuple gets created here which causes the move)
I've uglyfied your code to prove that point :)
struct List {
next: Option<Box<List>>
}
fn drain_lists(mut shorter: Option<Box<List>>,
mut longer: Option<Box<List>>) {
// Pair the elements in the two lists.
match(shorter, longer) {
(Some(node1), Some(node2)) => {
shorter = node1.next;
longer = node2.next;
},
(_, _) => return // without this you get the error
}
// Process the rest in the longer list.
while let Some(node) = longer {
// Actual work elided.
longer = node.next;
}
}
When I added the return for the default case, the code compiled.
One solution is to avoid the tuple and consequently the move of longer into the tuple.
fn actual_work(node1: &Box<List>, node2: &Box<List>) {
// Actual work elided
}
fn drain_lists(mut shorter: Option<Box<List>>, mut longer: Option<Box<List>>) {
while let Some(node1) = shorter {
if let Some(node2) = longer.as_ref() {
actual_work(&node1, node2);
}
shorter = node1.next;
longer = longer.map_or(None, move |l| {
l.next
});
}
// Process the rest in the longer list.
while let Some(node) = longer {
// Actual work elided.
longer = node.next;
}
}

How to process every value in a HashMap and optionally reject some?

I would like to process the values from a HashMap one by one, while maybe removing some of them.
For example, I would like to do an equivalent of:
use std::collections::HashMap;
fn example() {
let mut to_process = HashMap::new();
to_process.insert(1, true);
loop {
// get an arbitrary element
let ans = to_process.iter().next().clone(); // get an item from the hash
match ans {
Some((k, v)) => {
if condition(&k,&v) {
to_process.remove(&k);
}
}
None => break, // work finished
}
}
}
But this fails to compile:
error[E0502]: cannot borrow `to_process` as mutable because it is also borrowed as immutable
--> src/lib.rs:12:17
|
9 | let ans = to_process.iter().next().clone();
| ---------- immutable borrow occurs here
...
12 | to_process.remove(&k);
| ^^^^^^^^^^^------^^^^
| | |
| | immutable borrow later used by call
| mutable borrow occurs here
I know I really would need https://github.com/rust-lang/rust/issues/27804 (which is for HashSet but for HashMap would be the same)
and I cannot implement the provided solutions without having a non-mut and mutable reference still or using unsafe.
Is there a simple way I am missing?
Note If you need to alter keys or add kvps to the HashMap during processing, see #edwardw's answer. Otherwise ...
Use HashMap::retain. You can change your process function to return a bool indicating whether to keep that key value pair. For example
let mut to_process: HashMap<u32, String> = HashMap::new();
to_process.insert(1, "ok".to_string());
to_process.insert(2, "bad".to_string());
to_process.retain(process);
fn process(k: &u32, v: &mut String) -> bool {
// do stuff with k and v
v == "ok"
}
This looks like an awfully good fit for Iterator::filter_map:
The closure must return an Option<T>. filter_map creates an iterator which calls this closure on each element. If the closure returns Some(element), then that element is returned. If the closure returns None, it will try again, and call the closure on the next element, seeing if it will return Some.
The following process_and_maybe_add is very simple, but you get the idea:
use std::collections::HashMap;
fn main() {
let mut data = HashMap::new();
data.insert(1, "a");
data.insert(2, "b");
data.insert(3, "c");
let processed = data
.into_iter()
.filter_map(process_and_maybe_add)
.collect::<HashMap<_, _>>();
dbg!(processed);
}
fn process_and_maybe_add((k, v): (u32, &str)) -> Option<(u32, String)> {
if k % 2 != 0 {
Some((k + 100, v.to_owned() + v))
} else {
None
}
}

Iterating through a recursive structure using mutable references and returning the last valid reference

I'm trying to recurse down a structure of nodes, modifying them, and then returning the last Node that I get to. I solved the problems with mutable references in the loop using an example in the non-lexical lifetimes RFC. If I try to return the mutable reference to the last Node, I get a use of moved value error:
#[derive(Debug)]
struct Node {
children: Vec<Node>,
}
impl Node {
fn new(children: Vec<Self>) -> Self {
Self { children }
}
fn get_last(&mut self) -> Option<&mut Node> {
self.children.last_mut()
}
}
fn main() {
let mut root = Node::new(vec![Node::new(vec![])]);
let current = &mut root;
println!("Final: {:?}", get_last(current));
}
fn get_last(mut current: &mut Node) -> &mut Node {
loop {
let temp = current;
println!("{:?}", temp);
match temp.get_last() {
Some(child) => { current = child },
None => break,
}
}
current
}
Gives this error
error[E0382]: use of moved value: `*current`
--> test.rs:51:5
|
40 | let temp = current;
| ---- value moved here
...
51 | current
| ^^^^^^^ value used here after move
|
= note: move occurs because `current` has type `&mut Node`, which does not implement the `Copy` trait
If I return the temporary value instead of breaking, I get the error cannot borrow as mutable more than once.
fn get_last(mut current: &mut Node) -> &mut Node {
loop {
let temp = current;
println!("{:?}", temp);
match temp.get_last() {
Some(child) => { current = child },
None => return temp,
}
}
}
error[E0499]: cannot borrow `*temp` as mutable more than once at a time
--> test.rs:47:28
|
43 | match temp.get_last() {
| ---- first mutable borrow occurs here
...
47 | None => return temp,
| ^^^^ second mutable borrow occurs here
48 | }
49 | }
| - first borrow ends here
How can I iterate through the structure with mutable references and return the last Node? I've searched, but I haven't found any solutions for this specific problem.
I can't use Obtaining a mutable reference by iterating a recursive structure because it gives me a borrowing more than once error:
fn get_last(mut current: &mut Node) -> &mut Node {
loop {
let temp = current;
println!("{:?}", temp);
match temp.get_last() {
Some(child) => current = child,
None => current = temp,
}
}
current
}
This is indeed different from Cannot obtain a mutable reference when iterating a recursive structure: cannot borrow as mutable more than once at a time. If we look at the answer there, modified a bit, we can see that it matches on a value and is able to return the value that was matched on in the terminal case. That is, the return value is an Option:
fn back(&mut self) -> &mut Option<Box<Node>> {
let mut anchor = &mut self.root;
loop {
match {anchor} {
&mut Some(ref mut node) => anchor = &mut node.next,
other => return other, // transferred ownership to here
}
}
}
Your case is complicated by two aspects:
The lack of non-lexical lifetimes.
The fact that you want to take a mutable reference and "give it up" in one case (there are children) and not in the other (no children). This is conceptually the same as this:
fn maybe_identity<T>(_: T) -> Option<T> { None }
fn main() {
let name = String::from("vivian");
match maybe_identity(name) {
Some(x) => println!("{}", x),
None => println!("{}", name),
}
}
The compiler cannot tell that the None case could (very theoretically) continue to use name.
The straight-forward solution is to encode this "get it back" action explicitly. We create an enum that returns the &mut self in the case of no children, a helper method that returns that enum, and rewrite the primary method to use the helper:
enum LastOrNot<'a> {
Last(&'a mut Node),
NotLast(&'a mut Node),
}
impl Node {
fn get_last_or_self(&mut self) -> LastOrNot<'_> {
match self.children.is_empty() {
false => LastOrNot::Last(self.children.last_mut().unwrap()),
true => LastOrNot::NotLast(self),
}
}
fn get_last(mut current: &mut Node) -> &mut Node {
loop {
match { current }.get_last_or_self() {
LastOrNot::Last(child) => current = child,
LastOrNot::NotLast(end) => return end,
}
}
}
}
Note that we are using all of the techniques exposed in both Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in? and Cannot obtain a mutable reference when iterating a recursive structure: cannot borrow as mutable more than once at a time.
With an in-progress reimplementation of NLL, we can simplify get_last_or_self a bit to avoid the boolean:
fn get_last_or_self(&mut self) -> LastOrNot<'_> {
match self.children.last_mut() {
Some(l) => LastOrNot::Last(l),
None => LastOrNot::NotLast(self),
}
}
The final version of Polonius should allow reducing the entire problem to a very simple form:
fn get_last(mut current: &mut Node) -> &mut Node {
while let Some(child) = current.get_last() {
current = child;
}
current
}
See also:
Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in?
Cannot obtain a mutable reference when iterating a recursive structure: cannot borrow as mutable more than once at a time

Creating a simple linked list

I'm having difficulty getting the borrow checker working for a simple iterative linked list builder.
fn main() {
let v = vec![1,5,3,8,12,56,1230,2,1];
let nodes = Vec::<Node>::with_capacity(v.len());
let mut root: Option<&mut Box<Node>> = None;
let mut prev: &Option<&mut Box<Node>> = &None;
for i in v {
let curr = Some(&mut Box::new(Node { value: i, next: None }));
match *prev {
Some(ref mut p) => {
p.next = curr;
prev = &mut p.next;
},
None => {
root = curr;
prev = &mut root;
}
}
}
}
struct Node<'a> {
value: i32,
next: Option<&'a mut Box<Node<'a>>>,
}
The errors I'm receiving when I try to compile:
linked_list.rs:8:30: 8:69 error: borrowed value does not live long enough
linked_list.rs:8 let curr = Some(&mut Box::new(Node { value: i, next: None }));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: in expansion of for loop expansion
linked_list.rs:7:5: 19:6 note: expansion site
linked_list.rs:4:49: 20:2 note: reference must be valid for the block suffix following statement 2 at 4:48...
linked_list.rs:4 let mut root: Option<&mut Box<Node>> = None;
linked_list.rs:5 let mut prev: &Option<&mut Box<Node>> = &None;
linked_list.rs:6
linked_list.rs:7 for i in v {
linked_list.rs:8 let curr = Some(&mut Box::new(Node { value: i, next: None }));
linked_list.rs:9 match *prev {
...
linked_list.rs:8:9: 8:71 note: ...but borrowed value is only valid for the statement at 8:8
linked_list.rs:8 let curr = Some(&mut Box::new(Node { value: i, next: None }));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
linked_list.rs:8:9: 8:71 help: consider using a `let` binding to increase its lifetime
linked_list.rs:8 let curr = Some(&mut Box::new(Node { value: i, next: None }));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
linked_list.rs:10:18: 10:27 error: cannot borrow immutable anonymous field `(prev:core::option::Some).0` as mutable
linked_list.rs:10 Some(ref mut p) => {
^~~~~~~~~
note: in expansion of for loop expansion
linked_list.rs:7:5: 19:6 note: expansion site
linked_list.rs:15:17: 15:28 error: cannot assign to `root` because it is borrowed
linked_list.rs:15 root = curr;
^~~~~~~~~~~
note: in expansion of for loop expansion
linked_list.rs:7:5: 19:6 note: expansion site
linked_list.rs:16:29: 16:33 note: borrow of `root` occurs here
linked_list.rs:16 prev = &mut root;
^~~~
note: in expansion of for loop expansion
linked_list.rs:7:5: 19:6 note: expansion site
linked_list.rs:16:29: 16:33 error: cannot borrow `root` as mutable more than once at a time
linked_list.rs:16 prev = &mut root;
^~~~
note: in expansion of for loop expansion
linked_list.rs:7:5: 19:6 note: expansion site
linked_list.rs:16:29: 16:33 note: previous borrow of `root` occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `root` until the borrow ends
linked_list.rs:16 prev = &mut root;
^~~~
note: in expansion of for loop expansion
linked_list.rs:7:5: 19:6 note: expansion site
linked_list.rs:20:2: 20:2 note: previous borrow ends here
linked_list.rs:1 fn main() {
...
linked_list.rs:20 }
^
error: aborting due to 4 previous errors
What I'm trying to go for is fairly simple. We iterate through a Vec, creating a new node on each iteration. If prev is None this must be the start, so we make the root variable take ownership of that first node. If it's not, we update the previous node's next value to point to this node.
I'm new to Rust so I'm not sure where I'm going wrong. My interpretation is that the borrow checker isn't handling this well. It can't infer that the None branch in the match, containing the 'root' assignment, will only ever be called once, causing the two errors about root being borrowed twice. Am I correct?
Is this approach possible in Rust? Is there a more idiomatic way to do this sort of thing?
(A recursive approach is probably much easier but I'd like to complete an iterative one as a learning exercise.)
First of all, you should probably make sure you've read and understood the Rust Book chapters on Ownership and References and Borrowing. Your immediate problem is that you're borrowing things that aren't owned by anything, and will thus just disappear. You also have other problems like trying to mutate through an immutable pointer.
Let's get something that does, at least, work:
fn main() {
let v = vec![1,5,3,8,12,56,1230,2,1];
let mut root: Option<Box<Node>> = None;
for i in v.into_iter().rev() {
root = Some(Box::new(Node { value: i, next: root }));
}
println!("root: {}",
root.map(|n| n.to_string()).unwrap_or(String::from("None")));
}
struct Node {
value: i32,
next: Option<Box<Node>>,
}
impl std::fmt::Display for Node {
fn fmt(&self, fmt: &mut std::fmt::Formatter) -> Result<(), std::fmt::Error> {
let mut cur = Some(self);
let mut first = true;
try!(write!(fmt, "["));
while let Some(node) = cur {
if !first { try!(write!(fmt, ", ")); }
first = false;
try!(write!(fmt, "{}", node.value));
cur = node.next.as_ref().map(|n| &**n);
}
try!(write!(fmt, "]"));
Ok(())
}
}
This constructs a list and shows how you can iteratively display it. Note the complete lack of borrows in the construction code.
I have cheated somewhat, in that I've iterated the vector backwards to construct the list.
The problem with the original code is that, even if you strip out everything that isn't necessary, down to something like this:
let v = vec![1,5,3,8,12,56,1230,2,1];
let mut v = v.into_iter();
let mut root: Option<Box<Node>> = None;
if let Some(i) = v.next() {
root = Some(Box::new(Node { value: i, next: None }));
let mut prev: &mut Box<Node> = root.as_mut().unwrap();
for i in v {
let curr = Some(Box::new(Node { value: i, next: None }));
prev.next = curr;
prev = prev.next.as_mut().unwrap();
}
}
You still end up in a situation where the compiler sees you mutating a thing you've borrowed by a second path. It's not quite smart enough to realise that re-assigning prev doesn't actually create any aliases. On the other hand, if you break the loop into an equivalent recursion:
if let Some(i) = v.next() {
root = Some(Box::new(Node { value: i, next: None }));
fn step<It>(prev: &mut Box<Node>, mut v: It) where It: Iterator<Item=i32> {
if let Some(i) = v.next() {
let curr = Some(Box::new(Node { value: i, next: None }));
prev.next = curr;
step(prev.next.as_mut().unwrap(), v)
}
}
step(root.as_mut().unwrap(), v);
}
Then it's totally fine with it. Sadly, even with optimisations turned on, Rust doesn't perform tail call elimination in this case. So between borrow checker limitations and a lack of guaranteed tail call elimination, this design might be impossible to do in safe code.
I've run into this problem myself; loops and &mut pointers don't always play nicely with one another. You can work around this by switching to RefCell, with its associated runtime cost, although this then complicates iterating over such a list in a loop. Another alternative is to use usizes instead of pointers, and have all the nodes allocated into a Vec somewhere, although that introduces bounds checking overhead.
Failing all that, there's unsafe code, which lets you write more or less exactly what you would write in another language like C or C++, but without Rust's usual safety guarantees.
At the end of the day, writing data structures that are not just wrappers around an existing data structure in safe Rust without overhead is borderline impossible. It's why the fundamental data structures in Rust are all written using some amount of unsafe code.

Unable to borrow an iterator as mutable more than once at a time

I'm writing a simple tokenizer in Rust but I'm having trouble. I've simplified the code a bit for the sake of this question:
use std::iter::Peekable;
use std::str::Chars;
struct Example<'a> {
it: Peekable<Chars<'a>>,
}
impl<'a> Example<'a> {
fn tokenize_string(&mut self) {
loop {
match self.it.peek() {
None => break,
Some(_x) => self.it.next(),
};
}
}
}
The error I'm getting is:
error[E0499]: cannot borrow `self.it` as mutable more than once at a time
--> src/main.rs:13:29
|
11 | match self.it.peek() {
| ------- first mutable borrow occurs here
12 | None => break,
13 | Some(_x) => self.it.next(),
| ^^^^^^^ second mutable borrow occurs here
14 | };
| - first borrow ends here
I've been able to work around this by creating a copy of the iterator and calling peek() on that:
fn tokenize_string(&mut self) {
loop {
let mut iterator = self.it.clone();
match iterator.peek() {
None => break,
Some(_x) => self.it.next(),
};
}
}
Is this the best way to do this? It seems a little hack-ish.
Since you're working with str::chars(), and char is Copy, you can dereference to get a char instead of &char. :
fn tokenize_string(&mut self) {
loop {
let r = self.it.peek().cloned();
let n = match r {
Some(_) => self.it.next(),
None => break,
};
// whatever
}
}
If you just want to check if the iterator has returned something, use is_some():
let r = self.it.peek().is_some();
if r { ... } else { ... }
In general, however, I'm not sure if it is possible exactly in this manner without non-lexical lifetimes. You will need to put the code which checks iterator state and the code which works with the iterator based on the state with scopes, something like this:
let r = {
// work with self.it
};
if r { ... } else { ... }
Here any references into self must not escape the lexical block in r, so there is no direct match on a value which contains references into self. There's further examples of working around this in Rust borrow of a HashMap lasts beyond the scope it's in?.

Resources