Consider the following (contrived) way to increment x by 9.
fn main() {
let mut x = 0;
let mut f = || {
x += 4;
};
let _g = || {
f();
x += 5;
};
}
error[E0499]: cannot borrow `x` as mutable more than once at a time
--> x.rs:6:12
|
3 | let mut f = || {
| -- first mutable borrow occurs here
4 | x += 4;
| - first borrow occurs due to use of `x` in closure
5 | };
6 | let _g = || {
| ^^ second mutable borrow occurs here
7 | f();
| - first borrow later captured here by closure
8 | x += 5;
| - second borrow occurs due to use of `x` in closure
error: aborting due to previous error
For more information about this error, try `rustc --explain E0499`.
So, it does not work. How to make an algorithm like the above that would modify a variable in a closure and call another closure from it that also modifies the variable?
In traditional languages it's easy. What to do in Rust?
The accepted answer is the most idiomatic approach, but there is also an alternative that is useful in situations when the additional argument doesn't work, for example when you need to pass the closure to third-party code that will call it without arguments. In that case you can use a Cell, a form of interior mutability:
use std::cell::Cell;
fn main() {
let x = Cell::new(0);
let f = || {
x.set(x.get() + 4);
};
let g = || {
f();
x.set(x.get() + 5);
};
f();
g();
assert_eq!(x.get(), 13);
}
By design, closures have to enclose external objects that are used in it upon creation, in our case closures have to borrow the external x object. So as compiler explained to you, upon creation of the closure f it mutably borrows x, and you can't borrow it once again when you create closure g.
In order to compile it you can't enclose any external object that you want to change. Instead you can directly pass the objects as a closure's argument (arguably it's even more readable). This way you describe that a closure accepts some object of some type, but you don't yet use/pass any actual object. This object will be borrowed only when you call the closure.
fn main() {
let mut x = 0;
let f = |local_x: &mut i32| { // we don't enclose `x`, so no borrow yet
*local_x += 4;
};
let _g = |local_x: &mut i32| { // we don't enclose `x`, so no borrow yet
f(local_x);
*local_x += 5;
};
_g(&mut x); // finally we borrow `x`, and this borrow will later move to `f`,
// so no simultaneous borrowing.
println!("{}", x); // 9
}
To expand on Alex Larionov's answer: you should think of a closure as a callable structure, anything you capture is set as a field of the structure, then implicitly dereferenced inside the function body. The way those fields are used is also what determines whether the closure is Fn, FnMut or FnOnce, basically whether the method would take &self, &mut self or self if it were written longhand.
Here
fn main() {
let mut x = 0;
let mut f = || {
x += 4;
};
// ...
}
essentially translates to:
struct F<'a> { x: &'a mut u32 }
impl F<'_> {
fn call(&mut self) {
*self.x += 4
}
}
fn main() {
let mut x = 0;
let mut f = F { x: &mut x };
// ...
}
from this, you can see that as soon as f is created, x is mutably borrowed, with all that implies.
And with this partial desugaring, we can see essentially the same error: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e129f19f25dc61de8a6f42cdca1f67b5
If we use the relevant unstable features on nightly we can get just a bit closer. That is pretty much what rustc does for us under the hood.
Related
This question already has an answer here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed last month.
How do I move a vector reference into threads? The closest I get is the (minimized) code below. (I realize that the costly calculation still isn't parallel, as it is locked by the mutex, but one problem at a time.)
Base problem: I'm calculating values based on information saved in a vector. Then I'm storing the results as nodes per vector element. So vector in vector (but only one vector in the example code below). The calculation takes time so I would like to divide it into threads. The structure is big, so I don't want to copy it.
use std::sync::{Arc, Mutex};
use std::thread;
fn main() {
let n = Nodes::init();
n.calc();
println!("Result: nodes {:?}", n);
}
#[derive(Debug)]
struct Nodes {
nodes: Vec<Info>,
}
impl Nodes {
fn init() -> Self {
let mut n = Nodes { nodes: Vec::new() };
n.nodes.push(Info::init(1));
n.nodes.push(Info::init(2));
n
}
fn calc(&self) {
Nodes::calc_associative(&self.nodes);
}
fn calc_associative(nodes: &Vec<Info>) {
let mut handles = vec![];
let arc_nodes = Arc::new(nodes);
let counter = Arc::new(Mutex::new(0));
for _ in 0..2 {
let arc_nodes = Arc::clone(&arc_nodes);
let counter = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut idx = counter.lock().unwrap();
// costly calculation
arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
*idx += 1;
});
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
}
}
#[derive(Debug)]
struct Info {
length: u32,
}
impl Info {
fn init(length: u32) -> Self {
Info { length }
}
fn get_length(&self) -> u32 {
self.length
}
fn set_length(&mut self, x: u32) {
self.length = x;
}
}
The compiler complains that life time of the reference isn't fulfilled, but isn't that what Arc::clone() should do? Then Arc require a deref, but maybe there are better solutions before starting to dig into that...?
Compiling threads v0.1.0 (/home/freefox/proj/threads)
error[E0596]: cannot borrow data in an `Arc` as mutable
--> src/main.rs:37:17
|
37 | arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
| ^^^^^^^^^ cannot borrow as mutable
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Arc<&Vec<Info>>`
error[E0521]: borrowed data escapes outside of associated function
--> src/main.rs:34:26
|
25 | fn calc_associative(nodes: &Vec<Info>) {
| ----- - let's call the lifetime of this reference `'1`
| |
| `nodes` is a reference that is only valid in the associated function body
...
34 | let handle = thread::spawn(move || {
| __________________________^
35 | | let mut idx = counter.lock().unwrap();
36 | | // costly calculation
37 | | arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
38 | | *idx += 1;
39 | | });
| | ^
| | |
| |______________`nodes` escapes the associated function body here
| argument requires that `'1` must outlive `'static`
Some errors have detailed explanations: E0521, E0596.
For more information about an error, try `rustc --explain E0521`.
error: could not compile `threads` due to 2 previous errors
You wrap a reference with Arc. Now the type is Arc<&Vec<Info>>. There is still a reference here, so the variable could still be destroyed before the thread return and we have a dangling reference.
Instead, you should take a &Arc<Vec<Info>>, and on the construction of the Vec wrap it in Arc, or take &[Info] and clone it (let arc_nodes = Arc::new(nodes.to_vec());). You also need a mutex along the way (either Arc<Mutex<Vec<Info>>> or Arc<Vec<Mutex<Info>>>), since you want to change the items.
Or better, since you immediately join() the threads, use scoped threads:
fn calc_associative(nodes: &[Mutex<Info>]) {
let counter = std::sync::atomic::AtomicUsize::new(0); // Changed to atomic, prefer it to mutex wherever possible
std::thread::scope(|s| {
for _ in 0..2 {
s.spawn(|| {
let idx = counter.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let node = &mut *nodes[idx].lock().unwrap();
// costly calculation
node.set_length(node.get_length() * 2);
});
}
});
}
I want to use a double For loop to compare each element with each and if these properties are the same, make a change to the objects. Below is a little demo code of what I want to do.
But the Rust compiler tells me here that I can't have 2 mutable references from the same object. How can I implement the whole thing differently?
fn main() {
struct S {
a: i32
}
let mut v = vec![S{a: 1}, S{a: 1}, S{a: 1}];
let size = v.len();
for i in 0..size {
for j in 0..size {
if i == j {
continue;
}
let a = &mut v[i];
let b = &mut v[j];
if a.a == b.a {
a.a = 5;
}
}
}
}
Console:
error[E0499]: cannot borrow `v` as mutable more than once at a time
--> src\main.rs:35:26
|
34 | let a = &mut v[i];
| - first mutable borrow occurs here
35 | let b = &mut v[j];
| ^ second mutable borrow occurs here
36 |
37 | if a.a == b.a {
| --- first borrow later used here
First thing to note is that you don't need two mutable references since you are only mutating v[i].
Writing it like this will create two immutable borrows and inside the if a mutable borrow:
if v[i].a == v[j].a {
v[i].a = 5;
}
Being very explicit this is what is happening (but its nicer the other way):
let a = &v[i];
let b = &v[j];
if a.a == b.a {
let mut_a = &mut v[i];
mut_a.a = 5;
}
One of the rules of the borrow checker is that no two mutable borrows can exist to the same thing at the same time.
I'm still new with Rust but I have doubts about how mutability works for fields in structs. Specifically how we can modify fields that originally were immutable. For example:
struct Point {
x: isize,
y: isize,
}
impl Point {
fn new(x: isize, y: isize) -> Self {
Self { x, y }
}
fn set_x(&mut self, new_x: isize) {
self.x = new_x;
}
}
struct Line {
p: Point,
q: Point,
}
impl Line {
fn new(p: Point, q: Point) -> Self {
Self { p, q }
}
fn set_x_in_p(&mut self, new_x: isize) {
self.p.set_x(new_x);
}
}
fn main() {
// Both x and y are immutable
let p = Point::new(0, 0);
let q = Point::new(1, 1);
// Line IS mutable
let mut line = Line::new(p, q);
// Modifying point p (originally immutable) with a new x
line.set_x_in_p(999);
}
Instead with references we cannot
let x = 3;
let y = &mut x; // does not work because x originally is immutable
So, how does it work? Thanks.
In your example, p and q are indeed immutable, but then you move them into a Line instance with the constructor, since they're passed by value and don't implement Copy to enable implicit copying. This means that the original bindings (p and q) are no longer valid (the compiler will prevent you from using them) and the values are instead accessible only via the mutable line binding, which allows mutating its members. Essentially, it's not the values that are mutable, but rather their bindings. For example, the following code can be used to re-bind a value to change its mutability:
let x = String::from("hello world"); // using String as an example of a non-Copy type
let mut x = x; // this shadows the previous binding with a mutable one of the same name
x.make_ascii_uppercase(); // we can now mutate the string
let x = x; // shadow the mutable binding with an immutable one
println!("Result: {}", x);
This example works because we have direct control over the value, and can move/bind it as desired. Introducing references would limit what could be done - for example, the following examples wouldn't work:
let x = String::from("hello world");
let x_ref = &x; // create an immutable reference to x
let mut x_mut = x; // error - we can't move x while it's borrowed
let x_mut_ref = &mut x; // error - we can't create a mutable reference while any other references exist
I'd recommend reading the ownership and moves page of Rust by example, which explains it pretty well.
When you are declaring x, you are specifying it as immutable with the let instead of let mut. When you then declare y and initialize it as &mut x you are attempting to borrow x. In Rust, the one thing you can never do is simultaneously have shared ownership AND mutability.
Check out what Niko Matsakis has to say about ownership.
I would like to process the values from a HashMap one by one, while maybe removing some of them.
For example, I would like to do an equivalent of:
use std::collections::HashMap;
fn example() {
let mut to_process = HashMap::new();
to_process.insert(1, true);
loop {
// get an arbitrary element
let ans = to_process.iter().next().clone(); // get an item from the hash
match ans {
Some((k, v)) => {
if condition(&k,&v) {
to_process.remove(&k);
}
}
None => break, // work finished
}
}
}
But this fails to compile:
error[E0502]: cannot borrow `to_process` as mutable because it is also borrowed as immutable
--> src/lib.rs:12:17
|
9 | let ans = to_process.iter().next().clone();
| ---------- immutable borrow occurs here
...
12 | to_process.remove(&k);
| ^^^^^^^^^^^------^^^^
| | |
| | immutable borrow later used by call
| mutable borrow occurs here
I know I really would need https://github.com/rust-lang/rust/issues/27804 (which is for HashSet but for HashMap would be the same)
and I cannot implement the provided solutions without having a non-mut and mutable reference still or using unsafe.
Is there a simple way I am missing?
Note If you need to alter keys or add kvps to the HashMap during processing, see #edwardw's answer. Otherwise ...
Use HashMap::retain. You can change your process function to return a bool indicating whether to keep that key value pair. For example
let mut to_process: HashMap<u32, String> = HashMap::new();
to_process.insert(1, "ok".to_string());
to_process.insert(2, "bad".to_string());
to_process.retain(process);
fn process(k: &u32, v: &mut String) -> bool {
// do stuff with k and v
v == "ok"
}
This looks like an awfully good fit for Iterator::filter_map:
The closure must return an Option<T>. filter_map creates an iterator which calls this closure on each element. If the closure returns Some(element), then that element is returned. If the closure returns None, it will try again, and call the closure on the next element, seeing if it will return Some.
The following process_and_maybe_add is very simple, but you get the idea:
use std::collections::HashMap;
fn main() {
let mut data = HashMap::new();
data.insert(1, "a");
data.insert(2, "b");
data.insert(3, "c");
let processed = data
.into_iter()
.filter_map(process_and_maybe_add)
.collect::<HashMap<_, _>>();
dbg!(processed);
}
fn process_and_maybe_add((k, v): (u32, &str)) -> Option<(u32, String)> {
if k % 2 != 0 {
Some((k + 100, v.to_owned() + v))
} else {
None
}
}
Here is the code:
fn test(){
let mut numbers = vec![2];
let f = || {
for _ in numbers.iter(){
}
false
};
while false {
let res = f();
if res {
numbers.push(10);
}
}
}
The error is:
|
15 | let f = || {
| -- immutable borrow occurs here
16 | for _ in numbers.iter(){
| ------- first borrow occurs due to use of `numbers` in closure
...
22 | let res = f();
| - immutable borrow later used here
23 | if res {
24 | numbers.push(10);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
But if I change the while keyword to if, it can be compiled. How to fix this? I want to call the anonymous function in a loop.
We can simplify your example even more by replacing the closure by a simple immutable reference
let mut numbers = vec![2];
let r = &numbers;
while false {
println!("{:?}", r);
numbers.push(10);
}
Here we get this error:
error[E0502]: cannot borrow `numbers` as mutable because it is also borrowed as immutable
--> src/lib.rs:7:5
|
3 | let r = &numbers;
| -------- immutable borrow occurs here
...
6 | println!("{:?}", r); // use reference
| - immutable borrow later used here
7 | numbers.push(10);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
And like in your example, replacing the while with if makes the error go away. Why?
You probably know about the important Rust rule: Aliasing nand mutability. It states that, at any given time, a value can either be borrowed immutably arbitrarily many times or mutably exactly once.
The statement numbers.push(10) borrows numbers mutably temporarily (just for the statement). But we also have r which is an immutable reference. In order for numbers.push(10) to work, the compiler has to make sure that no other borrow exists at that time. But there exists the reference r! This reference cannot exist at the same time as numbers.push(10) exists.
Let's see for the if case first:
let mut numbers = vec![2];
let r = &numbers; // <------+ (creation of r)
// |
if false { // |
println!("{:?}", r); // <------+ (last use of r)
numbers.push(10);
}
While the lexical scope means the variable r is only dropped at the end of the function, due to non-lexical lifetimes, the compiler can see that the last use of r is in the println line. Then the compiler can mark r as "dead" after this line. And this in turn means, that there is no other borrow in the line numbers.push(10) and everything works out fine.
And for the loop case? Let's imagine the loop iterating three times:
let mut numbers = vec![2];
let r = &numbers; // <------+ (creation of r)
// |
// First iteration // |
println!("{:?}", r); // |
numbers.push(10); // | <== oh oh!
// |
// Second iteration // |
println!("{:?}", r); // |
numbers.push(10); // |
// |
// Third iteration // |
println!("{:?}", r); // <------+ (last use of r)
numbers.push(10);
As can be seen here, the time in which r is active overlaps numbers.push(10) (except the last one). And as a result, the compiler will produce an error because this code violates the central Rust rule.
And the explanation is the same for your closure case: the closure borrows numbers immutably and f(); uses that closure. In the loop case, the compiler is not able to shrink the "alive time" of the closure enough to make sure it doesn't overlap the mutable borrow for push.
How to fix?
Well, you could pass numbers into the closure each time:
let mut numbers = vec![2];
let f = |numbers: &[i32]| {
for _ in numbers.iter(){
}
false
};
while false {
let res = f(&numbers);
if res {
numbers.push(10);
}
}
This works because now, numbers is borrowed immutably also just temporarily for the f(&numbers); statement.
You can also use a RefCell as the other answer suggested, but that should be a last resort.
It's not exactly sure what you're trying to accomplish, but one way to solve this, without changing your code too drastically, would be to use std::cell::RefCell (described in the std and in the book):
use std::cell::RefCell;
fn test(){
let numbers = RefCell::new(vec![2]);
let f = || {
for _ in numbers.borrow().iter(){
}
false
};
while false {
let res = f();
if res {
numbers.borrow_mut().push(10);
}
}
}
Here's a little bit tweaked demo, which actually does something:
use std::cell::RefCell;
fn main() {
test();
}
fn test() {
let numbers = RefCell::new(vec![0]);
let f = || {
for n in numbers.borrow().iter() {
println!("In closure: {}", n);
}
println!();
true
};
let mut i = 1;
while i <= 3 {
let res = f();
if res {
numbers.borrow_mut().push(i);
}
i += 1;
}
println!("End of test(): {:?}", numbers.borrow());
}
Output:
In closure: 0
In closure: 0
In closure: 1
In closure: 0
In closure: 1
In closure: 2
End of test(): [0, 1, 2, 3]
Rust Playground demo