Cannot borrow value from a hashmap as mutable because it is also borrowed as immutable - rust

I want to get two values from a hashmap at the same time, but I can't escape the following error, I have simplified the code as follows, can anyone help me to fix this error.
#[warn(unused_variables)]
use hashbrown::HashMap;
fn do_cal(a: &[usize], b: &[usize]) -> usize {
a.iter().sum::<usize>() + b.iter().sum::<usize>()
}
fn do_check(i: usize, j:usize) -> bool {
i/2 < j - 10
}
fn do_expensive_cal(i: usize) -> Vec<usize> {
vec![i,i,i]
}
fn main() {
let size = 1000000;
let mut hash: HashMap<usize, Vec<usize>> = HashMap::new();
for i in 0..size{
if i > 0 {
hash.remove(&(i - 1));
}
if !hash.contains_key(&i){
hash.insert(i, do_expensive_cal(i));
}
let data1 = hash.get(&i).unwrap();
for j in i + 1..size {
if do_check(i, j) {
break
}
if !hash.contains_key(&j){
hash.insert(j, do_expensive_cal(j));
}
let data2 = hash.get(&j).unwrap();
let res = do_cal(data1, data2);
println!("res:{}", res);
}
}
}
Playground
error[E0502]: cannot borrow hash as mutable because it is also borrowed as immutable
--> src/main.rs:26:8
|
19 | let data1 = hash.get(&i).unwrap();
| ------------ immutable borrow occurs here
...
26 | hash.insert(j, vec![1,2,3]);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
...
29 | let res = do_cal(data1, data2);
| ----- immutable borrow later used here
For more information about this error, try rustc --explain E0502.
error: could not compile playground due to previous error

Consider this: the borrow checker doesn't know that hash.insert(j, …) will leave the data you inserted with hash.insert(i, …) alone. For the borrow checker, hash.insert(…) may do anything to any element in hash, including rewriting or removing it. So you can't be allowed to hold the reference data1 over hash.insert(j, …).
How to get over that? The easiest is probably to move let data1 = hash.get(…) so it doesn't have to live for so long:
let data1 = hash.get(&i).unwrap();
let data2 = hash.get(&j).unwrap();
let res = do_cal(data1, data2);
This will of course look up data1 every loop iteration (and it must, since hash.insert(j, …) may have resized and thus realocated the content of the hashmap, giving data1 a new storage location in the hashmap). For completeness's sake, there are ways to get around that, but I don't recommend you do any of this:
Clone: let data1 = hash.get(&i).unwrap().clone() (if your vecs are short, this may actually be reasonable…)
As a way of making the cloning cheap, you could use a HashMap<usize, Rc<Vec<usize>>> instead (where you only need to clone the Rc, no the entire Vec)
If you ever need mutable references to both arguments of do_call, you could combine the Rc with a RefCell: Rc<RefCell<Vec<…>>>
If you need to overengineer it even more, you could replace the Rcs with references obtained from allocating in a bump allocator, e.g. bumpalo.

Since the keys to the hash table are the integers 0..100 you can use a Vec to perform these steps, temporarily splitting the Vec into 2 slices to allow the mutation on one side. If you need a HashMap for later computations, you can then create a HashMap from the Vec.
The following code compiles but panics because the j - 10 calculation underflows:
fn do_cal(a: &[usize], b: &[usize]) -> usize {
a.iter().sum::<usize>() + b.iter().sum::<usize>()
}
fn do_check(i: usize, j:usize) -> bool {
i/2 < j - 10
}
fn main() {
let size = 100;
let mut v: Vec<Option<Vec<usize>>> = vec![None; size];
for i in 0..size {
let (v1, v2) = v.split_at_mut(i + 1);
if v1[i].is_none() {
v1[i] = Some(vec![1,2,3]);
}
let data1 = v1[i].as_ref().unwrap();
for (j, item) in (i + 1..).zip(v2.iter_mut()) {
if do_check(i, j) {
break
}
if item.is_none() {
*item = Some(vec![1,2,3]);
}
let data2 = item.as_ref().unwrap();
let res = do_cal(data1, data2);
println!("res:{}", res);
}
}
}

Related

How do I move a vector reference into threads? [duplicate]

This question already has an answer here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed last month.
How do I move a vector reference into threads? The closest I get is the (minimized) code below. (I realize that the costly calculation still isn't parallel, as it is locked by the mutex, but one problem at a time.)
Base problem: I'm calculating values based on information saved in a vector. Then I'm storing the results as nodes per vector element. So vector in vector (but only one vector in the example code below). The calculation takes time so I would like to divide it into threads. The structure is big, so I don't want to copy it.
use std::sync::{Arc, Mutex};
use std::thread;
fn main() {
let n = Nodes::init();
n.calc();
println!("Result: nodes {:?}", n);
}
#[derive(Debug)]
struct Nodes {
nodes: Vec<Info>,
}
impl Nodes {
fn init() -> Self {
let mut n = Nodes { nodes: Vec::new() };
n.nodes.push(Info::init(1));
n.nodes.push(Info::init(2));
n
}
fn calc(&self) {
Nodes::calc_associative(&self.nodes);
}
fn calc_associative(nodes: &Vec<Info>) {
let mut handles = vec![];
let arc_nodes = Arc::new(nodes);
let counter = Arc::new(Mutex::new(0));
for _ in 0..2 {
let arc_nodes = Arc::clone(&arc_nodes);
let counter = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut idx = counter.lock().unwrap();
// costly calculation
arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
*idx += 1;
});
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
}
}
#[derive(Debug)]
struct Info {
length: u32,
}
impl Info {
fn init(length: u32) -> Self {
Info { length }
}
fn get_length(&self) -> u32 {
self.length
}
fn set_length(&mut self, x: u32) {
self.length = x;
}
}
The compiler complains that life time of the reference isn't fulfilled, but isn't that what Arc::clone() should do? Then Arc require a deref, but maybe there are better solutions before starting to dig into that...?
Compiling threads v0.1.0 (/home/freefox/proj/threads)
error[E0596]: cannot borrow data in an `Arc` as mutable
--> src/main.rs:37:17
|
37 | arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
| ^^^^^^^^^ cannot borrow as mutable
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Arc<&Vec<Info>>`
error[E0521]: borrowed data escapes outside of associated function
--> src/main.rs:34:26
|
25 | fn calc_associative(nodes: &Vec<Info>) {
| ----- - let's call the lifetime of this reference `'1`
| |
| `nodes` is a reference that is only valid in the associated function body
...
34 | let handle = thread::spawn(move || {
| __________________________^
35 | | let mut idx = counter.lock().unwrap();
36 | | // costly calculation
37 | | arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
38 | | *idx += 1;
39 | | });
| | ^
| | |
| |______________`nodes` escapes the associated function body here
| argument requires that `'1` must outlive `'static`
Some errors have detailed explanations: E0521, E0596.
For more information about an error, try `rustc --explain E0521`.
error: could not compile `threads` due to 2 previous errors
You wrap a reference with Arc. Now the type is Arc<&Vec<Info>>. There is still a reference here, so the variable could still be destroyed before the thread return and we have a dangling reference.
Instead, you should take a &Arc<Vec<Info>>, and on the construction of the Vec wrap it in Arc, or take &[Info] and clone it (let arc_nodes = Arc::new(nodes.to_vec());). You also need a mutex along the way (either Arc<Mutex<Vec<Info>>> or Arc<Vec<Mutex<Info>>>), since you want to change the items.
Or better, since you immediately join() the threads, use scoped threads:
fn calc_associative(nodes: &[Mutex<Info>]) {
let counter = std::sync::atomic::AtomicUsize::new(0); // Changed to atomic, prefer it to mutex wherever possible
std::thread::scope(|s| {
for _ in 0..2 {
s.spawn(|| {
let idx = counter.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let node = &mut *nodes[idx].lock().unwrap();
// costly calculation
node.set_length(node.get_length() * 2);
});
}
});
}

Double for loop mutable references

I want to use a double For loop to compare each element with each and if these properties are the same, make a change to the objects. Below is a little demo code of what I want to do.
But the Rust compiler tells me here that I can't have 2 mutable references from the same object. How can I implement the whole thing differently?
fn main() {
struct S {
a: i32
}
let mut v = vec![S{a: 1}, S{a: 1}, S{a: 1}];
let size = v.len();
for i in 0..size {
for j in 0..size {
if i == j {
continue;
}
let a = &mut v[i];
let b = &mut v[j];
if a.a == b.a {
a.a = 5;
}
}
}
}
Console:
error[E0499]: cannot borrow `v` as mutable more than once at a time
--> src\main.rs:35:26
|
34 | let a = &mut v[i];
| - first mutable borrow occurs here
35 | let b = &mut v[j];
| ^ second mutable borrow occurs here
36 |
37 | if a.a == b.a {
| --- first borrow later used here
First thing to note is that you don't need two mutable references since you are only mutating v[i].
Writing it like this will create two immutable borrows and inside the if a mutable borrow:
if v[i].a == v[j].a {
v[i].a = 5;
}
Being very explicit this is what is happening (but its nicer the other way):
let a = &v[i];
let b = &v[j];
if a.a == b.a {
let mut_a = &mut v[i];
mut_a.a = 5;
}
One of the rules of the borrow checker is that no two mutable borrows can exist to the same thing at the same time.

fold string to build hashmap char counter in rust, but gives two-phase borrow error

I'm trying to build up a hashmap that counts character frequencies in a string. My approach is to fold over the characters in the string, incrementing the count for the current character at each iteration. Unfortunately, rust is telling me that I'm doing something wrong with borrows.
No variation of using clone() or breaking out the hm.get(&c) onto a separate line has any effect.
I don't see how to avoid this error about two-phase borrows. (relevant rust-lang github issue)
use std::collections::HashMap;
fn main() {
let some_str = "some string";
let hm = some_str.chars().fold(HashMap::new(), |mut hm, c| {
match hm.get(&c) {
None => hm.insert(c, 1),
Some(i) => hm.insert(c, i + 1),
};
hm
});
for (key, val) in hm.iter() {
println!("{}: {}", key, val);
}
}
which gives this error
warning: cannot borrow `hm` as mutable because it is also borrowed as immutable
--> dank.rs:9:24
|
7 | match hm.get(&c) {
| -- immutable borrow occurs here
8 | None => hm.insert(c, 1),
9 | Some(i) => hm.insert(c, i+1)
| ^^ - immutable borrow later used here
| |
| mutable borrow occurs here
|
= note: `#[warn(mutable_borrow_reservation_conflict)]` on by default
= warning: this borrowing pattern was not meant to be accepted, and may become a hard error in the future
= note: for more information, see issue #59159 <https://github.com/rust-lang/rust/issues/59159>
credit to #Stargateur for the answer to my question
The issue was that when we insert i + 1 back into the hashmap, the variable i is actually borrowed from the hashmap. If we copy i first, our immutable borrow of hm ends where we copy i, which is before we use a mutable reference to hm for the insert().
use std::collections::HashMap;
fn main() {
let some_str = "some string";
let hm = some_str.chars().fold(HashMap::new(), |mut hm, c| {
match hm.get(&c) { // borrow of hm
None => hm.insert(c, 1),
Some(i) => { // i is a reference
let j = *i; // i is Copy so we copy it, j is not a reference owned by hm, so hm is not borrowed anymore
hm.insert(c, j + 1) // we can borrow hm mutable
}
};
hm
});
for (key, val) in hm.iter() {
println!("{}: {}", key, val);
}
}
Further, a superior solution is to use the Entry api to get around this problem entirely with the following code:
use std::collections::HashMap;
fn main() {
let some_str = "some string";
let hm = some_str.chars().fold(HashMap::new(), |mut hm, c| {
*hm.entry(c).or_insert(0) += 1;
hm
});
for (key, val) in hm.iter() {
println!("{}: {}", key, val);
}
}
I think some_str.chars().counts(); is the easiest way to do this.
It's a built in function but the implementation is
#[cfg(feature = "use_std")]
fn counts(self) -> HashMap<Self::Item, usize>
where
Self: Sized,
Self::Item: Eq + Hash,
{
let mut counts = HashMap::new();
self.for_each(|item| *counts.entry(item).or_default() += 1);
counts
}

How to process every value in a HashMap and optionally reject some?

I would like to process the values from a HashMap one by one, while maybe removing some of them.
For example, I would like to do an equivalent of:
use std::collections::HashMap;
fn example() {
let mut to_process = HashMap::new();
to_process.insert(1, true);
loop {
// get an arbitrary element
let ans = to_process.iter().next().clone(); // get an item from the hash
match ans {
Some((k, v)) => {
if condition(&k,&v) {
to_process.remove(&k);
}
}
None => break, // work finished
}
}
}
But this fails to compile:
error[E0502]: cannot borrow `to_process` as mutable because it is also borrowed as immutable
--> src/lib.rs:12:17
|
9 | let ans = to_process.iter().next().clone();
| ---------- immutable borrow occurs here
...
12 | to_process.remove(&k);
| ^^^^^^^^^^^------^^^^
| | |
| | immutable borrow later used by call
| mutable borrow occurs here
I know I really would need https://github.com/rust-lang/rust/issues/27804 (which is for HashSet but for HashMap would be the same)
and I cannot implement the provided solutions without having a non-mut and mutable reference still or using unsafe.
Is there a simple way I am missing?
Note If you need to alter keys or add kvps to the HashMap during processing, see #edwardw's answer. Otherwise ...
Use HashMap::retain. You can change your process function to return a bool indicating whether to keep that key value pair. For example
let mut to_process: HashMap<u32, String> = HashMap::new();
to_process.insert(1, "ok".to_string());
to_process.insert(2, "bad".to_string());
to_process.retain(process);
fn process(k: &u32, v: &mut String) -> bool {
// do stuff with k and v
v == "ok"
}
This looks like an awfully good fit for Iterator::filter_map:
The closure must return an Option<T>. filter_map creates an iterator which calls this closure on each element. If the closure returns Some(element), then that element is returned. If the closure returns None, it will try again, and call the closure on the next element, seeing if it will return Some.
The following process_and_maybe_add is very simple, but you get the idea:
use std::collections::HashMap;
fn main() {
let mut data = HashMap::new();
data.insert(1, "a");
data.insert(2, "b");
data.insert(3, "c");
let processed = data
.into_iter()
.filter_map(process_and_maybe_add)
.collect::<HashMap<_, _>>();
dbg!(processed);
}
fn process_and_maybe_add((k, v): (u32, &str)) -> Option<(u32, String)> {
if k % 2 != 0 {
Some((k + 100, v.to_owned() + v))
} else {
None
}
}

Cannot borrow as mutable because it is also borrowed as immutable

I am learning Rust and I don't quite get why this is not working.
#[derive(Debug)]
struct Node {
value: String,
}
#[derive(Debug)]
pub struct Graph {
nodes: Vec<Box<Node>>,
}
fn mk_node(value: String) -> Node {
Node { value }
}
pub fn mk_graph() -> Graph {
Graph { nodes: vec![] }
}
impl Graph {
fn add_node(&mut self, value: String) {
if let None = self.nodes.iter().position(|node| node.value == value) {
let node = Box::new(mk_node(value));
self.nodes.push(node);
};
}
fn get_node_by_value(&self, value: &str) -> Option<&Node> {
match self.nodes.iter().position(|node| node.value == *value) {
None => None,
Some(idx) => self.nodes.get(idx).map(|n| &**n),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn some_test() {
let mut graph = mk_graph();
graph.add_node("source".to_string());
graph.add_node("destination".to_string());
let source = graph.get_node_by_value("source").unwrap();
let dest = graph.get_node_by_value("destination").unwrap();
graph.add_node("destination".to_string());
}
}
(playground)
This has the error
error[E0502]: cannot borrow `graph` as mutable because it is also borrowed as immutable
--> src/main.rs:50:9
|
47 | let source = graph.get_node_by_value("source").unwrap();
| ----- immutable borrow occurs here
...
50 | graph.add_node("destination".to_string());
| ^^^^^ mutable borrow occurs here
51 | }
| - immutable borrow ends here
This example from Programming Rust is quite similar to what I have but it works:
pub struct Queue {
older: Vec<char>, // older elements, eldest last.
younger: Vec<char>, // younger elements, youngest last.
}
impl Queue {
/// Push a character onto the back of a queue.
pub fn push(&mut self, c: char) {
self.younger.push(c);
}
/// Pop a character off the front of a queue. Return `Some(c)` if there /// was a character to pop, or `None` if the queue was empty.
pub fn pop(&mut self) -> Option<char> {
if self.older.is_empty() {
if self.younger.is_empty() {
return None;
}
// Bring the elements in younger over to older, and put them in // the promised order.
use std::mem::swap;
swap(&mut self.older, &mut self.younger);
self.older.reverse();
}
// Now older is guaranteed to have something. Vec's pop method // already returns an Option, so we're set.
self.older.pop()
}
pub fn split(self) -> (Vec<char>, Vec<char>) {
(self.older, self.younger)
}
}
pub fn main() {
let mut q = Queue {
older: Vec::new(),
younger: Vec::new(),
};
q.push('P');
q.push('D');
assert_eq!(q.pop(), Some('P'));
q.push('X');
let (older, younger) = q.split(); // q is now uninitialized.
assert_eq!(older, vec!['D']);
assert_eq!(younger, vec!['X']);
}
A MRE of your problem can be reduced to this:
// This applies to the version of Rust this question
// was asked about; see below for updated examples.
fn main() {
let mut items = vec![1];
let item = items.last();
items.push(2);
}
error[E0502]: cannot borrow `items` as mutable because it is also borrowed as immutable
--> src/main.rs:4:5
|
3 | let item = items.last();
| ----- immutable borrow occurs here
4 | items.push(2);
| ^^^^^ mutable borrow occurs here
5 | }
| - immutable borrow ends here
You are encountering the exact problem that Rust was designed to prevent. You have a reference pointing into the vector and are attempting to insert into the vector. Doing so might require that the memory of the vector be reallocated, invalidating any existing references. If that happened and you used the value in item, you'd be accessing uninitialized memory, potentially causing a crash.
In this particular case, you aren't actually using item (or source, in the original) so you could just... not call that line. I assume you did that for some reason, so you could wrap the references in a block so that they go away before you try to mutate the value again:
fn main() {
let mut items = vec![1];
{
let item = items.last();
}
items.push(2);
}
This trick is no longer needed in modern Rust because non-lexical lifetimes have been implemented, but the underlying restriction still remains — you cannot have a mutable reference while there are other references to the same thing. This is one of the rules of references covered in The Rust Programming Language. A modified example that still does not work with NLL:
let mut items = vec![1];
let item = items.last();
items.push(2);
println!("{:?}", item);
In other cases, you can copy or clone the value in the vector. The item will no longer be a reference and you can modify the vector as you see fit:
fn main() {
let mut items = vec![1];
let item = items.last().cloned();
items.push(2);
}
If your type isn't cloneable, you can transform it into a reference-counted value (such as Rc or Arc) which can then be cloned. You may or may not also need to use interior mutability:
struct NonClone;
use std::rc::Rc;
fn main() {
let mut items = vec![Rc::new(NonClone)];
let item = items.last().cloned();
items.push(Rc::new(NonClone));
}
this example from Programming Rust is quite similar
No, it's not, seeing as how it doesn't use references at all.
See also
Cannot borrow `*x` as mutable because it is also borrowed as immutable
Pushing something into a vector depending on its last element
Why doesn't the lifetime of a mutable borrow end when the function call is complete?
How should I restructure my graph code to avoid an "Cannot borrow variable as mutable more than once at a time" error?
Why do I get the error "cannot borrow x as mutable more than once"?
Why does Rust want to borrow a variable as mutable more than once at a time?
Try to put your immutable borrow inside a block {...}.
This ends the borrow after the block.
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn some_test() {
let mut graph = mk_graph();
graph.add_node("source".to_string());
graph.add_node("destination".to_string());
{
let source = graph.get_node_by_value("source").unwrap();
let dest = graph.get_node_by_value("destination").unwrap();
}
graph.add_node("destination".to_string());
}
}
So for anyone else banging their head against this problem and wanting a quick way out - use clones instead of references. Eg I'm iterating this list of cells and want to change an attribute so I first copy the list:
let created = self.cells
.into_iter()
.map(|c| {
BoardCell {
x: c.x,
y: c.y,
owner: c.owner,
adjacency: c.adjacency.clone(),
}
})
.collect::<Vec<BoardCell>>();
And then modify the values in the original by looping the copy:
for c in created {
self.cells[(c.x + c.y * self.size) as usize].adjacency[dir] = count;
}
Using Vec<&BoardCell> would just yield this error. Not sure how Rusty this is but hey, it works.

Resources