I've boiled a problem I'm seeing down to this example:
use std::collections::HashMap;
fn process(mut inputs: Vec<String>) -> Vec<String> {
// keep track of duplicate entries in the input Vec
let mut duplicates: HashMap<String, Vec<&mut String>> = HashMap::new();
for input in &mut inputs {
duplicates.entry(input.clone())
.or_insert_with(|| Vec::new())
.push(input);
}
// modify the input vector in place to append the number of each duplicate
for (key, instances) in duplicates {
for (i, instance) in instances.iter().enumerate() {
*instance = format!("{}_{}", instance, i);
}
}
return inputs;
}
fn main() {
println!("results: {:?}", process(vec![
String::from("test"),
String::from("another_test"),
String::from("test")
]));
}
I would expect this to print something like results: test_0, another_test_0, test_1 but am instead running into build issues:
error[E0308]: mismatched types
--> src/main.rs:13:25
|
13 | *instance = format!("{}_{}", instance, i);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&mut String`, found struct `String`
|
I'm still a bit new to rust and so haven't really found anything online that has helped. I'm hoping that I'm doing something silly. Thanks in advance for the help!
for (key, instances) in duplicates {
for (i, instance) in instances.iter().enumerate() {
*instance = format!("{}_{}", instance, i);
}
}
for (key, instances) in duplicates invokes the IntoIterator trait on HashMap<String, Vec<&mut String>>, with
type Item = (String, Vec<&mut String>);
so instances: Vec<&mut String>.1 instances.iter() then produces references to the elements of instances, so instance: &&mut String. So instance would need to be dereferenced twice in order to access the element in inputs, which, unfortunately, doesn't actually work:
error[E0594]: cannot assign to `**instance` which is behind a `&` reference
--> src/main.rs:14:13
|
13 | for (i, instance) in instances.iter().enumerate() {
| -------- help: consider changing this to be a mutable reference: `&mut &mut String`
14 | **instance = format!("{}_{}", instance, i);
| ^^^^^^^^^^ `instance` is a `&` reference, so the data it refers to cannot be written
The problem is that we are not allowed to modify the T value behind &&mut T. Here's the reason: immutable references can be freely copied, so there may be multiple &&mut T instances pointing to the same &mut T, defeating the anti-aliasing property of &mut.
Therefore, we have to change the type of instance to &mut &mut String, by using mut and .iter_mut():
for (key, mut instances) in duplicates {
for (i, instance) in instances.iter_mut().enumerate() {
**instance = format!("{}_{}", instance, i);
}
}
Since key isn't needed, we can use .values_mut(), so that instances itself becomes an &mut:
for instances in duplicates.values_mut() {
for (i, instance) in instances.iter_mut().enumerate() {
**instance = format!("{}_{}", instance, i);
}
}
1 We'll use var: type to denote that var is of type type.
In general, references (especially mutable ones) stored in containers are tedious to work with, so I recommend an alternative approach: (with an in-place interface for simplicity)
use std::{collections::HashMap, fmt::Write};
fn process(texts: &mut Vec<String>) {
let mut counter = HashMap::<String, usize>::new();
for text in texts.iter_mut() {
let count = counter.entry(text.clone()).or_insert(0);
write!(text, "_{}", count).unwrap();
*count += 1;
}
}
(playground)
Iterating returns you references over the values stored in your map's vecs.
As your map's vecs contain references to start with, you have to dereference twice, using **.
You also need to use iter_mut when iterating if you want to change the content.
All in one, you can change your loop like this:
for (key, instances) in duplicates.iter_mut() {
for (i, instance) in instances.iter_mut().enumerate() {
**instance = format!("{}_{}", instance, i);
}
}
playground
The problem you're facing is that immutability is transitive.
That is, coming from other languages you might expect that a & &mut Foo would let you dereference the outer ref', access the inner mutable ref, and update the Foo based on that. However if you consider the purpose of rust's unique references, this would be the same as allowing multiple mutable references: take a mutable reference, create any number of immutable references to it, and any holder of one such would be able to modify the original object, concurrently.
Therefore that's not acceptable. All references in a chain must be unique (&mut) in order to get mutable access to what's at the end of the chain, and so you need to instances.iter_mut(), and dereference twice (otherwise you're assigning to the &mut String at the first level, not the inner String), and for that you need instances itself to be mutable:
for (_, mut instances) in duplicates {
for (i, instance) in instances.iter_mut().enumerate() {
**instance = format!("{}_{}", instance, i);
}
}
alternatively since you don't care about duplicates' keys:
for instances in duplicates.values_mut() {
for (i, instance) in instances.iter_mut().enumerate() {
**instance = format!("{}_{}", instance, i);
}
}
Related
I thought I got the idea of move semantics until this code.
fn main() {
let v = Data {
body: vec![10, 40, 30],
};
p(&v);
}
fn p(d: &Data) {
for i in d.body {
// &d.body, Why d.body move?
println!("{}", i);
}
}
struct Data {
body: Vec<i32>,
}
error[E0507]: cannot move out of borrowed content
--> src/main.rs:9:14
|
9 | for i in d.body {
| ^^^^^^ cannot move out of borrowed content
error[E0507]: cannot move out of `d.body` which is behind a `&` reference
--> src/main.rs:9:14
|
8 | fn p(d: &Data) {
| ----- help: consider changing this to be a mutable reference: `&mut Data`
9 | for i in d.body {
| ^^^^^^
| |
| cannot move out of `d.body` which is behind a `&` reference
| `d` is a `&` reference, so the data it refers to cannot be moved
I passed a reference, and I accessed a field via auto-deref feature, so why is it a move?
What you are doing is field accessing on pointer.
Check Field Access Expression :
if the type of the expression to the left of the dot is a pointer, it
is automatically dereferenced as many times as necessary to make the
field access possible
Sample for how Rust evaluates Field Access Expression on Borrowed Content :
let d = Data { /*input*/}
let body = (&d).body // -> (*&d).body -> d.body
let ref_body = &(&d).body // -> &(*&).body -> &d.body -> &(d.body)
Note : d is still borrowed content, auto deref is just needed to access the fields.
Why move ?
Consider this code:
struct Data {
body: Vec<i32>,
id: i32,
}
fn p(mut d: &Data) {
let id = d.id;
}
This code will work as expected and there will be no moves in here so you will able to reuse d.id. In this situation:
Rust will try to copy the value of d.id. Since d.id is i32 and implements the Copy trait, it will copy the value to id.
Consider this code:
fn p(mut d: &Data) {
let id = d.id; // works
let body = d.body; // fails
}
This code will not work because:
Rust will try to copy d.body but Vec<i32> has no implementation of the Copy trait.
Rust will try to move body from d, and you will get the "cannot move out of borrowed content" error.
How does this effect the loop?
From the reference
A for expression is a syntactic construct for looping over elements provided by an implementation of std::iter::IntoIterator
A for loop is equivalent to the following block expression.
'label: for PATTERN in iter_expr {
/* loop body */
}
is equivalent to
{
let result = match IntoIterator::into_iter(iter_expr) {
mut iter => 'label: loop {
let mut next;
match Iterator::next(&mut iter) {
Option::Some(val) => next = val,
Option::None => break,
};
let PAT = next;
let () = { /* loop body */ };
},
};
result
}
This means your vector must have an implementation of IntoIterator because IntoIterator::into_iter(self) expects self as an argument. Luckily, both impl IntoIterator for Vec<T>, another is impl<'a, T> IntoIterator for &'a Vec<T> exist.
Why does this happen?
Simply:
When you use &d.body, your loop uses the &Vec implementation of IntoIterator.
This implementation returns an iterator which points at your vector's slice. This means you will get the reference of elements from your vector.
When you use d.body, your loop uses the Vec implementation of IntoIterator.
This implementation returns an iterator which is a consuming iterator. This means your loop will have the ownership of actual elements, not their references. For the consuming part this implementation needs the actual vector not the reference, so the move occurs.
You are accessing the field body in d. body itself is a Vec<i32> which is not a reference. If you would use d directly, no & would be necessary, but since you are accessing a field in d, you must specify that you want to have the reference to the field.
Basically d owns body. If you borrow d you cannot steal body, it belongs to d but you can borrow it.
This loop will be desugared into something similar to the following:
let mut iter = IntoIterator::into_iter(v);
loop {
match iter.next() {
Some(x) => {
// loop body
},
None => break,
}
}
As you can see, it's using into_iter, which moves the vector d.body.
I would like to remove items from a BTreeMap which have been found through iteration.
As it is not possible to remove items while iterating, I put the items to delete into a vector. The main issue is that it is not possible to use a vector of references, but only a vector of values. All the keys for which the entry has to be removed must then be cloned (assuming the key implements the Clone trait).
For instance, this short sample does not compile:
use std::collections::BTreeMap;
pub fn clean() {
let mut map = BTreeMap::<String, i32>::new();
let mut to_delete = Vec::new();
{
for (k, v) in map.iter() {
if *v > 10 {
to_delete.push(k);
}
}
}
for k in to_delete.drain(..) {
map.remove(k);
}
}
fn main() {}
It generates the following errors when it is compiled:
error[E0502]: cannot borrow `map` as mutable because it is also borrowed as immutable
--> src/main.rs:17:9
|
9 | for (k, v) in map.iter() {
| --- immutable borrow occurs here
...
17 | map.remove(k);
| ^^^ mutable borrow occurs here
18 | }
19 | }
| - immutable borrow ends here
Changing to_delete.push(k) with to_delete.push(k.clone()) makes this snippet compile correctly but it is quite costly if each key to delete must be cloned.
Is there a better solution?
TL;DR: you cannot.
As far as the compiler is concerned, the implementation of BTreeMap::remove might do this:
pub fn remove<Q>(&mut self, key: &Q) -> Option<V>
where
K: Borrow<Q>,
Q: Ord + ?Sized,
{
// actual deleting code, which destroys the value in the set
// now what `value` pointed to is gone and `value` points to invalid memory
// And now we access that memory, causing undefined behavior
key.borrow();
}
The compiler thus has to prevent using the reference to the value when the collection will be mutated.
To do this, you'd need something like the hypothetical "cursor" API for collections. This would allow you to iterate over the collection, returning a special type that hold the mutable innards of the collection. This type could give you a reference to check against and then allow you to remove the item.
I'd probably look at the problem from a bit different direction. Instead of trying to keep the map, I'd just create a brand new one:
use std::collections::BTreeMap;
pub fn main() {
let mut map = BTreeMap::new();
map.insert("thief", 5);
map.insert("troll", 52);
map.insert("gnome", 7);
let map: BTreeMap<_, _> =
map.into_iter()
.filter(|&(_, v)| v <= 10)
.collect();
println!("{:?}", map); // troll is gone
}
If your condition is equality on the field that makes the struct unique (a.k.a is the only field used in PartialEq and Hash), you can implement Borrow for your type and directly grab it / delete it:
use std::collections::BTreeMap;
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)]
struct Monster(String);
use std::borrow::Borrow;
impl Borrow<str> for Monster {
fn borrow(&self) -> &str { &self.0 }
}
pub fn main() {
let mut map = BTreeMap::new();
map.insert(Monster("thief".into()), 5);
map.insert(Monster("troll".into()), 52);
map.insert(Monster("gnome".into()), 7);
map.remove("troll");
println!("{:?}", map); // troll is gone
}
See also:
How to implement HashMap with two keys?
How to avoid temporary allocations when using a complex key for a HashMap?
As of 1.53.0, there is a BTreeMap.retain method which allows to remove/retain items based on return value of a closure, while iterating:
map.retain(|_, v| *v<=10);
I have an array of Elements and I want to iterate over it to do some stuff, then iterate over all Elements inside the loop to do something. There is a relation between elements so I want to iterate to all other elements to check something. The elements are mutable references for reasons. It's a bit broad, but I'm trying to be general (maybe I should not).
struct Element;
impl Element {
fn do_something(&self, _e: &Element) {}
}
fn main() {
let mut elements = [Element, Element, Element, Element];
for e in &mut elements {
// Do stuff...
for f in &mut elements {
e.do_something(f);
}
}
}
As expected, I got this error:
error[E0499]: cannot borrow `elements` as mutable more than once at a time
--> src/main.rs:13:18
|
10 | for e in &mut elements {
| -------------
| |
| first mutable borrow occurs here
| first borrow later used here
...
13 | for f in &mut elements {
| ^^^^^^^^^^^^^ second mutable borrow occurs here
I know it's a normal behavior in Rust, but what's the recommended way to avoid this error? Should I copy the elements first? Forget about loops and iterate in a different way? Learn about code design?
Is there a Rusty way to do this?
You can use indexed iteration instead of iterating with iterators. Then, inside the inner loop, you can use split_at_mut to obtain two mutable references into the same slice.
for i in 0..elements.len() {
for j in 0..elements.len() {
let (e, f) = if i < j {
// `i` is in the left half
let (left, right) = elements.split_at_mut(j);
(&mut left[i], &mut right[0])
} else if i == j {
// cannot obtain two mutable references to the
// same element
continue;
} else {
// `i` is in the right half
let (left, right) = elements.split_at_mut(i);
(&mut right[0], &mut left[j])
};
e.do_something(f);
}
}
You cannot do this, period. The rules of references state, emphasis mine:
At any given time, you can have either one mutable reference or any number of immutable references
On the very first iteration, you are trying to get two mutable references to the first element in the array. This must be disallowed.
Your method doesn't require mutable references at all (fn do_something(&self, e: &Element) {}), so the simplest thing is to just switch to immutable iterators:
for e in &elements {
for f in &elements {
e.doSomething(f);
}
}
If you truly do need to perform mutation inside the loop, you will also need to switch to interior mutability. This shifts the enforcement of the rules from compile time to run time, so you will now get a panic when you try to get two mutable references to the same item at the same time:
use std::cell::RefCell;
struct Element;
impl Element {
fn do_something(&mut self, _e: &mut Element) {}
}
fn main() {
let mut elements = [
RefCell::new(Element),
RefCell::new(Element),
RefCell::new(Element),
RefCell::new(Element),
];
for e in &elements {
for f in &elements {
// Note that this will panic as both `e` and `f`
// are the same value to start with
let mut e = e.borrow_mut();
let mut f = f.borrow_mut();
e.do_something(&mut f);
}
}
}
I have a HashMap<u32, u32> and I use it as an iterator:
for (key, mut value) in map {
if value == 0u32 {
value = 1u32;
}
println!("{}", value);
}
This compiles and prints out the expected values, but does it actually change the value in the map while I'm iterating it, or am I just shadowing the actual value and the value is written over after the iterator finishes?
Am I correct in assuming that mutating while iterating works with Rust iterators in general?
does it actually change the value in the map
No. In fact; the map no longer exists:
for (key, mut value) in map { /* ... */ }
println!("{:?}", map);
error[E0382]: use of moved value: `map`
--> src/main.rs:14:22
|
7 | for (key, mut value) in map {
| --- value moved here
...
14 | println!("{:?}", map);
| ^^^ value used here after move
|
= note: move occurs because `map` has type `std::collections::HashMap<u32, u32>`, which does not implement the `Copy` trait
You've transferred ownership of the map and all of the keys and values to the iterator. It's gone. Note that the map isn't even declared as mutable, so there's no way you could make changes to it.
If you wanted to modify all the values, you could:
let mut map: HashMap<u32, u32> = HashMap::new();
for (_key, value) in &mut map {
if *value == 0u32 {
*value = 1u32;
}
println!("{}", value);
}
Note how now we have a reference to the keys and values, not the value itself.
with Rust iterators in general
It depends on what iterators the item exposes. If there's an iterator of mutable references of some kind, then yes.
I would like to remove items from a BTreeMap which have been found through iteration.
As it is not possible to remove items while iterating, I put the items to delete into a vector. The main issue is that it is not possible to use a vector of references, but only a vector of values. All the keys for which the entry has to be removed must then be cloned (assuming the key implements the Clone trait).
For instance, this short sample does not compile:
use std::collections::BTreeMap;
pub fn clean() {
let mut map = BTreeMap::<String, i32>::new();
let mut to_delete = Vec::new();
{
for (k, v) in map.iter() {
if *v > 10 {
to_delete.push(k);
}
}
}
for k in to_delete.drain(..) {
map.remove(k);
}
}
fn main() {}
It generates the following errors when it is compiled:
error[E0502]: cannot borrow `map` as mutable because it is also borrowed as immutable
--> src/main.rs:17:9
|
9 | for (k, v) in map.iter() {
| --- immutable borrow occurs here
...
17 | map.remove(k);
| ^^^ mutable borrow occurs here
18 | }
19 | }
| - immutable borrow ends here
Changing to_delete.push(k) with to_delete.push(k.clone()) makes this snippet compile correctly but it is quite costly if each key to delete must be cloned.
Is there a better solution?
TL;DR: you cannot.
As far as the compiler is concerned, the implementation of BTreeMap::remove might do this:
pub fn remove<Q>(&mut self, key: &Q) -> Option<V>
where
K: Borrow<Q>,
Q: Ord + ?Sized,
{
// actual deleting code, which destroys the value in the set
// now what `value` pointed to is gone and `value` points to invalid memory
// And now we access that memory, causing undefined behavior
key.borrow();
}
The compiler thus has to prevent using the reference to the value when the collection will be mutated.
To do this, you'd need something like the hypothetical "cursor" API for collections. This would allow you to iterate over the collection, returning a special type that hold the mutable innards of the collection. This type could give you a reference to check against and then allow you to remove the item.
I'd probably look at the problem from a bit different direction. Instead of trying to keep the map, I'd just create a brand new one:
use std::collections::BTreeMap;
pub fn main() {
let mut map = BTreeMap::new();
map.insert("thief", 5);
map.insert("troll", 52);
map.insert("gnome", 7);
let map: BTreeMap<_, _> =
map.into_iter()
.filter(|&(_, v)| v <= 10)
.collect();
println!("{:?}", map); // troll is gone
}
If your condition is equality on the field that makes the struct unique (a.k.a is the only field used in PartialEq and Hash), you can implement Borrow for your type and directly grab it / delete it:
use std::collections::BTreeMap;
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)]
struct Monster(String);
use std::borrow::Borrow;
impl Borrow<str> for Monster {
fn borrow(&self) -> &str { &self.0 }
}
pub fn main() {
let mut map = BTreeMap::new();
map.insert(Monster("thief".into()), 5);
map.insert(Monster("troll".into()), 52);
map.insert(Monster("gnome".into()), 7);
map.remove("troll");
println!("{:?}", map); // troll is gone
}
See also:
How to implement HashMap with two keys?
How to avoid temporary allocations when using a complex key for a HashMap?
As of 1.53.0, there is a BTreeMap.retain method which allows to remove/retain items based on return value of a closure, while iterating:
map.retain(|_, v| *v<=10);