Editor's note: as of Rust 1.66.0, BTreeMap::pop_last has been stabilized.
In stable Rust 1.65 or older, is there a way to write a function equivalent to BTreeMap::pop_last?
The best I could come up with is:
fn map_pop_last<K, V>(m: &mut BTreeMap<K, V>) -> Option<(K, V)>
where
K: Ord + Clone,
{
let last = m.iter().next_back();
if let Some((k, _)) = last {
let k_copy = k.clone();
return m.remove_entry(&k_copy);
}
None
}
It works, but it requires that the key is cloneable. BTreeMap::pop_last from Rust nightly imposes no such constraint.
If I remove the cloning like this
fn map_pop_last<K, V>(m: &mut BTreeMap<K, V>) -> Option<(K, V)>
where
K: Ord,
{
let last = m.iter().next_back();
if let Some((k, _)) = last {
return m.remove_entry(k);
}
None
}
it leads to
error[E0502]: cannot borrow `*m` as mutable because it is also borrowed as immutable
--> ...
|
.. | let last = m.iter().next_back();
| -------- immutable borrow occurs here
.. | if let Some((k, _)) = last {
.. | return m.remove_entry(k);
| ^^------------^^^
| | |
| | immutable borrow later used by call
| mutable borrow occurs here
Is there a way to work around this issue without imposing additional constraints on map key and value types?
Is there a way to work around this issue without imposing additional constraints on map key and value types?
It doesn't appear doable in safe Rust, at least not with reasonable algorithmic complexity. (See Aiden4's answer for a solution that does it by re-building the whole map.)
But if you're allowed to use unsafe, and if you're determined enough that you want to delve into it, this code could do it:
// Safety: if key uses shared interior mutability, the comparison function
// must not use it. (E.g. it is not allowed to call borrow_mut() on a
// Rc<RefCell<X>> inside the key). It is extremely unlikely that such a
// key exists, but it's possible to write it, so this must be marked unsafe.
unsafe fn map_pop_last<K, V>(m: &mut BTreeMap<K, V>) -> Option<(K, V)>
where
K: Ord,
{
// We make a shallow copy of the key in the map, and pass a
// reference to the copy to BTreeMap::remove_entry(). Since
// remove_entry() is not dropping the key/value pair (it's returning
// it), our shallow copy will remain throughout the lifetime of
// remove_entry(), even if the key contains references.
let (last_key_ref, _) = m.iter().next_back()?;
let last_key_copy = ManuallyDrop::new(std::ptr::read(last_key_ref));
m.remove_entry(&last_key_copy)
}
Playground
Is there a way to work around this issue without imposing additional constraints on map key and value types?
You can't do it efficiently in safe rust, but it is possible:
fn map_pop_last<K, V>(m: &mut BTreeMap<K, V>) -> Option<(K, V)>
where
K: Ord,
{
let mut temp = BTreeMap::new();
std::mem::swap(m, &mut temp);
let mut iter = temp.into_iter();
let ret = iter.next_back();
m.extend(iter);
ret
}
playground
This will do a full traversal of the map but is safe.
We can even be sure that it is not possible to mutate the map in-place with the current offer of interfaces in safe Rust (version 1.59, at the time of writing):
To extract the key, we need to look the keys of the map. "Look at" implies borrowing here. This gives us a &K reference that also borrows the entire map.
Now, a problem arises: To call any of these remove* methods, we need to mutably borrow the map again, which we cannot do since we still hold the reference to the key that we are about to extract - these lifetimes overlap and it won't compile.
Another approach could be to try to get hold of the Entry, but to get one through the .entry method, we need have an owned K too.
Related
This should be a trivial task in any language. This isn't working in Rust.
use std::collections::HashMap;
fn do_it(map: &mut HashMap<String, String>) {
for (key, value) in map {
println!("{} / {}", key, value);
map.remove(key);
}
}
fn main() {}
Here's the compiler error:
error[E0382]: use of moved value: `*map`
--> src/main.rs:6:9
|
4 | for (key, value) in map {
| --- value moved here
5 | println!("{} / {}", key, value);
6 | map.remove(key);
| ^^^ value used here after move
|
= note: move occurs because `map` has type `&mut std::collections::HashMap<std::string::String, std::string::String>`, which does not implement the `Copy` trait
Why it is trying to move a reference? From the documentation, I didn't think moving/borrowing applied to references.
There are at least two reasons why this is disallowed:
You would need to have two concurrent mutable references to map — one held by the iterator used in the for loop and one in the variable map to call map.remove.
You have references to the key and the value within the map when trying to mutate the map. If you were allowed to modify the map in any way, these references could be invalidated, opening the door for memory unsafety.
A core Rust principle is Aliasing XOR Mutability. You can have multiple immutable references to a value or you can have a single mutable reference to it.
I didn't think moving/borrowing applied to references.
Every type is subject to Rust's rules of moving as well as mutable aliasing. Please let us know what part of the documentation says it isn't so we can address that.
Why it is trying to move a reference?
This is combined of two parts:
You can only have a single mutable reference, so mutable references don't implement the Copy trait
for loops take the value to iterate over by value
When you call for (k, v) in map {}, the ownership of map is transferred to the for loop and is now gone.
I'd perform an immutable borrow of the map (&*map) and iterate over that. At the end, I'd clear the whole thing:
fn do_it(map: &mut HashMap<String, String>) {
for (key, value) in &*map {
println!("{} / {}", key, value);
}
map.clear();
}
remove every value with a key that starts with the letter "A"
I'd use HashMap::retain:
fn do_it(map: &mut HashMap<String, String>) {
map.retain(|key, value| {
println!("{} / {}", key, value);
!key.starts_with("a")
})
}
This guarantees that key and value no longer exist when the map is actually modified, thus any borrow that they would have had is now gone.
This should be a trivial task in any language.
Rust is preventing you from mutating the map while you are iterating over it. In most languages this is allowed, but often the behaviour is not well-defined, and removal of the item can interfere with the iteration, compromising its correctness.
Why it is trying to move a reference?
HashMap implements IntoIterator, so your loop is equivalent to:
for (key, value) in map.into_iter() {
println!("{} / {}", key, value);
map.remove(key);
}
If you look at the definition of into_iter, you'll see that it takes self, not &self or &mut self. Your variable map is a mutable reference, and IntoIterator is implemented for &mut HashMap - the self in into_iter is the &mut HashMap, not the HashMap. Mutable references cannot be copied (since only one mutable reference to any data can exist at one time) so this mutable reference is moved.
The API is intentionally built that way so that you can't do anything dangerous while looping over a structure. Once the loop is complete, the ownership of the structure is relinquished and you can use it again.
One solution is to keep track of the items you intend to remove in a Vec and then remove them afterwards:
fn do_it(map: &mut HashMap<String, String>) {
let mut to_remove = Vec::new();
for (key, value) in &*map {
if key.starts_with("A") {
to_remove.push(key.to_owned());
}
}
for key in to_remove.iter() {
map.remove(key);
}
}
You may also use an iterator to filter the map into a new one. Perhaps something like this:
fn do_it(map: &mut HashMap<String, String>) {
*map = map.into_iter().filter_map(|(key, value)| {
if key.starts_with("A") {
None
} else {
Some((key.to_owned(), value.to_owned()))
}
}).collect();
}
But I just saw Shepmaster's edit - I had forgotten about retain, which is better. It's more concise and doesn't do unnecessary copying as I have done.
Rust actually supports a wide number of potential solutions to this problem, though I myself also found the situation to be a bit confusing at first, and again each time I need a more complicated treatment of my hashmaps.
To iterate through items while removing them, use .drain(). .drain() has the advantage of taking/owning rather than borrowing the values.
If you want to only conditionally remove some of them, use .drain_filter().
If you need to mutate every item but only want to remove some of them, you may mutate them in .drain_filter()'s closure argument, but this will check for removal after the changes.
If you need to check for removal before the changes, use a variable to store the result of the check, then return that variable at the end. A very slightly slower but maybe more clear alternative is just to mutate them in one for loop, then .drain_filter() them in another for loop or a map.
You may also simply allow the hashmap to drop at the end of the function by not borrowing it in the function argument, and initialize a new one if you need it. This completely removes the hashmap, obviously. Obviously you might want to keep the hashmap around so you don't reinitialize it over and over.
You may also call .clear() to remove all elements, after you're done iterating through them to print them.
use std::cell::RefCell;
use std::rc::Weak;
struct Elem {
attached_elem: Weak<RefCell<Elem>>,
value: i32,
}
impl Elem {
fn borrow_mut_attached_elem(&self) -> &mut Elem {
//what should this line be?
self.attached_elem.upgrade().unwrap().borrow_mut()
}
}
fn main(){}
I have tried some similar other lines but nothing has worked so far, even the experimental cell_leak feature for RefMut.
I don't mind changing the signature of the function I just want to reduce the overhead of getting a mutable reference to the attached_elem from an Elem.
what should this line be?
There is nothing you can put in that line to (safely) satisfy the function signature - and that's for good reason. While RefCell does allow obtaining &mut T from a RefCell<T> (that's why it exists), it must guarantee that only one mutable reference exist at a time. It does so by only providing a temporary reference whose lifetime is tied to the RefMut<T> wrapper. Once the wrapper is dropped, the value is marked as no longer borrowed, so the reference must not outlive it.
If Rust were to allow you to return a naked &mut Elem, you'd be able to use the reference after the RefCell has ceased being marked as borrowed. In that case, what's to stop you from calling borrow_mut_attached_elem() again, and obtain a second mutable reference to the same Elem?
So you'll definitely need to change the signature. If you just need to give outside code temporary access to &mut Elem, the easiest way is to accept a closure that will receive it. For example:
fn with_attached_elem<R>(&self, f: impl FnOnce(&mut Elem) -> R) -> R {
let rc = self.attached_elem.upgrade().unwrap();
let retval = f(&mut *rc.borrow_mut());
retval
}
You'd call it to do something with the element, e.g.:
elem.with_attached_elem(|e| e.value += 1);
with_attached_elem takes care to return the value returned by the closure, allowing you to collect data from &mut Elem and propagate it to the caller. For example, to pick up the value of the attached element you could use:
let value = elem.with_attached_elem(|e| e.value);
An example of how a range gets consumed is:
let coll = 1..10;
for i in coll {
println!("i is {}", &i);
}
println!("coll length is {}", coll.len());
This will fail with
error[E0382]: borrow of moved value: `coll`
--> src/main.rs:6:35
|
2 | let coll = 1..10;
| ---- move occurs because `coll` has type `std::ops::Range<i32>`, which does not implement the `Copy` trait
3 | for i in coll {
| ----
| |
| `coll` moved due to this implicit call to `.into_iter()`
| help: consider borrowing to avoid moving into the for loop: `&coll`
...
6 | println!("coll length is {}", coll.len());
| ^^^^ value borrowed here after move
|
note: this function consumes the receiver `self` by taking ownership of it, which moves `coll`
The usual way to fix this is to borrow the coll, but that doesn't work here:
error[E0277]: `&std::ops::Range<{integer}>` is not an iterator
--> src/main.rs:3:14
|
3 | for i in &coll {
| -^^^^
| |
| `&std::ops::Range<{integer}>` is not an iterator
| help: consider removing the leading `&`-reference
|
= help: the trait `std::iter::Iterator` is not implemented for `&std::ops::Range<{integer}>`
= note: required by `std::iter::IntoIterator::into_iter`
Why is that? Why is a borrowed range not an iterator, but the range is? Is it interpreting it differently?
To understand what is happening here it is helpful to understand how for loops work in Rust.
Basically a for loop is a short hand for using an iterator, so:
for item in some_value {
// ...
}
is basically a short-hand for
let mut iterator = some_value.into_iter();
while let Some(item) = iterator.next() {
// ... body of for loop here
}
So we can see that whatever we loop over with the for loop, Rust calls the into_iter method from the IntoIterator trait on. The IntoIterator trait looks (approximately) like this:
trait IntoIterator {
// ...
type IntoIter;
fn into_iter(self) -> Self::IntoIter;
}
So into_iter takes self by value and returns Self::IntoIter which is the type of the iterator. As Rust moves any arguments which are taken by value, the thing .into_iter() was called on is no longer available after the call (or after the for loop). That's why you can't use coll in your first code snippet.
So far so good, but why can we still use a collection if we loop over a reference of it as in the following?
for i in &collection {
// ...
}
// can still use collection here ...
The reason is that for a lot of collections C, the IntoIterator trait is implemented not just for the collection, but also for a shared reference to the collection &C and this implementation produces shared items. (Sometimes it is also implemented for mutable references &mut C which produces mutable references to items).
Now coming back to the example with the Range we can check how it implements IntoIterator.
Looking at the reference docs for Range, Range strangely does not seem to implement IntoIterator directly... but if we check the Blanket Implementations section on doc.rust-lang.org, we can see that every iterator implements the IntoIterator trait (trivially, by just returning itself):
impl<I> IntoIterator for I
where
I: Iterator
How does this help? Well, checking further up (under trait implementations) we see that Range does implement Iterator:
impl<A> Iterator for Range<A>
where
A: Step,
And thus Range does implement IntoIterator via the indirection of Iterator. However, there is no implementation of either Iterator for &Range<A> (this would be impossible) or of IntoIterator for &Range<A>. Therefore, we can use a for loop by passing Range by value, but not by reference.
Why can &Range not implement Iterator? An iterator needs to keep track of "where it is", which requires some kind of mutation, but we cannot mutate a &Range because we only have a shared reference. So this cannot work. (Note that &mut Range can and does implement Iterator - more on this later).
It would technically be possible to implement IntoIterator for &Range as that could produce a new iterator. But the likelihood that this would clash with the blanket iterator implementation of Range would be very high and things would be even more confusing. Besides, a Range is at most two integers and copying this is very cheap, so there is really no big value in implementing IntoIterator for &Range.
If you still want to use collection, you can clone it
for i in coll.clone() { /* ... */ }
// `coll` still available as the for loop used the clone
This brings up another question: If we can clone the range and it is (as claimed above) cheap to copy it, why doesn't Range implement the Copy trait? Then the .into_iter() call would copy the range coll (instead of moving it) and it could still be used after the loop. According to this PR the Copy trait implementation actually existed but was removed because the following was considered a footgun (hat tip to Michael Anderson for pointing this out):
let mut iter = 1..10;
for i in iter {
if i > 2 { break; }
}
// This doesn't work now, but if `Range` implemented copy,
// it would produce `[1,2,3,4,5,6,7,8,9]` instead of
// `[4,5,6,7,8,9]` as might have been expected
let v: Vec<_> = iter.collect();
Also note that &mut Range does implement iterator, so you can do
let mut iter = 1..10;
for i in &mut iter {
if i > 2 { break; }
}
// `[4,5,6,7,8,9]` as expected
let v: Vec<_> = iter.collect();
Finally, for completeness, it might be instructive to see which methods are actually called when we loop over a Range:
for item in 1..10 { /* ... */ }
is translated to
let mut iter = 1..10.into_iter();
// ˆˆˆˆˆˆˆˆˆ--- which into_iter() is this?
while let Some(item) = iter.next() { /* ... */ }
we can make this explicit using qualified method syntax:
let mut iter = std::iter::Iterator::into_iter(1..10);
// it's `Iterator`s method! ------^^^^^^^^^
while let Some(item) = iter.next() { /* ... */ }
Ranges are iterators that modify themselves to generate elements. Therefore, to loop over a range, it is necessary to modify it (or a copy of it, as shown below).
Vectors, on the other hand, are not iterators themselves. .into_iter() is called to create an iterator when a vector is looped over; the vector itself doesn't need to be consumed.
The solution here is to use clone to create a new iterator that can be looped over:
for i in coll.clone() {
println!("i is {}", i);
}
(Incidentally, the println! family of macros take references automatically.)
Let's say you have a vector:
let v = vec![1, 2, 3];
The method iter on Vec returns something that implements the Iterator trait. With a vector, there is also an implementation of the trait Borrow (and BorrowMut), that does not return a &Vec though. Instead, you get a slice &[T]. This slice can then be used to iterate over the elements of the vector.
However, the range (e.g. 1..10) implements IntoIterator already and does not need to be transformed into a slice or some other view into it. Therefore, you can consume the range itself by calling into_iter() (which you do implicitly). Now, it is as if you moved the range into some function and you cannot use your variable coll anymore. The borrowing syntax won't help, since this is only some special functionality of Vec.
In this case, you could construct a Vec from your range (with the collect method), clone the range when iterating over it or get the length before iterating (since getting the length doesn't consume the range itself).
Some references:
https://doc.rust-lang.org/std/vec/struct.Vec.html
https://doc.rust-lang.org/std/primitive.slice.html
https://doc.rust-lang.org/std/ops/struct.Range.html
I'm trying to modify the borrow of a mutable value, here is a minimal example:
fn main() {
let mut w: [char; 5] = ['h', 'e', 'l', 'l', 'o'];
let mut wslice: &mut [char] = &mut w;
advance_slice(&mut wslice);
advance_slice(&mut wslice);
}
fn advance_slice(s: &mut &mut [char]) {
let new: &mut [char] = &mut s[1..];
*s = new;
}
the compiler gives me this error:
error[E0623]: lifetime mismatch
--> src/main.rs:10:10
|
8 | fn advance_slice(s: &mut &mut [char]) {
| ----------------
| |
| these two types are declared with different lifetimes...
9 | let new: &mut [char] = &mut s[1..];
10 | *s = new;
| ^^^ ...but data from `s` flows into `s` here
I've tried to give the same lifetime to both borrow, without success.
Also this works if I remove the mutability of w.
This error message is indeed unfortunate, and I don't think it provides a good explanation of what's going on here. The issue is a bit involved.
The error is local to the advance_slice() function, so that's all we need to look at. The type of the slice items is irrelevant, so let's take this function definition:
fn advance_slice_mut<T>(s: &mut &mut [T]) {
let new = &mut s[1..];
*s = new;
}
The first line creates a new slice object starting after the first item of the original slice.
Why is this even allowed? Don't we have two mutable references to the same data now? The original slice *s includes the new slice new, and both allow modifying data. The reason this is legal is that *s is implicitly reborrowed when creating the subslice, and *s cannot be used again for the lifetime of that borrow, so we still have only one active reference to the data in the subslice. The reborrow is scoped to the function advance_slice_mut(), and thus has a shorter lifetime than the original slice, which is the root cause of the error you get – you are effectively trying to assign a slice which only lives until the end of the function to a memory location that lives longer than the function call.
This kind of implicit reborrow happens everytime you call a function that takes an argument by mutable reference, including in the implicit call to index_mut() in &mut s[1..]. Mutable references can't be copied, since this would create two mutable references to the same memory, and the Rust language designers decided that an implicit reborrow is the more ergonomic solution than moving mutable references by default. The reborrow does not happen for shared references, though, since they can be freely copied. This means that &s[1..] will have the same lifetime as the original scope, since it is perfectly fine for two overlapping immutable slices to coexist. This explains why your function definition works fine for immutable slices.
So how do we fix this problem? I believe that what you intend to do is perfectly safe – after reassigning the new slice to the old one, the old one is gone, so we don't have two concurrent mutable references to the same memory. To create a slice with the same lifetime as the original slice in safe code, we need to move the original slice out of the reference we got. We can do this by replacing it with an empty slice:
pub fn advance_slice_mut<T>(s: &mut &mut [T]) {
let slice = std::mem::replace(s, &mut []);
*s = &mut slice[1..];
}
Alternatively, we can also resort to unsafe code:
use std::slice::from_raw_parts_mut;
pub fn advance_slice_mut<T>(s: &mut &mut [T]) {
unsafe {
assert!(!s.is_empty());
*s = from_raw_parts_mut(s.as_mut_ptr().add(1), s.len() - 1);
}
}
I'm trying to implement Dijkstra's algorithm in Rust. I am getting an error because I'm mutating a HashMap inside a match expression on a value obtained from .get(). I understand why this is a violation of the borrowing and mutation principles of Rust, but I haven't found a workaround.
I have tried using .entry() and .and_modify() to perform an in-place modification, but I also need to insert and/or modify other keys besides the one being matched on.
Are there any constructs/functions I can use to safely enable this mutation within a get, or, failing that, is there another approach to this algorithm that steers clear of this issue?
(Unfinished) Code Snippet:
use std::collections::{HashMap, HashSet};
struct Graph {
vertices: Vec<i32>,
edges: HashMap<i32, HashMap<i32, i32>>, // start -> {end -> weight}
}
fn dijkstra(start: i32, g: Graph) -> HashMap<i32, HashMap<i32, (i32, Vec<i32>)>> {
let mut known_paths: HashMap<i32, (i32, Vec<i32>)> = HashMap::new(); // end -> (total_weight, path)
known_paths.insert(start, (0, Vec::new()));
let mut current = &start;
let mut unvisited: HashSet<i32> = g.edges.keys().cloned().collect();
while unvisited.len() > 0 {
match known_paths.get(current) {
Some((current_dist, current_path)) => if let Some(ref incident_edges) =
g.edges.get(current)
{
for (neighbor, dist) in incident_edges.iter() {
match known_paths.get(&neighbor) {
Some((old_distance, _old_path)) => if current_dist + dist < *old_distance {
let mut new_path = current_path.clone();
new_path.push(*current);
known_paths.insert(*neighbor, (current_dist + dist, new_path));
},
None => {
let mut new_path = current_path.clone();
new_path.push(*current);
known_paths.insert(*neighbor, (current_dist + dist, new_path));
}
}
}
},
None => panic!("Something went wrong with current={}", current),
}
}
HashMap::new()
}
Error:
error[E0502]: cannot borrow `known_paths` as mutable because it is also borrowed as immutable
--> src/lib.rs:26:29
|
17 | match known_paths.get(current) {
| ----------- immutable borrow occurs here
...
26 | known_paths.insert(*neighbor, (current_dist + dist, new_path));
| ^^^^^^^^^^^ mutable borrow occurs here
...
37 | }
| - immutable borrow ends here
error[E0502]: cannot borrow `known_paths` as mutable because it is also borrowed as immutable
--> src/lib.rs:31:29
|
17 | match known_paths.get(current) {
| ----------- immutable borrow occurs here
...
31 | known_paths.insert(*neighbor, (current_dist + dist, new_path));
| ^^^^^^^^^^^ mutable borrow occurs here
...
37 | }
| - immutable borrow ends here
No.
I am getting an error because I'm mutating a HashMap inside a match expression on a value obtained from .get(). I understand why this is a violation of the borrowing and mutation principles of Rust, but I haven't found a workaround.
I am afraid you have not actually understood the borrowing principles.
The key principle which underpins Rust's safety is: Mutation NAND Aliasing.
This principle is then enforced either at run-time or at compile-time, with a strong preference for compile-time when feasible since it is free of run-time overhead.
In your situation, you have:
A reference inside HashMap, obtained from HashMap::get().
You attempt to modify said HashMap while holding onto the reference.
Let's imagine, for a second, that some constructs allows this code to compile and run; what would happen? BOOM.
When inserting an element in the HashMap, the HashMap may:
Shuffle existing elements, since it is using Robin Hood hashing.
Transfer all elements to another (larger) heap-allocated array.
Therefore, HashMap::insert means that any existing reference to an element of the HashMap may become dangling and point to either another element, or random memory.
There is no safe way to insert into the HashMap while holding a reference to an element of the HashMap.
You need to find a solution which does not involve keeping a reference to the HashMap.
My advice would be to simply clone: match known_paths.get(current).clone(). Once you have a first working implementation of the algorithm, you can look into possibly improving its performance, as necessity dictates.