I'm learning Rust and I'm trying to cargo-cult this code into compiling:
use std::vec::Vec;
use std::collections::BTreeMap;
struct Occ {
docnum: u64,
weight: f32,
}
struct PostWriter<'a> {
bytes: Vec<u8>,
occurrences: BTreeMap<&'a [u8], Vec<Occ>>,
}
impl<'a> PostWriter<'a> {
fn new() -> PostWriter<'a> {
PostWriter {
bytes: Vec::new(),
occurrences: BTreeMap::new(),
}
}
fn add_occurrence(&'a mut self, term: &[u8], occ: Occ) {
let occurrences = &mut self.occurrences;
match occurrences.get_mut(term) {
Some(x) => x.push(occ),
None => {
// Add the term bytes to the big vector of all terms
let termstart = self.bytes.len();
self.bytes.extend(term);
// Create a new occurrences vector
let occs = vec![occ];
// Take the appended term as a slice to use as a key
// ERROR: cannot borrow `*occurrences` as mutable more than once at a time
occurrences.insert(&self.bytes[termstart..], occs);
}
}
}
}
fn main() {}
I get an error:
error[E0499]: cannot borrow `*occurrences` as mutable more than once at a time
--> src/main.rs:34:17
|
24 | match occurrences.get_mut(term) {
| ----------- first mutable borrow occurs here
...
34 | occurrences.insert(&self.bytes[termstart..], occs);
| ^^^^^^^^^^^ second mutable borrow occurs here
35 | }
36 | }
| - first borrow ends here
I don't understand... I'm just calling a method on a mutable reference, why would that line involve borrowing?
I'm just calling a method on a mutable reference, why would that line involve borrowing?
When you call a method on an object that's going to mutate the object, you can't have any other references to that object outstanding. If you did, your mutation could invalidate those references and leave your program in an inconsistent state. For example, say that you had gotten a value out of your hashmap and then added a new value. Adding the new value hits a magic limit and forces memory to be reallocated, your value now points off to nowhere! When you use that value... bang goes the program!
In this case, it looks like you want to do the relatively common "append or insert if missing" operation. You will want to use entry for that:
use std::collections::BTreeMap;
fn main() {
let mut map = BTreeMap::new();
{
let nicknames = map.entry("joe").or_insert(Vec::new());
nicknames.push("shmoe");
// Using scoping to indicate that we are done with borrowing `nicknames`
// If we didn't, then we couldn't borrow map as
// immutable because we could still change it via `nicknames`
}
println!("{:?}", map)
}
Because you're calling a method that borrows as mutable
I had a similar question yesterday about Hash, until I noticed something in the docs. The docs for BTreeMap show a method signature for insert starting with fn insert(&mut self..
So when you call .insert, you're implicitly asking that function to borrow the BTreeMap as mutable.
Related
I have a struct which maps ids to indices and vice versa.
struct IdMapping<'a> {
external_2_internal: HashMap<&'a str, usize>,
internal_2_external: HashMap<usize, String>,
}
impl<'a> IdMapping<'a> {
fn new() -> IdMapping<'a> {
IdMapping {
external_2_internal: HashMap::new(),
internal_2_external: HashMap::new(),
}
}
fn insert(&'a mut self, internal: usize, external: String) {
self.internal_2_external.insert(internal, external);
let mapped_external = self.internal_2_external.get(&internal).unwrap();
self.external_2_internal.insert(mapped_external, internal);
}
}
If I am using this structure the following way
fn map_ids<'a>(ids: Vec<String>) -> IdMapping<'a> {
let mut mapping = IdMapping::new();
for (i, id) in ids.iter().enumerate() {
mapping.insert(i, id.clone());
}
mapping
}
I receive the following compiler error:
error[E0499]: cannot borrow `mapping` as mutable more than once at a time
--> src/lib.rs:28:9
|
24 | fn map_ids<'a>(ids: Vec<String>) -> IdMapping<'a> {
| -- lifetime `'a` defined here
...
28 | mapping.insert(i, id.clone());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `mapping` was mutably borrowed here in the previous iteration of the loop
...
31 | mapping
| ------- returning this value requires that `mapping` is borrowed for `'a`
Playground link
Why can't I mutably borrow the mapping for its insert method each iteration of the loop? How should I implement this use case?
The problem is due to the self reference in your struct.
Let's first look at whether this is theoretically sound (assuming we're writing unsafe code):
HashMap uses a flat array (quadratic probing), so objects in the HashMap aren't address stable under insertion of a new element. This means that an insertion into internal_2_external may move the existing Strings around in memory.
String stores its content in a separate heap allocation, the heap allocation remains at the same location. So even if the String is moved around, a &str referencing it will remain pointing to valid memory.
So this would actually work if implemented in unsafe code.
While it's logically sound to do those operations, the type system is unable to recognise that you can move a String while keeping borrowed ranges of it valid. This means that if you borrow a &str from a String, the type system will prevent you from doing any mutation operation on your String, including moving it. As such, you can't do any mutation operation on the hash map either, giving rise to your error.
I can see two safe ways to work around this:
Make external_2_internal keep its own copy of the strings, i.e. useHashMap<String, usize>.
Keep the strings in a separate Vec:
struct IdMapping {
strings: Vec<String>,
external_2_internal: HashMap<usize /* index */, usize>,
internal_2_external: HashMap<usize, usize /* index */>,
}
Alternatively you could work with unsafe code, if you don't want to change the layout of your struct.
Running into an ownership issue when attempting to reference multiple values from a HashMap in a struct as parameters in a function call. Here is a PoC of the issue.
use std::collections::HashMap;
struct Resource {
map: HashMap<String, String>,
}
impl Resource {
pub fn new() -> Self {
Resource {
map: HashMap::new(),
}
}
pub fn load(&mut self, key: String) -> &mut String {
self.map.get_mut(&key).unwrap()
}
}
fn main() {
// Initialize struct containing a HashMap.
let mut res = Resource {
map: HashMap::new(),
};
res.map.insert("Item1".to_string(), "Value1".to_string());
res.map.insert("Item2".to_string(), "Value2".to_string());
// This compiles and runs.
let mut value1 = res.load("Item1".to_string());
single_parameter(value1);
let mut value2 = res.load("Item2".to_string());
single_parameter(value2);
// This has ownership issues.
// multi_parameter(value1, value2);
}
fn single_parameter(value: &String) {
println!("{}", *value);
}
fn multi_parameter(value1: &mut String, value2: &mut String) {
println!("{}", *value1);
println!("{}", *value2);
}
Uncommenting multi_parameter results in the following error:
28 | let mut value1 = res.load("Item1".to_string());
| --- first mutable borrow occurs here
29 | single_parameter(value1);
30 | let mut value2 = res.load("Item2".to_string());
| ^^^ second mutable borrow occurs here
...
34 | multi_parameter(value1, value2);
| ------ first borrow later used here
It would technically be possible for me to break up the function calls (using the single_parameter function approach), but it would be more convenient to pass the
variables to a single function call.
For additional context, the actual program where I'm encountering this issue is an SDL2 game where I'm attempting to pass multiple textures into a single function call to be drawn, where the texture data may be modified within the function.
This is currently not possible, without resorting to unsafe code or interior mutability at least. There is no way for the compiler to know if two calls to load will yield mutable references to different data as it cannot always infer the value of the key. In theory, mutably borrowing both res.map["Item1"] and res.map["Item2"] would be fine as they would refer to different values in the map, but there is no way for the compiler to know this at compile time.
The easiest way to do this, as already mentioned, is to use a structure that allows interior mutability, like RefCell, which typically enforces the memory safety rules at run-time before returning a borrow of the wrapped value. You can also work around the borrow checker in this case by dealing with mut pointers in unsafe code:
pub fn load_many<'a, const N: usize>(&'a mut self, keys: [&str; N]) -> [&'a mut String; N] {
// TODO: Assert that keys are distinct, so that we don't return
// multiple references to the same value
keys.map(|key| self.load(key) as *mut _)
.map(|ptr| unsafe { &mut *ptr })
}
Rust Playground
The TODO is important, as this assertion is the only way to ensure that the safety invariant of only having one mutable reference to any value at any time is upheld.
It is, however, almost always better (and easier) to use a known safe interior mutation abstraction like RefCell rather than writing your own unsafe code.
I have to iterate on keys, find the value in HashMap by key, possibly do some heavy computation in the found struct as a value (lazy => mutate the struct) and cached return it in Rust.
I'm getting the following error message:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:25:26
|
23 | fn it(&mut self) -> Option<&Box<Calculation>> {
| - let's call the lifetime of this reference `'1`
24 | for key in vec!["1","2","3"] {
25 | let result = self.find(&key.to_owned());
| ^^^^ `*self` was mutably borrowed here in the previous iteration of the loop
...
28 | return result
| ------ returning this value requires that `*self` is borrowed for `'1`
Here is the code in playground.
use std::collections::HashMap;
struct Calculation {
value: Option<i32>
}
struct Struct {
items: HashMap<String, Box<Calculation>> // cache
}
impl Struct {
fn find(&mut self, key: &String) -> Option<&Box<Calculation>> {
None // find, create, and/or calculate items
}
fn it(&mut self) -> Option<&Box<Calculation>> {
for key in vec!["1","2","3"] {
let result = self.find(&key.to_owned());
if result.is_some() {
return result
}
}
None
}
}
I can't avoid the loop as I have to check multiple keys
I have to make it mutable (self and the structure) as the possible calculation changes it
Any suggestion on how to change the design (as Rust forces to think in a bit different way that makes sense) or work around it?
PS. There are some other issues with the code, but let's split the problems and solve this one first.
You can't do caching with exclusive access. You can't treat Rust references like general-purpose pointers (BTW: &String and &Box<T> are double indirection, and very unidiomatic in Rust. Use &str or &T for temporary borrows).
&mut self means not just mutable, but exclusive and mutable, so your cache supports returning only one item, because the reference it returns has to keep self "locked" for as long as it exists.
You need to convince the borrow checker that the thing that find returns won't suddenly disappear next time you call it. Currently there's no such guarantee, because the interface doesn't stop you from calling e.g. items.clear() (borrow checker checks what function's interface allows, not what function actually does).
You can do that either by using Rc, or using a crate that implements a memory pool/arena.
struct Struct {
items: HashMap<String, Rc<Calculation>>,
}
fn find(&mut self, key: &str) -> Rc<Calculation>
This way if you clone the Rc, it will live for as long as it needs, independently of the cache.
You can also make it nicer with interior mutability.
struct Struct {
items: RefCell<HashMap<…
}
This will allow your memoizing find method to use a shared borrow instead of an exclusive one:
fn find(&self, key: &str) -> …
which is much easier to work with for the callers of the method.
Probably not the cleanest way to do that, but it compiles. The idea is not to store the value found in a temporary result, to avoid aliasing: if you store the result, self is kept borrowed.
impl Struct {
fn find(&mut self, key: &String) -> Option<&Box<Calculation>> {
None
}
fn it(&mut self) -> Option<&Box<Calculation>> {
for key in vec!["1","2","3"] {
if self.find(&key.to_owned()).is_some() {
return self.find(&key.to_owned());
}
}
None
}
}
I had similar issues, and I found a workaround by turning the for loop into a fold, which convinced the compiler that self was not mutably borrowed twice.
It works without using interior mutability or duplicated function call; the only downside is that if the result was found early, it will not short-circuit but continue iterating until the end.
Before:
for key in vec!["1","2","3"] {
let result = self.find(&key.to_owned());
if result.is_some() {
return result
}
}
After:
vec!["1", "2,", "3"]
.iter()
.fold(None, |result, key| match result {
Some(result) => Some(result),
None => self.find(&key.to_string())
})
Working playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=92bc73e4bac556ce163e0790c7d3f154
I would like to remove items from a BTreeMap which have been found through iteration.
As it is not possible to remove items while iterating, I put the items to delete into a vector. The main issue is that it is not possible to use a vector of references, but only a vector of values. All the keys for which the entry has to be removed must then be cloned (assuming the key implements the Clone trait).
For instance, this short sample does not compile:
use std::collections::BTreeMap;
pub fn clean() {
let mut map = BTreeMap::<String, i32>::new();
let mut to_delete = Vec::new();
{
for (k, v) in map.iter() {
if *v > 10 {
to_delete.push(k);
}
}
}
for k in to_delete.drain(..) {
map.remove(k);
}
}
fn main() {}
It generates the following errors when it is compiled:
error[E0502]: cannot borrow `map` as mutable because it is also borrowed as immutable
--> src/main.rs:17:9
|
9 | for (k, v) in map.iter() {
| --- immutable borrow occurs here
...
17 | map.remove(k);
| ^^^ mutable borrow occurs here
18 | }
19 | }
| - immutable borrow ends here
Changing to_delete.push(k) with to_delete.push(k.clone()) makes this snippet compile correctly but it is quite costly if each key to delete must be cloned.
Is there a better solution?
TL;DR: you cannot.
As far as the compiler is concerned, the implementation of BTreeMap::remove might do this:
pub fn remove<Q>(&mut self, key: &Q) -> Option<V>
where
K: Borrow<Q>,
Q: Ord + ?Sized,
{
// actual deleting code, which destroys the value in the set
// now what `value` pointed to is gone and `value` points to invalid memory
// And now we access that memory, causing undefined behavior
key.borrow();
}
The compiler thus has to prevent using the reference to the value when the collection will be mutated.
To do this, you'd need something like the hypothetical "cursor" API for collections. This would allow you to iterate over the collection, returning a special type that hold the mutable innards of the collection. This type could give you a reference to check against and then allow you to remove the item.
I'd probably look at the problem from a bit different direction. Instead of trying to keep the map, I'd just create a brand new one:
use std::collections::BTreeMap;
pub fn main() {
let mut map = BTreeMap::new();
map.insert("thief", 5);
map.insert("troll", 52);
map.insert("gnome", 7);
let map: BTreeMap<_, _> =
map.into_iter()
.filter(|&(_, v)| v <= 10)
.collect();
println!("{:?}", map); // troll is gone
}
If your condition is equality on the field that makes the struct unique (a.k.a is the only field used in PartialEq and Hash), you can implement Borrow for your type and directly grab it / delete it:
use std::collections::BTreeMap;
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)]
struct Monster(String);
use std::borrow::Borrow;
impl Borrow<str> for Monster {
fn borrow(&self) -> &str { &self.0 }
}
pub fn main() {
let mut map = BTreeMap::new();
map.insert(Monster("thief".into()), 5);
map.insert(Monster("troll".into()), 52);
map.insert(Monster("gnome".into()), 7);
map.remove("troll");
println!("{:?}", map); // troll is gone
}
See also:
How to implement HashMap with two keys?
How to avoid temporary allocations when using a complex key for a HashMap?
As of 1.53.0, there is a BTreeMap.retain method which allows to remove/retain items based on return value of a closure, while iterating:
map.retain(|_, v| *v<=10);
I would like to remove items from a BTreeMap which have been found through iteration.
As it is not possible to remove items while iterating, I put the items to delete into a vector. The main issue is that it is not possible to use a vector of references, but only a vector of values. All the keys for which the entry has to be removed must then be cloned (assuming the key implements the Clone trait).
For instance, this short sample does not compile:
use std::collections::BTreeMap;
pub fn clean() {
let mut map = BTreeMap::<String, i32>::new();
let mut to_delete = Vec::new();
{
for (k, v) in map.iter() {
if *v > 10 {
to_delete.push(k);
}
}
}
for k in to_delete.drain(..) {
map.remove(k);
}
}
fn main() {}
It generates the following errors when it is compiled:
error[E0502]: cannot borrow `map` as mutable because it is also borrowed as immutable
--> src/main.rs:17:9
|
9 | for (k, v) in map.iter() {
| --- immutable borrow occurs here
...
17 | map.remove(k);
| ^^^ mutable borrow occurs here
18 | }
19 | }
| - immutable borrow ends here
Changing to_delete.push(k) with to_delete.push(k.clone()) makes this snippet compile correctly but it is quite costly if each key to delete must be cloned.
Is there a better solution?
TL;DR: you cannot.
As far as the compiler is concerned, the implementation of BTreeMap::remove might do this:
pub fn remove<Q>(&mut self, key: &Q) -> Option<V>
where
K: Borrow<Q>,
Q: Ord + ?Sized,
{
// actual deleting code, which destroys the value in the set
// now what `value` pointed to is gone and `value` points to invalid memory
// And now we access that memory, causing undefined behavior
key.borrow();
}
The compiler thus has to prevent using the reference to the value when the collection will be mutated.
To do this, you'd need something like the hypothetical "cursor" API for collections. This would allow you to iterate over the collection, returning a special type that hold the mutable innards of the collection. This type could give you a reference to check against and then allow you to remove the item.
I'd probably look at the problem from a bit different direction. Instead of trying to keep the map, I'd just create a brand new one:
use std::collections::BTreeMap;
pub fn main() {
let mut map = BTreeMap::new();
map.insert("thief", 5);
map.insert("troll", 52);
map.insert("gnome", 7);
let map: BTreeMap<_, _> =
map.into_iter()
.filter(|&(_, v)| v <= 10)
.collect();
println!("{:?}", map); // troll is gone
}
If your condition is equality on the field that makes the struct unique (a.k.a is the only field used in PartialEq and Hash), you can implement Borrow for your type and directly grab it / delete it:
use std::collections::BTreeMap;
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)]
struct Monster(String);
use std::borrow::Borrow;
impl Borrow<str> for Monster {
fn borrow(&self) -> &str { &self.0 }
}
pub fn main() {
let mut map = BTreeMap::new();
map.insert(Monster("thief".into()), 5);
map.insert(Monster("troll".into()), 52);
map.insert(Monster("gnome".into()), 7);
map.remove("troll");
println!("{:?}", map); // troll is gone
}
See also:
How to implement HashMap with two keys?
How to avoid temporary allocations when using a complex key for a HashMap?
As of 1.53.0, there is a BTreeMap.retain method which allows to remove/retain items based on return value of a closure, while iterating:
map.retain(|_, v| *v<=10);