Passing both &mut self and &self to the same function - rust

Stripped down to the bare essentials, my problematic code looks as follows:
pub struct Item;
impl Item {
/// Partial copy. Not the same as simple assignment.
pub fn copy_from(&mut self, _other: &Item) {
}
}
pub struct Container {
items: Vec<Item>,
}
impl Container {
pub fn copy_from(&mut self, self_idx: usize, other: &Container, other_idx: usize) {
self.items[self_idx].copy_from(&other.items[other_idx]);
}
}
fn main() {
let mut container = Container { items: vec![Item, Item] };
container.copy_from(0, &container, 1);
}
This is of course rejected by the borrow checker:
error[E0502]: cannot borrow `container` as mutable because it is also borrowed as immutable
--> src/main.rs:21:5
|
21 | container.copy_from(0, &container, 1);
| ^^^^^^^^^^---------^^^^----------^^^^
| | | |
| | | immutable borrow occurs here
| | immutable borrow later used by call
| mutable borrow occurs here
For more information about this error, try `rustc --explain E0502`.
I understand why that happens, but I don't have a good solution.
I've considered adding a dedicated copy_from_self function that callers need to use in cases where self == other:
pub fn copy_from_self(&mut self, to_idx: usize, from_idx: usize) {
if to_idx != from_idx {
unsafe {
let from_item: *const Item = &self.items[from_idx];
self.items[to_idx].copy_from(&*from_item);
}
}
}
But this is un-ergonomic, bloats the API surface, and needs unsafe code inside.
Note that in reality, the internal items data structure is not a simple Vec, so any approach specific to Vec or slice will not work.
Is there an elegant, idiomatic solution to this problem?

If I understand the comments on the question correctly, a general solution seems to be impossible, so this answer is necessarily specific to my actual situation.
As mentioned, the actual data structure is not a Vec. If it were a Vec, we could use split_at_mut to at least implement copy_from_self safely.
But as it happens, my actual data structure is backed by a Vec, so I was able to add a helper function:
/// Returns a pair of mutable references to different items. Useful if you need to pass
/// a reference to one item to a function that takes `&mut self` on another item.
/// Panics if `a == b`.
fn get_mut_2(&mut self, a: usize, b: usize) -> (&mut T, &mut T) {
assert!(a != b);
if a < b {
let (first, second) = self.items.split_at_mut(b);
(&mut first[a], &mut second[0])
} else if a > b {
let (first, second) = self.items.split_at_mut(a);
(&mut second[0], &mut first[b])
} else {
panic!("cannot call get_mut_2 with the same index {} == {}", a, b);
}
}
Now we can implement copy_from_self without unsafe code:
pub fn copy_from_self(&mut self, to_idx: usize, from_idx: usize) {
let (to, from) = self.items.get_mut_2(to_idx, from_idx);
to.unwrap().copy_from(from.unwrap());
}

Related

Rust multiple mutable self borrowing from method calls

I am learning Rust. For my first program, I wrote this code to maintain data about a partial ordering:
use std::collections::{HashMap, HashSet};
struct Node {
num_before: usize,
successors: HashSet<String>,
}
impl Node {
fn new() -> Node {
Node {
num_before: 0,
successors: HashSet::new(),
}
}
}
pub struct PartialOrdering {
node: HashMap<String, Node>,
}
impl PartialOrdering {
pub fn new() -> PartialOrdering {
PartialOrdering {
node: HashMap::new(),
}
}
pub fn get_node(&mut self, name: &String) -> &mut Node {
self.node.entry(name.clone()).or_insert_with(Node::new)
}
pub fn add_order(&mut self, before: &String, after: &String) {
let mut before_node = self.get_node(before);
if after != before {
let mut after_node = self.get_node(after);
if before_node.successors.insert(after.clone()) {
after_node.num_before += 1;
}
}
}
}
Compiling this code produces this error:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> main.rs:35:25
|
33 | let before_node = self.get_node(before);
| ---- first mutable borrow occurs here
34 | if after != before {
35 | let mut after_node = self.get_node(after);
| ^^^^ second mutable borrow occurs here
36 | if before_node.successors.insert(after.clone()) {
| ---------------------- first borrow later used here
Admittedly I am new to the Rust borrowing rules, but this problem has me stumped. Please tell me what I am doing wrong, and how can I fix it?
The problem is that in Rust it is forbidden to take more than one mutable reference (&mut) to an object at a time (see here for details). Your get_node() takes &mut self and uses it to obtain an &mut Node contained in self (where self is a PartialOrdering). This causes the mutable borrow of self to exist for as long as the value returned by get_node() exists, preventing other calls to get_node(). This means that you can't have before_node: &mut Node and after_node: &mut Node in the same scope, unfortunately.

How to call a member method while inside block with mutable reference to self [duplicate]

Given the following program:
struct Data {
pub items: Vec<&'static str>,
}
trait Generator {
fn append(&mut self, s: &str) {
self.output().push_str(s);
}
fn data(&self) -> &Data;
fn generate_items(&mut self) {
for item in self.data().items.iter() {
match *item {
"foo" => self.append("it was foo\n"),
_ => self.append("it was something else\n"),
}
}
}
fn output(&mut self) -> &mut String;
}
struct MyGenerator<'a> {
data: &'a Data,
output: String,
}
impl<'a> MyGenerator<'a> {
fn generate(mut self) -> String {
self.generate_items();
self.output
}
}
impl<'a> Generator for MyGenerator<'a> {
fn data(&self) -> &Data {
self.data
}
fn output(&mut self) -> &mut String {
&mut self.output
}
}
fn main() {
let data = Data {
items: vec!["foo", "bar", "baz"],
};
let generator = MyGenerator {
data: &data,
output: String::new(),
};
let output = generator.generate();
println!("{}", output);
}
The following errors are produced trying to compile it:
error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
--> src/main.rs:15:26
|
13 | for item in self.data().items.iter() {
| ---- - immutable borrow ends here
| |
| immutable borrow occurs here
14 | match *item {
15 | "foo" => self.append("it was foo\n"),
| ^^^^ mutable borrow occurs here
error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
--> src/main.rs:16:22
|
13 | for item in self.data().items.iter() {
| ---- - immutable borrow ends here
| |
| immutable borrow occurs here
...
16 | _ => self.append("it was something else\n"),
| ^^^^ mutable borrow occurs here
What is the proper way to structure the code so that the mutable field output can be written to while iterating over the immutable field data? Assume the indirection through the Generator trait is being used to share similar logic with other structs, so accessing MyStruct's fields from the trait's default method implementations need to be done through accessor methods like this.
This is a common issue in Rust; the typical way of solving it is the replace dance. This involves making more of the data and methods use mutable references:
struct Data {
pub items: Vec<&'static str>,
}
trait Generator {
fn append(&mut self, s: &str) {
self.output().push_str(s);
}
fn data(&mut self) -> &mut Data;
fn generate_items(&mut self) {
// Take the data. The borrow on self ends after this statement.
let data = std::mem::replace(self.data(), Data { items: vec![] });
// Iterate over the local version. Now append can borrow all it wants.
for item in data.items.iter() {
match *item {
"foo" => self.append("it was foo\n"),
_ => self.append("it was something else\n"),
}
}
// Put the data back where it belongs.
std::mem::replace(self.data(), data);
}
fn output(&mut self) -> &mut String;
}
struct MyGenerator<'a> {
data: &'a mut Data,
output: String,
}
impl<'a> MyGenerator<'a> {
fn generate(mut self) -> String {
self.generate_items();
self.output
}
}
impl<'a> Generator for MyGenerator<'a> {
fn data(&mut self) -> &mut Data {
self.data
}
fn output(&mut self) -> &mut String {
&mut self.output
}
}
fn main() {
let mut data = Data {
items: vec!["foo", "bar", "baz"],
};
let generator = MyGenerator {
data: &mut data,
output: String::new(),
};
let output = generator.generate();
println!("{}", output);
}
The thing to realize is that the compiler is right to complain. Imagine if calling output() had the side effect of mutating the thing that is referenced by the return value of data() Then the iterator you're using in the loop could get invalidated. Your trait functions have the implicit contract that they don't do anything like that, but there is no way of checking this. So the only thing you can do is temporarily assume full control over the data, by taking it out.
Of course, this pattern breaks unwind safety; a panic in the loop will leave the data moved out.
Assume the indirection through the Generator trait is being used to share similar logic with other structs, so accessing MyStruct's fields from the trait's default method implementations need to be done through accessor methods like this.
Then it's impossible.
The compiler recognizes access to different fields when it sees such fields directly; it does not break abstraction boundaries to peek inside the functions called.
There have been discussions about adding attributes on the methods to specifically mention which field is accessed by which method:
the compiler would enforce that a method does not touch any field NOT mentioned in the attribute
the compiler could then use the knowledge that said method only operates on a subset of the fields
however... this is for non-virtual methods.
For a trait this gets significantly more complicated because a trait does not have fields, and each implementer may have a different set of fields!
So now what?
You will need to change your code:
you can split the trait in two, and require two objects (one to iterate, one to mutate)
you can "hide" the mutability of the append method, forcing users to use interior mutability
...
You can use RefCell:
RefCell uses Rust's lifetimes to implement 'dynamic borrowing', a
process whereby one can claim temporary, exclusive, mutable access to
the inner value. Borrows for RefCells are tracked 'at runtime',
unlike Rust's native reference types which are entirely tracked
statically, at compile time. Because RefCell borrows are dynamic it
is possible to attempt to borrow a value that is already mutably
borrowed; when this happens it results in thread panic.
use std::cell::{RefCell, RefMut};
struct Data {
pub items: Vec<&'static str>,
}
trait Generator {
fn append(&self, s: &str) {
self.output().push_str(s);
}
fn data(&self) -> &Data;
fn generate_items(&self) {
for item in self.data().items.iter() {
match *item {
"foo" => self.append("it was foo\n"),
_ => self.append("it was something else\n"),
}
}
}
fn output(&self) -> RefMut<String>;
}
struct MyGenerator<'a> {
data: &'a Data,
output: RefCell<String>,
}
impl<'a> MyGenerator<'a> {
fn generate(self) -> String {
self.generate_items();
self.output.into_inner()
}
}
impl<'a> Generator for MyGenerator<'a> {
fn data(&self) -> &Data {
self.data
}
fn output(&self) -> RefMut<String> {
self.output.borrow_mut()
}
}
fn main() {
let data = Data {
items: vec!["foo", "bar", "baz"],
};
let generator = MyGenerator {
data: &data,
output: RefCell::new(String::new()),
};
let output = generator.generate();
println!("{}", output);
}
Rust playground

Lifetime on trait returning iterator

I'm working with a trait requiring a function returning an iterator without consuming the object. The iterator itself returns copies of data values, not references. As the iterator implementation requires a reference to the object it is iterating over, I end up having to declare lots of lifetimes (more than I would have thought necessary, but could not get it to compile otherwise). I then run into trouble with borrow duration - a minimal "working" example is as follows:
pub trait MyTrait<'a> {
type IteratorType: Iterator<Item=u32>;
fn iter(&'a self) -> Self::IteratorType;
fn touch(&'a mut self, value: u32);
}
struct MyStruct {
data: Vec<u32>
}
struct MyIterator<'a> {
structref: &'a MyStruct,
next: usize,
}
impl<'a> Iterator for MyIterator<'a> {
type Item = u32;
fn next(&mut self) -> Option<u32> {
if self.next < self.structref.data.len() {
self.next += 1;
return Some(self.structref.data[self.next-1]);
} else {
return None;
}
}
}
impl<'a> MyTrait<'a> for MyStruct {
type IteratorType = MyIterator<'a>;
fn iter(&'a self) -> Self::IteratorType {
return MyIterator { structref: &self, next: 0 };
}
fn touch(&'a mut self, value: u32) {
}
}
fn touch_all<'a,T>(obj: &'a mut T) where T: MyTrait<'a> {
let data: Vec<u32> = obj.iter().collect();
for value in data {
obj.touch(value);
}
}
Compiling this gives me the error:
error[E0502]: cannot borrow `*obj` as mutable because it is also borrowed as immutable
|
39 | let data: Vec<u32> = obj.iter().collect();
| --- immutable borrow occurs here
40 | for value in data {
41 | obj.touch(value);
| ^^^ mutable borrow occurs here
42 | }
43 | }
| - immutable borrow ends here
By my limited understanding of lifetimes, I would have thought the immutable borrow only extends to the line where I make it - after all the iterator is consumed and I no longer hold any references to obj or data contained in it. Why does the lifetime of the borrow extend to the entire function, and how do I fix this?
Here is a sequence of steps on how I arrived here - running the code should provide the associated compiler errors.
no explicit lifetimes
IteratorType needs lifetime
Unconstrained lifetime parameter
To clarify: I'd like to be able to make calls like this:
fn main() {
let obj: MyStruct = MyStruct { data : vec![] };
touch_all(&mut obj);
}
rather than having to call
touch_all(&mut &obj);
which would be needed for the proposal by mcarton (1st and 2nd comment).

Mutating one field while iterating over another immutable field

Given the following program:
struct Data {
pub items: Vec<&'static str>,
}
trait Generator {
fn append(&mut self, s: &str) {
self.output().push_str(s);
}
fn data(&self) -> &Data;
fn generate_items(&mut self) {
for item in self.data().items.iter() {
match *item {
"foo" => self.append("it was foo\n"),
_ => self.append("it was something else\n"),
}
}
}
fn output(&mut self) -> &mut String;
}
struct MyGenerator<'a> {
data: &'a Data,
output: String,
}
impl<'a> MyGenerator<'a> {
fn generate(mut self) -> String {
self.generate_items();
self.output
}
}
impl<'a> Generator for MyGenerator<'a> {
fn data(&self) -> &Data {
self.data
}
fn output(&mut self) -> &mut String {
&mut self.output
}
}
fn main() {
let data = Data {
items: vec!["foo", "bar", "baz"],
};
let generator = MyGenerator {
data: &data,
output: String::new(),
};
let output = generator.generate();
println!("{}", output);
}
The following errors are produced trying to compile it:
error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
--> src/main.rs:15:26
|
13 | for item in self.data().items.iter() {
| ---- - immutable borrow ends here
| |
| immutable borrow occurs here
14 | match *item {
15 | "foo" => self.append("it was foo\n"),
| ^^^^ mutable borrow occurs here
error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
--> src/main.rs:16:22
|
13 | for item in self.data().items.iter() {
| ---- - immutable borrow ends here
| |
| immutable borrow occurs here
...
16 | _ => self.append("it was something else\n"),
| ^^^^ mutable borrow occurs here
What is the proper way to structure the code so that the mutable field output can be written to while iterating over the immutable field data? Assume the indirection through the Generator trait is being used to share similar logic with other structs, so accessing MyStruct's fields from the trait's default method implementations need to be done through accessor methods like this.
This is a common issue in Rust; the typical way of solving it is the replace dance. This involves making more of the data and methods use mutable references:
struct Data {
pub items: Vec<&'static str>,
}
trait Generator {
fn append(&mut self, s: &str) {
self.output().push_str(s);
}
fn data(&mut self) -> &mut Data;
fn generate_items(&mut self) {
// Take the data. The borrow on self ends after this statement.
let data = std::mem::replace(self.data(), Data { items: vec![] });
// Iterate over the local version. Now append can borrow all it wants.
for item in data.items.iter() {
match *item {
"foo" => self.append("it was foo\n"),
_ => self.append("it was something else\n"),
}
}
// Put the data back where it belongs.
std::mem::replace(self.data(), data);
}
fn output(&mut self) -> &mut String;
}
struct MyGenerator<'a> {
data: &'a mut Data,
output: String,
}
impl<'a> MyGenerator<'a> {
fn generate(mut self) -> String {
self.generate_items();
self.output
}
}
impl<'a> Generator for MyGenerator<'a> {
fn data(&mut self) -> &mut Data {
self.data
}
fn output(&mut self) -> &mut String {
&mut self.output
}
}
fn main() {
let mut data = Data {
items: vec!["foo", "bar", "baz"],
};
let generator = MyGenerator {
data: &mut data,
output: String::new(),
};
let output = generator.generate();
println!("{}", output);
}
The thing to realize is that the compiler is right to complain. Imagine if calling output() had the side effect of mutating the thing that is referenced by the return value of data() Then the iterator you're using in the loop could get invalidated. Your trait functions have the implicit contract that they don't do anything like that, but there is no way of checking this. So the only thing you can do is temporarily assume full control over the data, by taking it out.
Of course, this pattern breaks unwind safety; a panic in the loop will leave the data moved out.
Assume the indirection through the Generator trait is being used to share similar logic with other structs, so accessing MyStruct's fields from the trait's default method implementations need to be done through accessor methods like this.
Then it's impossible.
The compiler recognizes access to different fields when it sees such fields directly; it does not break abstraction boundaries to peek inside the functions called.
There have been discussions about adding attributes on the methods to specifically mention which field is accessed by which method:
the compiler would enforce that a method does not touch any field NOT mentioned in the attribute
the compiler could then use the knowledge that said method only operates on a subset of the fields
however... this is for non-virtual methods.
For a trait this gets significantly more complicated because a trait does not have fields, and each implementer may have a different set of fields!
So now what?
You will need to change your code:
you can split the trait in two, and require two objects (one to iterate, one to mutate)
you can "hide" the mutability of the append method, forcing users to use interior mutability
...
You can use RefCell:
RefCell uses Rust's lifetimes to implement 'dynamic borrowing', a
process whereby one can claim temporary, exclusive, mutable access to
the inner value. Borrows for RefCells are tracked 'at runtime',
unlike Rust's native reference types which are entirely tracked
statically, at compile time. Because RefCell borrows are dynamic it
is possible to attempt to borrow a value that is already mutably
borrowed; when this happens it results in thread panic.
use std::cell::{RefCell, RefMut};
struct Data {
pub items: Vec<&'static str>,
}
trait Generator {
fn append(&self, s: &str) {
self.output().push_str(s);
}
fn data(&self) -> &Data;
fn generate_items(&self) {
for item in self.data().items.iter() {
match *item {
"foo" => self.append("it was foo\n"),
_ => self.append("it was something else\n"),
}
}
}
fn output(&self) -> RefMut<String>;
}
struct MyGenerator<'a> {
data: &'a Data,
output: RefCell<String>,
}
impl<'a> MyGenerator<'a> {
fn generate(self) -> String {
self.generate_items();
self.output.into_inner()
}
}
impl<'a> Generator for MyGenerator<'a> {
fn data(&self) -> &Data {
self.data
}
fn output(&self) -> RefMut<String> {
self.output.borrow_mut()
}
}
fn main() {
let data = Data {
items: vec!["foo", "bar", "baz"],
};
let generator = MyGenerator {
data: &data,
output: RefCell::new(String::new()),
};
let output = generator.generate();
println!("{}", output);
}
Rust playground

HashMap borrow issue when trying to implement find or insert

I tried to implement own analogue of find_or_insert method that looks like this:
use std::collections::HashMap;
pub struct SomeManager {
next: i32,
types: HashMap<i32, i32>,
}
impl SomeManager {
pub fn get_type<'a>(&'a mut self, k: i32) -> &'a i32 {
match self.types.get(&k) {
Some(ref x) => return *x,
None => {
self.types.insert(k, self.next);
self.next += 1;
return self.types.get(&k).unwrap();
}
}
}
}
fn main() {}
Error:
error[E0502]: cannot borrow `self.types` as mutable because it is also borrowed as immutable
--> src/main.rs:13:17
|
10 | match self.types.get(&k) {
| ---------- immutable borrow occurs here
...
13 | self.types.insert(k, self.next);
| ^^^^^^^^^^ mutable borrow occurs here
...
18 | }
| - immutable borrow ends here
I know that there are some standard methods that implement this functionality, but I want this method to be as light as possible - it will be called very-very often and almost all of the time the values will already exist.
As I understand it, when we call self.types.get we borrow it to scope of match statement, so we can't call self.types.insert here. I have tried to move methods from None branch out of match statement, but it also fails.
The only working solution that I found requires invoking get twice:
pub fn get_type<'a>(&'a mut self, k: i32) -> &'a i32 {
let is_none = match self.types.get(&k) {
Some(ref x) => false,
None => true,
};
if is_none {
self.types.insert(k, self.next);
self.next += 1;
}
self.types.get(&k).unwrap()
}
How can I work around such situations?
There are a handful of methods on HashMap to achieve these sorts of complex cases. Most notably, for your case, HashMap::entry and Entry::or_insert_with:
pub fn get_type<'a>(&'a mut self, k: i32) -> &'a i32 {
self.types.entry(k).or_insert_with(|| {
let value = self.next;
self.next += 1;
value
})
}
In your case, however, there’s the borrow of self inside, so that won’t do.
We thus shift the borrow of self.next outside of the closure so the compiler can reason about it as disjoint from self.types. Problem solved with only one lookup, as it should be.
pub fn get_type<'a>(&'a mut self, k: i32) -> &'a i32 {
let next = &mut self.next;
self.types.entry(k).or_insert_with(|| {
let value = *next;
*next += 1;
value
})
}
Note that in your first case you're doing one lookup when the key exists in the map and three when it does not exist. Your last attempt does two lookups in either case. This is somewhat prettified version of the latter:
pub fn get_type<'a>(&'a mut self, k: i32) -> &'a i32 {
let contains = self.types.contains_key(&k);
if !contains {
self.types.insert(k, self.next);
self.next += 1;
}
self.types.get(&k).unwrap()
}
I don't think it is possible to avoid the second lookup without some support from the map's implementation because of borrowing restrictions.
In any case, using the solution by Chris Morgan is superior to the above one (for example, it may be more efficient and in fact require less lookups), so I suggest sticking with it.

Resources