To add the elements of two Vecs I wrote a function like
fn add_components(dest: &mut Vec<i32>, first: &Vec<i32>, second: &Vec<i32>){
for i in 0..first.len() {
dest[i] = first[i] + second[i];
}
}
And this works fine when dest is another Vec.
let mut new_components = Vec::with_capacity(components.len());
Vector::add_components(&mut new_comps, &components, &other_components);
But it blows up when I am trying to add in-place:
Vector::add_components(&mut components, &components, &other_components);
because now I borrow components as mutable and immutable at the same time. But this obviously is what I am trying to achieve.
Are there any conventional and general (meaning not only concerning Vecs) solutions to this problem which don't involve unsafe code and pointer magic?
Another example of this problem:
Suppose I want to overload AddAssign for a numeric type like
impl AddAssign<Output=&NumericType> for NumericType {
fn add_assign(&mut self, other: &NumericType) {
unimplemented!() // concrete implementation is not important
}
}
Notice that I want to take a reference as second argument to avoid copying. This works fine when adding two different objects, but adding an object to itself creates the exact same scenario:
let mut num = NumericType{};
num += &num
I am borrowing num mutably and immutably at the same time. So obviously this should work and is safe, but it also is against Rust's borrowing rules.
What are the best practices (apart from copying of course) to deal with this issue, which arises in many forms?
There is no generic solution to this. Rust can't generically abstract over mutability in borrow checking.
You will need to have two versions of the function for in-place and destination versions.
Rust has strict aliasing rules, so dest[i] = first[i] + second[i] actually compiles to different code depending on whether the compiler has a guarantee that dest and first are different. Don't try to fudge it with unsafe, because it will be Undefined Behavior and will get miscompiled.
Related
This question already has an answer here:
Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in?
(1 answer)
Closed 3 years ago.
I'm trying to write a function that finds returns a mutable reference to an existing element in a Vec, or inserts it if it doesn't exist and returns a mutable reference to the new element.
I've tried a couple of times, but the borrow checker isn't convinced. I've simplified the code I was trying to write to the example below, which gives the same errors.
fn mut_find_or_insert<T: PartialEq>(vec: &mut Vec<T>, val: T) -> &mut T {
if let Some(u) = vec.iter_mut().find(|u| **u == val) {
u
} else {
vec.push(val);
vec.last_mut().unwrap()
}
}
Playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=cb12c38bcf3682b15a247d14aab48b6b
Rust gives me the following compiler error (full message through the playground link):
error[E0499]: cannot borrow `*vec` as mutable more than once at a time
This seems to be something that should be possible to implement in rust, however it's not clear to me how I reimplement this to avoid borrow checker errors.
The reason this doesn't work as written is because of a limitation in the current borrow checker. This is very similar to NLL case #3, in which the compiler borrows somewhat overzealously for an entire match statement when the borrow is only used in one of the branches. With the experimental "Polonius" borrow checker (available on the nightly compiler with the -Z polonius flag), your code is accepted as-is.
Working in the stable compiler, it's probably a good idea to redesign your data structures as Sébastien Renauld's answer also suggests, but if you need to make this work with a Vec, you can work around it by briefly using an index to end the borrow:
fn mut_find_or_insert<T: PartialEq>(vec: &mut Vec<T>, val: T) -> &mut T {
if let Some(i) = vec.iter().position(|each| *each == val) {
&mut vec[i]
} else {
vec.push(val);
vec.last_mut().unwrap()
}
}
This works because the result of calling position is not a reference, so the borrow of vec is not held during the if let.
This is similar to the following questions, which manage to find the same limitation using early return from a loop:
Double mutable borrow error in a loop happens even with NLL on
"Variable does not live long enough" when returning a Result containing a reference but it does live long enough
Vec is an unordered, not very structured type. It has no way to look up the exact position of an item within it; the closest the default functions get to is contains(), which only tells you if the item is contained.
Furthermore, due to the fact that a Vec is not a Set, the behavior of "find the item or append and return " is undefined - "find the item", if there is a duplicate, needs to be defined further.
To solve this problem without changing to the correct type (HashSet is the type you really want for this. Note the existence of get_or_insert(), which is literally what you are after. It pays to use the proper structure for the job, rather than to try to make everything fit a Vec), we're going to have to build it ourselves. Keeping to your signature, it looks like this (Playground):
trait VecSeekOrAppend<T:PartialEq>:Sized {
fn get_or_insert(&mut self, item: T) -> &mut T;
}
impl<T> VecSeekOrAppend<T> for Vec<T>
where T: PartialEq + Clone {
fn get_or_insert(&mut self, item: T) -> &mut T {
if !self.contains(&item) {
self.push(item.clone());
}
for i in self.iter_mut() {
if i == &mut item {
return i;
}
}
unreachable!();
}
}
The reason your initial version does not work is due to the returned lifetime requirement; all methods returning a reference from a Vec require a lifetime validity for the duration of use. By returning such a &mut reference, if you attempt to do it in one go, the mutation of the Vec<_> will happen while there is already a mutable borrow of it.
Splitting the cycle in two, and performing the insertion (without keeping a reference) to then find the eference, allows us to sidestep this problem. Another way to perform this is to store items by a serializable or hashable identifier (the exact way HashMap and HashSet work) in order to innately provide this layer of indirection.
There is a rust feature in the works to ease some of this pain (non-lexical lifetimes), but, as you can see from the github issue, it's not in the near future.
My understanding of AsMut is that it is supposed to provide a generic way to take an argument which is "equivalent* to a mutable reference, i.e. it can be cheaply converted to a mutable reference.
However, I have encountered the following example code, I am trying to use AsMut just to be generic over slice, arr and Vec, but it seems to just be copying my array, rather than passing a mutable reference to it and modifying it in place:
pub fn uses_asmut<T, M>(mut m: M)
where M: AsMut<[T]> {
m.as_mut().swap(0,1);
}
#[test]
pub fn test_swap() {
let arr = [1,2];
uses_asmut(arr);
assert_eq!(arr, [2,1]);
}
(note, I know something must be wrong since I apparently pass ownership of arr to uses_asmut as an argument, but then the borrow checker doesn't complain on the next line when I use arr again! If I change it to uses_asmut(&mut arr) the test passes, but I think the code as written shouldn't even compile!)
The conversion should be cheap. But you're passing the array by value, and that was never said to be cheap. You could pass &mut arr to not copy the array.
When we say the conversion should be cheap, we mean "don't do something like str::from_utf8_mut()" that needs to scan the whole string. Indeed, the conversion from [T; N] to [T] is extremely cheap: so cheap, that it happens automatically by the compiler (coercion).
But it does not mean it is equivalent to mutable reference, because it is not a mutable reference. It is a generic type. If you want a mutable reference, take a mutable reference. You can even use AsMut, like &mut impl AsMut<[T]>.
There is no way AsMut could prevent you from moving (or copying) things but also not require you to type the &mut at the call site.
I am converting a variety of types to String when they are passed to a function. I'm not concerned about performance as much as ergonomics, so I want the conversion to be implicit. The original, less generic implementation of the function simply used &[impl Into<String>], but I think that it should be possible to pass a variety of types at once without manually converting each to a string.
The key is that ideally, all of the following cases should be valid calls to my function:
// String literals
perform_tasks(&["Hello", "world"]);
// Owned strings
perform_tasks(&[String::from("foo"), String::from("bar")]);
// Non-string types
perform_tasks(&[1,2,3]);
// A mix of any of them
perform_tasks(&["All", 3, String::from("types!")]);
Some various signatures I've attempted to use:
fn perform_tasks(items: &[impl Into<String>])
The original version fails twice; it can't handle numeric types without manual conversion, and it requires all of the arguments to be the same type.
fn perform_tasks(items: &[impl ToString])
This is slightly closer, but it still requires all of the arguments to be of one type.
fn perform_tasks(items: &[&dyn ToString])
Doing it this way is almost enough, but it won't compile unless I manually add a borrow on each argument.
And that's where we are. I suspect that either Borrow or AsRef will be involved in a solution, but I haven't found a way to get them to handle this situation. For convenience, here is a playground link to the final signature in use (without the needed references for it to compile), alongside the various tests.
The following way works for the first three cases if I understand your intention correctly.
pub fn perform_tasks<I, A>(values: I) -> Vec<String>
where
A: ToString,
I: IntoIterator<Item = A>,
{
values.into_iter().map(|s| s.to_string()).collect()
}
As the other comments pointed out, Rust does not support an array of mixed types. However, you can do one extra step to convert them into a &[&dyn fmt::Display] and then call the same function perform_tasks to get their strings.
let slice: &[&dyn std::fmt::Display] = &[&"All", &3, &String::from("types!")];
perform_tasks(slice);
Here is the playground.
If I understand your intention right, what you want is like this
fn main() {
let a = 1;
myfn(a);
}
fn myfn(i: &dyn SomeTrait) {
//do something
}
So it's like implicitly borrow an object as function argument. However, Rust won't let you to implicitly borrow some objects since borrowing is quite an important safety measure in rust and & can help other programmers quickly identified which is a reference and which is not. Thus Rust is designed to enforce the & to avoid confusion.
This question already has an answer here:
Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in?
(1 answer)
Closed 3 years ago.
I'm trying to write a function that finds returns a mutable reference to an existing element in a Vec, or inserts it if it doesn't exist and returns a mutable reference to the new element.
I've tried a couple of times, but the borrow checker isn't convinced. I've simplified the code I was trying to write to the example below, which gives the same errors.
fn mut_find_or_insert<T: PartialEq>(vec: &mut Vec<T>, val: T) -> &mut T {
if let Some(u) = vec.iter_mut().find(|u| **u == val) {
u
} else {
vec.push(val);
vec.last_mut().unwrap()
}
}
Playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=cb12c38bcf3682b15a247d14aab48b6b
Rust gives me the following compiler error (full message through the playground link):
error[E0499]: cannot borrow `*vec` as mutable more than once at a time
This seems to be something that should be possible to implement in rust, however it's not clear to me how I reimplement this to avoid borrow checker errors.
The reason this doesn't work as written is because of a limitation in the current borrow checker. This is very similar to NLL case #3, in which the compiler borrows somewhat overzealously for an entire match statement when the borrow is only used in one of the branches. With the experimental "Polonius" borrow checker (available on the nightly compiler with the -Z polonius flag), your code is accepted as-is.
Working in the stable compiler, it's probably a good idea to redesign your data structures as Sébastien Renauld's answer also suggests, but if you need to make this work with a Vec, you can work around it by briefly using an index to end the borrow:
fn mut_find_or_insert<T: PartialEq>(vec: &mut Vec<T>, val: T) -> &mut T {
if let Some(i) = vec.iter().position(|each| *each == val) {
&mut vec[i]
} else {
vec.push(val);
vec.last_mut().unwrap()
}
}
This works because the result of calling position is not a reference, so the borrow of vec is not held during the if let.
This is similar to the following questions, which manage to find the same limitation using early return from a loop:
Double mutable borrow error in a loop happens even with NLL on
"Variable does not live long enough" when returning a Result containing a reference but it does live long enough
Vec is an unordered, not very structured type. It has no way to look up the exact position of an item within it; the closest the default functions get to is contains(), which only tells you if the item is contained.
Furthermore, due to the fact that a Vec is not a Set, the behavior of "find the item or append and return " is undefined - "find the item", if there is a duplicate, needs to be defined further.
To solve this problem without changing to the correct type (HashSet is the type you really want for this. Note the existence of get_or_insert(), which is literally what you are after. It pays to use the proper structure for the job, rather than to try to make everything fit a Vec), we're going to have to build it ourselves. Keeping to your signature, it looks like this (Playground):
trait VecSeekOrAppend<T:PartialEq>:Sized {
fn get_or_insert(&mut self, item: T) -> &mut T;
}
impl<T> VecSeekOrAppend<T> for Vec<T>
where T: PartialEq + Clone {
fn get_or_insert(&mut self, item: T) -> &mut T {
if !self.contains(&item) {
self.push(item.clone());
}
for i in self.iter_mut() {
if i == &mut item {
return i;
}
}
unreachable!();
}
}
The reason your initial version does not work is due to the returned lifetime requirement; all methods returning a reference from a Vec require a lifetime validity for the duration of use. By returning such a &mut reference, if you attempt to do it in one go, the mutation of the Vec<_> will happen while there is already a mutable borrow of it.
Splitting the cycle in two, and performing the insertion (without keeping a reference) to then find the eference, allows us to sidestep this problem. Another way to perform this is to store items by a serializable or hashable identifier (the exact way HashMap and HashSet work) in order to innately provide this layer of indirection.
There is a rust feature in the works to ease some of this pain (non-lexical lifetimes), but, as you can see from the github issue, it's not in the near future.
I'm implementing Conway's game of life to teach myself Rust. The idea is to implement a single-threaded version first, optimize it as much as possible, then do the same for a multi-threaded version.
I wanted to implement an alternative data layout which I thought might be more cache-friendly. The idea is to store the status of two cells for each point on a board next to each other in memory in a vector, one cell for reading the current generation's status from and one for writing the next generation's status to, alternating the access pattern for each
generation's computation (which can be determined at compile time).
The basic data structures are as follows:
#[repr(u8)]
pub enum CellStatus {
DEAD,
ALIVE,
}
/** 2 bytes */
pub struct CellRW(CellStatus, CellStatus);
pub struct TupleBoard {
width: usize,
height: usize,
cells: Vec<CellRW>,
}
/** used to keep track of current pos with iterator e.g. */
pub struct BoardPos {
x_pos: usize,
y_pos: usize,
offset: usize,
}
pub struct BoardEvo {
board: TupleBoard,
}
The function that is causing me troubles:
impl BoardEvo {
fn evolve_step<T: RWSelector>(&mut self) {
for (pos, cell) in self.board.iter_mut() {
//pos: BoardPos, cell: &mut CellRW
let read: &CellStatus = T::read(cell); //chooses the right tuple half for the current evolution step
let write: &mut CellStatus = T::write(cell);
let alive_count = pos.neighbours::<T>(&self.board).iter() //<- can't borrow self.board again!
.filter(|&&status| status == CellStatus::ALIVE)
.count();
*write = CellStatus::evolve(*read, alive_count);
}
}
}
impl BoardPos {
/* ... */
pub fn neighbours<T: RWSelector>(&self, board: &BoardTuple) -> [CellStatus; 8] {
/* ... */
}
}
The trait RWSelector has static functions for reading from and writing to a cell tuple (CellRW). It is implemented for two zero-sized types L and R and is mainly a way to avoid having to write different methods for the different access patterns.
The iter_mut() method returns a BoardIter struct which is a wrapper around a mutable slice iterator for the cells vector and thus has &mut CellRW as Item type. It is also aware of the current BoardPos (x and y coordinates, offset).
I thought I'd iterate over all cell tuples, keep track of the coordinates, count the number of alive neighbours (I need to know coordinates/offsets for this) for each (read) cell, compute the cell status for the next generation and write to the respective another half of the tuple.
Of course, in the end, the compiler showed me the fatal flaw in my design, as I borrow self.board mutably in the iter_mut() method and then try to borrow it again immutably to get all the neighbours of the read cell.
I have not been able to come up with a good solution for this problem so far. I did manage to get it working by making all
references immutable and then using an UnsafeCell to turn the immutable reference to the write cell into a mutable one.
I then write to the nominally immutable reference to the writing part of the tuple through the UnsafeCell.
However, that doesn't strike me as a sound design and I suspect I might run into issues with this when attempting to parallelize things.
Is there a way to implement the data layout I proposed in safe/idiomatic Rust or is this actually a case where you actually have to use tricks to circumvent Rust's aliasing/borrow restrictions?
Also, as a broader question, is there a recognizable pattern for problems which require you to circumvent Rust's borrow restrictions?
When is it necessary to circumvent Rust's borrow checker?
It is needed when:
the borrow checker is not advanced enough to see that your usage is safe
you do not wish to (or cannot) write the code in a different pattern
As a concrete case, the compiler cannot tell that this is safe:
let mut array = [1, 2];
let a = &mut array[0];
let b = &mut array[1];
The compiler doesn't know what the implementation of IndexMut for a slice does at this point of compilation (this is a deliberate design choice). For all it knows, arrays always return the exact same reference, regardless of the index argument. We can tell that this code is safe, but the compiler disallows it.
You can rewrite this in a way that is obviously safe to the compiler:
let mut array = [1, 2];
let (a, b) = array.split_at_mut(1);
let a = &mut a[0];
let b = &mut b[0];
How is this done? split_at_mut performs a runtime check to ensure that it actually is safe:
fn split_at_mut(&mut self, mid: usize) -> (&mut [T], &mut [T]) {
let len = self.len();
let ptr = self.as_mut_ptr();
unsafe {
assert!(mid <= len);
(from_raw_parts_mut(ptr, mid),
from_raw_parts_mut(ptr.offset(mid as isize), len - mid))
}
}
For an example where the borrow checker is not yet as advanced as it can be, see What are non-lexical lifetimes?.
I borrow self.board mutably in the iter_mut() method and then try to borrow it again immutably to get all the neighbours of the read cell.
If you know that the references don't overlap, then you can choose to use unsafe code to express it. However, this means you are also choosing to take on the responsibility of upholding all of Rust's invariants and avoiding undefined behavior.
The good news is that this heavy burden is what every C and C++ programmer has to (or at least should) have on their shoulders for every single line of code they write. At least in Rust, you can let the compiler deal with 99% of the cases.
In many cases, there's tools like Cell and RefCell to allow for interior mutation. In other cases, you can rewrite your algorithm to take advantage of a value being a Copy type. In other cases you can use an index into a slice for a shorter period. In other cases you can have a multi-phase algorithm.
If you do need to resort to unsafe code, then try your best to hide it in a small area and expose safe interfaces.
Above all, many common problems have been asked about (many times) before:
How to iterate over mutable elements inside another mutable iteration over the same elements?
Mutating an item inside of nested loops
How can a nested loop with mutations on a HashMap be achieved in Rust?
What's the Rust way to modify a structure within nested loops?
Nesting an iterator's loops