I have this piece of code below where I iterate over a Vec (self.regions), and under the right conditions, I mutate the current element of the Vec
However, when processing a region, I must also call another method on self that borrows it, causing me to have both a mutable and shared reference of self at the same time, which is obviously impossible (and will not compile)
How can I change my code in order to make it work ?
I've thought of moving line 3 outside the loop and using a iter() instead of an iter_mut(), but then I don't see how to "keep the two iterators in sync" in order to make sure that both pieces of code refer to the same region
fn growth(&'a mut self) {
for region in self.regions.iter_mut() {
let candidates = self.neighbors(region); // Has to involve taking a &self
// Do stuff with `candidates` that involves mutating `region`
}
}
Edit
My post was lacking context, so here is the full function. self.regions is of type Vec<Vec<&'a Cell>>, and the signature of neighbors is fn neighbors(&self, region: &Vec<&Cell>) -> HashSet<&Cell>.
The objective of my program is to tesselate a grid into random polyominoes, by randomly selecting some sources and (the part that I posted) "growing them" into regions by iterating over each region and adding a random neighbor of the region to the region itself.
The neighbors function depends of the previous iterations' outcome, so it cannot be precomputed
fn growth(&'a mut self) -> bool {
let mut unchoosen_cells = 20; // More complex in reality
let mut steps: usize = 0;
while unchoosen_cells != 0 {
if steps > GROWTH_STEP_LIMIT { return false; }
for region in self.regions.iter_mut() {
if rand::random() { continue; }
let candidates = self.neighbors(region);
if let Some(cell) = candidates.iter().choose(&mut thread_rng()) {
// cell.in_region = true;
region.push(cell);
unchoosen_cells -= 1;
}
steps += 1;
}
}
true
}
After reading all the comments, I understand that my issue is the result of a poor design, and I will refactor my code in order to use indices instead of references
Thanks to all the people who took the time to help me, and I'm marking this as solved
Related
I am working on some software where I am managing a buffer of floats in a Vec<T> where T is either an f32 or f64. I sometimes need to interpret this buffer, or sections of it, as a mathematical vector. To this end, I am taking advantage of MatrixSlice and friends in nalgebra.
I can create a DVectorSliceMut, for example, the following way
fn as_vector<'a>(slice: &'a mut [f64]) -> DVectorSliceMut<'a, f64> {
DVectorSliceMut::from(slice)
}
However, sometimes I need to later extract the original slice from the DVectorSliceMut with the original lifetime 'a. Is there a way to do this?
The StorageMut trait has a as_mut_slice member function, but the lifetime of the returned slice is the lifetime of the reference to the Storage implementor, not the original slice. I am okay with a solution which consumes the DVectorSliceMut if necessary.
Update: Methods into_slice and into_slice_mut have been respectively added to the SliceStorage and SliceStorageMut traits as of nalgebra v0.28.0.
Given the current API of nalgebra (v0.27.1) there isn't much that you can do, except:
life with the shorter life-time of StorageMut::as_mut_slice
make a feature request for such a function at nalgebra (which seems you already did)
employ your own unsafe code to make StorageMut::ptr_mut into a &'a mut
You could go with the third option until nalgebra gets update and implement something like this in your own code:
use nalgebra::base::dimension::Dim;
use nalgebra::base::storage::Storage;
use nalgebra::base::storage::StorageMut;
fn into_slice<'a>(vec: DVectorSliceMut<'a, f64>) -> &'a mut [f64] {
let mut inner = vec.data;
// from nalgebra
// https://docs.rs/nalgebra/0.27.1/src/nalgebra/base/matrix_slice.rs.html#190
let (nrows, ncols) = inner.shape();
if nrows.value() != 0 && ncols.value() != 0 {
let sz = inner.linear_index(nrows.value() - 1, ncols.value() - 1);
unsafe { core::slice::from_raw_parts_mut(inner.ptr_mut(), sz + 1) }
} else {
unsafe { core::slice::from_raw_parts_mut(inner.ptr_mut(), 0) }
}
}
Methods into_slice and into_slice_mut which return the original slice have been respectively added to the SliceStorage and SliceStorageMut traits as of nalgebra v0.28.0.
My questions seems to be closely related to Rust error "cannot infer an appropriate lifetime for borrow expression" when attempting to mutate state inside a closure returning an Iterator, but I think it's not the same. So, this
use std::iter;
fn example(text: String) -> impl Iterator<Item = Option<String>> {
let mut i = 0;
let mut chunk = None;
iter::from_fn(move || {
if i <= text.len() {
let p_chunk = chunk;
chunk = Some(&text[..i]);
i += 1;
Some(p_chunk.map(|s| String::from(s)))
} else {
None
}
})
}
fn main() {}
does not compile. The compiler says it cannot determine the appropriate lifetime for &text[..i]. This is the smallest example I could come up with. The idea being, there is an internal state, which is a slice of text, and the iterator returns new Strings allocated from that internal state. I'm new to Rust, so maybe it's all obvious, but how would I annotate lifetimes so that this compiles?
Note that this example is different from the linked example, because there point was passed as a reference, while here text is moved. Also, the answer there is one and half years old by now, so maybe there is an easier way.
EDIT: Added p_chunk to emphasize that chunk needs to be persistent across calls to next and so cannot be local to the closure but should be captured by it.
Your code is an example of attempting to create a self-referential struct, where the struct is implicitly created by the closure. Since both text and chunk are moved into the closure, you can think of both as members of a struct. As chunk refers to the contents in text, the result is a self-referential struct, which is not supported by the current borrow checker.
While self-referential structs are unsafe in general due to moves, in this case it would be safe because text is heap-allocated and is not subsequently mutated, nor does it escape the closure. Therefore it is impossible for the contents of text to move, and a sufficiently smart borrow checker could prove that what you're trying to do is safe and allow the closure to compile.
The answer to the [linked question] says that referencing through an Option is possible but the structure cannot be moved afterwards. In my case, the self-reference is created after text and chunk were moved in place, and they are never moved again, so in principle it should work.
Agreed - it should work in principle, but it is well known that the current borrow checker doesn't support it. The support would require multiple new features: the borrow checker should special-case heap-allocated types like Box or String whose moves don't affect references into their content, and in this case also prove that you don't resize or mem::replace() the closed-over String.
In this case the best workaround is the "obvious" one: instead of persisting the chunk slice, persist a pair of usize indices (or a Range) and create the slice when you need it.
If you move the chunk Option into the closure, your code compiles. I can't quite answer why declaring chunk outside the closure results in a lifetime error for the borrow of text inside the closure, but the chunk Option looks superfluous anyways and the following code should be equivalent:
fn example(text: String) -> impl Iterator<Item = Option<String>> {
let mut i = 0;
iter::from_fn(move || {
if i <= text.len() {
let chunk = text[..i].to_string();
i += 1;
Some(Some(chunk))
} else {
None
}
})
}
Additionally, it seems unlikely that you really want an Iterator<Item = Option<String>> here instead of an Iterator<Item<String>>, since the iterator never yields Some(None) anyways.
fn example(text: String) -> impl Iterator<Item = String> {
let mut i = 0;
iter::from_fn(move || {
if i <= text.len() {
let chunk = text[..i].to_string();
i += 1;
Some(chunk)
} else {
None
}
})
}
Note, you can also go about this iterator without allocating a String for each chunk, if you take a &str as an argument and tie the lifetime of the output to the input argument:
fn example<'a>(text: &'a str) -> impl Iterator<Item = &'a str> + 'a {
let mut i = 0;
iter::from_fn(move || {
if i <= text.len() {
let chunk = &text[..i];
i += 1;
Some(chunk)
} else {
None
}
})
}
I faced the same problem as mentioned in this question. In short his problem is borrowing an object as mutable, due to its usage inside a closure, and borrowing it as immutable due to usage inside of a function (or macro in this case).
fn main() {
let mut count = 0;
let mut inc = || {
count += 2;
};
for _index in 1..5 {
inc();
println!("{}", count);
}
}
One solution to this problem is defining the closure inside the for loop instead of outside the for loop, or avoid capture of a variable by passing the mutable reference using the parameters of the closure:
1.
fn main() {
let mut count = 0;
for _index in 1..5 {
let mut inc = || {
count += 2;
};
inc();
println!("{}", count);
}
}
fn main() {
let mut count = 0;
let inc = | count: &mut i32| {
*count += 2;
};
for _index in 1..5 {
inc(&mut count);
println!("{}", count);
}
}
So I have the following questions on my mind:
Which one of these follows the best practice solutions?
Is there a 3rd way of doing things the right way?
According to my un understanding, closures are just anonymous functions, so defining them multiple times is as efficient as defining them a single time. But I am not able to find a definite answer to this question on the official rust references. Help!
Regarding which one is the right solutions, I would say it depends on the use case. They are so similar it shouldn't matter in most cases unless there is something else to sway the decision. I don't know of any third solution.
However, closures are not just anonymous functions but also anonymous structs: A closures is an anonymous struct that calls an anonymous function. The members of the struct are the references to borrows values. This is important because structs need to be initialized and potentially moved around, unlike functions. This means the more values your closure borrows, the more expensive it is to initialize and pass as an argument to functions (by value). Likewise, if you initialize your closure inside a loop, the initialization might happen every iteration (if it is not optimized out of the loop), making it less performant than initializing it outside the loop.
We can try and desugar the first example into the following code:
struct IncClusureStruct<'a> {
count: &'a mut i32,
}
fn inc_closure_fn<'a>(borrows: &mut IncClusureStruct<'a>) {
*borrows.count += 2
}
fn main() {
let mut count = 0;
for _index in 1..5 {
let mut inc_struct = IncClusureStruct { count: &mut count };
inc_closure_fn(&mut inc_struct);
println!("{}", count);
}
}
Note: The compiler doesn't necessarily do exactly like this, but it is a useful approximation.
Here you can see the closure struct IncClusureStructand its function inc_closure_fn, which together are used to provide the functionality of inc. You can see we initialize the struct in the loop and then call it immediately. If we were to desugar the second example, IncClusureStruct would have no members, but inc_closure_fn would take an additional argument that references the counter. The counter reference would then go to the function call instead of the struct initializer.
These two examples end up being the same efficiency-wise because the number of actual values passed to the function is the same in both cases: 1 reference. Initialize a struct with one member is the same as simply initializing the member itself, the wrapping struct is gone by the time you reach machine code. I tried this on Godbolt and as far as I can tell, the resulting assembly is the same.
However, optimizations don't catch all situations. So, if performance is important, benchmarking is the way to go.
This question already has answers here:
How can I swap in a new value for a field in a mutable reference to a structure?
(2 answers)
Closed 5 years ago.
Sometimes I run into a problem where, due to implementation details that should be invisible to the user, I need to "destroy" a &mut and replace it in-memory. This typically ends up happening in recursive methods or IntoIterator implementations on recursive structures. It typically follows the form of:
fn create_something(self);
pub fn do_something(&mut self) {
// What you want to do
*self = self.create_something();
}
One example that I happened to have in my current project is in a KD Tree I've written, when I "remove" a node, instead of doing logic to rearrange the children, I just destructure the node I need to remove and rebuild it from the values in its subtrees:
// Some recursive checks to identify is this is our node above this
if let Node{point, left, right} = mem::replace(self, Sentinel) {
let points = left.into_iter().chain(right.into_iter()).collect();
(*self) = KDNode::new(points);
Some(point)
} else {
None
}
Another more in-depth example is the IntoIterator for this KDTree, which has to move a curr value out of the iterator, test it, and then replace it:
// temporarily swap self.curr with a dummy value so we can
// move out of it
let tmp = mem::replace(&mut self.curr, (Sentinel,Left));
match tmp {
// If the next node is a Sentinel, that means the
// "real" next node was either the parent, or we're done
(Sentinel,_) => {
if self.stack.is_empty() {
None
} else {
self.curr = self.stack.pop().expect("Could not pop iterator parent stack");
self.next()
}
}
// If the next node is to yield the current node,
// then the next node is it's right child's leftmost
// descendent. We only "load" the right child, and lazily
// evaluate to its left child next iteration.
(Node{box right,point,..},Me) => {
self.curr = (right,Left);
Some(point)
},
// Left is an instruction to lazily find this node's left-most
// non-sentinel child, so we recurse down, pushing the parents on the
// stack as we go, and then say that our next node is our right child.
// If this child doesn't exist, then it will be taken care of by the Sentinel
// case next call.
(curr # Node{..},Left) => {
let mut curr = curr;
let mut left = get_left(&mut curr);
while !left.is_sentinel() {
self.stack.push((curr,Me));
curr = left;
left = get_left(&mut curr);
}
let (right,point) = get_right_point(curr);
self.curr = (right, Left);
Some(point)
}
As you can see, my current method is to just use mem::replace with a dummy value, and then just overwrite the dummy value later. However, I don't like this for several reasons:
In some cases, there's no suitable dummy value. This is especially true if there's no public/easy way to construct a "zero value" for one or more of your struct members (e.g. what if the struct held a MutexGuard?). If the member you need to dummy-replace is in another module (or crate), you may be bound by difficult constraints of its construction that are undesireable when trying to build a dummy type.
The struct may be rather large, in which case doing more moves than is necessary may be undesirable (in practice, this is unlikely to be a big problem, admittedly).
It just "feels" unclean, since the "move" is technically more of an "update". In fact, the simplest example might be something like *self = self.next.do_something() which will still have problems.
In some cases, such as that first remove snippet I showed, you could perhaps more cleanly represent it as a fn do_something(self) -> Self, but in other cases such as the IntoIterator example this can't be done because you're constrained by the trait definition.
Is there any better, cleaner way to do this sort of in-place update?
In any case we'll need assignment, mem::replace, mem::swap, or something like that. Because given a &mut reference to an object there is no way to move this object (or any of it's fields) out without replacing it's memory area with something valid, as long as Rust forbids references to uninitialized memory.
As for dummy values for replacement, you can always make them yourself for any type by using some wrapper type. For example, I often use Option for this purpose, where Some(T) is the value of type T, and None acts as dummy. This is what I mean:
struct Tree<T>(Option<Node<T>>);
enum Node<T> {
Leaf(T),
Children(Vec<Tree<T>>),
}
impl<T> Tree<T> where T: PartialEq {
fn remove(&mut self, value: &T) {
match self.0.take() {
Some(Node::Leaf(ref leaf_value)) if leaf_value == value =>
(),
node # Some(Node::Leaf(..)) =>
*self = Tree(node),
Some(Node::Children(node_children)) => {
let children: Vec<_> =
node_children
.into_iter()
.filter_map(|mut tree| { tree.remove(value); tree.0 })
.map(|node| Tree(Some(node)))
.collect();
if !children.is_empty() {
*self = Tree(Some(Node::Children(children)));
}
},
None =>
panic!("something went wrong"),
}
}
}
playground link
Consider the following code example, I have a vector of JoinHandlers in which I need it iterate over to join back to the main thread, however, upon doing so I am getting the error error: cannot move out of borrowed content.
let threads = Arc::new(Mutex::new(Vec::new()));
for _x in 0..100 {
let handle = thread::spawn(move || {
//do some work
}
threads.lock().unwrap().push((handle));
}
for t in threads.lock().unwrap().iter() {
t.join();
}
Unfortunately, you can't do this directly. When Mutex consumes the data structure you fed to it, you can't get it back by value again. You can only get &mut reference to it, which won't allow moving out of it. So even into_iter() won't work - it needs self argument which it can't get from MutexGuard.
There is a workaround, however. You can use Arc<Mutex<Option<Vec<_>>>> instead of Arc<Mutex<Vec<_>>> and then just take() the value out of the mutex:
for t in threads.lock().unwrap().take().unwrap().into_iter() {
}
Then into_iter() will work just fine as the value is moved into the calling thread.
Of course, you will need to construct the vector and push to it appropriately:
let threads = Arc::new(Mutex::new(Some(Vec::new())));
...
threads.lock().unwrap().as_mut().unwrap().push(handle);
However, the best way is to just drop the Arc<Mutex<..>> layer altogether (of course, if this value is not used from other threads).
As referenced in How to take ownership of T from Arc<Mutex<T>>? this is now possible to do without any trickery in Rust using Arc::try_unwrap and Mutex.into_inner()
let threads = Arc::new(Mutex::new(Vec::new()));
for _x in 0..100 {
let handle = thread::spawn(move || {
println!("{}", _x);
});
threads.lock().unwrap().push(handle);
}
let threads_unwrapped: Vec<JoinHandle<_>> = Arc::try_unwrap(threads).unwrap().into_inner().unwrap();
for t in threads_unwrapped.into_iter() {
t.join().unwrap();
}
Play around with it in this playground to verify.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=9d5635e7f778bc744d1fb855b92db178
while the drain is a good solution, you can also do the following thing
// with a copy
let built_words: Arc<Mutex<Vec<String>>> = Arc::new(Mutex::new(vec![]));
let result: Vec<String> = built_words.lock().unwrap().clone();
// using drain
let mut locked_result = built_words.lock().unwrap();
let mut result: Vec<String> = vec![];
result.extend(locked_result.drain(..));
I would prefer to clone the data to get the original value. Not sure if it has any performance overhead.