Rust error[E0373]: closure may outlive the current function, but it borrows `iteration_index` - rust

I have two vectors of itertools::MinMaxResult. I need to iterate through the first vector, and for each element, iterate through the second vector, checking if the minimum of the first vector is equal to the maximum of any elements of the second vector, and vice versa. Here is an MCVE of what I tried:
use itertools::MinMaxResult; // itertools = "0.8.0"
use itertools::MinMaxResult::*;
pub fn mcve() -> Vec<(usize, usize)> {
// dummy variables to make the MCVE compile
let num_rows = 0;
let num_cols = 0;
let row_minmax: Vec<MinMaxResult<&u64>> = vec![];
let col_minmax: Vec<MinMaxResult<&u64>> = vec![];
// Problematic code:
(0..num_rows)
.flat_map(|row_index| {
(0_usize..num_cols).filter_map(|col_index| {
match (row_minmax[row_index], col_minmax[col_index]) {
(MinMax(a, _b), MinMax(_c, d)) if a == d =>
Some((row_index, col_index)),
(MinMax(_a, b), MinMax(c, _d)) if b == c =>
Some((row_index, col_index)),
_ => None,
}
})
})
.collect::<Vec<(usize, usize)>>()
}
Link to the Playground with full code
I'm getting the following error:
error[E0373]: closure may outlive the current function, but it borrows `row_index`,
which is owned by the current function
--> src/main.rs:15:48
|
15 | (0_usize..num_cols).filter_map(|col_index| {
| ^^^^^^^^^^^ may outlive
borrowed value `row_index`
16 | match (row_minmax[row_index], col_minmax[col_index]) {
| --------- `row_index` is borrowed here
|
note: closure is returned here
--> src/main.rs:15:17
|
15 | / (0_usize..num_cols).filter_map(|col_index| {
16 | | match (row_minmax[row_index], col_minmax[col_index]) {
17 | | (MinMax(a, _b), MinMax(_c, d)) if a == d =>
Some((row_index, col_index)),
18 | | (MinMax(_a, b), MinMax(c, _d)) if b == c =>
Some((row_index, col_index)),
19 | | _ => None,
20 | | }
21 | | })
| |__________________^
help: to force the closure to take ownership of `row_index`
(and any other referenced variables), use the `move` keyword
|
15 | (0_usize..num_cols).filter_map(move |col_index| {
| ^^^^^^^^^^^^^^^^
If I add a move, as the compiler suggests, I get twice as many errors, so that doesn't help. How do I get rid of this error?

Your linked gist has a whole bunch of the typical "wrestling with the borrow-checker" errors. Here is how to solve them:
use move as the compiler proposes, but make sure that all moved values implement Copy.
Iteration indices of type usize already implement Copy, so there is nothing to do here.
The vectors are problematic, so just take immutable references of them, and pass the immutable references into the closures instead.
Your into_par_iter causes an unrelated ownership problem. Just use par_iter instead, then you will iterate over references, not the actual values, and you will not destroy your matrix while you are iterating over it.
This here compiles:
use itertools::MinMaxResult; // itertools = "0.8.0"
use itertools::MinMaxResult::*;
use itertools::Itertools;
use rayon::prelude::*; // rayon = "1.0.3"
pub fn find_saddle_points(input: &[Vec<u64>]) -> Vec<(usize, usize)> {
let flattened_matrix: Vec<(&u64)> = input.into_par_iter().flatten().collect();
if flattened_matrix.is_empty() {
vec![]
} else {
let num_rows = input.len();
let num_cols = input[0].len();
let row_minmax: Vec<MinMaxResult<&u64>> = input
.iter()
.map(|row| row.iter().minmax())
.collect::<Vec<MinMaxResult<&u64>>>();
let input_tranpose: Vec<Vec<u64>> = (0_usize..num_cols)
.into_par_iter()
.map(|col_index| {
(0_usize..num_rows)
.map(|row_index| input[row_index][col_index])
.collect::<Vec<u64>>()
})
.collect();
// don't take ownership!
// Use `par_iter` instead, then `col` will be a reference,
// and the borrow-checker is happy.
let col_minmax: Vec<MinMaxResult<&u64>> = input_tranpose
.par_iter()
.map(|col| col.iter().minmax())
.collect();
let rmm_ref = &row_minmax;
let cmm_ref = &col_minmax;
(0_usize..num_rows)
.flat_map(|row_index| {
(0_usize..num_cols).filter_map(move |col_index| {
match (rmm_ref[row_index], cmm_ref[col_index]) {
(MinMax(a, _b), MinMax(_c, d)) if a == d =>
Some((row_index, col_index)),
(MinMax(_a, b), MinMax(c, _d)) if b == c =>
Some((row_index, col_index)),
_ => None,
}
})
})
.collect::<Vec<(usize, usize)>>()
}
}
fn main(){}

Related

How to make multiple mutable references to the same element in an array using unsafe code?

I am trying to make a prime sieve in Rust.
I need several mutable references to the same element of the array, not mutable references to different parts of the array.
For the way this works, I know data races are not a relevant problem, so multiple mutable references are acceptable, but the Rust compiler does not accept my unsafe code.
I am using crossbeam 0.8.0.
fn extend_helper(
primes: &Vec<usize>,
true_block: &mut Vec<bool>,
segment_min: usize,
segment_len: usize,
) {
crossbeam::scope(|scope| {
for prime in primes {
let prime = prime.clone();
let segment_min = &segment_min;
let segment_len = &segment_len;
let shared = unsafe { &mut *true_block };
scope.spawn(move |_| {
let tmp = smallest_multiple_of_n_geq_m(prime, *segment_min) - *segment_min;
for j in (tmp..*segment_len).step_by(prime) {
shared[j] = false;
}
});
}
})
.unwrap();
}
fn smallest_multiple_of_n_geq_m(n: usize, m: usize) -> usize {
m + ((n - (m % n)) % n)
}
error[E0499]: cannot borrow `*true_block` as mutable more than once at a time
--> src/lib.rs:12:35
|
7 | crossbeam::scope(|scope| {
| ----- has type `&Scope<'1>`
...
12 | let shared = unsafe { &mut *true_block };
| ^^^^^^^^^^^^^^^^ `*true_block` was mutably borrowed here in the previous iteration of the loop
13 |
14 | / scope.spawn(move |_| {
15 | | let tmp = smallest_multiple_of_n_geq_m(prime, *segment_min) - *segment_min;
16 | | for j in (tmp..*segment_len).step_by(prime) {
17 | | shared[j] = false;
18 | | }
19 | | });
| |______________- argument requires that `*true_block` is borrowed for `'1`
warning: unnecessary `unsafe` block
--> src/lib.rs:12:26
|
12 | let shared = unsafe { &mut *true_block };
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
How I should write the unsafe in a way that Rust accepts?
Rust does not allow that in its model. You want AtomicBool with relaxed ordering.
This is a somewhat common edge case in most concurrency models - if you have two non-atomic writes of the same value to one location, is that well-defined? In Rust (and C++) it is not, and you need to explicitly use atomic.
On any platform you care about, the atomic bool store with relaxed ordering will have no impact.
For a reason why this matters, consider:
pub fn do_the_thing(x: &mut bool) {
if !*x { return };
// Do some stuff.
*x = true;
}
At setting x to true, the compiler is free to assume (due to no shared mutable references) that x is still false. It may implement this assignment as, e.g., inc x in assembly, moving its value from 0 to 1.
If two threads came through this and both reached *x = true, its value may become something other than 0 or 1, which could violate other assumed invariants.

Double for loop mutable references

I want to use a double For loop to compare each element with each and if these properties are the same, make a change to the objects. Below is a little demo code of what I want to do.
But the Rust compiler tells me here that I can't have 2 mutable references from the same object. How can I implement the whole thing differently?
fn main() {
struct S {
a: i32
}
let mut v = vec![S{a: 1}, S{a: 1}, S{a: 1}];
let size = v.len();
for i in 0..size {
for j in 0..size {
if i == j {
continue;
}
let a = &mut v[i];
let b = &mut v[j];
if a.a == b.a {
a.a = 5;
}
}
}
}
Console:
error[E0499]: cannot borrow `v` as mutable more than once at a time
--> src\main.rs:35:26
|
34 | let a = &mut v[i];
| - first mutable borrow occurs here
35 | let b = &mut v[j];
| ^ second mutable borrow occurs here
36 |
37 | if a.a == b.a {
| --- first borrow later used here
First thing to note is that you don't need two mutable references since you are only mutating v[i].
Writing it like this will create two immutable borrows and inside the if a mutable borrow:
if v[i].a == v[j].a {
v[i].a = 5;
}
Being very explicit this is what is happening (but its nicer the other way):
let a = &v[i];
let b = &v[j];
if a.a == b.a {
let mut_a = &mut v[i];
mut_a.a = 5;
}
One of the rules of the borrow checker is that no two mutable borrows can exist to the same thing at the same time.

Swapping 2 Vec properties from mutably referenced object in Rust

In my implementation of Conway's Game of Life in Rust, I've walked into a problem. I want to remember the previous state of the world. For this I've created an extra property on my Universe struct, which now looks like this:
pub struct Universe {
width: usize,
height: usize,
cells: Vec<Cell>,
previous_cells: Option<Vec<Cell>>,
}
The Option is there because at initialization, there is no previous state.
My idea would be that on every tick, we write the new state to the vector with the previous state. At the end, I swap the vectors. This resulted in the following code:
pub fn tick(&mut self) {
// get the previous state or create new one if it doesn't exist (i.e. this is the first tick/update)
let mut next = match self.previous_cells {
Some(cells) => cells,
None => {
let mut new_vec = Vec::with_capacity(self.width() * self.height());
new_vec.resize(new_vec.capacity(), Cell::Dead);
new_vec
}
};
// Calculate next state, not relevant for question
for row in 0..self.height as isize {
for column in 0..self.width as isize {
let index = self.get_index(row , column);
let target_cell = self.cells[index];
let live_neighbors = self.live_neighbor_count(row, column);
next[index] = match target_cell {
Cell::Alive => if live_neighbors == 2 || live_neighbors == 3 { Cell::Alive } else { Cell::Dead }
Cell::Dead => if live_neighbors == 3 { Cell::Alive } else { Cell::Dead }
}
}
}
// swap values
self.previous_cells = Some(self.cells);
self.cells = next;
}
Now I get the following errors from the compiler:
error[E0507]: cannot move out of `self.previous_cells.0` which is behind a mutable reference
--> src/lib.rs:61:30
|
61 | let mut next = match self.previous_cells {
| ^^^^^^^^^^^^^^^^^^^ help: consider borrowing here: `&self.previous_cells`
62 | Some(cells) => cells,
| -----
| |
| data moved here
| move occurs because `cells` has type `std::vec::Vec<Cell>`, which does not implement the `Copy` trait
error[E0507]: cannot move out of `self.cells` which is behind a mutable reference
--> src/lib.rs:83:36
|
83 | self.previous_cells = Some(self.cells);
| ^^^^^^^^^^ move occurs because `self.cells` has type `std::vec::Vec<Cell>`, which does not implement the `Copy` trait
I get why this happens, but I don't see a solution that does not include copying the entire Vec. That would be a solution, but is there maybe a better one?

Cannot borrow as mutable in a loop when calling a closure that borrows as immutable?

Here is the code:
fn test(){
let mut numbers = vec![2];
let f = || {
for _ in numbers.iter(){
}
false
};
while false {
let res = f();
if res {
numbers.push(10);
}
}
}
The error is:
|
15 | let f = || {
| -- immutable borrow occurs here
16 | for _ in numbers.iter(){
| ------- first borrow occurs due to use of `numbers` in closure
...
22 | let res = f();
| - immutable borrow later used here
23 | if res {
24 | numbers.push(10);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
But if I change the while keyword to if, it can be compiled. How to fix this? I want to call the anonymous function in a loop.
We can simplify your example even more by replacing the closure by a simple immutable reference
let mut numbers = vec![2];
let r = &numbers;
while false {
println!("{:?}", r);
numbers.push(10);
}
Here we get this error:
error[E0502]: cannot borrow `numbers` as mutable because it is also borrowed as immutable
--> src/lib.rs:7:5
|
3 | let r = &numbers;
| -------- immutable borrow occurs here
...
6 | println!("{:?}", r); // use reference
| - immutable borrow later used here
7 | numbers.push(10);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
And like in your example, replacing the while with if makes the error go away. Why?
You probably know about the important Rust rule: Aliasing nand mutability. It states that, at any given time, a value can either be borrowed immutably arbitrarily many times or mutably exactly once.
The statement numbers.push(10) borrows numbers mutably temporarily (just for the statement). But we also have r which is an immutable reference. In order for numbers.push(10) to work, the compiler has to make sure that no other borrow exists at that time. But there exists the reference r! This reference cannot exist at the same time as numbers.push(10) exists.
Let's see for the if case first:
let mut numbers = vec![2];
let r = &numbers; // <------+ (creation of r)
// |
if false { // |
println!("{:?}", r); // <------+ (last use of r)
numbers.push(10);
}
While the lexical scope means the variable r is only dropped at the end of the function, due to non-lexical lifetimes, the compiler can see that the last use of r is in the println line. Then the compiler can mark r as "dead" after this line. And this in turn means, that there is no other borrow in the line numbers.push(10) and everything works out fine.
And for the loop case? Let's imagine the loop iterating three times:
let mut numbers = vec![2];
let r = &numbers; // <------+ (creation of r)
// |
// First iteration // |
println!("{:?}", r); // |
numbers.push(10); // | <== oh oh!
// |
// Second iteration // |
println!("{:?}", r); // |
numbers.push(10); // |
// |
// Third iteration // |
println!("{:?}", r); // <------+ (last use of r)
numbers.push(10);
As can be seen here, the time in which r is active overlaps numbers.push(10) (except the last one). And as a result, the compiler will produce an error because this code violates the central Rust rule.
And the explanation is the same for your closure case: the closure borrows numbers immutably and f(); uses that closure. In the loop case, the compiler is not able to shrink the "alive time" of the closure enough to make sure it doesn't overlap the mutable borrow for push.
How to fix?
Well, you could pass numbers into the closure each time:
let mut numbers = vec![2];
let f = |numbers: &[i32]| {
for _ in numbers.iter(){
}
false
};
while false {
let res = f(&numbers);
if res {
numbers.push(10);
}
}
This works because now, numbers is borrowed immutably also just temporarily for the f(&numbers); statement.
You can also use a RefCell as the other answer suggested, but that should be a last resort.
It's not exactly sure what you're trying to accomplish, but one way to solve this, without changing your code too drastically, would be to use std::cell::RefCell (described in the std and in the book):
use std::cell::RefCell;
fn test(){
let numbers = RefCell::new(vec![2]);
let f = || {
for _ in numbers.borrow().iter(){
}
false
};
while false {
let res = f();
if res {
numbers.borrow_mut().push(10);
}
}
}
Here's a little bit tweaked demo, which actually does something:
use std::cell::RefCell;
fn main() {
test();
}
fn test() {
let numbers = RefCell::new(vec![0]);
let f = || {
for n in numbers.borrow().iter() {
println!("In closure: {}", n);
}
println!();
true
};
let mut i = 1;
while i <= 3 {
let res = f();
if res {
numbers.borrow_mut().push(i);
}
i += 1;
}
println!("End of test(): {:?}", numbers.borrow());
}
Output:
In closure: 0
In closure: 0
In closure: 1
In closure: 0
In closure: 1
In closure: 2
End of test(): [0, 1, 2, 3]
Rust Playground demo

Borrow checker issue in `zip`-like function with a callback

I'm trying to implement a function that steps though two iterators at the same time, calling a function for each pair. This callback can control which of the iterators is advanced in each step by returning a (bool, bool) tuple. Since the iterators take a reference to a buffer in my use case, they can't implement the Iterator trait from the stdlib, but instead are used though a next_ref function, which is identical to Iterator::next, but takes an additional lifetime parameter.
// An iterator-like type, that returns references to itself
// in next_ref
struct RefIter {
value: u64
}
impl RefIter {
fn next_ref<'a>(&'a mut self) -> Option<&'a u64> {
self.value += 1;
Some(&self.value)
}
}
// Iterate over two RefIter simultaneously and call a callback
// for each pair. The callback returns a tuple of bools
// that indicate which iterators should be advanced.
fn each_zipped<F>(mut iter1: RefIter, mut iter2: RefIter, callback: F)
where F: Fn(&Option<&u64>, &Option<&u64>) -> (bool, bool)
{
let mut current1 = iter1.next_ref();
let mut current2 = iter2.next_ref();
loop {
let advance_flags = callback(&current1, &current2);
match advance_flags {
(true, true) => {
current1 = iter1.next_ref();
current2 = iter2.next_ref();
},
(true, false) => {
current1 = iter1.next_ref();
},
(false, true) => {
current2 = iter1.next_ref();
},
(false, false) => {
return
}
}
}
}
fn main() {
let mut iter1 = RefIter { value: 3 };
let mut iter2 = RefIter { value: 4 };
each_zipped(iter1, iter2, |val1, val2| {
let val1 = *val1.unwrap();
let val2 = *val2.unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:28:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
28 | current1 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter2` as mutable more than once at a time
--> src/main.rs:29:28
|
23 | let mut current2 = iter2.next_ref();
| ----- first mutable borrow occurs here
...
29 | current2 = iter2.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:32:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
32 | current1 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:35:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
35 | current2 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
I understand why it complains, but can't find a way around it. I'd appreciate any help on the subject.
Link to this snippet in the playground.
Since the iterators take a reference to a buffer in my use case, they can't implement the Iterator trait from the stdlib, but instead are used though a next_ref function, which is identical to Iterator::next, but takes an additional lifetime parameter.
You are describing a streaming iterator. There is a crate for this, aptly called streaming_iterator. The documentation describes your problem (emphasis mine):
While the standard Iterator trait's functionality is based off of
the next method, StreamingIterator's functionality is based off of
a pair of methods: advance and get. This essentially splits the
logic of next in half (in fact, StreamingIterator's next method
does nothing but call advance followed by get).
This is required because of Rust's lexical handling of borrows (more
specifically a lack of single entry, multiple exit borrows). If
StreamingIterator was defined like Iterator with just a required
next method, operations like filter would be impossible to define.
The crate does not currently have a zip function, and certainly not the variant you have described. However, it's easy enough to implement:
extern crate streaming_iterator;
use streaming_iterator::StreamingIterator;
fn each_zipped<A, B, F>(mut iter1: A, mut iter2: B, callback: F)
where
A: StreamingIterator,
B: StreamingIterator,
F: for<'a> Fn(Option<&'a A::Item>, Option<&'a B::Item>) -> (bool, bool),
{
iter1.advance();
iter2.advance();
loop {
let advance_flags = callback(iter1.get(), iter2.get());
match advance_flags {
(true, true) => {
iter1.advance();
iter2.advance();
}
(true, false) => {
iter1.advance();
}
(false, true) => {
iter1.advance();
}
(false, false) => return,
}
}
}
struct RefIter {
value: u64
}
impl StreamingIterator for RefIter {
type Item = u64;
fn advance(&mut self) {
self.value += 1;
}
fn get(&self) -> Option<&Self::Item> {
Some(&self.value)
}
}
fn main() {
let iter1 = RefIter { value: 3 };
let iter2 = RefIter { value: 4 };
each_zipped(iter1, iter2, |val1, val2| {
let val1 = *val1.unwrap();
let val2 = *val2.unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
The issue with this code is that RefIter is being used in two ways, which are fundamentally at odds with one another:
Callers of next_ref recieve a reference to the stored value, which is tied to the lifetime of RefIter
RefIter's value needs to be mutable, so that it can be incremented on each call
This perfectly describes mutable aliasing (you're trying to modify 'value' while a reference to it is being held) - something which Rust is explicitly designed to prevent.
In order to make each_zipped work, you'll need to modify RefIter to avoid handing out references to data that you wish to mutate.
I've implemented one possibility below using a combination of RefCell and Rc:
use std::cell::RefCell;
use std::rc::Rc;
// An iterator-like type, that returns references to itself
// in next_ref
struct RefIter {
value: RefCell<Rc<u64>>
}
impl RefIter {
fn next_ref(&self) -> Option<Rc<u64>> {
let new_val = Rc::new(**self.value.borrow() + 1);
*self.value.borrow_mut() = new_val;
Some(Rc::clone(&*self.value.borrow()))
}
}
// Iterate over two RefIter simultaniously and call a callback
// for each pair. The callback returns a tuple of bools
// that indicate which iterators should be advanced.
fn each_zipped<F>(iter1: RefIter, iter2: RefIter, callback: F)
where F: Fn(&Option<Rc<u64>>, &Option<Rc<u64>>) -> (bool, bool)
{
let mut current1 = iter1.next_ref();
let mut current2 = iter2.next_ref();
loop {
let advance_flags = callback(&current1, &current2);
match advance_flags {
(true, true) => {
current1 = iter1.next_ref();
current2 = iter2.next_ref();
},
(true, false) => {
current1 = iter1.next_ref();
},
(false, true) => {
current2 = iter1.next_ref();
},
(false, false) => {
return
}
}
}
}
fn main() {
let iter1 = RefIter { value: RefCell::new(Rc::new(3)) };
let iter2 = RefIter { value: RefCell::new(Rc::new(4)) };
each_zipped(iter1, iter2, |val1, val2| {
// We can't use unwrap() directly, since we're only passed a reference to an Option
let val1 = **val1.iter().next().unwrap();
let val2 = **val2.iter().next().unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
This version of RefIter hands out Rcs to consumers, instead of references. This avoids the issue of mutable aliasing - updating value is done by placing
a new Rc into the outer RefCell. A side effect of this is that consumers are able to hold onto an 'old' reference to the buffer (through the returned Rc), even after RefIter has advanced.

Resources