This question already has an answer here:
Value does not live long enough
(1 answer)
Closed 4 years ago.
I'm trying to turn a matrix A into matrix B where B_11 would be the biggest element of the first row and the smallest element of the second row in a tuple. So the matrix [1,2;3,4] would become [(2,1), (2,2); (4,1), (4,2)].
pub fn find_saddle_points(input: &[Vec<u64>]) -> Vec<(usize, usize)> {
let answer = Vec::new();
println!("matrix: {:?}", input);
let x: Vec<Vec<_>> = input
.iter()
.map(|row| {
row.iter()
.map(move |_| {
let y = get_column(&input, 0);
println!("y is {:?}", y);
let x = get_lowest(&y);
(get_highest(row), x)
}).collect()
}).collect();
println!("x is {:?}", x);
answer
}
fn get_highest(row: &Vec<u64>) -> &u64 {
row.iter().max().unwrap()
}
fn get_lowest(column: &Vec<u64>) -> &u64 {
column.iter().min().unwrap()
}
fn get_column(matrix: &[Vec<u64>], index: usize) -> Vec<u64> {
matrix.iter().map(|row| row[index]).collect()
}
I'm getting an error messages when I try to use y:
error[E0597]: `y` does not live long enough
--> src/lib.rs:11:41
|
11 | let x = get_lowest(&y);
| ^ borrowed value does not live long enough
12 | (get_highest(row), x)
13 | }).collect()
| - `y` dropped here while still borrowed
...
17 | }
| - borrowed value needs to live until here
I don't understand why the value y has to live until after it's used. Isn't it enough that it lives until after the get_lowest function?
How would I fix this?
You can use the clone() function to pass new copies of the vectors to the functions get_highest and get_lowest and make the functions return integers and not references with the funciton to_owned().
pub fn find_saddle_points(input: &[Vec<u64>]) -> Vec<(usize, usize)> {
let answer = Vec::new();
println!("matrix: {:?}", input);
let x: Vec<Vec<_>> = input
.iter()
.map(|row| {
row.iter()
.map(move |_| {
let y = get_column(&input, 0);
println!("y is {:?}", y);
let x = get_lowest(y.clone());
(get_highest(row), x)
}).collect()
}).collect();
println!("x is {:?}", x);
answer
}
fn get_highest(row: Vec<u64>) -> u64 {
row.iter().max().unwrap().to_owned()
}
fn get_lowest(column: Vec<u64>) -> u64 {
column.iter().min().unwrap().to_owned()
}
fn get_column(matrix: &[Vec<u64>], index: usize) -> Vec<u64> {
matrix.iter().map(|row| row[index]).collect()
}
And as #shepmaster posted, you should avoid using references of Vectors as function parameters.
Shepmaster explained it well
TL;DR — you created the Vec y and then got a reference to the vector
via get_highest and get_lowest. When you exit the closure, Vec is
destroyed and the references would point at nothing. The compiler is
preventing you from making a massive mistake that would lead to memory
unsafety in a language like C or C++.
Related
CONTEXT
General overview
(Here is the github page with the minimal example of my problem, and the page of my whole project)
I'm very new to rust and I'm trying to simulate the behavior of a fluid in Rust. This is easy: computing large arrays with some functions for each timestep.
I'd like to parallelize the computations done each timestep using rayon. But the compiler doesn't want me to access a mutable struct containing an Array that I want to modify, even if I'm sure that each modification will be on different places in the array: which assure me that it's safe. (I think?).
So my question is: should I use unsafe rust in here? If so, how?
And: is it possible to make Rust understand that it's safe, or to do it properly without unsafe rust?
I saw this post which kind of resembled my problem, but couldn't find a way to use the solution for my problem.
I also tried to put unsafe {...} keywords everywhere, but the compilator still complains...
You may only need to read "Structs" subsection to understand the problem, but I will also put a "Function" subsection, in case it can be important. If you think it might not be necessary, you can skip to "Main function" subsection.
Structs
Here are my structs:
I'd like to keep them that way, as they would give me (I think) more flexibility with setters/getters, but I'm open to change the way it's implemented right now.
#[derive(Debug, PartialEq)]
struct vec2D {pub x: f64, pub y: f64}
#[derive(Debug, PartialEq)]
struct ScalarField2D {
s: Array2<f64>,
}
#[derive(Debug, PartialEq)]
struct VectorField2D {
x: ScalarField2D,
y: ScalarField2D
}
impl ScalarField2D {
// also a constructor new() but not shown for simplicity
fn get_pos(&self, x: usize, y: usize) -> f64
{return self.s[[y,x]];}
fn set_pos(&mut self, x: usize, y: usize, f: f64)
{self.s[[y,x]] = f;}
}
impl VectorField2D {
// also a constructor new() but not shown for simplicity
fn get_pos(&self, x: usize, y: usize) -> vec2D
{let vec_at_pos = vec2D {
x: self.x.get_pos(x, y),
y: self.y.get_pos(x, y)};
return vec_at_pos;}
fn set_pos(&mut self, x: usize, y: usize, vec: &vec2D)
{self.x.set_pos(x, y, vec.x);
self.y.set_pos(x, y, vec.y);}
}
Function
Here is my function: which takes a ScalarField2D struct, and computes a vector called the "gradient" at a particular position of the ScalarField2D array, and then returning this vector as a "vec2D" struct.
// computes the gradient of a scalar field at a given position
fn grad_scalar(a: &ScalarField2D,
x: i32, y: i32,
x_max: i32, y_max: i32) -> vec2D
{
let ip = ((x+1) % x_max) as usize;
// i-1 with Periodic Boundaries
let im = ((x - 1 + x_max) % x_max) as usize;
// j+1 with Periodic Boundaries
let jp = ((y+1) % y_max) as usize;
// j-1 with Periodic Boundaries
let jm = ((y - 1 + y_max) % y_max) as usize;
let (i, j) = (x as usize, y as usize);
let grad = vec2D {
x: (a.get_pos(ip, j) - a.get_pos(im, j))/(2.),
y: (a.get_pos(i, jp) - a.get_pos(i, jm))/(2.)};
return grad;
}
Main function
Here is my problem:
I try to iterate over all the possible x and y using (0..x_max).into_par_iter() (or y_max for y), compute the gradient associated with each position, and then set the value to the ScalarField2D struct using the set_pos method... Here is the main function, and the import commands, and I will show you the error message I get in the next subsection
use ndarray::prelude::*;
use rayon::prelude::*;
fn main() {
let (x_max, y_max) = (2usize, 50usize);
let (x_maxi32, y_maxi32) = (x_max as i32, y_max as i32);
let mut GD_grad_rho = VectorField2D::new(x_max, y_max);
let mut GD_rho = ScalarField2D::new(x_max, y_max);
let x_iterator = (0..x_max).into_par_iter();
x_iterator.map(|xi| {
let y_iterator = (0..y_max).into_par_iter();
y_iterator.map(|yi| {
// unsafe here?
GD_grad_rho
.set_pos(xi, yi,
&grad_scalar(&GD_rho,
xi as i32, yi as i32,
x_maxi32, y_maxi32));
});});
}
Error message
Here is the compilation error I get
error[E0596]: cannot borrow `GD_grad_rho` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:104:13
|
104 | / GD_grad_rho
105 | | .set_pos(xi, yi,
106 | | &grad_scalar(&GD_rho,
107 | | xi as i32, yi as i32,
108 | | x_maxi32, y_maxi32));
| |__________________________________________________________^ cannot borrow as mutable
error[E0596]: cannot borrow `GD_grad_rho` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:101:24
|
101 | y_iterator.map(|yi| {
| ^^^^ cannot borrow as mutable
...
104 | GD_grad_rho
| ----------- mutable borrow occurs due to use of `GD_grad_rho` in closure
For more information about this error, try `rustc --explain E0596`.
error: could not compile `minimal_example_para` due to 2 previous errors
If you want the complete thing, I created a github repo with everything in it.
Tests after susitsm answer
So I did something like this:
use ndarray::prelude::*;
use rayon::prelude::*;
fn grad_scalar(a: &Array2<f64>,
i: usize, j: usize) -> (f64, f64)
{
let array_shape = a.shape();
let imax = array_shape[0];
let jmax = array_shape[1];
// i-1 with Periodic Boundaries
let ip = ((i + 1) % imax);
// i-1 with Periodic Boundaries
let im = ((i + imax) - 1) % imax;
// j+1 with Periodic Boundaries
let jp = ((j + 1) % jmax);
// j-1 with Periodic Boundaries
let jm = ((j + jmax) - 1) % jmax;
let grad_i = (a[[ip, j]] - a[[im, j]])/2.;
let grad_j = (a[[i, jp]] - a[[i, jm]])/2.;
return (grad_i, grad_j);
}
fn main() {
let a = Array::<f64, Ix2>::ones((dim, dim));
let index_list: Vec<(_, _)> = (0..a.len())
.into_par_iter()
.map(|i| (i / a.dim().0, i % a.dim().1))
.collect();
let (r1, r2): (Vec<_>, Vec<_>) = (0..a.len())
.into_par_iter()
.map(|i| (index_list[i].0, index_list[i].1))
.map(|(i, j)| grad_scalar(&a, i, j))
.collect();
let grad_row = Array2::from_shape_vec(a.dim(), r1).unwrap();
let grad_col = Array2::from_shape_vec(a.dim(), r2).unwrap();
}
Which gives me the result I want, even though I wanted to access the values through my Structs. Which is not exactly what I want but we're getting closer
But I wonder about the efficiency for more arrays, I'll post a separate question for that
You can do something like this:
use ndarray::Array2;
use rayon::prelude::*;
fn main() {
let a: Vec<u64> = (0..20000).collect();
let a = Array2::from_shape_vec((100, 200), a).unwrap();
let stuff = |arr, i, j| (i + j, i * j);
let (x, y): (Vec<_>, Vec<_>) = (0..a.len())
.into_par_iter()
.map(|i| (i / a.dim().0, i % a.dim().1))
.map(|(i, j)| stuff(&a, i, j))
.collect();
let grad_x = Array2::from_shape_vec(a.dim(), x);
let grad_y = Array2::from_shape_vec(a.dim(), y);
let grad_vector_field = VectorField2D {
x: ScalarField2D { s: grad_x },
y: ScalarField2D { s: grad_y },
};
}
Or implement the FromParallelIterator<vec2D>
impl FromParallelIterator<vec2D> for VectorField2D {
fn from_par_iter<I>(par_iter: I) -> Self
where I: IntoParallelIterator<Item = T>
{
let (x, y): (Vec<_>, Vec<_>) = par_iter
.into_par_iter()
.map(|vec_2D| {
let vec2D { x, y } = vec_2D;
(x, y)
})
.collect();
Self {
x: ScalarField2D { s: Array2::from_shape_vec(a.dim(), x) },
y: ScalarField2D { s: Array2::from_shape_vec(a.dim(), y) },
}
}
}
This will enable using collect for your type when using parallel iterators
let vector_field: VectorField2D = (0..a.len())
.into_par_iter()
.map(|i| (index_list[i].0, index_list[i].1))
.map(|(i, j)| grad_scalar(&a, i, j))
.collect();
Suppose the following Rust snippet:
use std::borrow::Cow;
fn char_indices_from(s: &str, offset: usize) -> impl Iterator<Item=(usize, char)> + '_ {
s[offset..].char_indices().map(move |(i,c)| (i+offset,c))
}
fn main() {
let mut m = Cow::from("watermelons and stuff");
let offset = 2;
for (i, c) in char_indices_from(&m, offset) {
if i == 3 {
m = Cow::from("clouds and the sky");
break
}
}
}
This displeases the borrow checker:
error[E0506]: cannot assign to `m` because it is borrowed
--> src/main.rs:12:13
|
10 | for (i, c) in char_indices_from(&m, offset) {
| -----------------------------
| | |
| | borrow of `m` occurs here
| a temporary with access to the borrow is created here ...
11 | if i == 3 {
12 | m = Cow::from("clouds and the sky");
| ^ assignment to borrowed `m` occurs here
...
15 | }
| - ... and the borrow might be used here, when that temporary is dropped and runs the destructor for type `impl Iterator<Item = (usize, char)>`
Doing this, however, works just fine:
use std::borrow::Cow;
fn main() {
let mut m = Cow::from("watermelons and stuff");
let offset = 2;
for (i, c) in m[offset..].char_indices().map(|(i,c)| (i+offset, c)) {
if i == 3 {
m = Cow::from("clouds and the sky");
break
}
}
}
Those are some excellent diagnostics given by rustc. Nevertheless, I find myself confused as to how one would fix char_indices_from such that the first program satisfies Rust's borrowing rules.
Your assumption is that you can overwrite m because it's the last thing you do before break.
It's true that the Rust borrow checker is smart enough to figure this out; your second example proves this.
The borrow checker rightfully complains about the first example, though, because you forget destructors, meaning, the Drop trait. Because your return type is impl Iterator + '_, it has to assume this could be any type that implements Iterator and depends on the input lifetimes. Which includes types that use the borrowed values in their Drop implementation. This is also what the compiler tries to tell you.
You could fix that by replacing the impl return type with the actual type, proving to the borrow checker that there is no Drop implementation. Although you will also get problems with that, because your type contains a closure whose type cannot be named.
That's why usually these things return their own iterator type (for example the itertools crate, none of their functions have an impl return type).
So that's what I would do: implement your own iterator return type.
use std::{borrow::Cow, str::CharIndices};
struct CharIndicesFrom<'a> {
raw_indices: CharIndices<'a>,
offset: usize,
}
impl Iterator for CharIndicesFrom<'_> {
type Item = (usize, char);
fn next(&mut self) -> Option<Self::Item> {
self.raw_indices.next().map(|(i, c)| (i + self.offset, c))
}
}
fn char_indices_from(s: &str, offset: usize) -> CharIndicesFrom<'_> {
CharIndicesFrom {
raw_indices: s[offset..].char_indices(),
offset,
}
}
fn main() {
let mut m = Cow::from("watermelons and stuff");
let offset = 2;
for (i, c) in char_indices_from(&m, offset) {
if i == 3 {
m = Cow::from("clouds and the sky");
break;
}
}
}
I have a Cons list:
#[derive(Debug)]
pub enum Cons {
Empty,
Pair(i64, Box<Cons>),
}
I want to implement FromIterator<i64> for this type, in an efficient manner.
Attempt one is straightforward: implement a push method which recursively traverses the list and transforms a Cons::Empty into a Cons::Pair(x, Box::new(Cons::Empty)); repeatedly call this push method. This operation is O(n^2) in time and O(n) in temporary space for the stack frames.
Attempt two will combine the recursion with the iterator to improve the time performance: by pulling a single item from the iterator to construct a Cons::Pair and then recursing to construct the remainder of the list, we now construct the list in O(n) time and O(n) temporary space:
impl FromIterator<i64> for Cons {
fn from_iter<I>(iter: I) -> Self
where
I: IntoIterator<Item = i64>,
{
let mut iter = iter.into_iter();
match iter.next() {
Some(x) => Cons::Pair(x, Box::new(iter.collect())),
None => Cons::Empty,
}
}
}
In C, it would be possible to implement this method using O(n) operations and O(1) working space size. However, I cannot translate it into Rust. The idea is simple, but it requires storing two mutable pointers to the same value; something that Rust forbids. A failed attempt:
impl FromIterator<i64> for Cons {
fn from_iter<I>(iter: I) -> Self
where
I: IntoIterator<Item = i64>,
{
let mut iter = iter.into_iter();
let ret = Box::new(Cons::Empty);
let mut cursor = ret;
loop {
match iter.next() {
Some(x) => {
let mut next = Box::new(Cons::Empty);
*cursor = Cons::Pair(x, next);
cursor = next;
}
None => break,
}
}
return *ret;
}
}
error[E0382]: use of moved value: `next`
--> src/lib.rs:20:30
|
18 | let mut next = Box::new(Cons::Empty);
| -------- move occurs because `next` has type `Box<Cons>`, which does not implement the `Copy` trait
19 | *cursor = Cons::Pair(x, next);
| ---- value moved here
20 | cursor = next;
| ^^^^ value used here after move
error[E0382]: use of moved value: `*ret`
--> src/lib.rs:25:16
|
13 | let ret = Box::new(Cons::Empty);
| --- move occurs because `ret` has type `Box<Cons>`, which does not implement the `Copy` trait
14 | let mut cursor = ret;
| --- value moved here
...
25 | return *ret;
| ^^^^ value used here after move
Is it possible to perform this algorithm in safe Rust? How else could I implement an efficient FromIterator for my Cons type? I understand that I may be able to make some headway by switching Box to Rc, but I'd like to avoid this if possible.
You are attempting to have two owners of a single variable, but Rust only allows a single owner. You do this twice: once for ret and once for next. Instead, use mutable references.
I chose to introduce a last() method which can be used in an implementation of Extend and participate in more abstractions.
#[derive(Debug)]
pub enum Cons {
Empty,
Pair(i64, Box<Cons>),
}
impl Cons {
fn last(&mut self) -> &mut Self {
let mut this = self;
loop {
eprintln!("1 loop turn");
match this {
Cons::Empty => return this,
Cons::Pair(_, next) => this = next,
}
}
}
}
impl FromIterator<i64> for Cons {
fn from_iter<I>(iter: I) -> Self
where
I: IntoIterator<Item = i64>,
{
let mut this = Cons::Empty;
this.extend(iter);
this
}
}
impl Extend<i64> for Cons {
fn extend<I>(&mut self, iter: I)
where
I: IntoIterator<Item = i64>,
{
let mut this = self.last();
for i in iter {
eprintln!("1 loop turn");
*this = Cons::Pair(i, Box::new(Cons::Empty));
this = match this {
Cons::Empty => unreachable!(),
Cons::Pair(_, next) => next,
};
}
}
}
fn main() {
dbg!(Cons::from_iter(0..10));
}
This produces
Pair(0, Pair(1, Pair(2, Pair(3, Pair(4, Pair(5, Pair(6, Pair(7, Pair(8, Pair(9, Empty))))))))))
0 -> 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> ⏚
See also:
Adding an append method to a singly linked list
How to implement an addition method of linked list?
How do I return a mutable reference to the last element in a singly linked list to append an element?
Learn Rust With Entirely Too Many Linked Lists
I am trying to pass around a HashMap which stores values through a set of nested enums/structs. The problem of multiple mutability happens during iteration, even all references should be dropped.
The general idea is to have a vector of values, iterate through them and simplify them, keeping track of them within the HashMap. There are two stages of simplification.
The general flow looks something like
run(Vec<ComplexVal>)
-for each val->
val.fix_complex(holder)
-for each `smp` SimpleVal in val->
basicval = Simplifier::step(smp, holder)
holder.insert("name", basicval)
But the problem is that the holder is borrowed mutably in each stage, and there isn't supposed to be any reference from the ComplexVal to the holder and since the borrowchecker doesn't like multiple borrows, it fails.
Full playground snippet: here
It happens in this snippet:
pub fn run(&mut self, mut vals: Vec<ComplexVal>) {
let mut holder = Holder{hold:HashMap::new()};
// .. setup holder code omitted
let len = vals.len();
for _ in 0..len {
let mut val = vals.remove(0); // remove from vec, should drop after running
println!("Running {:?}", val);
match val {
ComplexVal::Cmplx1(mut c) => {
c.fix_complex(&mut holder)
},
//... more cases of different types of values omitted for simplicity
}
// val *should* be dropped here, and therefore the mutable borrow of holder?
}
println!("Holder: {:?}", holder);
}
}
The only thing I can think of is that it somehow is related to the BasicVal::Ref(&BasicVal) value when created.
I need to return a reference of type &BasicVal so I can't use a regular fn() -> &BasicVal as the reference would be dangling, so I pass a ret value which is to be modified and used as the storage for the return value.
I have also tried just returning the enum BasicVal::Ref(&BasicVal), but run into the same mutability issues.
The example below is a much more simple version which (sort of) demonstrates the same error, just thought I'd include this context in case someone has another idea on how to implement this which wouldn't have these issues
Code (edited)
Updated playground link
Edit: I made a mistake in not needing the lifetimes of both holder and ret to explicitly be the same, so I have made an updated example for it
use std::borrow::BorrowMut;
///////////////////////////////
use std::cell::{RefCell, RefMut};
use std::collections::HashMap;
#[derive(Debug)]
enum BasicVal<'a> {
Ref(&'a BasicVal<'a>),
Val1(BasicStruct),
}
#[derive(Debug)]
struct Holder<'b> {
hold: HashMap<String, RefCell<BasicVal<'b>>>,
}
#[derive(Debug)]
struct BasicStruct {
val: i32,
}
impl<'a> BasicVal<'a> {
pub fn empty() -> Self { BasicVal::Val1(BasicStruct { val: 0 }) }
}
// must match sig of modify_val_ref
fn modify_val<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
*ret = BasicVal::Val1(BasicStruct { val: 5 });
}
// must match sig of modify_val
fn modify_val_ref<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
ret = holder.hold.get("reference_val").unwrap().borrow_mut();
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
let mut v = RefCell::new(BasicVal::empty());
println!("Original {:?}", v);
modify_val(holder, v.borrow_mut());
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", holder.hold.get("Data"));
}
pub fn test_dropborrow() {
let mut holder = Holder { hold: HashMap::new() };
holder.hold.insert(
"reference_val".to_string(),
RefCell::new(BasicVal::Val1(BasicStruct { val: 8 })),
);
do_modify(&mut holder);
}
pub fn main() {
test_dropborrow();
}
Edit: Using just the holder for a temp return value gives me a multiple mutable borrow issue, so that workaround doesn't work. I have also tried it with a RefCell with the same issue.
fn modify_val<'f>(holder: &'f mut Holder<'f>) {
holder.hold.insert("$return".to_string(), BasicVal::Val1(BasicStruct{val: 5}));
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
modify_val(holder);
let mut v = holder.hold.remove("$return").unwrap();
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", v);
}
Error:
935 | fn do_modify<'f>(holder: &'f mut Holder<'f>) {
| -- lifetime `'f` defined here
936 |
937 | modify_val(holder);
| ------------------
| | |
| | first mutable borrow occurs here
| argument requires that `*holder` is borrowed for `'f`
938 | let mut v = holder.hold.remove("$return").unwrap();
| ^^^^^^^^^^^ second mutable borrow occurs here
Any help is greatly appreciated!!!
Figured it out, essentially the BasicVal<'a> was causing Holder to mutably borrow itself in successive iterations of the loop, so removing the lifetime was pretty much the only solution
I'm trying to implement a function that steps though two iterators at the same time, calling a function for each pair. This callback can control which of the iterators is advanced in each step by returning a (bool, bool) tuple. Since the iterators take a reference to a buffer in my use case, they can't implement the Iterator trait from the stdlib, but instead are used though a next_ref function, which is identical to Iterator::next, but takes an additional lifetime parameter.
// An iterator-like type, that returns references to itself
// in next_ref
struct RefIter {
value: u64
}
impl RefIter {
fn next_ref<'a>(&'a mut self) -> Option<&'a u64> {
self.value += 1;
Some(&self.value)
}
}
// Iterate over two RefIter simultaneously and call a callback
// for each pair. The callback returns a tuple of bools
// that indicate which iterators should be advanced.
fn each_zipped<F>(mut iter1: RefIter, mut iter2: RefIter, callback: F)
where F: Fn(&Option<&u64>, &Option<&u64>) -> (bool, bool)
{
let mut current1 = iter1.next_ref();
let mut current2 = iter2.next_ref();
loop {
let advance_flags = callback(¤t1, ¤t2);
match advance_flags {
(true, true) => {
current1 = iter1.next_ref();
current2 = iter2.next_ref();
},
(true, false) => {
current1 = iter1.next_ref();
},
(false, true) => {
current2 = iter1.next_ref();
},
(false, false) => {
return
}
}
}
}
fn main() {
let mut iter1 = RefIter { value: 3 };
let mut iter2 = RefIter { value: 4 };
each_zipped(iter1, iter2, |val1, val2| {
let val1 = *val1.unwrap();
let val2 = *val2.unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:28:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
28 | current1 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter2` as mutable more than once at a time
--> src/main.rs:29:28
|
23 | let mut current2 = iter2.next_ref();
| ----- first mutable borrow occurs here
...
29 | current2 = iter2.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:32:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
32 | current1 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:35:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
35 | current2 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
I understand why it complains, but can't find a way around it. I'd appreciate any help on the subject.
Link to this snippet in the playground.
Since the iterators take a reference to a buffer in my use case, they can't implement the Iterator trait from the stdlib, but instead are used though a next_ref function, which is identical to Iterator::next, but takes an additional lifetime parameter.
You are describing a streaming iterator. There is a crate for this, aptly called streaming_iterator. The documentation describes your problem (emphasis mine):
While the standard Iterator trait's functionality is based off of
the next method, StreamingIterator's functionality is based off of
a pair of methods: advance and get. This essentially splits the
logic of next in half (in fact, StreamingIterator's next method
does nothing but call advance followed by get).
This is required because of Rust's lexical handling of borrows (more
specifically a lack of single entry, multiple exit borrows). If
StreamingIterator was defined like Iterator with just a required
next method, operations like filter would be impossible to define.
The crate does not currently have a zip function, and certainly not the variant you have described. However, it's easy enough to implement:
extern crate streaming_iterator;
use streaming_iterator::StreamingIterator;
fn each_zipped<A, B, F>(mut iter1: A, mut iter2: B, callback: F)
where
A: StreamingIterator,
B: StreamingIterator,
F: for<'a> Fn(Option<&'a A::Item>, Option<&'a B::Item>) -> (bool, bool),
{
iter1.advance();
iter2.advance();
loop {
let advance_flags = callback(iter1.get(), iter2.get());
match advance_flags {
(true, true) => {
iter1.advance();
iter2.advance();
}
(true, false) => {
iter1.advance();
}
(false, true) => {
iter1.advance();
}
(false, false) => return,
}
}
}
struct RefIter {
value: u64
}
impl StreamingIterator for RefIter {
type Item = u64;
fn advance(&mut self) {
self.value += 1;
}
fn get(&self) -> Option<&Self::Item> {
Some(&self.value)
}
}
fn main() {
let iter1 = RefIter { value: 3 };
let iter2 = RefIter { value: 4 };
each_zipped(iter1, iter2, |val1, val2| {
let val1 = *val1.unwrap();
let val2 = *val2.unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
The issue with this code is that RefIter is being used in two ways, which are fundamentally at odds with one another:
Callers of next_ref recieve a reference to the stored value, which is tied to the lifetime of RefIter
RefIter's value needs to be mutable, so that it can be incremented on each call
This perfectly describes mutable aliasing (you're trying to modify 'value' while a reference to it is being held) - something which Rust is explicitly designed to prevent.
In order to make each_zipped work, you'll need to modify RefIter to avoid handing out references to data that you wish to mutate.
I've implemented one possibility below using a combination of RefCell and Rc:
use std::cell::RefCell;
use std::rc::Rc;
// An iterator-like type, that returns references to itself
// in next_ref
struct RefIter {
value: RefCell<Rc<u64>>
}
impl RefIter {
fn next_ref(&self) -> Option<Rc<u64>> {
let new_val = Rc::new(**self.value.borrow() + 1);
*self.value.borrow_mut() = new_val;
Some(Rc::clone(&*self.value.borrow()))
}
}
// Iterate over two RefIter simultaniously and call a callback
// for each pair. The callback returns a tuple of bools
// that indicate which iterators should be advanced.
fn each_zipped<F>(iter1: RefIter, iter2: RefIter, callback: F)
where F: Fn(&Option<Rc<u64>>, &Option<Rc<u64>>) -> (bool, bool)
{
let mut current1 = iter1.next_ref();
let mut current2 = iter2.next_ref();
loop {
let advance_flags = callback(¤t1, ¤t2);
match advance_flags {
(true, true) => {
current1 = iter1.next_ref();
current2 = iter2.next_ref();
},
(true, false) => {
current1 = iter1.next_ref();
},
(false, true) => {
current2 = iter1.next_ref();
},
(false, false) => {
return
}
}
}
}
fn main() {
let iter1 = RefIter { value: RefCell::new(Rc::new(3)) };
let iter2 = RefIter { value: RefCell::new(Rc::new(4)) };
each_zipped(iter1, iter2, |val1, val2| {
// We can't use unwrap() directly, since we're only passed a reference to an Option
let val1 = **val1.iter().next().unwrap();
let val2 = **val2.iter().next().unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
This version of RefIter hands out Rcs to consumers, instead of references. This avoids the issue of mutable aliasing - updating value is done by placing
a new Rc into the outer RefCell. A side effect of this is that consumers are able to hold onto an 'old' reference to the buffer (through the returned Rc), even after RefIter has advanced.