lifetime / borrow of nested structs error [duplicate] - rust

Question

This question already has an answer here:
How can I create my own data structure with an iterator that returns mutable references?
(1 answer)
Closed 6 months ago.
I'm having a lifetime issue when implementing an iterator on custom struct containing borrows of vecs.
I've been trying different solutions but can't fix it by myself (I'm still a beginner) and I want to understand what is going on.
here is a playground example of my issue, that should be simple enough.
This is the example :
struct SomeData;
struct CustomIterator<'a> {
pub vec: &'a mut Vec<SomeData>,
pub index: usize,
}
struct MultipleIterator<'a> {
iter1: CustomIterator<'a>,
iter2: CustomIterator<'a>,
}
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut SomeData, &'a mut SomeData);
fn next(&mut self) -> Option<Self::Item> {
Some((
match self.iter1.vec.get_mut(self.iter1.index) {
Some(mut data) => &mut data,
None => return None,
},
match self.iter2.vec.get_mut(self.iter2.index) {
Some(mut data) => &mut data,
None => return None,
}
))
}
}
I don't unerstand why I can't borrow out of the next function, since I am borrowing the struct anyway when calling next()

Answer 1

This is actually quite tricky to implement safely and requires a lot of care to do correctly.
First things first though:
match self.iter1.vec.get_mut(self.iter1.index) {
Some(mut data) => &mut data,
None => return None,
},
This is a problem because Vec::get_mut already returns a Option<&mut T>. So in the Some(mut data) arm, data already is a mutable reference. When you try to return &mut data, you're trying to return a &mut &mut T which doesn't work. Instead, just do this:
match self.iter1.vec.get_mut(self.iter1.index) {
Some(data) => data,
None => return None,
},
We can tidy this up even more with the ? operator which does the same thing. I'm gonna substitute SomeData for i32 from now on to demonstrate something later.
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut i32, &'a mut i32);
fn next(&mut self) -> Option<Self::Item> {
Some((
self.iter1.vec.get_mut(self.iter1.index)?,
self.iter2.vec.get_mut(self.iter2.index)?,
))
}
}
This still doesn't work and now we're getting to the core of the problem. The signature of next is
fn next(&mut self) -> Option<(&'a mut i32, &'a mut i32)>
which can be desugared to
fn next<'b>(&'b mut self) -> Option<(&'a mut i32, &'a mut i32)>
This means that the lifetime of &mut self ('b) is completely decoupled from the lifetime of the references we return ('a). Which makes total sense. If that wasn't the case, we couldn't do
let mut v = vec![1,2,3];
let mut iter = v.iter_mut();
let next1: &mut i32 = iter.next().unwrap();
let next2: &mut i32 = iter.next().unwrap();
because the lifetime of next1 would have to be the same lifetime of iter, i.e. that of v. But you can't have multiple mutable references to v at the same time, so next2 would be illegal.
So you're getting an error because Rust only knows that self is borrowed for 'b but you're telling it that you're returning a reference with lifetime 'a which it can't verify to be true.
And for good reason, because as it stands right now, your implementation isn't safe! Let's just throw caution to the wind and tell Rust that this is okay with unsafe:
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut i32, &'a mut i32);
fn next(&mut self) -> Option<Self::Item> {
unsafe {
Some((
&mut *(self.iter1.vec.get_mut(self.iter1.index)? as *mut _),
&mut *(self.iter2.vec.get_mut(self.iter2.index)? as *mut _),
))
}
}
}
It's not important what exactly this does, it basically just tells the compiler to shut up and trust me.
But now, we can do this:
let mut v1 = vec![1, 2, 3];
let mut v2 = vec![4, 5, 6];
let mut mi = MultipleIterator {
iter1: CustomIterator {
vec: &mut v1,
index: 0,
},
iter2: CustomIterator {
vec: &mut v2,
index: 0,
},
};
let next1 = mi.next().unwrap();
let next2 = mi.next().unwrap();
assert_eq!(next1, (&mut 1, &mut 4));
assert_eq!(next2, (&mut 1, &mut 4));
*next1.0 += 1;
assert_eq!(next1, (&mut 2, &mut 4));
assert_eq!(next2, (&mut 2, &mut 4));
We have broken Rust's most important rule: never have two mutable references to the same thing at once.
This can only be safe if your Iterator implementation can never return a mutable reference to something more than once. You could increment index each time, for example (although this still requires unsafe):
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut i32, &'a mut i32);
fn next(&mut self) -> Option<Self::Item> {
let next1 = self.iter1.vec.get_mut(self.iter1.index)?;
let next2 = self.iter2.vec.get_mut(self.iter2.index)?;
self.iter1.index += 1;
self.iter2.index += 1;
// SAFETY: this is safe because we will never return a reference
// to the same index more than once
unsafe { Some((&mut *(next1 as *mut _), &mut *(next2 as *mut _))) }
}
}
Here is an interesting related read from the nomicon; the "mutable slice" example being particularly relevant to your problem.