Lifetime may not live long enough - rust

I've defined a struct which only has a slice of element of type T.
pub struct MyStruct<'a, T> {
slice: &'a mut [T],
}
I want to define an Iterator for MyStruct which yields a MyStruct with a slice which has 10 elements.
impl<'a, E> Iterator for MyStruct<'a, E> {
type Item = MyStruct<'a, E>;
fn next(&mut self) -> Option<Self::Item> {
if self.slice.len() < 10 {
return None;
} else {
let (head, tail) = self.slice.split_at_mut(10);
self.slice = tail;
Some(MyStruct { slice: head })
}
}
}
But unfortunately I'm lifetime issues.
error: lifetime may not live long enough
--> src/lib.rs:12:32
|
5 | impl<'a, E> Iterator for MyStruct<'a, E> {
| -- lifetime `'a` defined here
...
8 | fn next(&mut self) -> Option<Self::Item> {
| - let's call the lifetime of this reference `'1`
...
12 | let (head, tail) = self.slice.split_at_mut(10);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`
I'm not an expert in lifetime and would appreciate it if someone could point out to me what I'm doing incorrectly here.

The question How can I create my own data structure with an iterator that returns mutable references? does a good job explaining why it is disallowed. But despite what said, or at least implied there that it is impossible to do that without unsafe code ("Now you may wonder: How did the standard library authors manage to write the mutable vector iterator? The answer is simple: They used unsafe code."), for pure slices it is actually possible.
The trick is simple: instead of using self.slice from the restrictive self reference, replace it with an empty slice and now we have the slice with the full lifetime:
fn next(&mut self) -> Option<Self::Item> {
if self.slice.len() < 10 {
return None;
} else {
let slice = std::mem::take(&mut self.slice);
let (head, tail) = slice.split_at_mut(10);
self.slice = tail;
Some(MyStruct { slice: head })
}
}

Related

Mutable methods with lifetimes and later references

Consider the following code:
struct Foo<'a> {
borrowed: &'a u8,
owned_one: Vec<u8>,
owned_two: Vec<u8>,
output: usize
}
impl<'a> Foo<'a> {
fn do_stuff(&mut self) {
self.output = self.owned_one.len();
let zipped = self.owned_one.iter().zip(self.owned_two.iter());
Self::subroutine(&zipped);
}
fn subroutine<Arg: Iterator<Item=(&'a u8, &'a u8)>>(_zipped: &Arg) {}
}
fn main() {
let num = 0u8;
let mut foo = Foo {
borrowed: &num,
owned_one: vec![0],
owned_two: vec![1],
output: 0
};
foo.do_stuff();
let _out = &foo.output;
}
(playground link)
It doesn't compile, producing the following error:
error: lifetime may not live long enough
--> src/lib.rs:12:9
|
8 | impl<'a> Foo<'a> {
| -- lifetime `'a` defined here
9 | fn do_stuff(&mut self) {
| - let's call the lifetime of this reference `'1`
...
12 | Self::subroutine(&zipped);
| ^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`
I don't fully understand the complaint - surely self would always have the lifetime assigned to the class we're implementing? - but I can understand that both arguments to zip() need to last the same time. So I change do_stuff to take &'a mut self.
struct Foo<'a> {
borrowed: &'a u8,
owned_one: Vec<u8>,
owned_two: Vec<u8>,
output: usize
}
impl<'a> Foo<'a> {
fn do_stuff(&'a mut self) {
self.output = self.owned_one.len();
let zipped = self.owned_one.iter().zip(self.owned_two.iter());
Self::subroutine(&zipped);
}
fn subroutine<Arg: Iterator<Item=(&'a u8, &'a u8)>>(_zipped: &Arg) {}
}
fn main() {
let num = 0u8;
let mut foo = Foo {
borrowed: &num,
owned_one: vec![0],
owned_two: vec![1],
output: 0
};
foo.do_stuff();
let _out = &foo.output;
}
However, now compilation fails with:
error[E0502]: cannot borrow `foo.output` as immutable because it is also borrowed as mutable
--> src/lib.rs:27:16
|
26 | foo.do_stuff();
| -------------- mutable borrow occurs here
27 | let _out = &foo.output;
| ^^^^^^^^^^^
| |
| immutable borrow occurs here
| mutable borrow later used here
Why has specifying a lifetime for self in the argument list for do_stuff meant that I suddenly can't take the immutable reference to foo later; and what can I do about it?
writing fn do_stuff(&'a mut self) in this context mean that the lifetime of this borrow of self, must life as long as what this self has borrow borrowed. That very often not what you want.
Your mistake is here fn subroutine<Arg: Iterator<Item=(&'a u8, &'a u8)>>(_zipped: &Arg) {}. Think of lexical meaning of your declaration. Your method return nothing but require a "parent" lifetime ? There is no reason to. The simple solution is simply to introduce a new lifetime for your Item. Like fn subroutine<'i, Arg: Iterator<Item=(&'i u8, &'i u8)>>(_zipped: &Arg) {}. Unless your function return something link to 'a, no lifetime of parent should be here.
Also, it's better to accept IntoIterator it's more generic, and there is no reason to take it by reference and finally when you have such complex generic better use where, and really really if you want to be pedantic you need two lifetime:
fn subroutine<'i, 'j, Arg>(_zipped: Arg)
where
Arg: IntoIterator<Item = (&'i u8, &'j u8)>,
{
}

Mutable iterator [duplicate]

This question already has answers here:
How can I create my own data structure with an iterator that returns mutable references?
(1 answer)
How to implement Iterator yielding mutable references [duplicate]
(2 answers)
Closed 2 months ago.
I am trying to write a mutable iterator over a vector, but I am unable to figure out what the compiler is trying to tell me. Here is my implementation
struct IterMut<'a> {
vec: &'a mut Vec<u32>,
index: usize,
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
let item = if self.index < self.vec.len() {
Some(&mut self.vec[self.index])
} else {
None
};
self.index += 1;
item
}
}
And the compiler error is:
error: lifetime may not live long enough
--> src/main.rs:60:9
|
48 | impl<'a> Iterator for IterMut<'a> {
| -- lifetime `'a` defined here
...
51 | fn next(&mut self) -> Option<Self::Item> {
| - let's call the lifetime of this reference `'1`
...
60 | item
| ^^^^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`
The non mutable version compiles and works.
struct Iter<'a> {
vec: &'a Vec<u32>,
index: usize,
}
impl<'a> Iterator for Iter<'a> {
type Item = &'a u32;
fn next(&mut self) -> Option<Self::Item> {
let item = if self.index < self.vec.len() {
Some(&self.vec[self.index])
} else {
None
};
self.index += 1;
item
}
}
Here is a playground link with the code to try out.
The problem is that your IterMut holds on to the elements already passed.
It can do so only by reborrowing the item but this now means that the item isn't borrowed from the original Vec any more but from the IterMut instead so it does not live long enough. (The reborrow only lives as long as the IterMut, not as long as the original Vec.
In nightly or once take_first_mut stabilizes you can do this:
#![feature(slice_take)]
struct IterMut<'a> {
vec: &'a mut [u32],
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
self.vec.take_first_mut()
}
}
Note I switched from &mut Vec<u32> to a slice because you have to be able to 'release' the items which is not easy with a Vec
As #ChayimFriedman pointed out on stable you can do it with mem::take in the meantime:
struct IterMut<'a> {
vec: &'a mut [u32],
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
let (first, rem) = std::mem::take(&mut self.vec).split_first_mut()?;
self.vec = rem;
Some(first)
}
}

Rust borrow checker and early returns

Rust-lang Playground link
struct Foo {
val: i32
}
impl Foo {
pub fn maybe_get(&mut self) -> Option<&mut i32> {
Some(&mut self.val)
}
pub fn definitely_get(&mut self) -> &mut i32 {
{ // Add closure to ensure things have a chance to get dropped
if let Some(val) = self.maybe_get() {
// Explicit return to avoid potential scope sharing with an else block or a match arms.
return val;
}
}
// One would think any mutable references would not longer be at play at this point
&mut self.val
}
}
I have some code that's similar but more complicated than what is provided above that I've been fighting with for quite a while. The borrow checker is unhappy with the implementation of definitely_get and has the following error
error[E0499]: cannot borrow `self.val` as mutable more than once at a time
--> src/main.rs:19:9
|
10 | pub fn definitely_get(&mut self) -> &mut i32 {
| - let's call the lifetime of this reference `'1`
11 | {
12 | if let Some(val) = self.maybe_get() {
| ---------------- first mutable borrow occurs here
13 | return val;
| --- returning this value requires that `*self` is borrowed for `'1`
...
19 | &mut self.val
| ^^^^^^^^^^^^^ second mutable borrow occurs here
It seems unreasonable for there to be no way to implement fallback logic with a mutable reference in Rust so I can't imagine there isn't a way.
I've managed to fix this with an unfortunately expensive alternative implementation due to how maybe_get is implemented in my non-trivial example.
impl Foo {
pub fn has_maybe_val(&self) -> bool {
// Non-trivial lookup...
true
}
pub fn maybe_get(&mut self) -> Option<&mut i32> {
// Same non-trivial lookup...
Some(&mut self.val)
}
pub fn definitely_get(&mut self) -> &mut i32 {
if self.has_maybe_val() {
self.maybe_get().unwrap() // Ouch!
} else {
&mut self.val
}
}
}

Why does changing a field from `&'a [u8]` to `&'a mut [u8]` cause a lifetime error?

This code compiles:
struct BufRef<'a> {
buf: &'a [u8],
}
struct Foo<'a> {
buf_ref: BufRef<'a>,
}
impl<'a> Iterator for Foo<'a> {
type Item = &'a [u8];
fn next(&mut self) -> Option<Self::Item> {
let result = &self.buf_ref.buf;
Some(result)
}
}
However, if I change BufRef to:
struct BufRef<'a> {
buf: &'a mut [u8],
}
The compiler says:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src\main.rs:13:16
|
13 | let result = &self.buf_ref.buf;
| ^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 12:5...
--> src\main.rs:12:5
|
12 | / fn next(&mut self) -> Option<Self::Item> {
13 | | let result = &self.buf_ref.buf;
14 | | Some(result)
15 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src\main.rs:13:16
|
13 | let result = &self.buf_ref.buf;
| ^^^^^^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 9:6...
--> src\main.rs:9:6
|
9 | impl<'a> Iterator for Foo<'a> {
| ^^
= note: ...so that the types are compatible:
expected std::iter::Iterator
found std::iter::Iterator
Why does changing the field to &'a mut [u8] cause the error?
Also, what does the compiler mean by this:
...so that the types are compatible:
expected std::iter::Iterator
found std::iter::Iterator
I think that what is misleading you is that your code has a collapsed reference.
Your next function is basically equivalent to this code:
fn next(&mut self) -> Option<&'a [u8]> {
let result: &&'a [u8] = &self.buf_ref.buf;
Some(result)
}
This works because the double reference collapses to a single reference. In this case the double reference only obfuscates the code. Just write:
fn next(&mut self) -> Option<Self::Item> {
Some(self.buf_ref.buf)
}
And this works because references are always Copy.
But now what happens when you change your definition to &'a mut? You are probably guessing right now... mutable references are not Copy, so the same simple code will give you an easy-to-read error message:
cannot move out of self.buf_ref.buf which is behind a mutable reference
Naturally, you can reborrow a mutable ref as a const one, and then try to return it, but unfortunately this will not work because the the re-borrow cannot use the same lifetime as the mutable variable, it must be strictly smaller (or you could alias the pointed values). The compiler assigns the lifetime of this re-borrow as that of the next function, but now you cannot return this borrow, because it is a local reference!
Unfortunately, I don't know of any safe way to make your code compile. In fact I'm quite sure that it would create an unsound API. That is, if you managed to compile your code, then this safe code would create undefined behavior:
fn main() {
let mut buf = vec![1,2,3];
let buf_ref = BufRef { buf: &mut buf };
let mut foo = Foo { buf_ref };
let x: &[u8] = foo.next().unwrap();
//note that x's lifetime is that of buf, foo is not borrowed
//x and foo.buf_ref.buf alias the same memory!
//but the latter is mutable
println!("{}", x[0]); //prints 1
foo.buf_ref.buf[0] = 4;
println!("{}", x[0]); //prints what?
}

Lifetime on trait returning iterator

I'm working with a trait requiring a function returning an iterator without consuming the object. The iterator itself returns copies of data values, not references. As the iterator implementation requires a reference to the object it is iterating over, I end up having to declare lots of lifetimes (more than I would have thought necessary, but could not get it to compile otherwise). I then run into trouble with borrow duration - a minimal "working" example is as follows:
pub trait MyTrait<'a> {
type IteratorType: Iterator<Item=u32>;
fn iter(&'a self) -> Self::IteratorType;
fn touch(&'a mut self, value: u32);
}
struct MyStruct {
data: Vec<u32>
}
struct MyIterator<'a> {
structref: &'a MyStruct,
next: usize,
}
impl<'a> Iterator for MyIterator<'a> {
type Item = u32;
fn next(&mut self) -> Option<u32> {
if self.next < self.structref.data.len() {
self.next += 1;
return Some(self.structref.data[self.next-1]);
} else {
return None;
}
}
}
impl<'a> MyTrait<'a> for MyStruct {
type IteratorType = MyIterator<'a>;
fn iter(&'a self) -> Self::IteratorType {
return MyIterator { structref: &self, next: 0 };
}
fn touch(&'a mut self, value: u32) {
}
}
fn touch_all<'a,T>(obj: &'a mut T) where T: MyTrait<'a> {
let data: Vec<u32> = obj.iter().collect();
for value in data {
obj.touch(value);
}
}
Compiling this gives me the error:
error[E0502]: cannot borrow `*obj` as mutable because it is also borrowed as immutable
|
39 | let data: Vec<u32> = obj.iter().collect();
| --- immutable borrow occurs here
40 | for value in data {
41 | obj.touch(value);
| ^^^ mutable borrow occurs here
42 | }
43 | }
| - immutable borrow ends here
By my limited understanding of lifetimes, I would have thought the immutable borrow only extends to the line where I make it - after all the iterator is consumed and I no longer hold any references to obj or data contained in it. Why does the lifetime of the borrow extend to the entire function, and how do I fix this?
Here is a sequence of steps on how I arrived here - running the code should provide the associated compiler errors.
no explicit lifetimes
IteratorType needs lifetime
Unconstrained lifetime parameter
To clarify: I'd like to be able to make calls like this:
fn main() {
let obj: MyStruct = MyStruct { data : vec![] };
touch_all(&mut obj);
}
rather than having to call
touch_all(&mut &obj);
which would be needed for the proposal by mcarton (1st and 2nd comment).

Resources