Mutable methods with lifetimes and later references - rust

Consider the following code:
struct Foo<'a> {
borrowed: &'a u8,
owned_one: Vec<u8>,
owned_two: Vec<u8>,
output: usize
}
impl<'a> Foo<'a> {
fn do_stuff(&mut self) {
self.output = self.owned_one.len();
let zipped = self.owned_one.iter().zip(self.owned_two.iter());
Self::subroutine(&zipped);
}
fn subroutine<Arg: Iterator<Item=(&'a u8, &'a u8)>>(_zipped: &Arg) {}
}
fn main() {
let num = 0u8;
let mut foo = Foo {
borrowed: &num,
owned_one: vec![0],
owned_two: vec![1],
output: 0
};
foo.do_stuff();
let _out = &foo.output;
}
(playground link)
It doesn't compile, producing the following error:
error: lifetime may not live long enough
--> src/lib.rs:12:9
|
8 | impl<'a> Foo<'a> {
| -- lifetime `'a` defined here
9 | fn do_stuff(&mut self) {
| - let's call the lifetime of this reference `'1`
...
12 | Self::subroutine(&zipped);
| ^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`
I don't fully understand the complaint - surely self would always have the lifetime assigned to the class we're implementing? - but I can understand that both arguments to zip() need to last the same time. So I change do_stuff to take &'a mut self.
struct Foo<'a> {
borrowed: &'a u8,
owned_one: Vec<u8>,
owned_two: Vec<u8>,
output: usize
}
impl<'a> Foo<'a> {
fn do_stuff(&'a mut self) {
self.output = self.owned_one.len();
let zipped = self.owned_one.iter().zip(self.owned_two.iter());
Self::subroutine(&zipped);
}
fn subroutine<Arg: Iterator<Item=(&'a u8, &'a u8)>>(_zipped: &Arg) {}
}
fn main() {
let num = 0u8;
let mut foo = Foo {
borrowed: &num,
owned_one: vec![0],
owned_two: vec![1],
output: 0
};
foo.do_stuff();
let _out = &foo.output;
}
However, now compilation fails with:
error[E0502]: cannot borrow `foo.output` as immutable because it is also borrowed as mutable
--> src/lib.rs:27:16
|
26 | foo.do_stuff();
| -------------- mutable borrow occurs here
27 | let _out = &foo.output;
| ^^^^^^^^^^^
| |
| immutable borrow occurs here
| mutable borrow later used here
Why has specifying a lifetime for self in the argument list for do_stuff meant that I suddenly can't take the immutable reference to foo later; and what can I do about it?

writing fn do_stuff(&'a mut self) in this context mean that the lifetime of this borrow of self, must life as long as what this self has borrow borrowed. That very often not what you want.
Your mistake is here fn subroutine<Arg: Iterator<Item=(&'a u8, &'a u8)>>(_zipped: &Arg) {}. Think of lexical meaning of your declaration. Your method return nothing but require a "parent" lifetime ? There is no reason to. The simple solution is simply to introduce a new lifetime for your Item. Like fn subroutine<'i, Arg: Iterator<Item=(&'i u8, &'i u8)>>(_zipped: &Arg) {}. Unless your function return something link to 'a, no lifetime of parent should be here.
Also, it's better to accept IntoIterator it's more generic, and there is no reason to take it by reference and finally when you have such complex generic better use where, and really really if you want to be pedantic you need two lifetime:
fn subroutine<'i, 'j, Arg>(_zipped: Arg)
where
Arg: IntoIterator<Item = (&'i u8, &'j u8)>,
{
}

Related

Using reference lifetimes correctly in iterator

I am trying to make an iterator out of some struct, s1, that has references within it with some lifetimes, say 'a and 'b. The iterator elements are new instances of the same struct type holding the same references. When I try to do this, the iterator elements appear to become subject to the lifetime of s1 instead of just 'a and 'b.
An example for clarity:
#[derive(Debug)]
pub struct MyStruct<'a, 'b> {
num: i32,
ref1: &'a i32,
ref2: &'b i32,
}
impl<'a, 'b> MyStruct<'a, 'b> {
pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct> {
(1..=n).map(|i| MyStruct { num: self.num + i, ref1: self.ref1, ref2: self.ref2 })
}
}
fn main() {
let i1 = 1;
let i2 = 2;
let s1 = MyStruct{ num: 0, ref1: &i1, ref2: &i2 };
let s_next: Vec<_> = s1.with_next_nums(3).collect();
drop(s1); // commenting this line the program compiles
println!("{:?}", s_next);
}
The error I get is:
error[E0505]: cannot move out of `s1` because it is borrowed
--> src/main.rs:19:10
|
18 | let s_next: Vec<_> = s1.with_next_nums(3).collect();
| -------------------- borrow of `s1` occurs here
19 | drop(s1); // commenting this line the program compiles
| ^^ move out of `s1` occurs here
20 | println!("{:?}", s_next);
| ------ borrow later used here
For more information about this error, try `rustc --explain E0505`.
error: could not compile `playground` due to previous error
So, because I am dropping s1, Rust assumes the elements of s_next will become invalid, even though s_next only holds references to i1 and i2.
I suppose it is a matter of lifetime annotation, but I don't know how to fix it. If I were just producing a single struct out of s1, then I could annotate the return type like MyStruct<'a, 'b> and it would work, but just using impl Iterator<Item=MyStruct<'a, 'b>> as a return type does not solve it.
The problem with your version is that because of the lifetime elision rules, since you didn't specify the lifetimes for 'a and 'b in the returned MyStruct, the compiler assumes they have the same lifetime as self. So, this as if you wrote:
pub fn with_next_nums<'c>(&'c self, n: i32) -> impl Iterator<Item = MyStruct<'c, 'c>>
To fix that you need to explicitly declare the lifetimes as 'a and 'b:
pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item = MyStruct<'a, 'b>>
Now there will be another error in the declaration of with_next_nums(), because it does borrow from self as the closure captures it. You can fix that by adding '_, as the compiler suggests, but then your iterator will borrow from self, so your code (that collects them into a Vec) will work, but similar code that does not collect() immediately will not:
fn main() {
let i1 = 1;
let i2 = 2;
let s1 = MyStruct {
num: 0,
ref1: &i1,
ref2: &i2,
};
// The iterator is still alive when we drop `s1`, but it borrows from it.
let s_next = s1.with_next_nums(3);
drop(s1);
let s_next: Vec<_> = s_next.collect();
println!("{:?}", s_next);
}
A better fix is to change the closure so it will only borrow the fields it needs and not self. You can do it like that:
pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item = MyStruct<'a, 'b>> {
let MyStruct { num, ref1, ref2 } = *self;
(1..=n).map(move |i| MyStruct {
num: num + i,
ref1,
ref2,
})
}
The problem is that in this case you need to specify the lifetime of both the iterator elements and the iterator itself. If you just specify the lifetimes of the iterator elements (with impl Iterator<Item=MyStruct<'a, 'b>>), the compiler assumes you will deal with lifetimes by hand and will give up trying to automatically assign a lifetime to the iterator. However, it does need to have the lifetime of the struct, because it depends on it, so it will fail to compile. The compiler error will actually give you the fix in this case:
error[E0700]: hidden type for `impl Iterator<Item = MyStruct<'a, 'b>>` captures lifetime that does not appear in bounds
--> src/main.rs:10:9
|
9 | pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct<'a, 'b>> {
| ----- hidden type `Map<RangeInclusive<i32>, [closure#src/main.rs:10:21: 10:24]>` captures the anonymous lifetime defined here
10 | (1..=n).map(|i| MyStruct { num: self.num + i, ref1: self.ref1, ref2: self.ref2 })
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
help: to declare that `impl Iterator<Item = MyStruct<'a, 'b>>` captures `'_`, you can add an explicit `'_` lifetime bound
|
9 | pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct<'a, 'b>> + '_ {
| ++++
For more information about this error, try `rustc --explain E0700`.
error: could not compile `playground` due to previous error
So, you just need to also add that + '_ to the return type to solve the issue:
impl<'a, 'b> MyStruct<'a, 'b> {
pub fn with_next_nums(&self, n: i32) -> impl Iterator<Item=MyStruct<'a, 'b>> + '_ {
(1..=n).map(|i| MyStruct { num: self.num + i, ref1: self.ref1, ref2: self.ref2 })
}
}

Trait object lifetime that matches a returning iterator's lifetime triggers a lifetime error

I have a trait with a function that returns Box<dyn Iterator<...>>. As I understand it, based off the definition, the lifetime of this returned iterator needs to match the trait object's. But in a function that drops both at the end, I get a lifetime error. Here's a minimal example, with assumptions added as comments:
trait Foo<'a> {
// The returned iterator's lifetime should match self's
fn bar(&'a self) -> Box<dyn Iterator<Item = usize> + 'a>;
}
struct FooImpl {
values: Vec<usize>
}
impl<'a> Foo<'a> for FooImpl {
fn bar(&'a self) -> Box<dyn Iterator<Item = usize> + 'a> {
Box::new(self.values.iter().cloned())
}
}
fn foo_bar_caller<'a>(foo: Box<dyn Foo<'a>>) {
// `foo` is moved into this fn
let _iter = foo.bar();
// `_iter` and `foo` should drop
}
fn main() {
let foo = FooImpl { values: vec![1, 2, 3, 4, 5] };
foo_bar_caller(Box::new(foo));
}
(rust playground with the code.) Running this gives the following lifetime error:
error[E0597]: `*foo` does not live long enough
--> src/main.rs:17:17
|
16 | fn foo_bar_caller<'a>(foo: Box<dyn Foo<'a>>) {
| -- lifetime `'a` defined here
17 | let _iter = foo.bar();
| ^^^^^^^^^
| |
| borrowed value does not live long enough
| argument requires that `*foo` is borrowed for `'a`
18 | // `_iter` and `foo` should drop
19 | }
| - `*foo` dropped here while still borrowed
For more information about this error, try `rustc --explain E0597`.
error: could not compile `playground` due to previous error
You probably meant to make the function Foo::bar generic over the reference lifetime, not the trait Foo:
trait Foo {
// The returned iterator's lifetime should match self's
fn bar<'a>(&'a self) -> Box<dyn Iterator<Item = usize> + 'a>;
}
By putting the lifetime on the trait Foo, you are unnecessarily (in this context, at least) restricting the lifetimes of the references you can use with a particular Foo::<'_>::bar implementation. A <T as Foo<'a>>::bar (T in the context of foo_bar_caller being Box<dyn Foo<'a>>) call can only take references with the lifetime 'a, but as #MaximilianBurszley points out in the comments, what you are actually passing to it is a temporary borrow that does not have the same lifetime 'a.
Playground
Note that you can make foo_bar_caller work with your existing code (with the trait Foo being generic over the lifetime) if you make the function take a reference with the same lifetime:
fn foo_bar_caller<'a>(foo: &'a dyn Foo<'a>) {
// `foo` is moved into this fn
let _iter = foo.bar();
// `_iter` and `foo` should drop
}
fn main() {
let foo = FooImpl { values: vec![1, 2, 3, 4, 5] };
foo_bar_caller(&foo);
}
Alternatively, you can use a higher-ranked trait bound (HRTB) to specify that foo_bar_caller must take in a value that implements Foo<'b> for all lifetimes 'b:
fn foo_bar_caller(foo: Box<dyn for<'b> Foo<'b>>) {
// `foo` is moved into this fn
let _iter = foo.bar();
// `_iter` and `foo` should drop
}
This allows the Foo<'_> implementation for the value foo to be valid for the lifetime for the temporary borrow created during the foo.bar() call (since it implements Foo<'_> for all lifetimes).
Playground for HRTB version

cannot infer an appropriate lifetime for lifetime parameter with multiple references with the same lifetime inside a struct [E0495]

I had gotten an error in my code about lifetime inferrence, and I have been able to reduce the code to the following:
use std::collections::HashMap;
struct A<'a> {
x: &'a mut HashMap<&'a str, i32>,
}
impl<'a> A<'a> {
fn new(x: &'a mut HashMap<&'a str, i32>) -> Self {
Self { x }
}
fn test(&mut self) {
let a = A::new(self.x);
}
}
fn main() {
}
The error that results from this is
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
--> src/main.rs:13:17
|
13 | let a = A::new(self.x);
| ^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 12:5...
--> src/main.rs:12:5
|
12 | / fn test(&mut self) {
13 | | let a = A::new(self.x);
14 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:13:24
|
13 | let a = A::new(self.x);
| ^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 7:6...
--> src/main.rs:7:6
|
7 | impl<'a> A<'a> {
| ^^
note: ...so that the expression is assignable
--> src/main.rs:13:24
|
13 | let a = A::new(self.x);
| ^^^^^^
= note: expected `&mut std::collections::HashMap<&str, i32>`
found `&mut std::collections::HashMap<&'a str, i32>`
I do not understand what is the error in this case, but I did find that if I add a lifetime 'b to struct A, such that the str reference in the HashMap has a lifetime of 'b, the codes does compile. Here is how the code looks like after the aforementioned change:
use std::collections::HashMap;
struct A<'a, 'b> {
x: &'a mut HashMap<&'b str, i32>,
}
impl<'a, 'b> A<'a, 'b> {
fn new(x: &'a mut HashMap<&'b str, i32>) -> Self {
Self { x }
}
fn test(&mut self) {
let a = A::new(self.x);
}
}
fn main() {
}
However, I have no idea why that change works. To my understanding, having both lifetimes be 'a means that A must live as long as the HashMap, and the HashMap must live as long as the &strs that are used as its keys, which I see no issue with. I also do not see how the change adds any additional information for the compiler. Can anyone please shed some light for me on this situation?
Change the A::test() function into
fn test(&'a mut self) { // Add lifetime specification
let a = A::new(self.x);
}
and it should work.
The compiler said it cannot infer lifetime for A::new(), and the first note mentioned "anonymous lifetime", that means the compiler doesn't know the lifetime of self.x in A::new(self.x). So we just need to tell the compiler that self has a lifetime of 'a.

Why does changing a field from `&'a [u8]` to `&'a mut [u8]` cause a lifetime error?

This code compiles:
struct BufRef<'a> {
buf: &'a [u8],
}
struct Foo<'a> {
buf_ref: BufRef<'a>,
}
impl<'a> Iterator for Foo<'a> {
type Item = &'a [u8];
fn next(&mut self) -> Option<Self::Item> {
let result = &self.buf_ref.buf;
Some(result)
}
}
However, if I change BufRef to:
struct BufRef<'a> {
buf: &'a mut [u8],
}
The compiler says:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src\main.rs:13:16
|
13 | let result = &self.buf_ref.buf;
| ^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 12:5...
--> src\main.rs:12:5
|
12 | / fn next(&mut self) -> Option<Self::Item> {
13 | | let result = &self.buf_ref.buf;
14 | | Some(result)
15 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src\main.rs:13:16
|
13 | let result = &self.buf_ref.buf;
| ^^^^^^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 9:6...
--> src\main.rs:9:6
|
9 | impl<'a> Iterator for Foo<'a> {
| ^^
= note: ...so that the types are compatible:
expected std::iter::Iterator
found std::iter::Iterator
Why does changing the field to &'a mut [u8] cause the error?
Also, what does the compiler mean by this:
...so that the types are compatible:
expected std::iter::Iterator
found std::iter::Iterator
I think that what is misleading you is that your code has a collapsed reference.
Your next function is basically equivalent to this code:
fn next(&mut self) -> Option<&'a [u8]> {
let result: &&'a [u8] = &self.buf_ref.buf;
Some(result)
}
This works because the double reference collapses to a single reference. In this case the double reference only obfuscates the code. Just write:
fn next(&mut self) -> Option<Self::Item> {
Some(self.buf_ref.buf)
}
And this works because references are always Copy.
But now what happens when you change your definition to &'a mut? You are probably guessing right now... mutable references are not Copy, so the same simple code will give you an easy-to-read error message:
cannot move out of self.buf_ref.buf which is behind a mutable reference
Naturally, you can reborrow a mutable ref as a const one, and then try to return it, but unfortunately this will not work because the the re-borrow cannot use the same lifetime as the mutable variable, it must be strictly smaller (or you could alias the pointed values). The compiler assigns the lifetime of this re-borrow as that of the next function, but now you cannot return this borrow, because it is a local reference!
Unfortunately, I don't know of any safe way to make your code compile. In fact I'm quite sure that it would create an unsound API. That is, if you managed to compile your code, then this safe code would create undefined behavior:
fn main() {
let mut buf = vec![1,2,3];
let buf_ref = BufRef { buf: &mut buf };
let mut foo = Foo { buf_ref };
let x: &[u8] = foo.next().unwrap();
//note that x's lifetime is that of buf, foo is not borrowed
//x and foo.buf_ref.buf alias the same memory!
//but the latter is mutable
println!("{}", x[0]); //prints 1
foo.buf_ref.buf[0] = 4;
println!("{}", x[0]); //prints what?
}

Lifetime on trait returning iterator

I'm working with a trait requiring a function returning an iterator without consuming the object. The iterator itself returns copies of data values, not references. As the iterator implementation requires a reference to the object it is iterating over, I end up having to declare lots of lifetimes (more than I would have thought necessary, but could not get it to compile otherwise). I then run into trouble with borrow duration - a minimal "working" example is as follows:
pub trait MyTrait<'a> {
type IteratorType: Iterator<Item=u32>;
fn iter(&'a self) -> Self::IteratorType;
fn touch(&'a mut self, value: u32);
}
struct MyStruct {
data: Vec<u32>
}
struct MyIterator<'a> {
structref: &'a MyStruct,
next: usize,
}
impl<'a> Iterator for MyIterator<'a> {
type Item = u32;
fn next(&mut self) -> Option<u32> {
if self.next < self.structref.data.len() {
self.next += 1;
return Some(self.structref.data[self.next-1]);
} else {
return None;
}
}
}
impl<'a> MyTrait<'a> for MyStruct {
type IteratorType = MyIterator<'a>;
fn iter(&'a self) -> Self::IteratorType {
return MyIterator { structref: &self, next: 0 };
}
fn touch(&'a mut self, value: u32) {
}
}
fn touch_all<'a,T>(obj: &'a mut T) where T: MyTrait<'a> {
let data: Vec<u32> = obj.iter().collect();
for value in data {
obj.touch(value);
}
}
Compiling this gives me the error:
error[E0502]: cannot borrow `*obj` as mutable because it is also borrowed as immutable
|
39 | let data: Vec<u32> = obj.iter().collect();
| --- immutable borrow occurs here
40 | for value in data {
41 | obj.touch(value);
| ^^^ mutable borrow occurs here
42 | }
43 | }
| - immutable borrow ends here
By my limited understanding of lifetimes, I would have thought the immutable borrow only extends to the line where I make it - after all the iterator is consumed and I no longer hold any references to obj or data contained in it. Why does the lifetime of the borrow extend to the entire function, and how do I fix this?
Here is a sequence of steps on how I arrived here - running the code should provide the associated compiler errors.
no explicit lifetimes
IteratorType needs lifetime
Unconstrained lifetime parameter
To clarify: I'd like to be able to make calls like this:
fn main() {
let obj: MyStruct = MyStruct { data : vec![] };
touch_all(&mut obj);
}
rather than having to call
touch_all(&mut &obj);
which would be needed for the proposal by mcarton (1st and 2nd comment).

Resources