Here's a minimal example of the issue I'm having:
struct S<'a> {
value: &'a mut Option<()>,
}
impl<'a> S<'a> {
fn f<'b>(&'b mut self) {
*self.value = Some(());
let value: &'a () = self.value.as_ref().unwrap();
}
}
Playground
Error:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/lib.rs:8:34
|
8 | let value: &'a () = self.value.as_ref().unwrap();
| ^^^^^^
|
note: first, the lifetime cannot outlive the lifetime `'b` as defined on the method body at 6:7...
--> src/lib.rs:6:7
|
6 | fn f<'b>(&'b mut self) {
| ^^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:8:23
|
8 | let value: &'a () = self.value.as_ref().unwrap();
| ^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 5:6...
--> src/lib.rs:5:6
|
5 | impl<'a> S<'a> {
| ^^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:8:23
|
8 | let value: &'a () = self.value.as_ref().unwrap();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The issue is that I'm expecting f's value variable to have the lifetime 'a as I've annotated, but the borrow checker is only content if it's 'b. I need it to be 'a because in my real code I derive a value from it that expects the 'a lifetime. And it also makes sense to me that it should be 'a because (and this is the crux of my confusion:) in self.value I've dereferenced 'b away and should be left with &'a mut Option<()>, i.e. a value with (just) the lifetime 'a.
I can solve it by specifying 'b: 'a but it's actually the opposite (if anything I need 'a: 'b); and also if 'b: 'a then this function mutably borrows indefinitely and I can't touch the S after calling f which is a problem Playground.
I'm not sure if it's overconservative or if I'm actually trying to do something unsound. Looking at the lifetimes at the call-site it makes perfect sense though.
Some information about my real use case: what I have is a buffer of type B, and from that buffer I can derive a value of type T. T isn't just derived from B, it also has a reference to it (imagine fn derive<'a>(self: &'a B) -> T<'a>). T values expire and require management, so I made a manager struct M, and this is where the trouble is: M contains a mutable reference to a B and a cached T. Therefore inside M, the B reference has some lifetime, and T has it too since T refers to B. It's something like this:
struct B { ... }
struct T<'a> { ... }
struct M<'a> {
b: &mut 'a Option<B>,
t: T<'a>,
}
So as in my minimal example, I try to obtain the reference of the B (which I expected to have the lifetime of the buffer but it has the lifetime of M) so I could derive from it and then assign the result to the cached T, but I don't have the right lifetime.
Related
Consider the following struct:
struct State<'a> {
parent: Option<&'a mut State<'a>>,
// ...
}
My state stores some values that I might need later. Now I want to implement substates, i.e. allow the manipulation of those values in the substate without touching the parent state, but forwarding lookup of values not in the substate to its parent. Unfortunately, I need a mutable reference to every parent state at all times. I tried the following, but it doesn't work (Playground):
impl<'a> State<'a> {
fn substate<'b>(&'b mut self) -> State<'b>
where
'a: 'b,
{
State::<'b> { parent: Some(self) }
}
}
This gives the following error message:
error[E0308]: mismatched types
--> src/main.rs:10:36
|
10 | State::<'b> { parent: Some(self) }
| ^^^^ lifetime mismatch
|
= note: expected mutable reference `&mut State<'b>`
found mutable reference `&mut State<'a>`
note: the lifetime `'b` as defined here...
--> src/main.rs:6:17
|
6 | fn substate<'b>(&'b mut self) -> State<'b>
| ^^
note: ...does not necessarily outlive the lifetime `'a` as defined here
--> src/main.rs:5:6
|
5 | impl<'a> State<'a> {
| ^^
I don't understand why the compiler wants 'b to outlive 'a. In fact, the parent of a state will always live longer than its substate, so in my case the opposite is always true. So why can't the compiler just downgrade the "longer" lifetime 'a into the "shorter" lifetime 'b?
The error message on nightly is better (thanks to NLL stabilization that may be removed before hitting stable, but now it is there):
error: lifetime may not live long enough
--> src/main.rs:10:31
|
5 | impl<'a> State<'a> {
| -- lifetime `'a` defined here
6 | fn substate<'b>(&'b mut self) -> State<'b>
| -- lifetime `'b` defined here
...
10 | State::<'b> { parent: Some(self) }
| ^^^^^^^^^^ this usage requires that `'b` must outlive `'a`
|
= help: consider adding the following bound: `'b: 'a`
= note: requirement occurs because of a mutable reference to `State<'_>`
= note: mutable references are invariant over their type parameter
= help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance
It is not enough that 'a outlives 'b, it should be exactly the same since it is invariant and invariant lifetimes cannot be shortened. Look at the link the compiler gives or the reference entry about variance to better understand what variance is and why mutable references are invariant.
The function signature is unsound, and here's an example of why:
struct State<'a> {
parent: Option<&'a mut State<'a>>,
}
impl<'a> State<'a> {
fn substate<'b>(&'b mut self) -> State<'b>
where
'a: 'b,
{
/* implementation hidden but assumed to be implemented as described */
}
}
// [root]
let mut root = State { parent: None };
// [foo] -> [root]
let mut foo = root.substate();
{
// [bar] -> [foo] -> [root]
let mut bar = foo.substate();
// [bar] -> [foo] -> [tmp]
let mut tmp = State { parent: None };
bar.parent.as_mut().unwrap().parent = Some(&mut tmp);
// tmp is dropped
}
// [foo] -> {uninitialized memory}
drop(foo);
(Full playground example)
The function signature allows us to store a reference inside foo here that lives shorter than foo itself does, so after we leave the block foo points to uninitialized memory, which is undefined behavior and bad.
If you run the above code with cargo miri, it will give this error, confirming that it is indeed undefined behavior:
error: Undefined Behavior: type validation failed at .parent.<enum-variant(Some)>.0: encountered a dangling reference (use-after-free)
--> src/main.rs:40:10
|
40 | drop(foo);
| ^^^ type validation failed at .parent.<enum-variant(Some)>.0: encountered a dangling reference (use-after-free)
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
= note: inside `main` at src/main.rs:40:10
note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace
error: aborting due to previous error
I'm trying to write a method on a struct that returns a closure. This closure should take a &[u8] with an arbitrary lifetime 'inner as an argument and return the same type, &'inner [u8]. To perform its function, the closure also needs a (shared) reference to a member of the struct &self. Here is my code:
#![warn(clippy::pedantic)]
// placeholder for a large type that I can't afford to clone
struct Opaque(usize);
struct MyStruct<'a>(Vec<&'a Opaque>);
impl<'a> MyStruct<'a> {
pub fn get_func_for_index(
&'a self,
n: usize,
) -> Option<impl for<'inner> Fn(&'inner [u8]) -> &'inner [u8] + 'a> {
// the reference to the struct member, captured by the closure
let opaque: &'a Opaque = *self.0.get(n)?;
Some(move |i: &[u8]| {
// placeholder: do something with the input using the &Opaque
&i[opaque.0..]
})
}
}
fn main() {
let o1 = Opaque(1);
let o5 = Opaque(5);
let o7 = Opaque(7);
let x = MyStruct(vec![&o1, &o5, &o7]);
let drop_five = x.get_func_for_index(1 /*Opaque(5)*/).unwrap();
let data: Vec<u8> = Vec::from(&b"testing"[..]);
assert_eq!(drop_five(&data[..]), b"ng");
}
If I try to compile it with rustc 1.54.0 (a178d0322 2021-07-26), I get the following error:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> /tmp/lifetimes.rs:16:14
|
16 | &i[opaque.0..]
| ^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 14:14...
--> /tmp/lifetimes.rs:14:14
|
14 | Some(move |i: &[u8]| {
| ______________^
15 | | // placeholder: do something with the input using the &Opaque
16 | | &i[opaque.0..]
17 | | })
| |_________^
note: ...so that reference does not outlive borrowed content
--> /tmp/lifetimes.rs:16:14
|
16 | &i[opaque.0..]
| ^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 6:6...
--> /tmp/lifetimes.rs:6:6
|
6 | impl<'a> MyStruct<'a> {
| ^^
note: ...so that return value is valid for the call
--> /tmp/lifetimes.rs:10:17
|
10 | ) -> Option<impl for<'inner> Fn(&'inner [u8]) -> &'inner [u8] + 'a> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: higher-ranked subtype error
--> /tmp/lifetimes.rs:7:5
|
7 | / pub fn get_func_for_index(
8 | | &'a self,
9 | | n: usize,
10 | | ) -> Option<impl for<'inner> Fn(&'inner [u8]) -> &'inner [u8] + 'a> {
| |_______________________________________________________________________^
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0495`.
It's quite a mouthful and I don't really understand what it's trying to tell me. The first part (first, the lifetime...) makes sense to me, the returned slice must not outlive the closure argument. The second part (but, the lifetime...) however seems weird to me - the + 'a annotation in the method signature refers to the closure itself (because it captures &'a self.foo), not to the value the closure returns.
Is it possible to change the code to model this correctly in rust, or is this construct just not supported at this time?
The problem is here:
Some(move |i: &[u8]| {
Every & has a lifetime on it, explicit or not. What is the lifetime of &[u8]? Clearly it should be "a lifetime chosen by the caller of the closure" (that is, a higher-ranked lifetime). But when the compiler encounters a reference type with a free lifetime parameter, even in the argument list of a closure, it will not assume that the lifetime is higher-ranked. The error message you get about the "anonymous lifetime #1" is the compiler confusedly trying to make a non-higher-ranked lifetime work.
The compiler could, in theory, work "backwards" from the impl Fn in the return type and recognize that the type of the closure needs to have this higher ranked lifetime. It's not quite clever enough to do that in this case, but there is a way to convince it: use a local function with a bounded type parameter to constrain the type of the closure to exactly the bound you want.
pub fn get_func_for_index(
&self, // note 1
n: usize,
) -> Option<impl 'a + for<'inner> Fn(&'inner [u8]) -> &'inner [u8]> { // note 2
// the reference to the struct member, captured by the closure
let opaque: &'a Opaque = *self.0.get(n)?;
// helper function to constrain the closure type
fn check<F: Fn(&[u8]) -> &[u8]>(f: F) -> F { // note 3
f
}
Some(check(move |i| {
// placeholder: do something with the input using the &Opaque
&i[opaque.0..]
}))
}
Playground
Please note the following:
&'a self is too conservative for this function because 'a is the lifetime parameter of the reference self contains, not the lifetime for which self is borrowed. In general, you should almost never write &'a self or &'a mut self where 'a is a named lifetime from an outer scope.
I find the + 'a easy to miss at the end of a long trait, especially a Fn trait with a return type. I recommend fronting the lifetime (putting it first) in cases like this to make clear that it relates more to the impl than to the &'inner [u8]. This is a stylistic choice.
Fn(&[u8]) -> &[u8] is actually exactly the same as for<'inner> Fn(&'inner [u8]) -> &'inner [u8], because the elision rules for Fn traits are the same as for regular functions. Either way is fine; I find the shorter version easier to read.
Similar questions
Expected bound lifetime parameter, found concrete lifetime [E0271]
How to declare a higher-ranked lifetime for a closure argument?
Why does this closure require inlining or `dyn`? What does `dyn` do here?
I'm trying to put a borrowed value behind a Mutex but I'm having trouble with the borrow checker. Here's a simplified code that demonstrate the problem I encounter:
use std::{
marker::PhantomData,
sync::{Arc, Mutex},
};
struct Test<'a> {
d: PhantomData<Mutex<&'a u8>>,
}
impl<'a> Test<'a> {
pub fn new() -> Self {
Test { d: PhantomData }
}
pub fn test(&'a self) {}
}
fn main() {
let t = Arc::new(Test::new());
let t2 = t.clone();
std::thread::spawn(move || {
t2.test();
});
}
This fails to compile with the following error
error[E0597]: `t2` does not live long enough
--> src/main.rs:21:9
|
19 | let t2 = t.clone();
| -- lifetime `'1` appears in the type of `t2`
20 | std::thread::spawn(move || {
21 | t2.test();
| ^^-------
| |
| borrowed value does not live long enough
| argument requires that `t2` is borrowed for `'1`
22 | });
| - `t2` dropped here while still borrowed
I guess the compiler thinks t2 might be borrowed to somewhere else when calling test(). It seems if I modify the type of the d field in struct Test to anything excluding Mutex, such as d: Option<&'a u8>, it works fine. What is so special about Mutex since it's just a wrapper around an UnsafeCell?
What is so special about Mutex since it's just a wrapper around an UnsafeCell?
The difference is variance.
&'a T is covariant in the lifetime 'a: You can coerce an immutable reference with a longer lifetime to one with a strictly shorter lifetime, because it is always safe to pass &'long T where &'short T is expected. This is why the code compiles without the UnsafeCell.
But UnsafeCell<&'a T> is invariant in 'a because it has interior mutability: If you could pass UnsafeCell<&'long T> to code that takes UnsafeCell<&'short T>, that code could write a short-lived reference into your long-lived cell. So it is not safe to coerce an UnsafeCell to have a different lifetime.
(The same is true for any type that lets you mutate the reference it contains, e.g. Mutex<&'a T> or &mut &'a T.)
I have a struct which contains some stuff. I implement the Iterator trait for that struct, and return a tuple of references to internal data in the struct. That necessitates that I annotate at least some things with lifetimes. What I want is to minimize the lifetime annotation, especially when it comes to other structs which have the original struct as a member.
some code:
pub struct LogReader<'a> {
data:String,
next_fn:fn(&mut LogReader)->Option<(&'a str,&'a [ConvertedValue])>,
//...
}
pub struct LogstreamProcessor {
reader: LogReader, // doesn't work without polluting LogstreamProcessor with lifetimes
//...
}
impl<'a> Iterator for LogReader<'a > {
type Item = (&'a str,&'a[ConvertedValue]);
fn next(&mut self) -> Option<(&'a str,&'a[ConvertedValue])>{(self.next_fn)(self)}
}
impl <'a> LogReader<'a> {
pub fn new(textFile:Option<bool>) -> LogReader<'a> {
LogReader {
next_fn:if textFile.unwrap_or(false) { LogReader::readNextText }else{ LogReader::readNextRaw },
data: "blah".to_string()
}
}
fn readNextText(&mut self)->Option<(&str,&[ConvertedValue])>{unimplemented!();}
fn readNextRaw(&mut self)->Option<(&str,&[ConvertedValue])>{unimplemented!();}
}
Can I limit the lifetime pollution from a struct?
Generically, if you're using them in any of your struct's fields, then you can't. They are made explicit for very good reasons (see Why are explicit lifetimes needed in Rust?), and once you have a struct containing objects that require explicit lifetimes, then they must be propagated.
Note that usually this isn't a concern to consumers of the struct, since the concrete lifetimes are then imposed by the compiler:
struct NameRef<'a>(&'a str);
let name = NameRef("Jake"); // 'a is 'static
One could also slightly mitigate the "noise" on the implementation of next by using the definition of Self::Item.
impl<'a> Iterator for LogReader<'a > {
type Item = (&'a str,&'a[ConvertedValue]);
fn next(&mut self) -> Option<Self::Item> {
(self.next_fn)(self)
}
}
However, your concern actually hides a more serious issue: Unlike you've mentioned, the values returned from next are not necessarily internal data from the struct. They actually live for as long as the generic lifetime 'a, and nothing inside LogReader is actually bound by that lifetime.
This means two things:
(1) I could pass a function that gives something completely different, and it would work just fine:
static NO_DATA: &[()] = &[()];
fn my_next_fn<'a>(reader: &mut LogReader<'a>) -> Option<(&'a str, &'a[ConvertedValue])> {
Some(("wat", NO_DATA))
}
(2) Even if I wanted my function to return something from the log reader's internal data, it wouldn't work because the lifetimes do not match at all. Let's try it out anyway to see what happens:
static DATA: &[()] = &[()];
fn my_next_fn<'a>(reader: &mut LogReader<'a>) -> Option<(&'a str, &'a[ConvertedValue])> {
Some((&reader.data[0..4], DATA))
}
fn main() {
let mut a = LogReader {
data: "This is DATA!".to_owned(),
next_fn: my_next_fn
};
println!("{:?}", a.next());
}
The compiler would throw you this:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> src/main.rs:26:12
|
26 | Some((&reader.data[0..4], DATA))
| ^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 25:88...
--> src/main.rs:25:89
|
25 | fn my_next_fn<'a>(reader: &mut LogReader<'a>) -> Option<(&'a str, &'a[ConvertedValue])> {
| _________________________________________________________________________________________^ starting here...
26 | | Some((&reader.data[0..4], DATA))
27 | | }
| |_^ ...ending here
note: ...so that reference does not outlive borrowed content
--> src/main.rs:26:12
|
26 | Some((&reader.data[0..4], DATA))
| ^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the body at 25:88...
--> src/main.rs:25:89
|
25 | fn my_next_fn<'a>(reader: &mut LogReader<'a>) -> Option<(&'a str, &'a[ConvertedValue])> {
| _________________________________________________________________________________________^ starting here...
26 | | Some((&reader.data[0..4], DATA))
27 | | }
| |_^ ...ending here
note: ...so that expression is assignable (expected std::option::Option<(&'a str, &'a [()])>, found std::option::Option<(&str, &[()])>)
--> src/main.rs:26:5
|
26 | Some((&reader.data[0..4], DATA))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...where the anonymous lifetime #1 is the log reader's lifetime. Forcing &mut LogReader to also have a lifetime 'a (&'a mut LogReader<'a>) would lead to further lifetime issues when attempting to implement Iterator. This basically narrows down to the fact that 'a is incompatible with references to values of LogReader themselves.
So, how should we fix that?
but that doesn't change the fact that the return type has references and so lifetime annotations come into it
Although that is not accurate (since lifetime elision can occur in some cases), that gives a hint to the solution: either avoid returning references at all or delegate data to a separate object, so that 'a can be bound to that object's lifetime. The final part of the answer to your question is in Iterator returning items by reference, lifetime issue.
Consider the following example
trait MyTrait<'a> {
type N: 'a;
fn func(&'a self) -> Self::N;
}
fn myfunc<'a, T: 'a + MyTrait<'a>>(g: T) {
g.func();
}
fn main() {}
Compiling this small program fails with:
error[E0597]: `g` does not live long enough
--> src/main.rs:8:5
|
8 | g.func();
| ^ borrowed value does not live long enough
9 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the lifetime 'a as defined on the function body at 7:1...
--> src/main.rs:7:1
|
7 | fn myfunc<'a, T: 'a + MyTrait<'a>>(g: T) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As far as I understand, the lifetime parameter 'a is not restricted and could be arbitrary. However, g is a parameter and its lifetime is only the function scope, therefore it does not satisfy the condition of lifetime 'a in the definition of method func.
What I really want is that the associated type N is always restricted to the lifetime of self in MyTrait. That's why I came up with the explicit lifetime parameter 'a of MyTrait. I want function myfunc to work, i.e. 'a should somehow be restricted to the lifetime of of the parameter g.
What is the "correct" way to solve this problem?
A very simple example is
struct MyPtr<'a> {
x: &'a usize,
}
struct MyStruct {
data: Vec<usize>,
}
impl<'a> MyTrait<'a> for MyStruct {
type N = MyPtr<'a>;
fn func(&'a self) -> Self::N {
MyPtr { x: &self.data[0] }
}
}
Note that this is extremely simplified, of course. The idea is that N always contains a reference to something contained in MyTrait and should therefore never outlive MyTrait.
What you want is not to bind a generic lifetime, but to allow "any" lifetime:
fn myfunc<T: for<'a> MyTrait<'a>>(g: T) {
g.func();
}
Fully working example in the playground.
The best source for an explanation is How does for<> syntax differ from a regular lifetime bound?.