Playing around with async/await I stumbled across this lifetime puzzle. I assume that predicate takes a reference/value and produces static future - it has no reference to its arguments and captures no values. Instead I got complain about lifetime being too short.
Why it fails and where's the lifetime '2?
use futures::future;
use std::future::Future;
fn validate<F, Fut>(_: F)
where
F: FnMut(&str) -> Fut,
Fut: Future<Output = bool>,
{
}
fn predicate(_: impl AsRef<str>) -> impl Future<Output = bool> {
future::ready(true)
}
fn main() {
validate(|x| predicate(x));
}
Error message:
error: lifetime may not live long enough
--> src\main.rs:16:18
|
16 | validate(|x| predicate(x));
| -- ^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
| ||
| |return type of closure is impl futures::Future
| has type `&'1 str`
error: aborting due to previous error
The error message here is a little confusing because it's talking about two lifetimes, '1 and '2, but doesn't actually say what '2 refers to.
There are two lifetimes that matter here:
the &str argument, which the message labels with lifetime '1
the future type variable Fut, which is labeled with lifetime '2 in the message, though it doesn't clearly state that.
The inferred lifetimes (using the lifetime elision rules) for predicate are like this:
fn predicate<'x>(_: impl AsRef<str> + 'x) -> impl Future<Output = bool> + 'x {
future::ready(true)
}
Even though the implementation doesn't actually use the string input, it would be weird if it didn't, so this is probably correct for a real implementation. The compiler has inferred that the return value of a function uses the input, so they must have compatible lifetimes.
However, the lifetimes for validate are inferred to be different:
fn validate<'1, '2, F, Fut>(_: F)
where
F: FnMut(&'1 str) -> Fut,
Fut: Future<Output = bool> + '2,
{
}
This function accepts arguments with broader requirements than predicate, which means it can't guarantee to enforce the constraints that predicate needs.
You can fix that by making it explicit in the type of validate:
fn validate<'a, F, Fut>(_: F)
where
F: FnMut(&'a str) -> Fut,
Fut: Future<Output = bool> + 'a,
{
}
Constraining both types with the 'a lifetime communicates that validate expects F to use data from the &str, and therefore will ensure that the string lives long enough. Any implementation of validate that broke this contract would also not compile.
Related
I have started to learn Rust. Currently I'm trying to learn how to properly use lifetime annotations and think I have understood the basics quite well. However, I have on several occasions encountered the following structure:
fn<'a> foo(a: &'a str, ...) -> &str + 'a
The str is not relevant it can be any type really, my question is specifically what &str + 'a mean (I might not be using it correctly, which is why I'm asking about it) as opposed to &'a str. As a real world example I have encountered it in this tutorial for async rust where they write:
fn foo_expanded<'a>(x: &'a u8) -> impl Future<Output = u8> + 'a
I'm speculating that it might have to do with that Future is a trait and not a type, but I have been unable to verify it in any official documentation and have not found any source on what the syntax mean.
First of all, the syntax shown in your post is not allowed.
fn<'a> foo(a: &'a str, ...) -> &str + 'a
There are two reasons:
lifetime generics must be specified after the function name.
the displayed way of specifying return lifetimes is allowed only for traits, not complete types.
Otherwise you would get one of the two following errors:
error[E0178]: expected a path on the left-hand side of `+`, not `&str`
--> ./ex_056.rs:11:43
|
11 | fn _get<'a>(ms: &'a MyStruct, s: &str) -> &str + 'a {
| ^^^^^^^^^ help: try adding parentheses: `&(str + 'a)`
or
error[E0404]: expected trait, found builtin type `str`
--> ./ex_056.rs:15:31
|
15 | fn _get2<'a>(s: &'a str) -> &(str + 'a) {
| ^^^ not a trait
Thus it's not valid.
As a crude guess, I imagine that you have been misled by not a complete type but just a trait object. Since such a notation was allowed in 2015 but now it is deprecated, as you can see in the following warning:
warning: trait objects without an explicit `dyn` are deprecated
--> ./ex_056.rs:15:31
|
15 | fn _get2<'a>(s: &'a str) -> &(str + 'a) {
| ^^^^^^^^ help: use `dyn`: `dyn str + 'a`
|
= warning: this is accepted in the current edition (Rust 2015) but is a hard error in Rust 2021!
= note: for more information, see <https://doc.rust-lang.org/nightly/edition-guide/rust-2021/warnings-promoted-to-error.html>
Your first example (&str + 'a) is not valid. The + 'a notation can only be applied to a trait.
Your second example: impl Future<Output = u8> + 'a means that foo_expanded returns some unknown type that implements the trait Future<Output = u8> and that this unknown type may contain references with the 'a lifetime. Therefore you won't be able to use the returned value once the 'a lifetime expires.
The implementation of std::mem::drop is documented to be the following:
pub fn drop<T>(_x: T) { }
As such, I would expect the closure |_| () (colloquially known as the toilet closure) to be a potential 1:1 replacement to drop, in both directions. However, the code below shows that drop isn't compatible with a higher ranked trait bound on the function's parameter, whereas the toilet closure is.
fn foo<F, T>(f: F, x: T)
where
for<'a> F: FnOnce(&'a T),
{
dbg!(f(&x));
}
fn main() {
foo(|_| (), "toilet closure"); // this compiles
foo(drop, "drop"); // this does not!
}
The compiler's error message:
error[E0631]: type mismatch in function arguments
--> src/main.rs:10:5
|
1 | fn foo<F, T>(f: F, x: T)
| ---
2 | where
3 | for<'a> F: FnOnce(&'a T),
| ------------- required by this bound in `foo`
...
10 | foo(drop, "drop"); // this does not!
| ^^^
| |
| expected signature of `for<'a> fn(&'a _) -> _`
| found signature of `fn(_) -> _`
error[E0271]: type mismatch resolving `for<'a> <fn(_) {std::mem::drop::<_>} as std::ops::FnOnce<(&'a _,)>>::Output == ()`
--> src/main.rs:10:5
|
1 | fn foo<F, T>(f: F, x: T)
| ---
2 | where
3 | for<'a> F: FnOnce(&'a T),
| ------------- required by this bound in `foo`
...
10 | foo(drop, "drop"); // this does not!
| ^^^ expected bound lifetime parameter 'a, found concrete lifetime
Considering that drop is supposedly generic with respect to any sized T, it sounds unreasonable that the "more generic" signature fn(_) -> _ is not compatible with for<'a> fn (&'a _) -> _. Why is the compiler not admitting the signature of drop here, and what makes it different when the toilet closure is placed in its stead?
The core of the issue is that drop is not a single function, but rather a parameterized set of functions that each drop some particular type. To satisfy a higher-ranked trait bound (hereafter hrtb), you'd need a single function that can simultaneously take references to a type with any given lifetime.
We'll use drop as our typical example of a generic function, but all this applies more generally too. Here's the code for reference: fn drop<T>(_: T) {}.
Conceptually, drop is not a single function, but rather one function for every possible type T. Any particular instance of drop takes only arguments of a single type. This is called monomorphization. If a different T is used with drop, a different version of drop is compiled. That's why you can't pass a generic function as an argument and use that function in full generality (see this question)
On the other hand, a function like fn pass(x: &i32) -> &i32 {x} satisfies the hrtb for<'a> Fn(&'a i32) -> &'a i32. Unlike drop, we have a single function that simultaneously satisfies Fn(&'a i32) -> &'a i32 for every lifetime 'a. This is reflected in how pass can be used.
fn pass(x: &i32) -> &i32 {
x
}
fn two_uses<F>(f: F)
where
for<'a> F: Fn(&'a i32) -> &'a i32, // By the way, this can simply be written
// F: Fn(&i32) -> &i32 due to lifetime elision rules.
// That applies to your original example too.
{
{
// x has some lifetime 'a
let x = &22;
println!("{}", f(x));
// 'a ends around here
}
{
// y has some lifetime 'b
let y = &23;
println!("{}", f(y));
// 'b ends around here
}
// 'a and 'b are unrelated since they have no overlap
}
fn main() {
two_uses(pass);
}
(playground)
In the example, the lifetimes 'a and 'b have no relation to each other: neither completely encompasses the other. So there isn't some kind of subtyping thing going on here. A single instance of pass is really being used with two different, unrelated lifetimes.
This is why drop doesn't satisfy for<'a> FnOnce(&'a T). Any particular instance of drop can only cover one lifetime (ignoring subtyping). If we passed drop into two_uses from the example above (with slight signature changes and assuming the compiler let us), it would have to choose some particular lifetime 'a and the instance of drop in the scope of two_uses would be Fn(&'a i32) for some concrete lifetime 'a. Since the function would only apply to single lifetime 'a, it wouldn't be possible to use it with two unrelated lifetimes.
So why does the toilet closure get a hrtb? When inferring the type for a closure, if the expected type hints that a higher-ranked trait bound is needed, the compiler will try to make one fit. In this case, it succeeds.
Issue #41078 is closely related to this and in particular, eddyb's comment here gives essentially the explanation above (though in the context of closures, rather than ordinary functions). The issue itself doesn't address the present problem though. It instead addresses what happens if you assign the toilet closure to a variable before using it (try it out!).
It's possible that the situation will change in the future, but it would require a pretty big change in how generic functions are monomorphized.
In short, both lines should fail. But since one step in old way of handling hrtb lifetimes, namely the leak check, currently has some soundness issue, rustc ends up (incorrectly) accepting one and leaving the other with a pretty bad error message.
If you disable the leak check with rustc +nightly -Zno-leak-check, you'll be able to see a more sensible error message:
error[E0308]: mismatched types
--> src/main.rs:10:5
|
10 | foo(drop, "drop");
| ^^^ one type is more general than the other
|
= note: expected type `std::ops::FnOnce<(&'a &str,)>`
found type `std::ops::FnOnce<(&&str,)>`
My interpretation of this error is that the &x in the body of the foo function only has a scope lifetime confined to the said body, so f(&x) also has the same scope lifetime which can't possibly satisfy the for<'a> universal quantification required by the trait bound.
The question you present here is almost identical to issue #57642, which also has two contrasting parts.
The new way to process hrtb lifetimes is by using so-called universes. Niko has a WIP to tackle the leak check with universes. Under this new regime, both parts of issue #57642 linked above is said to all fail with far more clear diagnoses. I suppose the compiler should be able to handle your example code correctly by then, too.
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.
The following code works fine:
fn get<F: Fn(&[u8]) -> u8>(f: F) -> u8 {
f(&[1, 2, 3])
}
However, when I add explicit lifetime information to it, it doesn't:
fn get<'inp, F: Fn(&'inp [u8]) -> u8>(f: F) -> u8 {
f(&[1, 2, 3])
}
What lifetime does the compiler infer in the working code?
I'm using Rust 1.18.0.
The error message is:
error: borrowed value does not live long enough
--> test.rs:4:8
|
4 | f(&[1, 2, 3])
| ^^^^^^^^^ does not live long enough
5 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the lifetime 'inp as defined on the body at 3:49...
--> test.rs:3:50
|
3 | fn get<'inp, F: Fn(&'inp [u8]) -> u8>(f: F) -> u8 {
| __________________________________________________^
4 | | f(&[1, 2, 3])
5 | | }
| |_^
Lifetimes in trait bounds are a bit special and the Fn family of traits has a special lifetime elision rule. We'll dive into that, but first, here it the correct explicitly annotated version:
fn get<F: for<'inp> Fn(&'inp [u8]) -> u8>(f: F) -> u8 {
f(&[1, 2, 3])
}
Oh gosh, what is this for<'inp> doing there? It's a so called higher ranked trait bound (HRTB) and it's used here to make 'inp universally quantiefied in regards to f. In order to fully understand that, we need to understand a bit of theory.
Who has the choice?
Let's take a look at an easy example:
fn bar<'a>(x: &'a u8) {}
Here, bar() is generic of the lifetime 'a. The syntax above reads: "choose any 'a and there is a bar() that will work with 'a". This means that we can choose any 'a we want, and bar() works! Who are "we"? "We" are the caller -- the one calling bar. This will be important later: the caller chooses the generic parameters. We can call bar() with a &'static u8 as well as with a reference that doesn't live as long.
Now you might ask: are there situations where the caller doesn't choose the generic parameter, but someone else does? Yes, there are! Sadly, it's a bit more difficult to understand, because it doesn't occur too often in today's Rust code. But let's try:
trait Bar<'a> {
fn bar(&self, x: &'a u8);
}
This is similar to the bar() function above, but now the lifetime parameter is defined on the trait, not the function. Let's use the trait:
fn use_bar<'a, B: Bar<'a>>(b: B) {
let local = 0u8;
b.bar(&local);
}
This doesn't compile, printing the same error as above. Why? The method b.bar() expects a reference of lifetime 'a. But who chooses 'a here? Exactly: the caller -- the caller of use_bar(), not the caller of bar()! So the caller of use_bar() could choose the 'static lifetime; in that case, it's easy to see that our &local doesn't fulfill the lifetime requirements.
Note that the caller of use_bar() chooses 'a as well as B. Once use_bar() is instantiated, B is a fixed type and B::bar() works only for one specific lifetime. This means the caller of bar() can't choose the lifetime, but bar() itself chose it!
What do we want instead? We want use_bar() to choose the lifetime of the bar() call. And we can do that with the for<> syntax:
fn use_bar<B: for<'a> Bar<'a>>(b: B) {
let local = 0u8;
b.bar(&local);
}
This works. What we say here is: "for any lifetime parameter 'a, B has to implement the trait Bar<'a>". Instead of: "there needs to exist a lifetime parameter 'a for which B implements Bar<'a>". It's all about who chooses which parameter.
Let's use the real names for it:
a generic parameter is universally quantified if the caller can choose it
a generic parameter is existentially quantified if the callee can choose it
What does Rust do?
To return to your example:
fn get<'inp, F: Fn(&'inp [u8]) -> u8>(f: F) -> u8 {
f(&[1, 2, 3])
}
Here we have the same problem as above: the lifetime parameter of f is existentially quantified. The caller of f cannot choose the lifetime parameter. We can fix that with the for<> syntax as shown above.
When you omit the lifetimes:
fn get<F: Fn(&[u8]) -> u8>(f: F) -> u8 {
f(&[1, 2, 3])
}
The Rust compiler will do something special for the Fn family of traits. Your F: Fn(&[u8]) desugars to F: for<'a> Fn<(&'a [u8],)>. If you use Fn* traits with parameters that involve lifetimes, those lifetimes are automatically universally quantified, because that's usually what you want with higher order functions.
The infered code seems to be:
fn get<F : for<'inp> Fn(&'inp[u8]) -> u8>(f: F) -> u8 {
f(&[1,2,3])
}
Explanation