I am having an issue working with lifetime parameters for structs. I am not 100% sure how to describe the problem, but I created a trivial case that shows my compile time error.
struct Ref;
struct Container<'a> {
r : &'a Ref
}
struct ContainerB<'a> {
c : Container<'a>
}
trait ToC {
fn to_c<'a>(&self, r : &'a Ref) -> Container<'a>;
}
impl<'a> ToC for ContainerB<'a> {
fn to_c(&self, r : &'a Ref) -> Container<'a> {
self.c
}
}
The error I am getting with this is
test.rs:16:3: 18:4 error: method `to_c` has an incompatible type for trait: expected concrete lifetime, but found bound lifetime parameter 'a
test.rs:16 fn to_c(&self, r : &'a Ref) -> Container<'a> {
test.rs:17 self.c
test.rs:18 }
test.rs:16:48: 18:4 note: expected concrete lifetime is the lifetime 'a as defined on the block at 16:47
test.rs:16 fn to_c(&self, r : &'a Ref) -> Container<'a> {
test.rs:17 self.c
test.rs:18 }
error: aborting due to previous error
I have tried many variations and just can't get this thing to compile. I found another post here (How to fix: expected concrete lifetime, but found bound lifetime parameter) but is appears to get around the problem instead of solving it. I can't really see why the problem even originates. The &Ref is being passed along via moves so it should just work right?
Any ideas? Thanks for all the help.
Let’s compare the two definitions. First, the trait method:
fn to_c<'a>(&self, r: &'a Ref) -> Container<'a>;
And the implementation:
fn to_c(&self, r: &'a Ref) -> Container<'a>;
See the difference? The latter doesn’t have <'a>. <'a> has been specified elsewhere; the fact that it has the same name does not matter: it is a different thing entirely.
Functionally, your trait definition says that the returned container will have a reference inside it to something from r, but nothing from self. It may use self inside the method, but it may not store any references to it in the returned value.
Your method definition, however, is using a 'a that ties the lifetimes of r and the returned Container to self (that is, to the object itself, not the reference—the ρ₂ in &'ρ₁ T<'ρ₂>—it’s a subtle but sometimes significant difference), whereas the trait definition had no such connection.
The two can be made to match by inserting the <'a> in the method definition in the implementation. But bear in mind that that is shadowing the 'a from ContainerB<'a>; it is not the same 'a! We’re better to give it another name; for convenience, I’ll make the change the other way round, changing it on the impl instead of the method (either would do):
impl<'b> ToC for ContainerB<'b> {
fn to_c<'a>(&self, r: &'a Ref) -> Container<'a> {
self.c
}
}
But now of course you have a problem: the return value is of type Container<'b> (because that’s what the field c in a ContainerB<'b> is), but your signature demands Container<'a> (something using a reference from r, not from self).
One way which would fix it is specifying the lifetime of &self as 'a in both the trait definition and the implementation; in the implementation, this would then demand that 'b was greater than or equal to 'a (by virtue of the fact that you have successfully taken a reference with lifetime 'a to an object with lifetime 'b, and the object must outlive the reference) and so due to the subtyping ('a is a subtype of 'b) Container<'b> would be safely coerced to Container<'a>.
These sorts of lifetime matters are difficult to think about when you’re not familiar with them; but in time they become quite natural.
Related
I'm trying to have a MyType that supports a From<&[u8]> trait, but I'm running into "lifetime problems":
Here's a minimally viable example:
struct MyType {
i: i32
}
impl MyType {
fn from_bytes(_buf: &[u8]) -> MyType {
// for example...
MyType { i: 3 }
}
}
impl From<&[u8]> for MyType {
fn from(bytes: &[u8]) -> Self {
MyType::from_bytes(bytes)
}
}
fn do_smth<T>() -> T where T: From<&[u8]>
{
// for example...
let buf : Vec<u8> = vec![1u8, 2u8];
T::from(buf.as_slice())
}
(...and here's a Rust playground link)
For reasons I cannot understand, the Rust compiler is telling me:
error[E0637]: `&` without an explicit lifetime name cannot be used here
--> src/lib.rs:17:36
|
17 | fn do_smth<T>() -> T where T: From<&[u8]>
| ^ explicit lifetime name needed here
I'm not an expert on lifetimes and I don't understand why this piece of code needs one. What would be the best way to fix this?
Might Rust be thinking that the type T could be a &[u8] itself? But, in that case, the lifetime should be inferred to be the same as the input to From::<&[u8]>::from(), no?
One fix I was given was to do:
fn do_smth<T>() -> T where for<'a> T: From<&'a [u8]>
{
// for example...
let buf : Vec<u8> = vec![1u8, 2u8];
T::from(buf.as_slice())
}
...but I do not understand this fix, nor do I understand why lifetimes are needed in the first place.
Rust first wants you to write:
fn do_smth<'a, T>() -> T
where
T: From<&'a [u8]>,
{
// for example...
let buf: Vec<u8> = vec![1u8, 2u8];
T::from(&buf)
}
where you make explicit that this function can be called for any lifetime 'a and any type T such that T implements From<&'a [u8]>.
But Rust then complains:
error[E0597]: `buf` does not live long enough
--> src/lib.rs:24:13
|
18 | fn do_smth<'a, T>() -> T
| -- lifetime `'a` defined here
...
24 | T::from(&buf)
| --------^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `buf` is borrowed for `'a`
25 | }
| - `buf` dropped here while still borrowed
You promised that this function could work with any lifetime, but this turns out to not be true, because in the body of the function you create a fresh reference to the local Vec which has a different lifetime, say 'local. Your function only works when 'a equals 'local, but you promise that it also works for all other lifetimes. What you need is a way to express that these lifetimes are the same, and the only way I think that is possible is by changing the local reference to an argument:
fn do_smth<'a, T>(buf: &'a [u8]) -> T
where
T: From<&'a [u8]>,
{
T::from(buf)
}
And then it compiles.
If instead of the function promising it can work with any lifetime, you want to make the caller promise it can work with any lifetime, you can instead use HRTBs to make the caller promise it.
fn do_smth<T>() -> T
where
for<'a> T: From<&'a [u8]>,
{
// for example...
let buf: Vec<u8> = vec![1u8, 2u8];
T::from(&buf)
}
Now, since you can use any lifetime, a local one also works and the code compiles.
Lifetimes represent a "duration" (metaphorically), or, more pragmatically, a scope, in which a variable is valid. Outside of one's lifetime, the variable should be considered as having been freed from memory, even though you haven't done that explicitly, because that's how Rust manages memory.
It becomes a bit more complex when Rust tries to ensure that, once a variable is done for, no other parts of the code that could have had access to that variable still have access. These shared accesses are called borrows, and that's why borrows have lifetimes too. The main condition Rust enforces on them is that a borrow's lifetime is always shorter (or within, depending on how you see it) than its original variable, ie. you can't share something for more time than you actually own it.
Rust therefore enforces all borrows (as well as all variables, really) to have an established lifetime at compile-time. To lighten things, Rust has default rules about what a lifetime should be if it was not explicitly defined by the user, that is, when you talk about a type that involves a lifetime, Rust let's you not write that lifetime explicitly under certain conditions. However, this is not a "lifetime inference", in the sense of inferring types: Rust will not try to make sense out of explicit lifetimes, it's a lot less smart about it. In particular, this lifetime explicitation can fail, in the sense that Rust will not be able to figure out the right lifetime it has to assign even though it was possible to find out that worked.
Back to business: your first error simply stems from the fact that Rust has no rule to make a lifetime if it wasn't provided in the position pointed out by the error. As I said, Rust won't try to infer what the right lifetime would be, it simply checks whether not explicitly putting a lifetime there implicitly means something or not. So, you simply need to put one.
Your first reflex might be to make your function generic over the missing lifetime, which is often the right thing to do (and even the only possible action), that is, do something like that:
fn do_smth<'a, T>() -> T
where
T: From<&'a [u8]>
{
// for example...
let buf : Vec<u8> = vec![1, 2];
T::from(buf.as_slice())
}
What this means is that do_smth is generic over the lifetime 'a, just like it is generic over the type T. This has two consequences:
Rust will proceed to a monomorphisation of your function for each call, meaning it will actually provide a concrete implementation of your function for each type T and each lifetime 'a that is required. In particular, it will automatically find out what is the right lifetime. This might seem contradictory with what I said earlier, about Rust not inferring lifetimes. The difference is that type inference and monomorphisation, although similar, are not the same step, and so the compiler does not work lifetimes in the same way. Don't worry about this until you have understood the rest.
The second consequence, which is a bit disastrous, is that your function exposes the following contract: for any type T, and any lifetime 'a, such that T: From<&'a [u8]>, do_smth can produce a type T. If you think about it, it means that even if T only implements From<&'a [u8]> for a lifetime 'a that is already finished (or, if you see lifetimes as scopes, for a lifetime 'a that is disjoint from do_smth's scope), you can produce an element of type T. This is not what you actually meant: you don't want the caller to give you an arbitrary lifetime. Instead, you know that the lifetime of the borrow of the slice is the one you chose it to be, within your function (because you own the underlying vector), and you want that the type T to be buildable from that slice. That is, you want T: From<&'a [u8]> for a 'a that you have chosen, not one provided by the caller.
This last point should make you understand why the previous snippet of code is unsound, and won't compile. Your function should not take a lifetime as argument, just a type T with certain constraints. But then, how do you encode the said conditions? That's where for<'a> comes into play. If you have a type T such that T: for<'a> From<&'a [u8]>, it means that for all 'a, T: From<&'a [u8]>. In particular, it is true for the lifetime of your slice. This is why the following works
fn do_smth<T>() -> T
where
T: for<'a> From<&'a [u8]>
{
// for example...
let buf: Vec<u8> = vec![1, 2];
T::from(buf.as_slice())
}
Note that, as planned, this version of do_smth is not generic over a lifetime, that is, the caller does not provide a lifetime to the function.
I'm trying to write a constructor function which takes a generic value implementing some trait by argument, and then boxes it (the point is to then initialize something with these boxes, but the following example is simplified):
struct Struct {}
trait Trait {}
impl Trait for Struct {}
fn f(arg: impl Trait) -> Box<dyn Trait> {
Box::new(arg)
}
fn main() {
let x = Struct {};
f(x);
}
Here, the compiler whines that arg might not live long enough. This makes perfect sense to me, because the only requirement on arg is impl Trait, meaning it might in fact be a reference which implement the trait, in which case it cannot be boxed safely.
What confuses me is the following solution, adding a 'static bound:
fn f(arg: impl Trait + 'static) -> Box<dyn Trait> {
Box::new(arg)
}
Now, of course it works, but in principle we are now constrained to passing in values with a 'static lifetime. Yet, the compiler lets me pass in x without a problem.
Here are my questions, more specifically:
Doesn't x have a lifetime, residing on the stack? Why is it possible to pass it to f when arg has a 'static lifetime bound? Do these bounds only concern the lifetime of references?
Is this solution valid in general, or will I face cases where the compiler will refuse my stack-allocated arguments?
Is there a nicer way of saying "any type that impls Trait where the type is not a reference?
Note that 1) is mostly answered by Why does Rust require a `'static` lifetime for this variable? but I am left confused as to if this is the idiomatic way of boxing arguments.
EDIT:
Now that I understand better, I am left wondering what is the idiomatic solution to fix the compiler error in the case of populating a struct:
struct OtherStruct {
x: Box<dyn Trait>,
}
impl OtherStruct {
fn new(arg: impl Trait) -> Self { // Does not compile
Self { x: Box::new(arg) }
}
}
The only solutions I see so far are 1) adding a lifetime parameter to OtherStruct (not so great), adding a 'static lifetime bound to arg (I'm not sure if this is OK?)
You have a couple misconceptions here.
Doesn't x have a lifetime, residing on the stack? Why is it possible to pass it to f when arg has a 'static lifetime bound? Do these bounds only concern the lifetime of references?
When you are doing f(x), because you don't take a reference to x, you are moving the value of x into the function, which shifts its lifetime. If you tried to use x again after calling f(x), Rust will fail to compile your code and tell you this.
As for the + 'static bound... Box<dyn Trait> is shorthand for Box<dyn Trait + 'static>, which is why the compiler was giving you an error. The type system needs to know the lifetime of the implementation inside of the box so that it can check it. You could give the box a different lifetime, if you wish, explicitly:
fn f<'a>(arg: impl Trait + 'a) -> Box<dyn Trait + 'a> {
Box::new(arg)
}
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.
I believe that this function declaration tells Rust that the lifetime of the function's output is the same as the lifetime of it's s parameter:
fn substr<'a>(s: &'a str, until: u32) -> &'a str;
^^^^
It seems to me that the compiler only needs to know this(1):
fn substr(s: &'a str, until: u32) -> &'a str;
What does the annotation <'a> after the function name mean? Why does the compiler need it, and what does it do with it?
(1): I know it needs to know even less, due to lifetime elision. But this question is about specifying lifetime explicitly.
Let me expand on the previous answers…
What does the annotation <'a> after the function name mean?
I wouldn't use the word "annotation" for that. Much like <T> introduces a generic type parameter, <'a> introduces a generic lifetime parameter. You can't use any generic parameters without introducing them first and for generic functions this introduction happens right after their name. You can think of a generic function as a family of functions. So, essentially, you get one function for every combination of generic parameters. substr::<'x> would be a specific member of that function family for some lifetime 'x.
If you're unclear on when and why we have to be explicit about lifetimes, read on…
A lifetime parameter is always associated with all reference types. When you write
fn main() {
let x = 28374;
let r = &x;
}
the compiler knows that x lives in the main function's scope enclosed with curly braces. Internally, it identifies this scope with some lifetime parameter. For us, it is unnamed. When you take the address of x, you'll get a value of a specific reference type. A reference type is kind of a member of a two dimensional family of reference types. One axis is the type of what the reference points to and the other axis is a lifetime that is used for two constraints:
The lifetime parameter of a reference type represents an upper bound for how long you can hold on to that reference
The lifetime parameter of a reference type represents a lower bound for the lifetime of the things you can make the reference point to.
Together, these constraints play a vital role in Rust's memory safety story. The goal here is to avoid dangling references. We would like to rule out references that point to some memory region we are not allowed to use anymore because that thing it used to point to does not exist anymore.
One potential source of confusion is probably the fact that lifetime parameters are invisible most of the time. But that does not mean they are not there. References always have a lifetime parameter in their type. But such a lifetime parameter does not have to have a name and most of the time we don't need to mention it anyways because the compiler can assign names for lifetime parameters automatically. This is called "lifetime elision". For example, in the following case, you don't see any lifetime parameters being mentioned:
fn substr(s: &str, until: u32) -> &str {…}
But it's okay to write it like this. It's actually a short-cut syntax for the more explicit
fn substr<'a>(s: &'a str, until: u32) -> &'a str {…}
Here, the compiler automatically assigns the same name to the "input lifetime" and the "output lifetime" because it's a very common pattern and most likely exactly what you want. Because this pattern is so common, the compiler lets us get away without saying anything about lifetimes. It assumes that this more explicit form is what we meant based on a couple of "lifetime elision" rules (which are at least documented here)
There are situations in which explicit lifetime parameters are not optional. For example, if you write
fn min<T: Ord>(x: &T, y: &T) -> &T {
if x <= y {
x
} else {
y
}
}
the compiler will complain because it will interpret the above declaration as
fn min<'a, 'b, 'c, T: Ord>(x: &'a T, y: &'b T) -> &'c T { … }
So, for each reference a separate lifetime parameter is introduced. But no information on how the lifetime parameters relate to each other is available in this signature. The user of this generic function could use any lifetimes. And that's a problem inside its body. We're trying to return either x or y. But the type of x is &'a T. That's not compatible with the return type &'c T. The same is true for y. Since the compiler knows nothing about how these lifetimes relate to each other, it's not safe to return these references as a reference of type &'c T.
Can it ever be safe to go from a value of type &'a T to &'c T? Yes. It's safe if the lifetime 'a is equal or greater than the lifetime 'c. Or in other words 'a: 'c. So, we could write this
fn min<'a, 'b, 'c, T: Ord>(x: &'a T, y: &'b T) -> &'c T
where 'a: 'c, 'b: 'c
{ … }
and get away with it without the compiler complaining about the function's body. But it's actually unnecessarily complex. We can also simply write
fn min<'a, T: Ord>(x: &'a T, y: &'a T) -> &'a T { … }
and use a single lifetime parameter for everything. The compiler is able to deduce 'a as the minimum lifetime of the argument references at the call site just because we used the same lifetime name for both parameters. And this lifetime is precisely what we need for the return type.
I hope this answers your question. :)
Cheers!
What does the annotation <'a> after the function name mean?
fn substr<'a>(s: &'a str, until: u32) -> &'a str;
// ^^^^
This is declaring a generic lifetime parameter. It's similar to a generic type parameter (often seen as <T>), in that the caller of the function gets to decide what the lifetime is. Like you said, the lifetime of the result will be the same as the lifetime of the first argument.
All lifetime names are equivalent, except for one: 'static. This lifetime is pre-set to mean "guaranteed to live for the entire life of the program".
The most common lifetime parameter name is probably 'a, but you can use any letter or string. Single letters are most common, but any snake_case identifier is acceptable.
Why does the compiler need it, and what does it do with it?
Rust generally favors things to be explicit, unless there's a very good ergonomic benefit. For lifetimes, lifetime elision takes care of something like 85+% of cases, which seemed like a clear win.
Type parameters live in the same namespace as other types — is T a generic type or did someone name a struct that? Thus type parameters need to have an explicit annotation that shows that T is a parameter and not a real type. However, lifetime parameters don't have this same problem, so that's not the reason.
Instead, the main benefit of explicitly listing type parameters is because you can control how multiple parameters interact. A nonsense example:
fn better_str<'a, 'b, 'c>(a: &'a str, b: &'b str) -> &'c str
where
'a: 'c,
'b: 'c,
{
if a.len() < b.len() {
a
} else {
b
}
}
We have two strings and say that the input strings may have different lifetimes, but must both outlive the lifetime of the result value.
Another example, as pointed out by DK, is that structs can have their own lifetimes. I made this example also a bit of nonsense, but it hopefully conveys the point:
struct Player<'a> {
name: &'a str,
}
fn name<'p, 'n>(player: &'p Player<'n>) -> &'n str {
player.name
}
Lifetimes can be one of the more mind-bending parts of Rust, but they are pretty great when you start to grasp them.
The <'a> annotation just declares the lifetimes used in the function, exactly like generic parameters <T>.
fn subslice<'a, T>(s: &'a [T], until: u32) -> &'a [T] { \\'
&s[..until as usize]
}
Note that in your example, all lifetimes can be inferred.
fn subslice<T>(s: &[T], until: u32) -> &[T] {
&s[..until as usize]
}
fn substr(s: &str, until: u32) -> &str {
&s[..until as usize]
}
playpen example