This question already has answers here:
The compiler suggests I add a 'static lifetime because the parameter type may not live long enough, but I don't think that's what I want
(2 answers)
Closed 4 years ago.
I'm trying to build a very basic example of an asynchronous callback function in Rust:
extern crate tokio;
extern crate tokio_core;
extern crate tokio_io;
use std::error::Error;
use tokio::prelude::future::ok;
use tokio::prelude::Future;
fn async_op() -> impl Future<Item = i32, Error = Box<Error>> {
ok(12)
}
fn main() {
let fut = async_op().and_then(|result| {
println!("result: {:?}", result);
});
tokio::run(fut);
}
This always results in the compiler error:
error[E0106]: missing lifetime specifier
--> src/main.rs:9:54
|
9 | fn async_op() -> impl Future<Item = i32, Error = Box<Error>> {
| ^^^^^ expected lifetime parameter
|
= help: this function's return type contains a borrowed value, but there is no value for it to be borrowed from
= help: consider giving it a 'static lifetime
Why is there a lifetime error in the first place? Why is it only for the Error but not for the Item?
I am also unsure about "help: consider giving it a 'static lifetime" ‒ AFAICT this would result in a lifetime of the return value over the entire program execution, which is definitely not what I would want in a more complex example.
Everything in Rust has a lifetime bound. If it contains references, it is the lifetime of the shortest-lived reference, otherwise it is 'static, which should be interpreted as “does not depend on anything that could have shorter lifetime”.
Therefore the lifetime of i32 is known. It is 'static, because it does not contain any references.
But Error, which is std::error::Error is a trait, and one which does not require any lifetime bound. That means you could implement it for a reference, or a type including a reference with a lifetime. And since the code is supposed to be valid for any substitution you might make anywhere downstream, the compiler insists that you give it a lifetime that is lower bound for usability of the return value.
Giving it 'static lifetime is sensible here. It does not mean the return value would be valid over the entire program execution, it only means the the return value is not limited to any shorter lifetime (which basically means that it does not depend on anything outside the Box except possibly static things) and will stay valid as long as something holds on to it.
I believe the correct syntax is:
fn async_op() -> impl Future<Item = i32, Error = Box<Error + 'static>>
Note that it is really only limiting the content of the Box. That's the only thing the compiler does not see into and is concerned it might cease to be valid at some point. And thus you promise it that the content of the Box won't become invalid until it drops the Box.
Related
I'm trying to have a MyType that supports a From<&[u8]> trait, but I'm running into "lifetime problems":
Here's a minimally viable example:
struct MyType {
i: i32
}
impl MyType {
fn from_bytes(_buf: &[u8]) -> MyType {
// for example...
MyType { i: 3 }
}
}
impl From<&[u8]> for MyType {
fn from(bytes: &[u8]) -> Self {
MyType::from_bytes(bytes)
}
}
fn do_smth<T>() -> T where T: From<&[u8]>
{
// for example...
let buf : Vec<u8> = vec![1u8, 2u8];
T::from(buf.as_slice())
}
(...and here's a Rust playground link)
For reasons I cannot understand, the Rust compiler is telling me:
error[E0637]: `&` without an explicit lifetime name cannot be used here
--> src/lib.rs:17:36
|
17 | fn do_smth<T>() -> T where T: From<&[u8]>
| ^ explicit lifetime name needed here
I'm not an expert on lifetimes and I don't understand why this piece of code needs one. What would be the best way to fix this?
Might Rust be thinking that the type T could be a &[u8] itself? But, in that case, the lifetime should be inferred to be the same as the input to From::<&[u8]>::from(), no?
One fix I was given was to do:
fn do_smth<T>() -> T where for<'a> T: From<&'a [u8]>
{
// for example...
let buf : Vec<u8> = vec![1u8, 2u8];
T::from(buf.as_slice())
}
...but I do not understand this fix, nor do I understand why lifetimes are needed in the first place.
Rust first wants you to write:
fn do_smth<'a, T>() -> T
where
T: From<&'a [u8]>,
{
// for example...
let buf: Vec<u8> = vec![1u8, 2u8];
T::from(&buf)
}
where you make explicit that this function can be called for any lifetime 'a and any type T such that T implements From<&'a [u8]>.
But Rust then complains:
error[E0597]: `buf` does not live long enough
--> src/lib.rs:24:13
|
18 | fn do_smth<'a, T>() -> T
| -- lifetime `'a` defined here
...
24 | T::from(&buf)
| --------^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `buf` is borrowed for `'a`
25 | }
| - `buf` dropped here while still borrowed
You promised that this function could work with any lifetime, but this turns out to not be true, because in the body of the function you create a fresh reference to the local Vec which has a different lifetime, say 'local. Your function only works when 'a equals 'local, but you promise that it also works for all other lifetimes. What you need is a way to express that these lifetimes are the same, and the only way I think that is possible is by changing the local reference to an argument:
fn do_smth<'a, T>(buf: &'a [u8]) -> T
where
T: From<&'a [u8]>,
{
T::from(buf)
}
And then it compiles.
If instead of the function promising it can work with any lifetime, you want to make the caller promise it can work with any lifetime, you can instead use HRTBs to make the caller promise it.
fn do_smth<T>() -> T
where
for<'a> T: From<&'a [u8]>,
{
// for example...
let buf: Vec<u8> = vec![1u8, 2u8];
T::from(&buf)
}
Now, since you can use any lifetime, a local one also works and the code compiles.
Lifetimes represent a "duration" (metaphorically), or, more pragmatically, a scope, in which a variable is valid. Outside of one's lifetime, the variable should be considered as having been freed from memory, even though you haven't done that explicitly, because that's how Rust manages memory.
It becomes a bit more complex when Rust tries to ensure that, once a variable is done for, no other parts of the code that could have had access to that variable still have access. These shared accesses are called borrows, and that's why borrows have lifetimes too. The main condition Rust enforces on them is that a borrow's lifetime is always shorter (or within, depending on how you see it) than its original variable, ie. you can't share something for more time than you actually own it.
Rust therefore enforces all borrows (as well as all variables, really) to have an established lifetime at compile-time. To lighten things, Rust has default rules about what a lifetime should be if it was not explicitly defined by the user, that is, when you talk about a type that involves a lifetime, Rust let's you not write that lifetime explicitly under certain conditions. However, this is not a "lifetime inference", in the sense of inferring types: Rust will not try to make sense out of explicit lifetimes, it's a lot less smart about it. In particular, this lifetime explicitation can fail, in the sense that Rust will not be able to figure out the right lifetime it has to assign even though it was possible to find out that worked.
Back to business: your first error simply stems from the fact that Rust has no rule to make a lifetime if it wasn't provided in the position pointed out by the error. As I said, Rust won't try to infer what the right lifetime would be, it simply checks whether not explicitly putting a lifetime there implicitly means something or not. So, you simply need to put one.
Your first reflex might be to make your function generic over the missing lifetime, which is often the right thing to do (and even the only possible action), that is, do something like that:
fn do_smth<'a, T>() -> T
where
T: From<&'a [u8]>
{
// for example...
let buf : Vec<u8> = vec![1, 2];
T::from(buf.as_slice())
}
What this means is that do_smth is generic over the lifetime 'a, just like it is generic over the type T. This has two consequences:
Rust will proceed to a monomorphisation of your function for each call, meaning it will actually provide a concrete implementation of your function for each type T and each lifetime 'a that is required. In particular, it will automatically find out what is the right lifetime. This might seem contradictory with what I said earlier, about Rust not inferring lifetimes. The difference is that type inference and monomorphisation, although similar, are not the same step, and so the compiler does not work lifetimes in the same way. Don't worry about this until you have understood the rest.
The second consequence, which is a bit disastrous, is that your function exposes the following contract: for any type T, and any lifetime 'a, such that T: From<&'a [u8]>, do_smth can produce a type T. If you think about it, it means that even if T only implements From<&'a [u8]> for a lifetime 'a that is already finished (or, if you see lifetimes as scopes, for a lifetime 'a that is disjoint from do_smth's scope), you can produce an element of type T. This is not what you actually meant: you don't want the caller to give you an arbitrary lifetime. Instead, you know that the lifetime of the borrow of the slice is the one you chose it to be, within your function (because you own the underlying vector), and you want that the type T to be buildable from that slice. That is, you want T: From<&'a [u8]> for a 'a that you have chosen, not one provided by the caller.
This last point should make you understand why the previous snippet of code is unsound, and won't compile. Your function should not take a lifetime as argument, just a type T with certain constraints. But then, how do you encode the said conditions? That's where for<'a> comes into play. If you have a type T such that T: for<'a> From<&'a [u8]>, it means that for all 'a, T: From<&'a [u8]>. In particular, it is true for the lifetime of your slice. This is why the following works
fn do_smth<T>() -> T
where
T: for<'a> From<&'a [u8]>
{
// for example...
let buf: Vec<u8> = vec![1, 2];
T::from(buf.as_slice())
}
Note that, as planned, this version of do_smth is not generic over a lifetime, that is, the caller does not provide a lifetime to the function.
I started learning Rust a few days back.
This is an extract from the famous book Programming Rust by Jim Blandy.
For the code
fn g<'a>(p: &'a i32) { ... }
let x = 10;
g(&x);
The book says
Rust Choose the smallest possible lifetime for &x, that of the call to g. This meets all constraints: it doesn't outlive x, and encloses the entire call to g. So code must muster.
Q1. What is meant by the smallest possible lifetime for &x?
For the code
fn f(p: &'static i32) { ... }
let x = 10;
f(&x);
Q2. Why does this code fail? According to my understanding, &'static is used for static global variables which live for the full program. link
A 'static lifetime is a special concept. It specifies that the variable referenced by this needs to exist for the entire lifetime of the program. Using this is a rare case, and requires even rarer precautions to fulfill.
In practice, a &'static reference may only happen in two cases:
A const declaration
A static declaration
Both effectively accomplish the same thing, in different ways; the differences aren't important to this question, however. In both cases, the outcome is a variable that is available for the entire lifetime of the program and will not be relocated, thus guaranteeing &'static if borrowed.
Now that we've covered this, let's cover both of your questions.
Q1. What is meant by the smallest possible lifetime for &x?
When you define a function as fn g<'a>(p: &'a i32) { ... }, you are requiring p to be valid for a lifetime 'a; this lifetime is determined by the compiler so that 'a is the smallest possible. If the reference is never used outside of the function scope, 'a will be the lifetime of execution of that function, for example. If you use or reference this borrow outside of the function, the lifetime will (evidently) be larger.
The definition of "smallest possible" is simple: the compiler will infer the lifetime based from the time you start that reference, to the last time you use that reference. Dependent borrows also count, and this typically comes back to bite people when dealing with collections.
The reason it is the smallest possible is so that you don't run into crazy situations where you don't have a borrow but it is borrowed anyway; this typically happens when you try to provide your own, incorrect, lifetime hints. There are plenty of cases where it is usually best to let the compiler decide; the other case is struct implementations such as the following:
struct Foo<'a> {
item: &'a u32
}
impl<'a> Foo<'a> {
pub fn compare<'b>(&self, other: &'b u32) {
...
}
}
A common fault in situations like this is to describe other to the compiler as 'a, not defining the second 'b lifetime, and thus (accidentally) requiring other to be borrowed for the lifetime of the struct itself.
Q2. Why does this code fail? According to my understanding, &'static is used for static global variables which live for the full program.
let x = 10;
This assignment does not have a 'static lifetime. It has an anonymous lifetime defined as less than 'static, because it is not strictly defined as global. The only way to get a 'static borrow on anything is if that source element is defined as const or static.
You can convince yourself of this with this snippet (playground):
fn f(p: &'static i32) {
println!("{}", p)
}
const FOO:i32 = 3;
static BAR:i32 = 4;
fn main() {
f(&FOO); // Works
f(&BAR); // Also works
}
f(&x);
A 'static lifetime requirement on a reference requires this argument to be declared for the global lifetime of the program, but x cannot fulfill this condition as it is declared midway through execution.
To be able to use this, declare x as const or static so its lifetime will be 'static and the code will work fine.
I'm trying to understand why the code below compiles. I wasn't expecting to be able to construct Wrapper<String> since T: 'static and runtime-allocated strings don't live for the entire lifetime of the program.
I think the reason this is allowed is because I'm setting T to a non-reference type (String). When I use a &str, or a struct containing a reference, the compiler issues the error I'd expect.
However, I haven't been able to find anything in the Rust docs that confirms my hypothesis, so maybe I don't fully understand the rules. Will all non-reference types satisfy the 'static lifetime bound on Wrapper<T>, or are there some that will fail?
use rand::Rng;
struct Wrapper<T>
where
T: 'static,
{
value: T,
}
fn wrap_string() -> Wrapper<String> {
// Use rng to avoid construcing string at compile time
let mut rng = rand::thread_rng();
let n: u8 = rng.gen();
let text = format!("The number is {}", n);
Wrapper { value: text }
}
fn main() {
let wrapped = wrap_string();
std::mem::drop(wrapped);
}
From the Background section of RFC 2093:
[...] in order for a reference type &'a T to be "well formed" (valid),
the compiler must know that the type T "outlives" the lifetime 'a --
meaning that all references contained in the type T must be valid for
the lifetime 'a. So, for example, the type i32 outlives any lifetime,
including 'static, since it has no references at all.
So I'd say the answer to your question is: yes, any type which has no references (or which only contains static references) satisfies the 'static bound.
Side note: according to that RFC, bounds like T: 'static and T: 'a are known as outlives requirements.
You can think of a type bound T: 'x as "Instances of T cannot suddenly become invalid because something that lives shorter than 'x was dropped.". However, this does not affect how long the instance of T itself lives.
So, a reference becomes invalid if the referenced thing is dropped. Which means that the referenced thing must live at least as long as 'x - for the entire run of the program in the case of 'static.
But something that own all its data - an i32 or a String for example - never becomes invalid because something else is dropped. An integer is good until it is dropped itself. So it satisfies the 'static bound.
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.