If I have the two functions
// implicit
fn foo(x: &i32) {
}
// explicit
fn bar<'a>(x: &'a i32) {
}
When would foo return an error and bar be the correct function header? I'm confused as to why I would explicitly declare a lifetime:
The 'a reads ‘the lifetime a’. Technically, every reference has some
lifetime associated with it, but the compiler lets you elide them in
common cases.
I understand what a lifetime is, but what does explicitly specifying a lifetime 'a do for me? For reference I'm using the Rust book as reading material
Practically speaking, the #1 reason you'll have to write lifetime annotations is because the compiler asks you so. It will reject function signatures which are not covered by lifetime elision rules.
I assume you would like an simple example where lifetimes are mandatory. Imagine the following scenario:
struct Blah<'a> {
hoy: &'a u8
}
fn want_a_hoy(blah: &Blah) -> &u8 {
blah.hoy
}
The intention is obvious, but the compiler doesn't handle it:
<anon>:7:35: 7:38 error: missing lifetime specifier [E0106]
<anon>:7 fn want_a_hoy(blah: &Blah) -> &u8 {
^~~
<anon>:7:35: 7:38 help: see the detailed explanation for E0106
<anon>:7:35: 7:38 help: this function's return type contains a borrowed value, but
the signature does not say which one of `blah`'s 2 elided
lifetimes it is borrowed from
In this case, annotations solve the problem:
fn want_a_hoy<'a, 'b>(blah: &'b Blah<'a>) -> &'a u8 {
blah.hoy
}
Here you're specifying 'a twice (on Blah<'a> and &'a). This is the same lifetime! So what you're saying to the compiler here is: "This function takes a reference to a blah containing an inner reference. I will return something which lives exactly as long as the inner reference of the blah." In this case, the signature gives a strong hint that you're likely to return something coming from the innards of the blah.
Related
I've worked down a real-life example in a web app, which I've solved using unnecessary heap allocation, to the following example:
// Try replacing with (_: &String)
fn make_debug<T>(_: T) -> impl std::fmt::Debug {
42u8
}
fn test() -> impl std::fmt::Debug {
let value = "value".to_string();
// try removing the ampersand to get this to compile
make_debug(&value)
}
pub fn main() {
println!("{:?}", test());
}
As is, compiling this code gives me:
error[E0597]: `value` does not live long enough
--> src/main.rs:9:16
|
5 | fn test() -> impl std::fmt::Debug {
| -------------------- opaque type requires that `value` is borrowed for `'static`
...
9 | make_debug(&value)
| ^^^^^^ borrowed value does not live long enough
10 | }
| - `value` dropped here while still borrowed
I can fix this error in at least two ways:
Instead of passing in a reference to value in test(), pass in value itself
Instead of the parameter T, explicitly state the type of the argument for make_debug as &String or &str
My understanding of what's happening is that, when there is a parameter, the borrow checker is assuming that any lifetime on that parameter affects the output impl Debug value.
Is there a way to keep the code parameterized, continue passing in a reference, and get the borrow checker to accept it?
I think this is due to the rules around how impl trait opaque types capture lifetimes.
If there are lifetimes inside an argument T, then an impl trait has to incorporate them. Additional lifetimes in the type signature follow the normal rules.
For more information please see:
https://github.com/rust-lang/rust/issues/43396#issuecomment-349716967
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#lifetime-parameters
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#assumption-3-there-should-be-an-explicit-marker-when-a-lifetime-could-be-embedded-in-a-return-type
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#scoping-for-type-and-lifetime-parameters
A more complete answer
Original goal: the send_form function takes an input parameter of type &T which is rendered to a binary representation. That binary representation is owned by the resulting impl Future, and no remnant of the original &T remains. Therefore, the lifetime of &T need not outlive the impl Trait. All good.
The problem arises when T itself, additionally, contains references with lifetimes. If we were not using impl Trait, our signature would look something like this:
fn send_form<T>(self, data: &T) -> SendFormFuture;
And by looking at SendFormFuture, we can readily observe that there is no remnant of T in there at all. Therefore, even if T has lifetimes of its own to deal with, we know that all references are used within the body of send_form, and never used again afterward by SendFormFuture.
However, with impl Future as the output, we get no such guarantees. There's no way to know if the concrete implementation of Future in fact holds onto the T.
In the case where T has no references, this still isn't a problem. Either the impl Future references the T, and fully takes ownership of it, or it doesn't reference it, and no lifetime issues arise.
However, if T does have references, you could end up in a situation where the concrete impl Future is holding onto a reference stored in the T. Even though the impl Future has ownership of the T itself, it doesn't have ownership of the values referenced by the T.
This is why the borrow check must be conservative, and insist that any references inside T must have a 'static lifetime.
The only workaround I can see is to bypass impl Future and be explicit in the return type. Then, you can demonstrate to the borrow checker quite easily that the output type does not reference the input T type at all, and any references in it are irrelevant.
The original code in the actix web client for send_form looks like:
https://docs.rs/awc/0.2.1/src/awc/request.rs.html#503-522
pub fn send_form<T: Serialize>(
self,
value: &T,
) -> impl Future<
Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
Error = SendRequestError,
> {
let body = match serde_urlencoded::to_string(value) {
Ok(body) => body,
Err(e) => return Either::A(err(Error::from(e).into())),
};
// set content-type
let slf = self.set_header_if_none(
header::CONTENT_TYPE,
"application/x-www-form-urlencoded",
);
Either::B(slf.send_body(Body::Bytes(Bytes::from(body))))
}
You may need to patch the library or write your own function that does the same thing but with a concrete type. If anyone else knows how to deal with this apparent limitation of impl trait I'd love to hear it.
Here's how far I've gotten on a rewrite of send_form in awc (the actix-web client library):
pub fn send_form_alt<T: Serialize>(
self,
value: &T,
// ) -> impl Future<
// Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
// Error = SendRequestError,
) -> Either<
FutureResult<String, actix_http::error::Error>,
impl Future<
Item = crate::response::ClientResponse<impl futures::stream::Stream>,
Error = SendRequestError,
>,
> {
Some caveats so far:
Either::B is necessarily an opaque impl trait of Future.
The first param of FutureResult might actually be Void or whatever the Void equivalent in Rust is called.
In chapter 19.2 of The Rust Programming Language, the following example compiles without any error. I found out from issue #1834 that there is a new lifetime elision rule that implicitly makes 's longer than 'c.
Although I couldn't find a detailed explanation of this new elision rule, I guess that it is not more than just an implicit version of the longer, more explicit constraint: <'c, 's: 'c>. I think however my confusion is probably not about this new elision rule but of course I could be wrong about this.
My understanding is, that parse_context takes ownership of context as it has not been borrowed but actually moved to the function. That alone implies to me that the lifetime of context should match the lifetime of the function it is owned by regardless of the lifetimes and constraint we defined in Context, and Parser.
Based on those definitions, the part where context outlives the temporary Parser makes perfect sense to me (after all, we defined a longer lifetime), but the part where the &str reference is not dropped when context goes out of scope at the end of parse_context and I can still safely return it -- makes me puzzled.
What have I missed? How can the compiler reason about the lifetime of the returned &str?
UPDATED EXAMPLE
struct Context<'s>(&'s str);
struct Parser<'c, 's>
{
context: &'c Context<'s>,
}
impl<'c, 's> Parser<'c, 's>
{
fn parse(&self) -> Result<(), &'s str>
{
Err(self.context.0)
}
}
fn parse_context(context: Context) -> Result<(), &str>
{
Parser { context: &context }.parse()
}
fn main()
{
let mut s = String::new();
s += "Avail";
s += "able?";
if let Err(text) = parse_context(Context(&s))
{
println!("{}", text);
}
}
You are not returning a reference to the owned value of the function. You are returning a copy of the reference passed in.
Your function is an elaborate version of the identity function:
fn parse_context(s: &str) -> &str {
s
}
In your real code, you take a reference to a struct containing a string slice, then another reference to the string slice, but all of those references are thrown away.
For example, there's an unneeded reference in parse:
fn parse(&self) -> Result<(), &'s str> {
Err( self.context.0)
// ^ no & needed
}
Additionally, if you enable some more lints, you'll be forced to add more lifetimes to your function signature, which might make things more clear:
#![deny(rust_2018_idioms)]
fn parse_context(context: Context<'_>) -> Result<(), &'_ str> {
Parser { context: &context }.parse()
}
See also:
What are Rust's exact auto-dereferencing rules?
Although I couldn't find a detailed explanation of this new elision rule,
T: 'a inference in structs in the edition guide.
struct Context<'s>(&'s str);
→ Values of type Context hold a string with some lifetime 's. This lifetime is implicitly at least as long as the lifetime of the context, but it may be longer.
struct Parser<'c, 's>
{
context: &'c Context<'s>,
}
→ Values of type Parser hold a a reference to context with some lifetime 'c. This context holds a string with some other lifetime 's.
impl<'c, 's> Parser<'c, 's>
{
fn parse(&self) -> Result<(), &'s str>
{
Err(self.context.0)
}
}
→ Function parse returns a value with lifetime 's, ie. with the same lifetime as the string that is stored inside the context, which is not the same as the lifetime of the context itself.
fn parse_context(context: Context) -> Result<(), &str>
{
Parser { context: &context }.parse()
}
→ I'm not sure exactly where this is specified, but obviously the compiler infers that the lifetime for the returned string is the same as the 's parameter used for the context. Note that even though the context itself is moved into parse_context, this only affects the context itself, not the string that it contains.
fn main()
{
let mut s = String::new();
→ Create a new string valid until the end of main
s += "Avail";
s += "able?";
if let Err(text) = parse_context(Context(&s))
→ Create a new context and move it into parse_context. It will automatically be dropped at the end of parse_context. It holds a reference to string s which is valid until the end of main and parse_context returns a string with the same lifetime as s ⇒ text is valid until the end of main.
{
println!("{}", text);
}
}
→ No problem: text is valid until the end of main.
Thank to the comments of Jmb and some bits of Shepmaster's answer it is indeed clear to me now that my confusion was about the RAII rules and the ownership.
That is, the string was created in the main scope, the anonymous Context instance is not taking ownership of the string it is only borrowing a reference, therefore even when the instance is dropped at the end of parse_context along with the borrowed reference, a reference copied to the Err object still exists and points to the existing string -- hence we are using the constrained lifetime variables and the compiler is able to reason about the lifetime of the internal string reference.
I tried to compile this code (Playground):
trait Family<'a> {
type Out;
}
struct U32Family;
impl<'a> Family<'a> for U32Family {
type Out = u32;
}
trait Iterator {
type Item;
fn next<'s>(&'s mut self) -> <Self::Item as Family<'s>>::Out
where
Self::Item: Family<'s>;
}
struct Foo;
impl Iterator for Foo {
type Item = U32Family;
fn next<'s>(&'s mut self) -> <Self::Item as Family<'s>>::Out
where
Self::Item: Family<'s>,
{
0u32 // <-- in real code, this is somehow calculated
}
}
But sadly, it results in this error:
error[E0308]: mismatched types
--> src/main.rs:28:9
|
24 | fn next<'s>(&'s mut self) -> <Self::Item as Family<'s>>::Out
| ------------------------------- expected `<U32Family as Family<'s>>::Out` because of return type
...
28 | 0u32
| ^^^^ expected associated type, found u32
|
= note: expected type `<U32Family as Family<'s>>::Out`
found type `u32`
I really don't understand why. Obviously, in this code snippet, <U32Family as Family<'s>>::Out is exactly u32. But Rust seems to think that it's not always the same. Why? And how can I make it compile?
Some notes:
There are a bunch of similar situations where a similar error occurs, but I think this is different from everything I've seen so far.
I cannot use type Out: for<'a> Family<'a>;. So that's not a workaround that works for me.
If I remove the lifetime parameter of Family, everything works.
If I replace Family<'s> with Family<'static> in the function signature, everything works.
EDIT: I can work around this problem by adding:
impl U32Family {
fn from<'a>(v: u32) -> <Self as Family<'a>>::Out {
v
}
}
Then I can just say Self::Item::from(0u32) in the body of next(). (Playground)
I think it's clear why the error in next() is gone: U32Family::from always takes u32 as argument. Hardcoded. Never changing. The bigger question about this workaround is: why does the from() method compile fine? So in from() the compiler somehow knows that <Self as Family<'a>>::Out is always u32, but if I try the same in next(), somehow the compiler doesn't understand that <Self::Item as Family<'s>>::Out is u32. Now I'm even more confused.
EDIT2: first, I suspected that specialization is the problem. For example, you might write:
impl Family<'static> for U32Family {
type Out = char;
}
Then of course, the compiler would be right in assuming that u32 is not always the same as <Self::Item as Family<'s>>::Out for any 's. However, I think this is not the problem.
First of all, impls that can be specialized need to be marked with the default keyword. I did not do that, so I should be able to assume the associated type is in fact u32 (the RFC talks about something very similar). But additionally, specialization based on lifetimes is not allowed.
So by now I tend to think this is a compiler error. But I'd love to get another answer!
I think the problem is that it is a "coincidence" that <Self::Item as Family<'s>>::Out is u32 for all 's. The compiler can prove it for any 's you want, but it can't even express the concept that it is true for all 's.
The work-around you have found is the right approach: add a method to U32Family which converts a u32 into a <Self as Family<'a>>::Out. The body of the method is entirely inside the scope of 'a, so the compiler can prove that the conversion is type-correct for that 'a, and therefore that the method is type-correct. Then, at the call-site, you're telling the compiler to use its knowledge about the method.
struct U32Family;
...
impl Iterator for Foo {
type Item = U32Family;
So next() must return Option<U32Family>, whose only possible values are None and Some(U32Family{})
You probably want Item = <U32Family as Family<'static>::Out which fixes this issue but creates some lifetime issues. (The Item needs a lifetime because Family has one, but you only accept a lifetime on next())
I am trying to write a function returning a boxed closure that can work on references to types with any
lifetime. When writing a specific instance, everything works fine. But when writing a generic
version, I run into lifetime problems.
struct Parameter<'a> {
s: &'a str,
}
fn main() {
let closure = generate_closure_gen();
let string = String::from("Hello World!");
let parameter = Parameter { s: &string }; // Error: string does not live long enough
closure(¶meter);
}
// This one works fine
// Desugared version for Box<Fn(&Parameter)>
fn generate_closure() -> Box<for <'a, 'r> Fn(&'r Parameter<'a>)> {
Box::new(|c: &Parameter| {})
}
// This one gives lifetime errors
fn generate_closure_gen<C>() -> Box<Fn(&C)> {
Box::new(|c: &C| {})
}
I don't see why the closure needs the type parameter to live longer than it (there is no storage or anything ...). And it works for the non-generic version with HRTB, it just feels like it should be possible to make it work with the generic version.
Also, if I try to write the specific version using the generic version, I get a type error
// Desugared version for Box<Fn(&Parameter)>
fn generate_closure_2() -> Box<for <'a, 'r> Fn(&'r Parameter<'a>)> {
generate_closure_gen()
}
src/main.rs:22:5: 22:27 error: mismatched types:
expected `Box<for<'r, 'r> core::ops::Fn(&'r Parameter<'r>) + 'static>`,
found `Box<for<'r> core::ops::Fn(&'r _) + 'static>`
(expected concrete lifetime,
found bound lifetime parameter ) [E0308]
src/main.rs:22 generate_closure_gen()
^~~~~~~~~~~~~~~~~~~~~~
src/main.rs:22:5: 22:27 help: run `rustc --explain E0308` to see a detailed explanation
Any idea on how to make this work?
(playpen link)
Type parameters have a lifetime bound. That lifetime bound is the shortest of all of the implementor's lifetime parameters. You omitted it on generate_closure_gen, so the compiler inferred it, but if we explicitly wrote it out, the function definition would look like this:
fn generate_closure_gen<'a, C: 'a>() -> Box<Fn(&C)> {
Box::new(|c: &C| {})
}
Making this change doesn't solve our problem, though.
To understand why, we need to figure out what C is inferred to be. You call the closure with a &'y Parameter<'x>, and the closure accepts for<'b> &'b C, so C is Parameter<'x>. Parameter<'x> has a lifetime parameter, which will have an influence on the lifetime bound on C.
Lifetime parameters in generic functions must be substituted with lifetimes that start before the function call. In this case, this means that the lifetime of any C we pass to the closure must be valid before the call to generate_closure_gen. That's because C is bound to a specific lifetime, not to any lifetime; i.e. when C is Parameter<'x>, the 'x must be known in advance; we can't have a different 'x each time we call the closure. In other words, what you'd like to have is something like this:
fn generate_closure_gen<C: for<'a> 'a>() -> Box<Fn(&C)> {
Box::new(|c| {})
}
But unfortunately, that isn't legal as of Rust 1.7.
This is a follow-up to my question on how to create & use a list of callbacks.
I'm trying to create (and store, near an event loop) a list of callback functions that will be called at some indeterminate point in the future.
struct ComplexThing {
calls: Vec<Box<FnMut()>>,
}
impl ComplexThing {
fn call<'a, T: FnMut() + 'a>(&'a mut self, func: T) {
self.calls.push(Box::new(func));
}
}
Errors with:
calls.rs:30:25: 30:39 error: the parameter type `T` may not live long enough [E0310]
calls.rs:30 self.calls.push(Box::new(func));
^~~~~~~~~~~~~~
calls.rs:30:39: 30:39 help: consider adding an explicit lifetime bound `T: 'static`...
calls.rs:30:25: 30:39 note: ...so that the type `T` will meet its required lifetime bounds
calls.rs:30 self.calls.push(Box::new(func));
^~~~~~~~~~~~~~
I tried adding it to the struct, which fixed the error about lifetimes on the call to push,
struct ComplexThing<'a> {
calls: Vec<Box<FnMut() + 'a>>,
}
impl ComplexThing {
fn call<'a, T: FnMut() + 'a>(&'a mut self, func: T) {
self.calls.push(Box::new(func));
}
}
… but gets me:
calls.rs:28:6: 28:18 error: wrong number of lifetime parameters: expected 1, found 0 [E0107]
calls.rs:28 impl ComplexThing {
^~~~~~~~~~~~
Which, yes, I suppose the struct has a <'a>, and I'm not specifying it. If I add it,
impl ComplexThing<'a> {
I get,
calls.rs:28:19: 28:21 error: use of undeclared lifetime name `'a` [E0261]
calls.rs:28 impl ComplexThing<'a> {
I don't know if I should be specifying it on the struct ComplexThing or not. If I leave it off (and I would greatly prefer to, I think),
I think there's something crucial about how Rust notates lifetimes that I'm not getting here. The FnMut that ComplexThing is (presently) trying to store in a Box is (in the design in my head) owned by the instance of ComplexThing; it's lifetime should be less than that of .ComplexThing — i.e., one of two things would happen:
What will be a private function of the ComplexThing will end up removing the Box<FnMut> from the Vec (and thus, take ownership of it), run the FnMut, and then exit, thus freeing the FnMut.
The ComplexThing is deallocated, in which case the Vec and any Box<FnMut>'s are deallocated with it.
The question "How do I store a closure in Rust?"'s answer made me think a Box<FnMut> wouldn't need lifetime annotations, but the answer I got on how to create & use a list of callbacks makes me think I do.
My best guess is that Box is just storing a pointer to an object that I don't really own, and that I need to either create a copy of the FnMut on the heap, or move-construct one there, and then that copy is the one I can own. (Otherwise, if it's like a closure that's on the stack, I need to make sure that closure doesn't go out of scope before my Box does, which is why Rust is having me annotate lifetimes.)
'a on ComplexThing is a generic lifetime parameter and needs to be defined just as much as a generic type parameter. If you had a struct Foo<T>, you would not be able to write impl Foo<T> because there is no concrete type T in scope; if you want it to be generic, you need to define it, impl<T> Foo<T>. This allows you to write constraints too, such as impl<T: Clone> Foo<T> to implement methods on Foo<T> only where T is a type that implements Clone.
So then, the answer is simply that you need to define the lifetime 'a as generic:
impl<'a> ComplexThing<'a> { … }