I am trying to better understand lifetime elisions for &mut self and corresponding output parameters.
Let's say I have a function signature of the form
fn foo(&mut self) -> &mut bar
From the lifetime section in the Rustonomicon I understand that this gets expanded to
fn foo<'a>(&'a mut self) -> &'a mut bar
which has the side effect that self gets mutably borrowed as long as the corresponding object exists.
What I would like to know is what happens if I specify a function with signature
fn foo<'a>(&mut self) -> &'a mut bar
In my understanding, &mut self is elided and according to elision rules gets its own input lifetime. So we would end up with something like
fn foo<'b, 'a>(&'b mut self) -> &'a mut bar
According to elision rules, 'b gets assigned to all elided output lifetimes. But 'a is not elided. So what is the lifetime 'a? Code with such a signature compiles fine. But I don't quite understand the lifetime implications for 'a.
You've correctly un-elided the lifetimes.
So what is the lifetime 'a?
Whenever a function has a lifetime parameter, it is always: "whatever the caller wants it to be" (subject to bounds, which there aren't in this case).
Code with such a signature compiles fine.
Did you try writing a body for that function? Given those lifetimes, it can't actually do very much. For example,
impl Foo {
fn foo<'b, 'a>(&'b mut self) -> &'a mut Bar {
&mut self.bar
}
}
will fail to compile with errors telling you (indirectly) that this is only valid if 'b outlives 'a, which it doesn't.
If you add that "outlives" relation to the signature, then the code can compile:
impl Foo {
fn foo<'b, 'a>(&'b mut self) -> &'a mut Bar
where
'b: 'a
{
&mut self.bar
}
}
However, this is almost never any more useful than the simpler
fn foo<'a>(&'a mut self) -> &'a mut Bar {
because references can generally be used as if they have shorter lifetimes than their type specifies, so 'a here can serve the purpose of 'a and the purpose of 'b in the more complex declaration.
Like any other generic parameter, 'a can be freely chosen by the caller as long as it satisfies any generic bounds on the trait (which there aren't any of, in your example). Just as you have something like
// caller chooses the value of B in `fn collect<B>(self) -> B` to be Vec<_>
let items: Vec<_> = my_data.into_iter().collect();
you can have
// caller chooses the value of 'a in `fn foo<'a>(&mut self) -> &'a mut bar` to be 's
's: {
let b = obj.foo()
}
Of course, Rust won't even compile foo if it can't prove that 'a can indeed be freely chosen by the caller, which, in the absence of other lifetime bounds, means we must have 'a: 'static, and the body of foo must be compatible with that.
Related
I want to create a wrapper around (nested) slices for easy operations on multidimensional data, owned by a different struct.
The most basic version of the mutable version of my slice wrapper might look like this:
struct MySliceMut<'a> {
data: Vec<&'a mut [f32]>,
}
impl<'a, 'b> MySliceMut<'a> {
fn get(&'b mut self) -> &'a mut [&'b mut [f32]] {
self.data.as_mut_slice()
}
}
Now if I want to implement a trait, for instance AddAssign, Rust does not seem to infer the lifetime of &mut self from the implementing type. The compiler complains that &mut self might outlive 'a:
impl<'a> AddAssign<MySlice<'a>> for MySliceMut<'a> { // lifetime 'a
fn add_assign(&mut self, rhs: MySlice<'a>) { // lifetime '1
let a = self.get(); // lifetime may not live long enough, '1 must outlive 'a
let b = rhs.get();
// do inplace addition here
}
}
Full Code - Rust Playground
I tried to figure out the issue with the lifetimes, but can't find it. Would the trait impl require any additional annotations?
struct MySlice<'a> {
data: Vec<&'a [f32]>,
}
impl<'a, 'b> MySlice<'a> {
fn get(&'b self) -> &'a [&'b [f32]] {
self.data.as_slice()
}
}
Problem with your code is that fn get(&'b self) returns variable with wrong lifetime. Associated lifetime 'a of MySlice<'a> is lifetime of inner slice. Associated lifetime 'b of fn get(...) is lifetime of the self. So I guess the function probably should return &'b [&'a [f32]] instead.
-- Edited --
Make sure to change fn get(...) of MySliceMut either.
I'm designing a connection retry function. It retries if the error was ConnectionClosed. In the full implementation it has other things that justify it being a recursion, but this is off topic. Anyways, since it's a connection, it makes sense to be async. However, async recursives force me to use BoxFuture, and we know that asyncs capture and return all the arguments.
use std::future::Future;
use futures::future::{BoxFuture, FutureExt};
struct Message;
struct Client;
enum Error {
ConnectionClosed
}
impl Client {
fn send_and_expect<'a>(
&'a mut self,
message: &'a Message
) -> BoxFuture<'a, Result<(), Error>> {
async move {
Ok(())
}.boxed()
}
//With this function I can wrap any async function f that grabs data from the internet
pub fn connection_retrier<'a, T>(
f: fn(&'a mut Self, &'a Message) -> T,
f_self: &'a mut Self,
f_m: &'a Message,
) -> BoxFuture<'a, Result<(),Error>>
where
T: Future<Output = Result<(), Error>> + 'a + Send
{
/*
By making f: fn(&'a mut Self, &'a Message) -> T, we tell the compiler that
`f` is a specific function: one that specifically requires the `Self` and `Message`
arguments to live as long as `'a`, where `'a` is the min of the lifetimes of the `f_self` and `f_m`
arguments passed to `connection_retrier`.
Thus this forces `f_self` and `f_m` to live as long as `connection_retrier`.
The call `let r = f(f_self, f_m).await` returns a `Future` that lives as long as `'a`.
I think this is the problem. The result of the first call borrows `f_self` and `f_m`
for at least the entire lifetime of `r`. This would make it impossible to use
`f_self` and `f_m` again inside `connection_retrier`. As you see, I tried making sure
that `r` destructs before we use `f_self` in `connection_retrier` again, but somehow
`r` is marked to live as long as `connection_retrier`, because `f_self` is still considered
to be borrowed.
*/
async move {
let ok: bool;
{
let r = f(f_self, f_m).await;
match r {
Ok(_) => ok = true,
Err(Error::ConnectionClosed) => {
ok = false;
}
}
}
match ok {
true => Ok(()),
false => Client::connection_retrier(f, f_self, f_m).await
}
}.boxed()
}
async fn send_with_retry<'a> (
&'a mut self,
message: &'a Message,
) -> Result<(), Error>
{
Client::connection_retrier(
Client::send_and_expect,
self,
message
).await
}
}
Error:
error[E0499]: cannot borrow `*f_self` as mutable more than once at a time
--> src/lib.rs:58:56
|
24 | f: fn(&'a mut Self, &'a Message) -> T,
| - lifetime `'1` appears in the type of `f`
...
48 | let r = f(f_self, f_m).await;
| --------------
| | |
| | first mutable borrow occurs here
| argument requires that `*f_self` is borrowed for `'1`
...
58 | false => Client::connection_retrier(f, f_self, f_m).await
| ^^^^^^ second mutable borrow occurs here
Playground
I understand why the error occurs: Client::connection_retrier(f, f_self, f_m).await, let's call it r, holds a mutable reference to f_self, so I cannot use it again while it's being held. However, after I check that this result r is Error::ConnectionClosed, I don't need it anymore, so there should be a way to discard it so I can reborrow it mutably.
The problem
As #moy2010 said, the problem is here:
pub fn connection_retrier<'a, T>(
f: fn(&'a mut Self, &'a Message) -> T,
f_self: &'a mut Self,
f_m: &'a Message,
) -> BoxFuture<'a, Result<(), Error>>
where T: Future<Output = Result<(), Error>> + 'a + Send
This signature defines f to be a function that accepts arguments with exactly the lifetime 'a, which is alive throughout the body of connection_retrier.
When you are able to temporarily pass a mutable reference to a function and then 'recover' it later, this is called a reborrow. Reborrowing from a mutable reference works by creating a new mutable reference of shorter lifetime to the data it points to, and preventing use of the original reference until this lifetime has ended. Usually a reborrow happens during a function call, in which case that lifetime is just long enough for the function call to be made.
As you can see, reborrowing always produces a reference with lifetime shorter than the original. But f needs a reference whose lifetime is exactly 'a, so the compiler correctly deduces that the call to f cannot reborrow f_self and instead, f_self is permanently moved into f. Hence the error when you attempt to use it again.
Solutions
If you can change the signature of send_and_expect so that the returned future only borrows from message (i.e. not the Client), then #moy2010 has already provided a correct solution in the comments to their answer. I will proceed on the assumption that send_and_expect must borrow from both its arguments, as indicated by the signature in the question.
Since you would like to reborrow f_self before passing it to f, what you really need is to define f to be a function that can be called for any lifetime (not just 'a). You might think that means we can just change the declaration of f to for<'b> fn(&'b mut Self, &'b Message) -> T. But wait! Your where bound specifies that T borrows from the lifetime 'a, not 'b, so this signature doesn't line up with that of send_and_expect. But also notice that we can't change the bound to say 'b instead, because 'b is not in scope outside of the declaration of f! So there is no way to express the correct bound on T here. There are, however, a few workarounds.
Use a Boxed trait object
The simplest solution is just to make the return type of f a boxed trait object instead of a generic parameter, like this:
pub fn connection_retrier<'a>(
f: for<'b> fn(&'b mut Self, &'b Message) -> BoxFuture<'b, Result<(), Error>>,
f_self: &'a mut Self,
f_m: &'a Message,
) -> BoxFuture<'a, Result<(), Error>>
Now we can specify what the return value borrows from inline, avoiding the problem that 'b isn't in scope for a where bound. Since send_and_expect already returns a BoxFuture anyway in your example, this may be the best solution for you.
Use a trait implemented on a ZST instead of a fn pointer
Resorting to boxing and dynamic dispatch to satisfy a compile-time issue might strike some as distasteful, and there is a small performance penalty. This is a trick that can be used to make this work with static dispatch, at the cost of some verbosity. We will have to define our own trait, ConnectFn:
trait ConnectFn<'a> {
type ConnectFuture: Future<Output=Result<(), Error>> + Send + 'a;
fn connect(&self, client: &'a mut Client, message: &'a Message) -> Self::ConnectFuture;
}
Then, we can make the signature of connection_retrier the following:
pub fn connection_retrier<'a, F>(
f: F,
f_self: &'a mut Self,
f_m: &'a Message,
) -> BoxFuture<'a, Result<(), Error>>
where F: for<'b> ConnectFn<'b>
And replace f(f_self, f_m) in the body with f.connect(f_self, f_m). This avoids the need for the return type of f to appear anywhere in connection_retrier's signature, circumventing the issue we had before.
We can then replace our function send_and_expect with a zero-sized type SendAndExpect which implements ConnectFn, like so:
struct SendAndExpect;
impl<'a> ConnectFn<'a> for SendAndExpect
{
type ConnectFuture = BoxFuture<'a, Result<(), Error>>;
fn connect(&self, client: &'a mut Client, message: &'a Message) -> Self::ConnectFuture {
/* the body of Client::send_and_expect goes here. `self` is ignored */
}
}
With this, we can call connection_retrier like so:
Client::connection_retrier(SendAndExpect, &mut client, &message)
Which is pretty nice. The definition of SendAndExpect itself can be made even nicer, close to that of a regular fn, using macros, but this is left as an exercise to the reader.
Sidenote: You might think that it would be possible to have a blanket implementation of ConnectFn on suitable fn pointers and avoid the need for a new type, but sadly you would be wrong. You'll either run into the same issue with impossible where clause bounds or trait solver limitations when dealing with universally quantified types.
The reason lies within these three lines:
pub fn connection_retrier<'a, T>(
f: fn(&'a mut Self, &'a Message) -> T,
f_self: &'a mut Self,
For f, you are telling the compiler that it will receive a mutable reference with a lifetime equal to that of the mutable reference that connection_retrier receives for f_self (in this case it is 'a).
When you change f's signature to f: fn(&mut Self, &Message) -> T, you are letting the compiler determine the lifetime, which it correctly does by assigning it a lifetime shorter than 'a and hence why it compiles.
How are lifetimes handled when there is a trait object passed to a function?
struct Planet<T> {
i: T,
}
trait Spinner<T> {
fn spin(&self, value: T);
}
impl<T> Spinner<T> for Planet<T> {
fn spin(&self, value: T) {}
}
// foo2 fails: Due to lifetime of local variable being less than 'a
fn foo2<'a>(t: &'a Spinner<&'a i32>) {
let x: i32 = 10;
t.spin(&x);
}
// foo1 passes: But here also the lifetime of local variable is less than 'a?
fn foo1<'a>(t: &'a Planet<&'a i32>) {
let x: i32 = 10;
t.spin(&x);
}
(Playground)
This code results in this error:
error[E0597]: `x` does not live long enough
--> src/main.rs:16:17
|
16 | t.spin(&x);
| ^ borrowed value does not live long enough
17 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the lifetime 'a as defined on the function body at 14:5...
--> src/main.rs:14:5
|
14 | fn foo2<'a>(t: &'a Spinner<&'a i32>) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The function signature of foo1 is nearly same as foo2. One receiving the reference to struct and the other a trait object.
I read this is where Higher Ranked Trait Bounds comes in. Modifying foo2 as foo2(t: &for<'a> Spinner<&'a i32>) compiles the code, but I don't understand why.
Why won't 'a shrink for x?
Citing the Nomicon:
How on earth are we supposed to express the lifetimes on F's trait bound? We need to provide some lifetime there, but the lifetime we care about can't be named until we enter the body of call! Also, that isn't some fixed lifetime; call works with any lifetime &self happens to have at that point.
Can this please be elaborated?
In short: foo1 compiles because most types are variant over their generic parameters and the compiler can still chose a Spinner impl for t. foo2 doesn't compile because traits are invariant over their generic parameters and the Spinner impl is already fixed.
Some explanation
Let's take a look at a third version of foo:
fn foo3<'a>(t: &'a Planet<&'a i32>) {
let x: i32 = 10;
Spinner::<&'a i32>::spin(t, &x);
}
This results in the same error as your foo2. What's going in there?
By writing Spinner::<&'a i32>::spin, we force the compiler to use a specific implementation of the Spinner trait. And the signature of Spinner::<&'a i32>::spin is fn spin(&self, value: &'a i32). Period. The lifetime 'a is given by the caller; foo can't choose it. Thus we have to pass a reference that lives for at least 'a. That's why the compiler error happens.
So why does foo1 compile? As a reminder:
fn foo1<'a>(t: &'a Planet<&'a i32>) {
let x: i32 = 10;
t.spin(&x);
}
Here, the lifetime 'a is also given by the caller and cannot be chosen by foo1. But, foo1 can chose which impl of Spinner to use! Note that impl<T> Spinner<T> for Planet<T> basically defines infinitely many specific implementations (one for each T). So the compiler also knows that Planet<&'x i32> does implement Spinner<&'x i32> (where 'x is the specific lifetime of x in the function)!
Now the compiler just has to figure out if it can turn Planet<&'a i32> into Planet<&'x i32>. And yes, it can, because most types are variant over their generic parameters and thus Planet<&'a i32> is a subtype of Planet<&'x i32> if 'a is a subtype of 'x (which it is). So the compiler just "converts" t to Planet<&'x i32> and then the Spinner<&'x i32> impl can be used.
Fantastic! But now to the main part: why doesn't foo2 compile then? Again, as a reminder:
fn foo2<'a>(t: &'a Spinner<&'a i32>) {
let x: i32 = 10;
t.spin(&x);
}
Again, 'a is given by the caller and foo2 cannot chose it. Unfortunately, now we already have a specific implementation! Namely Spinner<&'a i32>. We can't just assume that the thing we were passed also implements Spinner<&'o i32> for any other lifetime 'o != 'a! Traits are invariant over their generic parameters.
In other words: we know we have something that can handle references which live at least as long as 'a. But we can't assume that the thing we were handed can also handle lifetimes shorter than 'a!
As an example:
struct Star;
impl Spinner<&'static i32> for Star {
fn spin(&self, value: &'static i32) {}
}
static SUN: Star = Star;
foo2(&SUN);
In this example, 'a of foo2 is 'static. And in fact, Star implements Spinner only for 'static references to i32.
By the way: this is not specific to trait objects! Let's look at this fourth version of foo:
fn foo4<'a, S: Spinner<&'a i32>>(t: &'a S) {
let x: i32 = 10;
t.spin(&x);
}
Same error once again. The problem is, again, that the Spinner impl is already fixed! As with the trait object, we only know that S implements Spinner<&'a i32>, not necessarily more.
HRTB to the rescue?
Using higher ranked trait bounds resolves the issue:
fn foo2(t: &for<'a> Spinner<&'a i32>)
and
fn foo4<S: for<'a> Spinner<&'a i32>>(t: &S)
As it's hopefully clear from the explanation above, this works because we the specific impl of Spinner isn't fixed anymore! Instead, we again have infinitely many impls to choose from (one for each 'a). Thus we can choose the impl where 'a == 'x.
I want a callback on changes inside a list, so I created simple example:
struct Foo;
struct FooList {
list: Vec<Foo>,
on_change_cb: Vec<Box<FnMut(& mut [Foo])>>,
}
impl FooList {
/*
pub fn register_on_change_cb2<F>(&mut self, cb: F) where F: FnMut(&mut [Foo]) {
self.on_change_cb.push(Box::new(cb));
}*/
pub fn register_on_change_cb(&mut self, cb: Box<FnMut(&mut [Foo])>) {
self.on_change_cb.push(cb);
}
pub fn push(&mut self, foo: Foo) {
self.list.push(foo);
self.on_change();
}
fn on_change(&mut self) {
for cb in &mut self.on_change_cb {
cb(&mut self.list);
}
}
}
I don't give any explicit hint to the compiler about lifetimes here: Vec<Box<FnMut(& mut [Foo])>>, so what lifetimes will the compiler use here? If I change the code like this:
struct FooList<'a> {
list: Vec<Foo>,
on_change_cb: Vec<Box<FnMut(&'a mut [Foo])>>,
}
impl<'a> FooList<'a> {
I get a compile time error:
error[E0495]: cannot infer an appropriate lifetime for borrow
expression due to conflicting requirements
How can I explicitly set the lifetimes in some way such that the lifetime of & mut [Foo] for the callback is less than, but not equal to the lifetime of the whole FooList object?
I have commented register_on_change_cb2, I want to allow calling register_on_change_cb without usage of Box::new but failed. If you uncomment register_on_change_cb2, you get the error:
error[E0310]: the parameter type F may not live long enough
How can I fix this error without the requirement of a 'static lifetime for callback? I just want to call Box::new on my side.
I'm going to try to answer your questions 1 and 3, because question 2 is either redundant or orthogonal to the others, and I can't tell what you really want to achieve by it. Perhaps it deserves a question of its own.
If you have a function that takes a reference, but it doesn't need any lifetime information about the reference, it must be able to accept a reference of any lifetime. Here's the explicit syntax for that (this is what the compiler infers from the code you wrote):
on_change_cb: Vec<Box<for<'b> FnMut(&'b mut [Foo])>>,
This is called a higher ranked trait bound or HRTB for short. They're mostly useful for the Fn traits, which is why they exist.
If the type of on_change_cb is Vec<Box<FnMut(&mut [Foo])>>, which doesn't carry any lifetime information, then it must not contain any references (except 'static references). You need to say that the type implementing FnMut may also contain (non-'static) references, as long as they outlive some lifetime 'a:
struct FooList<'a> {
list: Vec<Foo>,
on_change_cb: Vec<Box<FnMut(&mut [Foo]) + 'a>>,
}
This reads something like: "For each FooList object, there is a lifetime 'a such that every callback in the FooList contains only references that live for at least 'a." This interpretation may make it easier to write the prototype for register_on_change_cb2: it takes a callback that also contains only references that live for at least 'a.
impl<'a> FooList<'a> {
pub fn register_on_change_cb2<F>(&mut self, cb: F)
where F: FnMut(&mut [Foo]) + 'a
{
self.on_change_cb.push(Box::new(cb));
}
(I think I have the variance of 'a correct now -- a previous version of this answer had it wrong.)
The 'a lifetime lets the compiler guarantee that you never put a callback in the Box (and therefore the Vec) unless it lasts at least as long as the FooList itself. This is important because closures can capture references to values in the enclosing scope, as in the following code (playground link):
let longlived = String::from("hello");
let mut list = FooList {
list: Vec::new(),
on_change_cb: Vec::new(),
};
list.register_on_change_cb2(|_| println!("{}", longlived)); // ok
let shortlived = String::from("hello");
list.register_on_change_cb2(|_| println!("{}", shortlived)); // `shortlived` does not live long enough
list.push(Foo);
In this example, you can't insert the closure that captures shortlived because it doesn't outlive the (inferred) lifetime 'a. But you can insert the closure that captures longlived, because the compiler can infer a lifetime 'a that satisfies both constraints:
'a must outlive list, because list is of type FooList<'a>.
longlived must outlive 'a, because |_| println!("{}", longlived), which borrows longlived, is bounded by 'a in the call to register_on_change_cb2.
If you want to say that the callbacks don't borrow anything by-reference, the 'a lifetime is unnecessary, and in that case you could just add the 'static bound that the compiler suggests:
pub fn register_on_change_cb2<F>(&mut self, cb: F)
where F: FnMut(&mut [Foo]) + 'static
I have a trait which says that any implementation of Foo needs to provide a method bar which returns an object of some type which implements Iterator<Item = u32>:
trait Foo {
type FooIterator: Iterator<Item = u32>;
fn bar(&self) -> FooIterator;
}
For this case, I believe that the default lifetime elision means that the iterator returned by bar is required to live on its own, without being tied to the lifetime of the Foo it is iterating over. User Habnabit on #rust irc suggested the following way to say that the lifetime of the FooIterator is less than the lifetime of the Foo. i.e. it allows the implementation of the FooIterator to keep a reference to the Foo that it comes from:
trait Foo<'a> {
type FooIterator: Iterator<Item = u32> + 'a;
fn bar<'b: 'a>(&'b self) -> Self::FooIterator;
}
What I really want is the case where the function bar takes an additional argument, and the implementation of FooIterator is allowed to keep a reference to both the Foo and the additional argument. i.e. the lifetime of FooIterator is bounded by the lifetime of the Foo and the lifetime of the additional argument.
My literal translation of this idea would be
trait Zip {}
trait Foo<'a, 'c> {
type FooIterator: Iterator<Item = u32> + 'a + 'c;
// Foo.bar() returns an iterator that has a lifetime less than the Foo
fn bar<'b: 'a, 'd: 'c>(&'b self, &'d Zip) -> Self::FooIterator;
}
But I was told there there is no "good" way to do this. What would be the best way to implement this idiom? What would the above code do exactly?
What you're looking for is associated type constructors, a planned feature that is not yet implemented in Rust. With associated type constructors, your code would look like this:
trait Zip {}
trait Foo {
type FooIterator<'a, 'c>: Iterator<Item = u32> + 'a + 'c;
// Foo.bar() returns an iterator that has a lifetime less than the Foo
fn bar<'a, 'b: 'a, 'c, 'd: 'c>(&'b self, &'d Zip) -> Self::FooIterator<'a, 'c>;
}
Actually, I'm not sure all those lifetimes are necessary, because a &'a T can be coerced to a &'b T where 'a: 'b. Thus, the following might be good enough:
trait Zip {}
trait Foo {
type FooIterator<'a, 'c>: Iterator<Item = u32> + 'a + 'c;
// Foo.bar() returns an iterator that has a lifetime less than the Foo
fn bar<'a, 'c>(&'a self, &'c Zip) -> Self::FooIterator<'a, 'c>;
}
Depending on how you want to use this trait, you may be able to make it work by implementing it for &'a Struct instead of for Struct, thus "hoisting" the responsibility for finding the right lifetime from the trait into the caller.
Remove the lifetime annotation from the trait and change bar so it takes self, plus another argument of the same lifetime:
trait Foo {
type FooIterator: Iterator<Item = u32>;
fn bar(self, other: Self) -> Self::FooIterator;
}
(Removing 'a from the trait is possible because bar consumes the reference instead of reborrowing it -- self doesn't have to outlive the return value anymore because it's been moved into it.)
Then impl it for a reference of lifetime 'a:
impl<'a> Foo for &'a Vec<u32> {
type FooIterator = ...; // something presumably containing 'a
fn bar(self, other: Self) -> Self::FooIterator {
...
}
}
This works because the compiler can limit the lifetime 'a to one for which the impl applies.
Here's a playground link where bar is basically a wrapper around .chain().
I'm ignoring the Zip trait for now because how to incorporate it depends on what it provides. Instead, I suppose that bar only accepts an argument of the same type as Self. However, you can probably add it as well, maybe using the same technique if you need to.