Storing FnMut in struct gives lifetime problems - rust

I'm trying to store a FnMut in a struct:
struct OpenVPNSocket {
socket_send_callback: Option<Box<dyn FnMut(Vec<u8>) -> Result<(), ()>>>,
}
impl OpenVPNSocket {
fn set_socket_send<F: FnMut(Vec<u8>) -> Result<(), ()>>(&mut self, callback: Box<F>) {
self.socket_send_callback = Some(callback);
}
}
I get this error:
error[E0310]: the parameter type `F` may not live long enough
--> src/lib.rs:8:42
|
7 | fn set_socket_send<F: FnMut(Vec<u8>) -> Result<(), ()>>(&mut self, callback: Box<F>) {
| -- help: consider adding an explicit lifetime bound...: `F: 'static +`
8 | self.socket_send_callback = Some(callback);
| ^^^^^^^^ ...so that the type `F` will meet its required lifetime bounds
I understand lifetime as something to do with references. However I don't use references. I don't see why my struct cannot store a Box. A Box lives as long as it's used.
UPDATE:
I have this example:
use std::sync::Arc;
pub type OnConsume = Arc<dyn Fn() -> Option<u8> + Send + Sync>;
struct Test {
callback: OnConsume
}
impl Test {
fn set_on_consume(&mut self, f: OnConsume) {
self.callback = f;
}
}
which works. What is the difference from the previous one?

In Rust, values also have lifetimes. Take, for example, this struct:
struct RefWrapper<'a> {
some_ref: &'a u32
}
An instance of RefWrapper is not a reference, but contains a lifetime. Since you're moving the box into the struct, which could live for the duration of the program (the method makes no guarantees as to when the struct instance could be dropped), the function must live for the maximum lifetime, the static lifetime.

All trait objects have lifetimes, the default implicit lifetime for boxed trait objects is 'static so your struct's socket_send_callback actually has an implicit + 'static bound. Shown in context:
struct OpenVPNSocket {
socket_send_callback: Option<Box<dyn FnMut(Vec<u8>) -> Result<(), ()> + 'static>>,
}
Since the boxed trait object has to be bounded by a 'static lifetime when you write a function to set this field the value itself must have a 'static lifetime which is why the compiler suggests adding that explicit bound. Fixed example with added bound:
impl OpenVPNSocket {
// notice the added 'static bound for F
fn set_socket_send<F: FnMut(Vec<u8>) -> Result<(), ()> + 'static>(&mut self, callback: Box<F>) {
self.socket_send_callback = Some(callback);
}
}
With this change your code will compile. If you want to accept trait objects that aren't bounded by 'static lifetimes then you can do that by making your OpenVPNSocket generic over lifetimes. This alternative solution also compiles:
struct OpenVPNSocket<'a> {
socket_send_callback: Option<Box<dyn FnMut(Vec<u8>) -> Result<(), ()> + 'a>>,
}
impl<'a> OpenVPNSocket<'a> {
fn set_socket_send<F: FnMut(Vec<u8>) -> Result<(), ()> + 'a>(&mut self, callback: Box<F>) {
self.socket_send_callback = Some(callback);
}
}
The reason why this code works is because you define the type once and use it in multiple places, and in all places it has the implicit 'static bound. Desugared:
use std::sync::Arc;
pub type OnConsume = Arc<dyn Fn() -> Option<u8> + Send + Sync + 'static>;
struct Test {
callback: OnConsume
}
impl Test {
fn set_on_consume(&mut self, f: OnConsume) {
self.callback = f;
}
}
However you can do the exact same thing in your prior code as well:
type Callback = Box<dyn FnMut(Vec<u8>) -> Result<(), ()>>;
struct OpenVPNSocket {
socket_send_callback: Option<Callback>,
}
impl OpenVPNSocket {
fn set_socket_send(&mut self, callback: Callback) {
self.socket_send_callback = Some(callback);
}
}
The above also compiles, and the implicit 'static bound is still there.

Related

Rust - implementing trait for Deref: the parameter type `T` may not live long enough

I have a trait:
trait Foo {
fn bar(&self) -> Cow<str>;
}
And I want to implement it for any type that implements Deref with a target of a type that implements Foo. Basically:
impl<T: Foo, D: std::ops::Deref<Target = T>> Foo for D {
fn bar(&self) -> Cow<str> {
<T as Foo>::bar(std::ops::Deref::deref(self))
}
}
Unfortunately, this gives the error the parameter type T may not live long enough.
My understanding is that T could have a reference within it that has a short lifetime, and the lifetime bound of the return value of Cow<str> is linked to the lifetime of &self due to lifetime elision, which would cause problems.
I'm not sure how I can fix this, since I'm not able to bound any of the lifetimes in bar. I can try to make sure T lives as long as &self, but this doesn't work.
impl<'a, T: Foo + 'a, D: std::ops::Deref<Target = T>> Foo for D {
fn bar(&'a self) -> Cow<'a, str> {
<T as Foo>::bar(std::ops::Deref::deref(self))
}
}
I get the error method not compatible with trait since the lifetimes don't match the trait defenition anymore. I've tried all sorts of different ways of adding lifetime bounds and I always get one of those two errors.
I am able to implement Foo for a specific type that implements Deref:
impl<T: Foo> Foo for Box<T> {
fn bar(&self) -> Cow<str> {
<T as Foo>::bar(self)
}
}
I'm not sure why that works but the original example doesn't.
The Box version works because of the deref coercion the compiler will do when it sees a reference and expects a different reference.
You can use the same mechanic when using a generic implementor of Deref to ensure that it Derefs to an owned type you can simply add a 'static lifetime bound on T like this:
impl<T: Foo + 'static, D: std::ops::Deref<Target = T>> Foo for D {
fn bar(&self) -> Cow<str> {
<T as Foo>::bar(self)
}
}
playground
Note: there is rarely a need to call methods of std::ops traits directly, they're all just the methods behind Rusts operators, deref for example is the method behind unary *
Update:
Since there is an additional requirement that T might not be static we have to thread through the lifetime like you tried in your second example, like the error you're getting suggest you have to adjust the trait to take a lifetime as well:
use std::borrow::Cow;
trait Foo<'a> {
fn bar(&self) -> Cow<'a, str>;
}
impl<'a, T: Foo<'a>, D: std::ops::Deref<Target = T>> Foo<'a> for D {
fn bar(&self) -> Cow<'a, str> {
<T as Foo>::bar(self)
}
}
struct S<'a> {
val: &'a str,
}
impl<'a> Foo<'a> for S<'a> {
fn bar(&self) -> Cow<'a, str> {
todo!()
}
}
fn main() {
let val = String::from("test");
let s = S { val: &val }; // error: `val` does not live long enough
let b = Box::new(s);
let cow = Foo::bar(&b); // argument requires that `val` is borrowed for `'static`
}

Why is a reference leaked from inside a lambda despite using clone?

I'm trying to implement the clone_into_box pattern in a piece of code the uses callbacks, and I'm running into an error I'm having trouble understanding.
Basically, I'm cloning the parameter of a lambda, but the borrow checker still complains I'm leaking a reference to said parameter.
My question: is this a case of the borrow checker being overly conservative (and if so, how can I re-write this in a way that would play nicely with the borrow checker), or is there something I'm missing here and a reference is indeed being leaked despite the clone?
Code:
pub trait CloneIntoBox{
fn clone_into_box<'a>(&self) -> Box<dyn CloneIntoBox + 'a>;
}
impl<'a> Clone for Box<dyn CloneIntoBox + 'a> {
fn clone(&self) -> Self {
self.as_ref().clone_into_box()
}
}
#[derive(Clone)]
pub struct StructWithBox<'a> {
pub my_box: Box<dyn CloneIntoBox + 'a>,
}
#[derive(Clone)]
struct StructThatCanBeClonedIntoBox {
pub data: u32,
}
impl CloneIntoBox for StructThatCanBeClonedIntoBox {
fn clone_into_box<'a>(&self) -> Box<dyn CloneIntoBox + 'a> {
Box::new(self.clone())
}
}
pub type WalkCallback<'a> = dyn FnMut(&StructWithBox) + 'a;
pub fn walk(data: Vec<u32>, cb: &mut WalkCallback) {
for d in data{
let instance = StructWithBox{my_box: Box::new(StructThatCanBeClonedIntoBox{data:d})};
cb(&instance);
}
}
fn main() {
let data = vec![1, 2];
let mut result = vec![];
walk(data, &mut|param| result.push((*param).clone()));
}
[Playground link]
Gives:
error[E0521]: borrowed data escapes outside of closure
--> src/main.rs:39:28
|
38 | let mut result = vec![];
| ---------- `result` declared here, outside of the closure body
39 | walk(data, &mut|param| result.push((*param).clone()));
| ----- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `param` escapes the closure body here
| |
| `param` is a reference that is only valid in the closure body
The problem is the 'a in StructWithBox. Consider the following implementation of walk():
impl<'b> CloneIntoBox for &'b u32 {
fn clone_into_box<'a>(&self) -> Box<dyn CloneIntoBox + 'a> {
Box::new(&42)
}
}
pub fn walk(data: Vec<u32>, cb: &mut WalkCallback) {
let data = 123u32;
let instance = StructWithBox {
my_box: Box::new(&data),
};
cb(&instance);
}
I haven't found a way to make this actually unsound: the only thing you can return from the impl CloneIntoBox for &u32 is a 'static reference, since 'a is decided by the caller. But the borrow checker doesn't know that: it assumes that 'a can be any lifetime in the callback, and then we push it into the vector outside the callback, but it can be freed after the callback is finished. So you can consider this overly conservative, or not.
To fix that you can make 'a always 'static in the callback:
pub type WalkCallback<'a> = dyn FnMut(&StructWithBox<'static>) + 'a;
Or get rid of the lifetime altogether:
pub trait CloneIntoBox {
fn clone_into_box(&self) -> Box<dyn CloneIntoBox>;
}
impl Clone for Box<dyn CloneIntoBox> {
fn clone(&self) -> Self {
self.as_ref().clone_into_box()
}
}
#[derive(Clone)]
pub struct StructWithBox {
pub my_box: Box<dyn CloneIntoBox>,
}
#[derive(Clone)]
struct StructThatCanBeClonedIntoBox {
pub data: u32,
}
impl CloneIntoBox for StructThatCanBeClonedIntoBox {
fn clone_into_box(&self) -> Box<dyn CloneIntoBox> {
Box::new(self.clone())
}
}
pub type WalkCallback<'a> = dyn FnMut(&StructWithBox) + 'a;
is equivalent to to
pub type WalkCallback<'a> = dyn for<'b> FnMut(&StructWithBox<'b>) + 'a;
which requires the callback to be able to take a StructWithBox of any lifetime 'c, as long as it is alive when the callback is called. Since the StructWithBox may not be valid any longer than that, it can't be pushed to a vector that lives longer than the callback.
Changing this to
pub type WalkCallback<'a, 'b> = dyn FnMut(&StructWithBox<'b>) + 'a;
allows the callback to only accept lifetimes longer than 'b.
In this case, 'b is inferred to be as long as result lives for, so that StructWithBox<'b> can be pushed to result.

Generic parameter with reference used as function pointer argument

I am having trouble figuring out what lifetime parameter will work for this, so my current workarounds include transmutes or raw pointers. I have a structure holding a function pointer with a generic as a parameter:
struct CB<Data> {
cb: fn(Data) -> usize
}
I would like to store an instance of that, parameterized by some type containing a reference, in some other structure that implements a trait with one method, and use that trait method to call the function pointer in CB.
struct Holder<'a> {
c: CB<Option<&'a usize>>
}
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
(self.c.cb)(Some(v))
}
}
impl<'a> Exec for Holder<'a> {
fn exec(&self, v: &usize) -> usize
{
self.exec_aux(v)
}
}
This gives me a lifetime error for the 'Exec' impl of Holder:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
Simply calling exec_aux works fine as long as I don't define that Exec impl:
fn main() {
let h = Holder { c: CB{cb:cbf}};
let v = 12;
println!("{}", h.exec_aux(&v));
}
Also, making CB not generic also makes this work:
struct CB {
cb: fn(Option<&usize>) -> usize
}
The parameter in my actual code is not a usize but something big that I would rather not copy.
The lifetimes in your Exec trait are implicitly this:
trait Exec {
fn exec<'s, 'a>(&'s self, v: &'a usize) -> usize;
}
In other words, types that implement Exec need to accept any lifetimes 's and 'a. However, your Holder::exec_aux method expects a specific lifetime 'a that's tied to the lifetime parameter of the Holder type.
To make this work, you need to add 'a as a lifetime parameter to the Exec trait instead, so that you can implement the trait specifically for that lifetime:
trait Exec<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize;
}
impl<'a> Exec<'a> for Holder<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize
{
self.exec_aux(v)
}
}
The problem here is that the Exec trait is too generic to be used in this way by Holder. First, consider the definition:
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
This definition will cause the compiler to automatically assign two anonymous lifetimes for &self and &v in exec. It's basically the same as
fn exec<'a, 'b>(&'a self, v: &'b usize) -> usize;
Note that there is no restriction on who needs to outlive whom, the references just need to be alive for the duration of the method call.
Now consider the definition
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
// ... doesn't matter
}
}
Since we know that &self is a &Holder<'a> (this is what the impl refers to), we need to have at least a &'a Holder<'a> here, because &'_ self can't have a lifetime shorter than 'a in Holder<'a>. So this is saying that the two parameters have the same lifetime: &'a self, &'a usize.
Where it all goes wrong is when you try to combine the two. The trait forces you into the following signature, which (again) has two distinct implicit lifetimes. But the actual Holder which you then try to call a method on forces you to have the same lifetimes for &self and &v.
fn exec(&self, v: &usize) -> usize {
// Holder<'a> needs `v` to be `'a` when calling exec_aux
// But the trait doesn't say so.
self.exec_aux(v)
}
One solution is to redefine the trait as
trait Exec<'a> {
fn exec(&'a self, v: &'a usize) -> usize;
}
and then implement it as
impl<'a> Exec<'a> for Holder<'a> {
fn exec(&'a self, v: &'a usize) -> usize {
self.exec_aux(v)
}
}

lifetime must be valid for the static lifetime so that the types are compatible

I'm aware of Where did the 'static lifetime come from and Cannot infer an appropriate lifetime for autoref due to conflicting requirements。
But I still do not understand the problem I've encountered:
use std::ops::Index;
trait Stack<T> {
fn as_slice(&self) -> &[T];
}
impl<T> Index<usize> for Stack<T> {
type Output = T;
fn index(&self, i: usize) -> &T {
&self.as_slice()[i]
}
}
trait Core {
fn stack(&self) -> &Stack<usize>;
fn bad(&mut self) -> usize {
self.stack()[0]
}
fn good(&mut self) -> usize {
self.stack().as_slice()[0]
}
}
fn main() {}
In the code above, good() gives no error, but bad() complains with:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/main.rs:18:14
|
18 | self.stack()[0]
| ^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 17:5...
--> src/main.rs:17:5
|
17 | / fn bad(&mut self) -> usize {
18 | | self.stack()[0]
19 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:18:9
|
18 | self.stack()[0]
| ^^^^
= note: but, the lifetime must be valid for the static lifetime...
= note: ...so that the types are compatible:
expected std::ops::Index<usize>
found std::ops::Index<usize>
There is no Box in this code, and I do not know where the static lifetime comes from.
Edit: through try and error I found the compiler assume Stack + 'static. The following code compiles. But why? Please point me to some document.
impl<'b, T> Index<usize> for Stack<T> + 'b {
type Output = T;
fn index<'a>(&'a self, i: usize) -> &'a T {
&self.as_slice()[i]
}
}
You are implementing a Trait (Index) for a Trait Object (Stack<T>).
The Rust reference states:
Since a trait object can contain references, the lifetimes of those references need to be expressed as part of the trait object. This lifetime is written as Trait + 'a. There are defaults that allow this lifetime to usually be inferred with a sensible choice.
If You don't define a lifetime the compiler assume a default, in this case it is assumed 'static (see here for a detailed explanation)
Your code is equivalent to:
impl<T> Index<usize> for Stack<T> + 'static {
type Output = T;
fn index(&self, i: usize) -> &T {
&self.as_slice()[i]
}
}
To resolve the cannot infer an appropriate lifetime for autoref due to conflicting requirements compilation error just declare that stack() method returns a trait object with 'static lifetime.
trait Core {
fn stack(&self) -> &'static Stack<usize>;
fn bad(&mut self) -> usize {
self.stack()[0]
}
fn good(&mut self) -> usize {
self.stack().as_slice()[0]
}
}
Otherwise declare a generic lifetime for the Stack<T> trait object that impl Index:
impl<'a, T> Index<usize> for Stack<T> + 'a {
type Output = T;
fn index(&self, i: usize) -> &T {
&self.as_slice()[i]
}
}
trait Core {
fn stack(&self) -> &Stack<usize>;
fn bad(&mut self) -> usize {
self.stack()[0]
}
fn good(&mut self) -> usize {
self.stack().as_slice()[0]
}
}
At this point you should ask: why using as_slice() in good() works and using index() in bad() does not?
To make same sense of it try read the comments embedded in the MVCE below.
use std::ops::Index;
trait Stack<T> {
fn as_slice(&self) -> &[T];
}
// equivalent to: impl<T> Index<usize> for Stack<T>
// just to expose the conflicting requirements error
// the right declaration is:
// impl<'a, T> Index<usize> for Stack<T> + 'a
impl<T> Index<usize> for Stack<T> + 'static {
type Output = T;
fn index(&self, i: usize) -> &T {
&self.as_slice()[i]
}
}
trait Core {
fn stack(&self) -> &Stack<usize>;
fn bad<'a>(&'a mut self) -> usize {
//self.stack()[0] syntactic sugar for:
*self.stack().index(0)
// self.stack() returns a trait object with a lifetime bound != 'static
// but Stack impl for Index requires a 'static lifetime bound:
// COMPILE ERROR: cannot infer an appropriate lifetime for
// autoref due to conflicting requirements
}
fn good<'a>(&'a mut self) -> usize {
// self.stack() returns a trait object with 'a lifetime bound
// this is congruent with as_slice() lifetime requirements
// see Catasta::as_slice() impl below
// NO COMPILE ERROR
self.stack().as_slice()[0]
}
}
struct Catasta<T> {
pila: [T;4]
}
impl<T> Stack<T> for Catasta<T> {
fn as_slice<'a>(&'a self) -> &'a [T] {
&self.pila
}
}
struct RealCore {
stk: Catasta<usize>
}
impl Core for RealCore {
fn stack(&self) -> &Stack<usize> {
&self.stk
}
}
fn main() {
let mut core = RealCore {stk: Catasta {pila: [100, 2, 3, 4]} };
println!("pos [0] item: {}", core.good());
}

How do you write a trait that returns an iterator?

Broadly speaking my goal is this:
For some known type Bar...
Have a trait Foo with a function: get_iterator<T>() -> T where T: Iterator<Item = Bar>
The instance of the iterator borrows the original object Foo is implemented on.
I imagine it working like this:
let mut foo = Foo;
let bar = foo.get_iterator();
foo.mutable_call(); // <-- This fails, because foo is borrowed in bar
for x in bar {
...
}
So, that's the goal, and here's my attempt, which I can't seem to get working:
struct ValsFromT<'a, T: 'a> {
parent:&'a T,
offset: usize,
}
struct Val;
trait HasValsIterator<T> {
fn val_iterator(&self) -> T where T: Iterator<Item = Val>;
}
struct Foo;
impl<'a> Iterator for ValsFromT<'a, Foo> {
type Item = Val;
fn next(&mut self) -> Option<Val> {
return None;
}
}
impl<'a> HasValsIterator<ValsFromT<'a, Foo>> for Foo {
fn val_iterator(&'a self) -> ValsFromT<'a, Foo> {
return ValsFromT {
offset: 0usize,
parent: self
};
}
}
fn takes_vals<T>(instance:T) where T: HasValsIterator<T> {
// ...
}
#[test]
fn test_foo() {
let x = Foo;
takes_vals(x);
}
(playpen: http://is.gd/wys3fx)
We're getting the dreaded concrete/bound lifetime error here, because of trying to return an iterator instance that references self from the trait function:
<anon>:22:3: 27:4 error: method `val_iterator` has an incompatible type for trait:
expected bound lifetime parameter ,
found concrete lifetime [E0053]
<anon>:22 fn val_iterator(&'a self) -> ValsFromT<'a, Foo> {
<anon>:23 return ValsFromT {
<anon>:24 offset: 0usize,
<anon>:25 parent: self
<anon>:26 };
<anon>:27 }
<anon>:22:3: 27:4 help: see the detailed explanation for E0053
Is there some way of doing this?
Unfortunately, Veedrac's suggestion doesn't work directly. You will get the following error if you'd try to use val_iterator() method on instance inside takes_vals():
<anon>:31:25: 31:39 error: the trait `core::iter::Iterator` is not implemented for the type `U` [E0277]
<anon>:31 let iter = instance.val_iterator();
^~~~~~~~~~~~~~
<anon>:31:25: 31:39 help: see the detailed explanation for E0277
<anon>:31:25: 31:39 note: `U` is not an iterator; maybe try calling `.iter()` or a similar method
error: aborting due to previous error
playpen: application terminated with error code 101
This (and some other further errors) requires changing the signature of the function to this one:
fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U>
However, even this doesn't work yet:
<anon>:31:16: 31:24 error: `instance` does not live long enough
<anon>:31 let iter = instance.val_iterator();
^~~~~~~~
<anon>:30:97: 32:2 note: reference must be valid for the lifetime 'a as defined on the block at 30:96...
<anon>:30 fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U> {
<anon>:31 let iter = instance.val_iterator();
<anon>:32 }
<anon>:30:97: 32:2 note: ...but borrowed value is only valid for the scope of parameters for function at 30:96
<anon>:30 fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U> {
<anon>:31 let iter = instance.val_iterator();
<anon>:32 }
Remember that the trait requires that val_iterator() accepts the target by reference with lifetime 'a. This lifetime in this function is an input parameter. However, when val_iterator() is called on instance, the only lifetime which can be specified for the reference is the one of instance which is strictly smaller than any possible 'a as a parameter, because it is a local variable. Therefore, it is not possible to pass instance by value; you can only pass it by reference for lifetimes to match:
fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: &'a T) where T: HasValsIterator<'a, U> {
let iter = instance.val_iterator();
}
This works.
I'd like to add that using associated types instead of type parameters would be more correct semantically:
trait HasValsIterator<'a> {
type Iter: Iterator<Item=Val> + 'a;
fn val_iterator(&'a self) -> Self::Iter;
}
impl<'a> HasValsIterator<'a> for Foo {
type Iter = ValsFromT<'a, Foo>;
fn val_iterator(&'a self) -> ValsFromT<'a, Foo> { ... }
}
fn takes_vals<'a, T: 'a>(instance: &'a T) where T: HasValsIterator<'a> {
...
}
I say that this is more correct because the type of the iterator is determined by the implementor, that is, it is "output" type, which are modeled by associated types. As you can see, takes_vals() signature also shrank considerably.
Ideally, HasValsIterator trait should have been defined like this:
trait HasValsIterator {
type Iter<'a>: Iterator<Item=Val> + 'a
fn val_iterator<'a>(&'a self) -> Iter<'a>;
}
This way, val_iterator() would in any situation, including when HasValsIterator implementor is passed by value. However, Rust is not there yet.

Resources