How do I pass reference parameters to boxed closures? - rust

I want to store a callback that can take different types of parameters (both owned values and references), and can also modify its environment (hence the FnMut). When invoking the callback with a reference, I'd like the compiler to enforce that the parameter is only valid in the closure body. I've tried to implement this using boxed closures.
A minimum example shown below:
fn main() {
let mut caller = Caller::new();
let callback = |x: &Foo| println!("{:?}", x);
caller.register(callback);
let foo = Foo{
bar: 1,
baz: 2,
};
//callback(&foo); // works
caller.invoke(&foo); // borrowed value does not live long enough
}
struct Caller<'a, T> {
callback: Box<dyn FnMut(T) + 'a>
}
impl<'a, T> Caller<'a, T> {
fn new() -> Self {
Caller {
callback: Box::new(|_| ()),
}
}
fn register(&mut self, cb: impl FnMut(T) + 'a) {
self.callback = Box::new(cb);
}
fn invoke(&mut self, x: T) {
(self.callback)(x);
}
}
#[derive(Debug, Clone)]
struct Foo {
bar: i32,
baz: i32,
}
I want to understand why this works if I directly call callback() but the compiler complains about lifetimes if I invoke it through a struct than owns the closure. Perhaps it has something to do with the Box? I can get this to work if I define foo before caller, but I'd like to avoid this.

This is yet another example of the compiler's type inference quirks when working with closures and bounds of a similar sort (issue #41078). Although this Caller<'a, T> may seem to be well capable of handling invoke calls for a given generic T, the given example passes a reference &'b Foo (where 'b would be some anonymous lifetime of that value). And due to this limitation, T was inferred to be a &Foo of one expected lifetime, which is different from a reference of any lifetime to a value of type Foo (for<'a> &'a Foo), and incompatible with the reference passed to the invoke call.
By not passing the closure to Caller, the compiler would be able to correctly infer the expected parameter type of the callback, including reference lifetime.
One way to overcome this is to redefine Caller to explicitly receive a reference value as the callback parameter. This changes the behavior of the inferred type &T into a higher-ranked lifetime bound, as hinted above.
Playground
fn main() {
let mut caller = Caller::new();
let callback = |x: &Foo| { println!("{:?}", x) };
caller.register(callback);
let foo = Foo { bar: 1, baz: 2 };
caller.invoke(&foo);
}
struct Caller<'a, T> {
callback: Box<dyn FnMut(&T) + 'a>,
}
impl<'a, T> Caller<'a, T> {
fn new() -> Self {
Caller {
callback: Box::new(|_| ()),
}
}
fn register(&mut self, cb: impl FnMut(&T) + 'a) {
self.callback = Box::new(cb);
}
fn invoke(&mut self, x: &T) {
(self.callback)(x);
}
}
One way to make this clearer would be to use the expanded definition of invoke:
fn register<F>(&mut self, cb: F)
where
F: for<'b> FnMut(&'b T) + 'a
{
self.callback = Box::new(cb);
}
See also:
Why is "one type is more general than the other" in an Option containing a closure?
Type mismatches resolving a closure that takes arguments by reference
How to declare a lifetime for a closure argument?

Related

Rust - implementing trait for Deref: the parameter type `T` may not live long enough

I have a trait:
trait Foo {
fn bar(&self) -> Cow<str>;
}
And I want to implement it for any type that implements Deref with a target of a type that implements Foo. Basically:
impl<T: Foo, D: std::ops::Deref<Target = T>> Foo for D {
fn bar(&self) -> Cow<str> {
<T as Foo>::bar(std::ops::Deref::deref(self))
}
}
Unfortunately, this gives the error the parameter type T may not live long enough.
My understanding is that T could have a reference within it that has a short lifetime, and the lifetime bound of the return value of Cow<str> is linked to the lifetime of &self due to lifetime elision, which would cause problems.
I'm not sure how I can fix this, since I'm not able to bound any of the lifetimes in bar. I can try to make sure T lives as long as &self, but this doesn't work.
impl<'a, T: Foo + 'a, D: std::ops::Deref<Target = T>> Foo for D {
fn bar(&'a self) -> Cow<'a, str> {
<T as Foo>::bar(std::ops::Deref::deref(self))
}
}
I get the error method not compatible with trait since the lifetimes don't match the trait defenition anymore. I've tried all sorts of different ways of adding lifetime bounds and I always get one of those two errors.
I am able to implement Foo for a specific type that implements Deref:
impl<T: Foo> Foo for Box<T> {
fn bar(&self) -> Cow<str> {
<T as Foo>::bar(self)
}
}
I'm not sure why that works but the original example doesn't.
The Box version works because of the deref coercion the compiler will do when it sees a reference and expects a different reference.
You can use the same mechanic when using a generic implementor of Deref to ensure that it Derefs to an owned type you can simply add a 'static lifetime bound on T like this:
impl<T: Foo + 'static, D: std::ops::Deref<Target = T>> Foo for D {
fn bar(&self) -> Cow<str> {
<T as Foo>::bar(self)
}
}
playground
Note: there is rarely a need to call methods of std::ops traits directly, they're all just the methods behind Rusts operators, deref for example is the method behind unary *
Update:
Since there is an additional requirement that T might not be static we have to thread through the lifetime like you tried in your second example, like the error you're getting suggest you have to adjust the trait to take a lifetime as well:
use std::borrow::Cow;
trait Foo<'a> {
fn bar(&self) -> Cow<'a, str>;
}
impl<'a, T: Foo<'a>, D: std::ops::Deref<Target = T>> Foo<'a> for D {
fn bar(&self) -> Cow<'a, str> {
<T as Foo>::bar(self)
}
}
struct S<'a> {
val: &'a str,
}
impl<'a> Foo<'a> for S<'a> {
fn bar(&self) -> Cow<'a, str> {
todo!()
}
}
fn main() {
let val = String::from("test");
let s = S { val: &val }; // error: `val` does not live long enough
let b = Box::new(s);
let cow = Foo::bar(&b); // argument requires that `val` is borrowed for `'static`
}

Generic parameter with reference used as function pointer argument

I am having trouble figuring out what lifetime parameter will work for this, so my current workarounds include transmutes or raw pointers. I have a structure holding a function pointer with a generic as a parameter:
struct CB<Data> {
cb: fn(Data) -> usize
}
I would like to store an instance of that, parameterized by some type containing a reference, in some other structure that implements a trait with one method, and use that trait method to call the function pointer in CB.
struct Holder<'a> {
c: CB<Option<&'a usize>>
}
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
(self.c.cb)(Some(v))
}
}
impl<'a> Exec for Holder<'a> {
fn exec(&self, v: &usize) -> usize
{
self.exec_aux(v)
}
}
This gives me a lifetime error for the 'Exec' impl of Holder:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
Simply calling exec_aux works fine as long as I don't define that Exec impl:
fn main() {
let h = Holder { c: CB{cb:cbf}};
let v = 12;
println!("{}", h.exec_aux(&v));
}
Also, making CB not generic also makes this work:
struct CB {
cb: fn(Option<&usize>) -> usize
}
The parameter in my actual code is not a usize but something big that I would rather not copy.
The lifetimes in your Exec trait are implicitly this:
trait Exec {
fn exec<'s, 'a>(&'s self, v: &'a usize) -> usize;
}
In other words, types that implement Exec need to accept any lifetimes 's and 'a. However, your Holder::exec_aux method expects a specific lifetime 'a that's tied to the lifetime parameter of the Holder type.
To make this work, you need to add 'a as a lifetime parameter to the Exec trait instead, so that you can implement the trait specifically for that lifetime:
trait Exec<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize;
}
impl<'a> Exec<'a> for Holder<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize
{
self.exec_aux(v)
}
}
The problem here is that the Exec trait is too generic to be used in this way by Holder. First, consider the definition:
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
This definition will cause the compiler to automatically assign two anonymous lifetimes for &self and &v in exec. It's basically the same as
fn exec<'a, 'b>(&'a self, v: &'b usize) -> usize;
Note that there is no restriction on who needs to outlive whom, the references just need to be alive for the duration of the method call.
Now consider the definition
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
// ... doesn't matter
}
}
Since we know that &self is a &Holder<'a> (this is what the impl refers to), we need to have at least a &'a Holder<'a> here, because &'_ self can't have a lifetime shorter than 'a in Holder<'a>. So this is saying that the two parameters have the same lifetime: &'a self, &'a usize.
Where it all goes wrong is when you try to combine the two. The trait forces you into the following signature, which (again) has two distinct implicit lifetimes. But the actual Holder which you then try to call a method on forces you to have the same lifetimes for &self and &v.
fn exec(&self, v: &usize) -> usize {
// Holder<'a> needs `v` to be `'a` when calling exec_aux
// But the trait doesn't say so.
self.exec_aux(v)
}
One solution is to redefine the trait as
trait Exec<'a> {
fn exec(&'a self, v: &'a usize) -> usize;
}
and then implement it as
impl<'a> Exec<'a> for Holder<'a> {
fn exec(&'a self, v: &'a usize) -> usize {
self.exec_aux(v)
}
}

Can I borrow self immutably for self.callbacks: Vec<Box<FnMut>>?

pub struct Notifier<'a, T> {
callbacks: Vec<Box<'a + FnMut(&T)>>
}
impl<'a, T> Notifier<'a, T>{
fn add_callback<F: 'a + FnMut(&T)>(&mut self, callback: F) {
self.callbacks.push(Box::new(callback));
}
fn trigger(&mut self, payload: T) {
for callback in &mut self.callbacks {
callback(&payload);
}
}
}
struct A {
x: i64
}
impl A {
fn foo(&mut self, x: &i64) {
self.x = x + 1;
}
}
fn test() {
let mut bar = A {x: 3};
let mut notifier = Notifier{callbacks: Vec::new()};
notifier.add_callback(|x| bar.foo(x));
}
Playground
This is a simple observer pattern implemented using callbacks. It works.
However, the fact that trigger(&mut self... causes much trouble in my later coding (How to update self based on reference of value from hashmap from self). Is it possible to make trigger(&self ... instead?
I'm using rustc 1.19.0-nightly.
If you want to change the interior of a struct without having the struct mutable, you should use a Cell:
Values of the Cell<T> and RefCell<T> types may be mutated through shared references (i.e. the common &T type), whereas most Rust types can only be mutated through unique (&mut T) references. We say that Cell<T> and RefCell<T> provide 'interior mutability', in contrast with typical Rust types that exhibit 'inherited mutability'.
Playground

How do you write a trait that returns an iterator?

Broadly speaking my goal is this:
For some known type Bar...
Have a trait Foo with a function: get_iterator<T>() -> T where T: Iterator<Item = Bar>
The instance of the iterator borrows the original object Foo is implemented on.
I imagine it working like this:
let mut foo = Foo;
let bar = foo.get_iterator();
foo.mutable_call(); // <-- This fails, because foo is borrowed in bar
for x in bar {
...
}
So, that's the goal, and here's my attempt, which I can't seem to get working:
struct ValsFromT<'a, T: 'a> {
parent:&'a T,
offset: usize,
}
struct Val;
trait HasValsIterator<T> {
fn val_iterator(&self) -> T where T: Iterator<Item = Val>;
}
struct Foo;
impl<'a> Iterator for ValsFromT<'a, Foo> {
type Item = Val;
fn next(&mut self) -> Option<Val> {
return None;
}
}
impl<'a> HasValsIterator<ValsFromT<'a, Foo>> for Foo {
fn val_iterator(&'a self) -> ValsFromT<'a, Foo> {
return ValsFromT {
offset: 0usize,
parent: self
};
}
}
fn takes_vals<T>(instance:T) where T: HasValsIterator<T> {
// ...
}
#[test]
fn test_foo() {
let x = Foo;
takes_vals(x);
}
(playpen: http://is.gd/wys3fx)
We're getting the dreaded concrete/bound lifetime error here, because of trying to return an iterator instance that references self from the trait function:
<anon>:22:3: 27:4 error: method `val_iterator` has an incompatible type for trait:
expected bound lifetime parameter ,
found concrete lifetime [E0053]
<anon>:22 fn val_iterator(&'a self) -> ValsFromT<'a, Foo> {
<anon>:23 return ValsFromT {
<anon>:24 offset: 0usize,
<anon>:25 parent: self
<anon>:26 };
<anon>:27 }
<anon>:22:3: 27:4 help: see the detailed explanation for E0053
Is there some way of doing this?
Unfortunately, Veedrac's suggestion doesn't work directly. You will get the following error if you'd try to use val_iterator() method on instance inside takes_vals():
<anon>:31:25: 31:39 error: the trait `core::iter::Iterator` is not implemented for the type `U` [E0277]
<anon>:31 let iter = instance.val_iterator();
^~~~~~~~~~~~~~
<anon>:31:25: 31:39 help: see the detailed explanation for E0277
<anon>:31:25: 31:39 note: `U` is not an iterator; maybe try calling `.iter()` or a similar method
error: aborting due to previous error
playpen: application terminated with error code 101
This (and some other further errors) requires changing the signature of the function to this one:
fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U>
However, even this doesn't work yet:
<anon>:31:16: 31:24 error: `instance` does not live long enough
<anon>:31 let iter = instance.val_iterator();
^~~~~~~~
<anon>:30:97: 32:2 note: reference must be valid for the lifetime 'a as defined on the block at 30:96...
<anon>:30 fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U> {
<anon>:31 let iter = instance.val_iterator();
<anon>:32 }
<anon>:30:97: 32:2 note: ...but borrowed value is only valid for the scope of parameters for function at 30:96
<anon>:30 fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U> {
<anon>:31 let iter = instance.val_iterator();
<anon>:32 }
Remember that the trait requires that val_iterator() accepts the target by reference with lifetime 'a. This lifetime in this function is an input parameter. However, when val_iterator() is called on instance, the only lifetime which can be specified for the reference is the one of instance which is strictly smaller than any possible 'a as a parameter, because it is a local variable. Therefore, it is not possible to pass instance by value; you can only pass it by reference for lifetimes to match:
fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: &'a T) where T: HasValsIterator<'a, U> {
let iter = instance.val_iterator();
}
This works.
I'd like to add that using associated types instead of type parameters would be more correct semantically:
trait HasValsIterator<'a> {
type Iter: Iterator<Item=Val> + 'a;
fn val_iterator(&'a self) -> Self::Iter;
}
impl<'a> HasValsIterator<'a> for Foo {
type Iter = ValsFromT<'a, Foo>;
fn val_iterator(&'a self) -> ValsFromT<'a, Foo> { ... }
}
fn takes_vals<'a, T: 'a>(instance: &'a T) where T: HasValsIterator<'a> {
...
}
I say that this is more correct because the type of the iterator is determined by the implementor, that is, it is "output" type, which are modeled by associated types. As you can see, takes_vals() signature also shrank considerably.
Ideally, HasValsIterator trait should have been defined like this:
trait HasValsIterator {
type Iter<'a>: Iterator<Item=Val> + 'a
fn val_iterator<'a>(&'a self) -> Iter<'a>;
}
This way, val_iterator() would in any situation, including when HasValsIterator implementor is passed by value. However, Rust is not there yet.

Tying a trait lifetime variable to &self lifetime

I'd like to do something along the following lines:
trait GetRef<'a> {
fn get_ref(&self) -> &'a [u8];
}
struct Foo<'a> {
buf: &'a [u8]
}
impl <'a> GetRef<'a> for Foo<'a> {
fn get_ref(&self) -> &'a [u8] {
&self.buf[1..]
}
}
struct Bar {
buf: Vec<u8>
}
// this is the part I'm struggling with:
impl <'a> GetRef<'a> for Bar {
fn get_ref(&'a self) -> &'a [u8] {
&self.buf[1..]
}
The point of the explicit lifetime variable in the GetRef trait is to allow the return value of get_ref() on a Foo object to outlive the Foo itself, tying the return value's lifetime to that of the lifetime of Foo's buffer.
However, I haven't found a way to implement GetRef for Bar in a way that the compiler accepts. I've tried several variations of the above, but can't seem to find one that works. Is there any there any reason that this fundamentally cannot be done? If not, how can I do this?
Tying a trait lifetime variable to &self lifetime
Not possible.
Is there any there any reason that this fundamentally cannot be done?
Yes. An owning vector is something different than a borrowed slice. Your trait GetRef only makes sense for things that already represent a “loan” and don't own the slice. For an owning type like Bar you can't safely return a borrowed slice that outlives Self. That's what the borrow checker prevents to avoid dangling pointers.
What you tried to do is to link the lifetime parameter to the lifetime of Self. But the lifetime of Self is not a property of its type. It just depends on the scope this value was defined in. And that's why your approach cannot work.
Another way of looking at it is: In a trait you have to be explicit about whether Self is borrowed by a method and its result or not. You defined the GetRef trait to return something that is not linked to Self w.r.t. lifetimes. So, no borrowing. So, it's not implementable for types that own the data. You can't create a borrowed slice referring to a Vec's elements without borrowing the Vec.
If not, how can I do this?
Depends on what exactly you mean by “this”. If you want to write a “common denominator” trait that can be implemented for both borrowed and owning slices, you have to do it like this:
trait GetRef {
fn get_ref(&self) -> &[u8];
}
The meaning of this trait is that get_ref borrows Self and returns a kind of “loan” because of the current lifetime elision rules. It's equivalent to the more explicit form
trait GetRef {
fn get_ref<'s>(&self) -> &'s [u8];
}
It can be implemented for both types now:
impl<'a> GetRef for Foo<'a> {
fn get_ref(&self) -> &[u8] { &self.buf[1..] }
}
impl GetRef for Bar {
fn get_ref(&self) -> &[u8] { &self.buf[1..] }
}
You could make different lifetimes for &self and result in your trait like that:
trait GetRef<'a, 'b> {
fn get_ref(&'b self) -> &'a [u8];
}
struct Foo<'a> {
buf: &'a [u8]
}
impl <'a, 'b> GetRef<'a, 'b> for Foo<'a> {
fn get_ref(&'b self) -> &'a [u8] {
&self.buf[1..]
}
}
struct Bar {
buf: Vec<u8>
}
// Bar, however, cannot contain anything that outlives itself
impl<'a> GetRef<'a, 'a> for Bar {
fn get_ref(&'a self) -> &'a [u8] {
&self.buf[1..]
}
}
fn main() {
let a = vec!(1 as u8, 2, 3);
let b = a.clone();
let tmp;
{
let x = Foo{buf: &a};
tmp = x.get_ref();
}
{
let y = Bar{buf: b};
// Bar's buf cannot outlive Bar
// tmp = y.get_ref();
}
}

Resources