Using callbacks on trait objects - rust

I’m trying to use a callback function on a trait object. I reduced my problem to the following code (playpen):
trait Caller {
fn call(&self, call: fn(&Caller)) where Self: Sized {
call(self)
}
}
struct Type;
impl Caller for Type {}
fn callme(_: &Caller) {}
fn main() {
let caller: Box<Caller> = Box::new(Type);
caller.call(callme); // does not work
//callme(&*caller); // works
}
which results in
<anon>:14:12: 14:24 error: the trait `core::marker::Sized` is not implemented for the type `Caller` [E0277]
<anon>:14 caller.call(callme); // does not work
^~~~~~~~~~~~
Adding a Sized bound to Caller results in:
<anon>:3:14: 3:18 error: cannot convert to a trait object because trait `Caller` is not object-safe [E0038]
I really don’t understand why I need the Sized bound on the trait. Funnily it works if I use the callback directly. How do I get this to work?
Edit: Thanks to the answer I now came up with a nice solution
trait Caller {
fn borrow(&self) -> &Caller;
fn call(&self, call: fn(&Caller)) {
call(self.borrow())
}
}
struct Type;
impl Caller for Type {
fn borrow(&self) -> &Caller { self }
}
fn callme(_: &Caller) {}
fn main() {
let caller: Box<Caller> = Box::new(Type);
caller.call(callme);
}

The argument call in fn call takes a trait object &Caller, so calling it requires coercing the self reference (of type &Self) to a &Caller trait object. The coercion is only possible when &Self is a thin pointer rather than a fat pointer like a trait object or a &[T] slice. &Self is a thin pointer exactly when Self: Sized. The compiler defaults to Self in traits not being Sized, and so the extra restriction is required. The Sized trait represents that the type has a size that is known at compile time, there is no need to store extra info (next to the pointer, making it "fat") to compute it at runtime.
Unfortunately, this leaves a hole: AFAIK, it's actually not possible to have such a method be a default method and still be able to call it on trait objects, since a trait object &Caller has Self = Caller which isn't Sized. However, it should work if the method is implemented manually for each type:
trait Caller {
fn call(&self, call: fn(&Caller));
}
struct Type;
impl Caller for Type {
fn call(&self, call: fn(&Caller)) {
call(self)
}
}
fn callme(_: &Caller) {}
fn main() {
let caller: Box<Caller> = Box::new(Type);
caller.call(callme);
}
The call method declaration in the trait no longer needs the where Self: Sized since it isn't trying to do the trait object coercion itself, and the concrete implementations have much more control over how the &Caller trait object is obtained. For Sized types, it works directly, like the original where Self: Sized code.

Related

`impl Trait` return type causes wrong lifetime elision

The impl Trait syntax for return types seems to cause the compiler to incorrectly assume that the lifetime of the input argument must match the output in some situations. Consider the function
fn take_by_trait<T: InTrait>(_: T) -> impl OutTrait {}
If the input type contains a lifetime the compiler complains when the output outlives the it even though they are completely independent. This will not happen if the input type is not generic, or if the output is a Box<dyn OutTrait>.
Full code:
trait InTrait {}
struct InStruct<'a> {
_x: &'a bool, // Just a field with some reference
}
impl<'a> InTrait for InStruct<'a> {}
trait OutTrait {}
impl OutTrait for () {}
fn take_by_type(_: InStruct) -> impl OutTrait {}
fn take_by_trait<T: InTrait>(_: T) -> impl OutTrait {}
fn take_by_trait_output_dyn<T: InTrait>(_: T) -> Box<dyn OutTrait> {
Box::new(())
}
fn main() {
let _ = {
let x = true;
take_by_trait(InStruct{ _x: &x }) // DOES NOT WORK
// take_by_type(InStruct { _x: &x }) // WORKS
// take_by_trait_output_dyn(InStruct { _x: &x }) // WORKS
};
}
Is there some elided lifetime here that I could qualify to make this work, or do I need to do heap allocation?
The impl Trait semantic means that the function returns some type that implements the Trait, but the caller cannot make any assumptions about which type that would be or what lifetimes it would use.
For all the compiler knows the take_by_trait function can be used in many different modules, or, probably, in other crates. Now, your implementation would work fine in all use cases. It can be rewritten like
fn take_by_trait<T: InTrait>(_: T) -> () {}
This is a perfectly fine function and will work just fine. But then at some point you might want to add another implementation for the OutTrait and change your take_by_trait function a little.
trait OutTrait { fn use_me(&self) {} }
impl<T: InTrait> OutTrait for T {}
fn take_by_trait<T: InTrait>(v: T) -> impl OutTrait {v}
If we expand the generic parameter and impl definition, we get this code:
fn take_by_trait<'a>(v: InStruct<'a>) -> InStruct<'a> {v}
fn main() {
let v = {
let x = true;
take_by_trait(InStruct{ _x: &x })
};
v.use_me();
}
This obviously cannot work because x is dropped before println! tries to access its value. So by adding a new implementation for the OutTrait you broke code that uses this function, potentially somewhere in a crate that depends on yours. That's why the compiler is reluctant to allow you defining such things.
So, again, the issue with impl OutTrait is just that the compiler cannot make any assumptions about the returned type and its lifetime, so it uses the maximum possible bound, which produces the borrow checker error you see.
EDIT: I've modified the code a little so that the signature of the function would not change and the code actually compiles and produces the same lifetime error: playground
impl Trait in return position implicitly captures the any lifetime appearing in generic parameters. This is not something you can express in normal Rust, only with impl Trait.
On stable as far as I know there is no way to avoid that. On nightly you can use type_alias_impl_trait:
#![feature(type_alias_impl_trait)]
type Ret = impl OutTrait;
fn take_by_trait<T: InTrait>(v: T) -> Ret {}
See issues Unclear compiler error when impl Trait return value captures non-'static argument (#82171), impl Trait capturing lifetime of type parameter (#79415), False-positive "temporary dropped while borrowed" involving return-position impl Trait (#98997), impl Trait + 'static is not static if returned from generic function (#76882).

Understanding "the trait X cannot be made into an object" for `&mut Box<Self>` parameter

I've got this code snippet (playground):
struct TeddyBear {
fluffiness: u8,
}
trait Scruffy {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy>;
}
impl Scruffy for TeddyBear {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy> {
// do something about the TeddyBear's fluffiness
}
}
It doesn't compile. The error is:
the trait Scruffy cannot be made into an object
, along with the hint:
because method scruff_up's self parameter cannot be dispatched on.
I checked the "E0038" error description, but I haven't been able to figure out which category my error falls into.
I also read the "object-safety" entry in "The Rust Reference", and I believe this matches the "All associated functions must either be dispatchable from a trait object", but I'm not sure, partly because I'm not sure what "receiver" means in that context.
Can you please clarify for me what's the problem with this code and why it doesn't work?
The problem is when you pass it in as a reference, because the inner type may not be well-sized (e.g. a trait object, like if you passed in a Box<Fluffy>) the compiler doesn't have enough information to figure out how to call methods on it. If you restrict it to sized objects (like your TeddyBear) it should compile
trait Scruffy {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy> where Self: Sized;
}
A receiver is the self (&self, &mut self, self: &mut Box<Self> and so on).
Note that the list you cited from the reference lists both Box<Self> and &mut Self, but does not list &mut Box<Self> nor it says that combinations of these types are allowed.
This is, indeed, forbidden. As for the why, it is a little more complex.
In order for a type to be a valid receiver, it needs to hold the following condition:
Given any type Self that implements Trait and the receiver type Receiver, the receiver type should implement DispatchFromDyn<dyn Trait> for itself with all Self occurrences of Self replaced with dyn Trait.
For instance:
&self (has the type &Self) has to implement DispatchFromDyn<&dyn Trait>, which it does.
Box<Self> has to implement DispatchFromDyn<Box<dyn Trait>>, which it does.
But in order for &mut Box<Self> to be an object-safe receiver, it would need to impl DispatchFromDyn<&mut Box<dyn Trait>>. What you want is kind of blanket implementation DispatchFromDyn<&mut T> for &mut U where U: DispatchFromDyn<T>.
This impl will never exist. Because it is unsound (even ignoring coherence problems).
As explained in the code in rustc that calculates this:
The only case where the receiver is not dispatchable, but is still a valid receiver type (just not object-safe), is when there is more than one level of pointer indirection. E.g., self: &&Self, self: &Rc<Self>, self: Box<Box<Self>>. In these cases, there is no way, or at least no inexpensive way, to coerce the receiver from the version where Self = dyn Trait to the version where Self = T, where T is the unknown erased type contained by the trait object, because the object that needs to be coerced is behind a pointer.
The problem is inherent to how Rust handles dyn Trait.
dyn Trait is a fat pointer: it is actually two words sized. One is a pointer to the data, and the other is a pointer to the vtable.
When you call a method on dyn Trait, the compiler looks up in the vtable, find the method for the concrete type (which is unknown at compilation time, but known at runtime), and calls it.
This all may be very abstract without an example:
trait Trait {
fn foo(&self);
}
impl Trait for () {
fn foo(&self) {}
}
fn call_foo(v: &dyn Trait) {
v.foo();
}
fn create_dyn_trait(v: &impl Trait) {
let v: &dyn Trait = v;
call_foo(v);
}
The compiler generates code like:
trait Trait {
fn foo(&self);
}
impl Trait for () {
fn foo(&self) {}
}
struct TraitVTable {
foo: fn(*const ()),
}
static TRAIT_FOR_UNIT_VTABLE: TraitVTable = TraitVTable {
foo: unsafe { std::mem::transmute(<() as Trait>::foo) },
};
type DynTraitRef = (*const (), &'static TraitVTable);
impl Trait for dyn Trait {
fn foo(self: DynTraitRef) {
(self.1.foo)(self.0)
}
}
fn call_foo(v: DynTraitRef) {
v.foo();
}
fn create_dyn_trait(v: &impl Trait) {
let v: DynTraitRef = (v as *const (), &TRAIT_FOR_UNIT_VTABLE);
call_foo(v);
}
Now suppose that the pointer to the value is behind an indirection. I'll use Box<&self> because it's simple but demonstrates the concept best, but the concept applies to &mut Box<Self> too: they have the same layout. How will we write foo() for impl Trait for dyn Trait?
trait Trait {
fn foo(self: Box<&Self>);
}
impl Trait for () {
fn foo(self: Box<&Self>) {}
}
struct TraitVTable {
foo: fn(Box<*const ()>),
}
static TRAIT_FOR_UNIT_VTABLE: TraitVTable = TraitVTable {
foo: unsafe { std::mem::transmute(<() as Trait>::foo) },
};
type DynTraitRef = (*const (), &'static TraitVTable);
impl Trait for dyn Trait {
fn foo(self: Box<DynTraitRef>) {
let concrete_foo: fn(Box<*const ()>) = self.1.foo;
let data: *const () = self.0;
concrete_foo(data) // We need to wrap `data` in `Box`! Error.
}
}
You may think "then the compiler should just insert a call to Box::new()!" But besides Box not being the only one here (what with Rc, for example?) and we will need some trait to abstract over this behavior, Rust never performs any hard work implicitly. This is a design choice, and an important one (as opposed to e.g. C++, where an innocent-looking statement like auto v1 = v; can allocate and copy 10GB by a copy constructor). Converting a type to dyn Trait and back is done implicitly: the first one by a coercion, the second one when you call a method of the trait. Thus, the only thing that Rust does for that is attaching a VTable pointer in the first case, or discarding it in the second case. Even allowing only references (&&&Self, no need to call a method, just take the address of a temporary) exceeds that. And it can have severe implications in unexpected places, e.g. register allocation.
So, what to do? You can take &mut self or self: Box<Self>. Which one to choose depends on whether you need ownership (use Box) or not (use a reference). And anyway, &mut Box<Self> is not so useful (its only advantage over &mut T is that you can replace the box and not just its contents, but when you do that that's usually a mistake).

Why does a reference to a trait in a generic function have to implement `Sized`?

I have a function that returns a reference to a trait (trait_ref()) and another function that takes a reference to a generic trait implementation (take_trait_ref_generic).
However, it's not possible to pass the reference I get from the first function to the second one. Rustc complains that "the trait std::marker::Sized is not implemented for SomeTrait".
Even though that's true, why does it have to implement Sized? It's a reference anyway.
trait SomeTrait {}
struct TraitImpl;
impl SomeTrait for TraitImpl {}
struct Container {
trait_impl: TraitImpl,
}
impl Container {
fn trait_ref(&self) -> &SomeTrait {
&self.trait_impl
}
}
fn take_trait_ref_generic<T: SomeTrait>(generic_trait_ref: &T) {}
fn main() {
let container = Container { trait_impl: TraitImpl };
/*Not possible*/
take_trait_ref_generic(container.trait_ref());
}
By default, all generic types on functions implicitly have the Sized bound, regardless of how they are used. You need to explicitly opt-out of that requirement using ?Sized:
fn take_trait_ref_generic<T>(generic_trait_ref: &T)
where
T: ?Sized + SomeTrait
{}

Rust Trait object conversion

The following code won't compile due to two instances of this error:
error[E0277]: the trait bound Self: std::marker::Sized is not satisfied
I don't understand why Sized is required in this instance as both &self and &Any are pointers and the operation does not require knowledge of the size of the structure that implements the trait, it only requires knowledge of the pointer itself and the type it is converting from and to, which it will have because &self is generic when implemented inside a trait.
I think this may be an instance of the compiler enforcing unnecessary constraints and I've considered filing an issue with the rust-lang GitHub repo but I figured I should probably see if someone here knows something I don't before I go file an issue.
use std::any::Any;
trait Component: Any {
fn as_any(&self) -> &Any {
self
}
fn as_any_mut(&mut self) -> &mut Any {
self
}
}
The alternative to this is to make as_any() and as_any_mut() required functions for the structs that implement this trait, but for those structures the implementation would always be exactly as displayed here down to each individual character, resulting in several instances of identical boilerplate code.
Dynamically sized types can also implement traits. In particular, when you define an object-safe trait, the compiler also defines a dynamically sized type with the same name as the trait, which lets you use object types such as &Component.
Object types such as &Component or &Any are not just ordinary pointers; they're fat pointers. A fat pointer combines a pointer to the data and another piece of data: for object types, it's a pointer to the vtable; for slices, it's the length of the slice.
When casting from a regular pointer (e.g. a &Button) to an object type, the compiler statically knows which vtable to put in the fat pointer (e.g. Button's vtable for Any). On the other hand, Rust doesn't support casting from an object type to another object type (e.g. from &Component to &Any), because there's not enough data in an object to initialize the new fat pointer. This is why the compiler adds this note to the error message:
= note: required for the cast to the object type `std::any::Any + 'static`
There are two ways to fix this:
Require that all types implementing Component be Sized:
trait Component: Any + Sized {
fn as_any(&self) -> &Any {
self
}
fn as_any_mut(&mut self) -> &mut Any {
self
}
}
This has the consequence that you will not be able to use object types such as &Component or Box<Component> at all.
Make the as_any and as_any_mut methods only available when Self is Sized:
trait Component: Any {
fn as_any(&self) -> &Any
where Self: Sized
{
self
}
fn as_any_mut(&mut self) -> &mut Any
where Self: Sized
{
self
}
}
This way, you can still use object types for the trait, but you will not be able to call as_any and as_any_mut on them.
I found what I consider to be a great solution that didn't require new compiler features.
pub trait Component {
// ...
}
pub trait ComponentAny: Component + Any {
fn as_any(&self) -> &Any;
fn as_any_mut(&mut self) -> &mut Any;
}
impl<T> ComponentAny for T
where T: Component + Any
{
fn as_any(&self) -> &Any {
self
}
fn as_any_mut(&mut self) -> &mut Any {
self
}
}
From here, I just change all my APIs to accept ComponentAny instead of Component. Because Any is automatically implemented for any 'static type, ComponentAny is now automatically implemented for any 'static type that implements Component. Thanks to Is there a way to combine multiple traits in order to define a new trait? for the idea.

What is 'core::kinds::Sized` is not implemented for the type `Self' in rust?

This used to work:
struct Foo<'a, T> {
parent:&'a (Array<T> + 'a)
}
impl<'a, T> Foo<'a, T> { //'
pub fn new<T>(parent:&Array<T>) -> Foo<T> {
return Foo {
parent: parent
};
}
}
trait Array<T> {
fn as_foo(&self) -> Foo<T> {
return Foo::new(self);
}
}
fn main() {
}
Now it errors:
:15:21: 15:25 error: the trait core::kinds::Sized is not implemented for the type Self
:15 return Foo::new(self);
I can kind of guess what's wrong; it's saying that my impl of Foo<'a, T> is for T, not Sized? T, but I'm not trying to store a Sized? element in it; I'm storing a reference to a Sized element in it. That should be a pointer, fixed size.
I don't see what's wrong with what I'm doing, or why it's wrong?
For example, I should (I think...) be able to store a &Array in my Foo, no problem. I can't see any reason this would force my Foo instance to be unsized.
playpen link: http://is.gd/eZSZYv
There's two things going on here: trait objects coercions (the error), and object safety (fixing it).
The error
As suggested by the error message, the difficult part of the code is the Foo::new(self), and this is because pub fn new<T>(parent: &Array<T>) -> ..., that is, self is being coerced to an &Array<T> trait object. I'll simplify the code to:
trait Array {
fn as_foo(&self) {
let _ = self as &Array; // coerce to a trait object
}
}
fn main() {}
which gives the same thing:
<anon>:3:13: 3:27 error: the trait `core::kinds::Sized` is not implemented for the type `Self`
<anon>:3 let _ = self as &Array; // coerce to a trait object
^~~~~~~~~~~~~~
Self is the stand-in name for the type that implements the trait. Unlike most generic parameters, Self is possibly-unsized (?Sized) by default, since RFC 546 and #20341 for the purposes of allowing e.g. impl Array<T> for Array<T> to work by default more often (we'll come to this later).
The variable self has type &Self. If Self is a sized type, then this is a normal reference: a single pointer. If Self is an unsized type (like [T] or a trait), then &Self (&[T] or &Trait) is a slice/trait object: a fat pointer.
The error appears because the only references &T that can be cast to a trait object are when T is sized: Rust doesn't support making fat pointers fatter, only thin pointer → fat pointer is valid. Hence, since the compiler doesn't know that Self will always be Sized (remember, it's special and ?Sized by default) it has to assume the worst: that the coercion is not legal, and so it's disallowed.
Fixing it
It seems logical that the fix we're looking for is to ensure that Self: Sized when we want to do a coercion. The obvious way to do this would be to make Self always Sized, that is, override the default ?Sized bound as follows:
trait Array: Sized {
fn as_foo(&self) {
let _ = self as &Array; // coerce to a trait object
}
}
fn main() {}
Looks good!
Except there's the small point that it doesn't work; but at least it's for a difference reason, we're making progress! Trait objects can only be made out of traits that are "object safe" (i.e. safe to be made into a trait object), and having Sized Self is one of the things that breaks object safety:
<anon>:3:13: 3:17 error: cannot convert to a trait object because trait `Array` is not object-safe [E0038]
<anon>:3 let _ = self as &Array; // coerce to a trait object
^~~~
<anon>:3:13: 3:17 note: the trait cannot require that `Self : Sized`
<anon>:3 let _ = self as &Array; // coerce to a trait object
^~~~
<anon>:3:13: 3:17 note: the trait cannot require that `Self : Sized`
<anon>:3 let _ = self as &Array; // coerce to a trait object
^~~~
(I filed the double printing of the note as #20692.)
Back to the drawing board. There's a few other "easy" possibilities for a solution:
define an extension trait trait ArrayExt: Sized + Array { fn as_foo(&self) { ... } } and implement it for all Sized + Array types
just use a free function fn array_as_foo<A: Array>(x: &A) { ... }
However, these don't necessarily work for every use case, e.g. specific types can't customise the behaviour by overloading the default method. However, fortunately there is a fix!
The Turon Trick
(Named for Aaron Turon, who discovered it.)
Using generalised where clauses we can be highly specific about when Self should implement Sized, restricting it to just the method(s) where it is required, without infecting the rest of the trait:
trait Array {
fn as_foo(&self) where Self: Sized {
let _ = self as &Array; // coerce to a trait object
}
}
fn main() {}
This compiles just fine! By using the where clause like this, the compiler understands that (a) the coercion is legal because Self is Sized so self is a thin pointer, and (b) that the method is illegal to call on a trait object anyway, and so doesn't break object safety. To see it being disallowed, changing the body of as_foo to
let x = self as &Array; // coerce to a trait object
x.as_foo();
gives
<anon>:4:7: 4:15 error: the trait `core::kinds::Sized` is not implemented for the type `Array`
<anon>:4 x.as_foo();
^~~~~~~~
as expected.
Wrapping it all up
Making this change to the original unsimplified code is as simple adding that where clause to the as_foo method:
struct Foo<'a, T> { //'
parent:&'a (Array<T> + 'a)
}
impl<'a, T> Foo<'a, T> {
pub fn new(parent:&Array<T>) -> Foo<T> {
return Foo {
parent: parent
};
}
}
trait Array<T> {
fn as_foo(&self) -> Foo<T> where Self: Sized {
return Foo::new(self);
}
}
fn main() {
}
which compiles without error. (NB. I had to remove the unnecessary <T> in pub fn new<T> because that was causing inference failures.)
(I have some in-progress blog posts that go into trait objects, object safety and the Turon trick, they will appear on /r/rust in the near future: first one.)

Resources