I want trait implementations in Rust to be able to return arbitrary iterators (of specific item type) that may reference the original object with a lifetime 'a without having to explicitly mention 'a in the trait generics and everywhere where the trait is used or otherwise introducing significant trait bound bloat to user code. The only simple way I've figured to do this is that the trait has to be implemented for &'a MyStruct instead of MyStruct (this approach is used in some places in the standard library), but the significant drawback is that in generic code wrappers cannot “own” implementations of the trait (MyStruct) without exposing the lifetime in trait bounds all over the code. So nothing gained when ownership is needed.
Another way I figured out that should work (just done the simple test below so far) is to use higher-ranked lifetime bounds on a “base trait” that would implement the iterator-generation functions. In the code below Foo is the main trait, FooInterfaceGen is the iterator-generator trait that has its lifetime “hidden” through for <'a> when assumed as a super-trait of Foo. The FooInterface generated by FooInterfaceGen would be the trait for an appropriate type of iterator when modified to that application of the idea. However, it seems impossible to make additional trait bounds on the specific implementation FooInterfaceGen::Interface. The code below works, but when you uncomment the Asdf trait bound in the function footest, the compiler complains that
the trait `for<'a> Asdf` is not implemented for `<_ as FooInterfaceGen<'a>>::Interface
But I have implemented Asdf! It's as if the compiler is ignoring the 'a in the expression <T as FooInterfaceGen<'a>> and just applying for<'a> to the right-hand-side. Any ideas if this is a compiler bug, a known restriction, or of any ways around it?
trait FooInterface<'a> {
fn foo(&self) -> u32;
}
trait FooInterfaceGen<'a> {
type Interface : FooInterface<'a>;
fn gen(&'a self) -> Self::Interface;
}
trait Foo : for<'a> FooInterfaceGen<'a> { }
struct S2;
struct S1(S2);
impl<'a> FooInterfaceGen<'a> for S1 {
type Interface = &'a S2;
fn gen(&'a self) -> Self::Interface { &self.0 }
}
impl Foo for S1 { }
impl<'a> FooInterface<'a> for &'a S2 {
fn foo(&self) -> u32 { 42 }
}
trait Asdf {}
impl<'a> Asdf for &'a S2 {}
fn footest<T : Foo>(a : &T) -> u32
/* where for<'a> <T as FooInterfaceGen<'a>>::Interface : Asdf */ {
a.gen().foo()
}
fn main() {
let q = S1(S2);
println!("{}", footest(&q));
}
(Regarding some alternative implementations, maybe there's a technical reason for it, but otherwise I really don't understand the reason behind the significant trait bound bloat that Rust code easily introduces. Assuming a trait should in any reasonable situation automatically assume all the trait bound as well, also in generic code, not just specific code, without having to copy-paste an increasing number of where-clauses all over the code.)
The error seems to be a known compiler bug: https://github.com/rust-lang/rust/issues/89196
Related
TL;RD. I would like to inherit a trait from From like this: pub trait VectorLike<'a, T: 'a + Clone>: From<&'a [T]> {} but in such a way that VectorLike doesn't have a lifetime parameter.
Loner version. I'm writing generic code that supports various vector kinds (e.g. standard Vec and SmallVec from the smallvec crate). In order for my functions to work with different vectors I'm making a trait that encompasses everything that is common to them. It goes like this:
pub trait VectorLike<T: Clone>:
ops::Deref<Target = [T]> +
IntoIterator<Item = T> +
FromIterator<T> +
{
fn pop(&mut self) -> Option<T>;
fn push(&mut self, value: T);
}
Everything above works fine.
However I run into problem when I try to support creating vector from slices like this: Vec::from(&[1, 2, 3][..]). In order to allow this Vec implements From<&'_ [T]> trait. So naturally I'm trying to add it as a parent trait for VectorLike:
pub trait VectorLike<T: Clone>:
// ...
From<&[T]> +
{
// ...
}
... and I get “expected named lifetime parameter” error. Rust compiler suggests to fix the code like this:
pub trait VectorLike<'a, T: 'a + Clone>:
// ...
From<&'a [T]> +
{
// ...
}
Now I have to specify explicit lifetime wherever VectorLike is used. This seems completely unnecessary:
This lifetime contains no valuable information: the vector does not inherit the lifetime, it copies all elements.
Lifetime specification of this sort is not required when Vec is used directly, e.g. this works: fn make_vector(elements: &[i32]) -> Vec<i32> { Vec::<i32>::from(elements) }.
I can workaround this limitation by adding a new function instead of implementing From: pub trait VectorLike<T: Clone>: /* ... */ { fn from_slice(s: &[T]) -> Self; }. This way my trait is functionally equivalent and can be used without lifetime specifiers.
So is there a way to remove the superfluous lifetime specifier here?
P.S. For now I'm using the “new function” option as a workaround, but it has drawbacks. For example, I find it confusing when the function has a different name, but giving it the same name leads to ambiguity which needs to be resolved with verbose constructs like this: <SmallVec::<[T; 8]> as From<&[T]>>::from(slice).
Every reference in Rust must have an associated lifetime.
The reason you usually don't have to write them out is because the compiler is very good at lifetime elision.
Whenever this doesn't work, you need to carefully consider where the lifetime you need is actually coming form.
In this case, the constraint you're trying to impose can be though of as
"for any lifetime 'a that you give me, VectorLike<T> implements From<&'a [T]>".
This is a higher-ranked trait bound that can be expressed using for<'a> syntax:
pub trait VectorLike<T: Clone>:
...
for<'a> From<&'a [T]>
{
...
}
Playground
I've got this code snippet (playground):
struct TeddyBear {
fluffiness: u8,
}
trait Scruffy {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy>;
}
impl Scruffy for TeddyBear {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy> {
// do something about the TeddyBear's fluffiness
}
}
It doesn't compile. The error is:
the trait Scruffy cannot be made into an object
, along with the hint:
because method scruff_up's self parameter cannot be dispatched on.
I checked the "E0038" error description, but I haven't been able to figure out which category my error falls into.
I also read the "object-safety" entry in "The Rust Reference", and I believe this matches the "All associated functions must either be dispatchable from a trait object", but I'm not sure, partly because I'm not sure what "receiver" means in that context.
Can you please clarify for me what's the problem with this code and why it doesn't work?
The problem is when you pass it in as a reference, because the inner type may not be well-sized (e.g. a trait object, like if you passed in a Box<Fluffy>) the compiler doesn't have enough information to figure out how to call methods on it. If you restrict it to sized objects (like your TeddyBear) it should compile
trait Scruffy {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy> where Self: Sized;
}
A receiver is the self (&self, &mut self, self: &mut Box<Self> and so on).
Note that the list you cited from the reference lists both Box<Self> and &mut Self, but does not list &mut Box<Self> nor it says that combinations of these types are allowed.
This is, indeed, forbidden. As for the why, it is a little more complex.
In order for a type to be a valid receiver, it needs to hold the following condition:
Given any type Self that implements Trait and the receiver type Receiver, the receiver type should implement DispatchFromDyn<dyn Trait> for itself with all Self occurrences of Self replaced with dyn Trait.
For instance:
&self (has the type &Self) has to implement DispatchFromDyn<&dyn Trait>, which it does.
Box<Self> has to implement DispatchFromDyn<Box<dyn Trait>>, which it does.
But in order for &mut Box<Self> to be an object-safe receiver, it would need to impl DispatchFromDyn<&mut Box<dyn Trait>>. What you want is kind of blanket implementation DispatchFromDyn<&mut T> for &mut U where U: DispatchFromDyn<T>.
This impl will never exist. Because it is unsound (even ignoring coherence problems).
As explained in the code in rustc that calculates this:
The only case where the receiver is not dispatchable, but is still a valid receiver type (just not object-safe), is when there is more than one level of pointer indirection. E.g., self: &&Self, self: &Rc<Self>, self: Box<Box<Self>>. In these cases, there is no way, or at least no inexpensive way, to coerce the receiver from the version where Self = dyn Trait to the version where Self = T, where T is the unknown erased type contained by the trait object, because the object that needs to be coerced is behind a pointer.
The problem is inherent to how Rust handles dyn Trait.
dyn Trait is a fat pointer: it is actually two words sized. One is a pointer to the data, and the other is a pointer to the vtable.
When you call a method on dyn Trait, the compiler looks up in the vtable, find the method for the concrete type (which is unknown at compilation time, but known at runtime), and calls it.
This all may be very abstract without an example:
trait Trait {
fn foo(&self);
}
impl Trait for () {
fn foo(&self) {}
}
fn call_foo(v: &dyn Trait) {
v.foo();
}
fn create_dyn_trait(v: &impl Trait) {
let v: &dyn Trait = v;
call_foo(v);
}
The compiler generates code like:
trait Trait {
fn foo(&self);
}
impl Trait for () {
fn foo(&self) {}
}
struct TraitVTable {
foo: fn(*const ()),
}
static TRAIT_FOR_UNIT_VTABLE: TraitVTable = TraitVTable {
foo: unsafe { std::mem::transmute(<() as Trait>::foo) },
};
type DynTraitRef = (*const (), &'static TraitVTable);
impl Trait for dyn Trait {
fn foo(self: DynTraitRef) {
(self.1.foo)(self.0)
}
}
fn call_foo(v: DynTraitRef) {
v.foo();
}
fn create_dyn_trait(v: &impl Trait) {
let v: DynTraitRef = (v as *const (), &TRAIT_FOR_UNIT_VTABLE);
call_foo(v);
}
Now suppose that the pointer to the value is behind an indirection. I'll use Box<&self> because it's simple but demonstrates the concept best, but the concept applies to &mut Box<Self> too: they have the same layout. How will we write foo() for impl Trait for dyn Trait?
trait Trait {
fn foo(self: Box<&Self>);
}
impl Trait for () {
fn foo(self: Box<&Self>) {}
}
struct TraitVTable {
foo: fn(Box<*const ()>),
}
static TRAIT_FOR_UNIT_VTABLE: TraitVTable = TraitVTable {
foo: unsafe { std::mem::transmute(<() as Trait>::foo) },
};
type DynTraitRef = (*const (), &'static TraitVTable);
impl Trait for dyn Trait {
fn foo(self: Box<DynTraitRef>) {
let concrete_foo: fn(Box<*const ()>) = self.1.foo;
let data: *const () = self.0;
concrete_foo(data) // We need to wrap `data` in `Box`! Error.
}
}
You may think "then the compiler should just insert a call to Box::new()!" But besides Box not being the only one here (what with Rc, for example?) and we will need some trait to abstract over this behavior, Rust never performs any hard work implicitly. This is a design choice, and an important one (as opposed to e.g. C++, where an innocent-looking statement like auto v1 = v; can allocate and copy 10GB by a copy constructor). Converting a type to dyn Trait and back is done implicitly: the first one by a coercion, the second one when you call a method of the trait. Thus, the only thing that Rust does for that is attaching a VTable pointer in the first case, or discarding it in the second case. Even allowing only references (&&&Self, no need to call a method, just take the address of a temporary) exceeds that. And it can have severe implications in unexpected places, e.g. register allocation.
So, what to do? You can take &mut self or self: Box<Self>. Which one to choose depends on whether you need ownership (use Box) or not (use a reference). And anyway, &mut Box<Self> is not so useful (its only advantage over &mut T is that you can replace the box and not just its contents, but when you do that that's usually a mistake).
I'm trying to implement common trait for a bunch of types created from binary data (read from a disk). Majority of trait methods could use default implementations and only conversions etc. would be needed to be implemented separately. I would like to use TryFrom<&[u8]> trait for conversions from binary data to my types but I don't know how to express (in the context of trait) that lifetime of &[u8] and lifetimes of values of my types created from it are not related. Here is minimal example of the problem.
use std::convert::TryFrom;
struct Foo;
// Value of Foo can be created from &[u8] but it doesn't borrow anything.
impl TryFrom<&[u8]> for Foo {
type Error = ();
fn try_from(v: &[u8]) -> Result<Self, ()> {
Ok(Foo)
}
}
trait Bar<'a>
where
Self: TryFrom<&'a [u8], Error = ()>, // `&` without an explicit lifetime name cannot be used here
{
fn baz() -> Self {
let vec = Vec::new();
Self::try_from(&vec).unwrap() // ERROR: vec does not live long enough (nothing is borrowed)
}
}
Alternative solution would be to make conversions as trait methods but it would be nicer to use common std traits. Is there a way to achieve this? (Or I could use const generics but I don't want to rely on nightly compiler.)
What you want are "higher ranked trait bounds" (HRTB, or simply hearty boy). They look like this: for<'a> T: 'a. This example just means: "for every possible lifetime 'a, T must ...". In your case:
trait Bar
where
Self: for<'a> TryFrom<&'a [u8], Error = ()>,
You can also specify that requirement as super trait bound directly instead of where clause:
trait Bar: for<'a> TryFrom<&'a [u8], Error = ()> { ... }
And yes, now it just means that all implementors of Bar have to implement TryFrom<&'a [u8], Error = ()> for all possible lifetimes. That's what you want.
Working Playground
I have a trait for which I want to require that implementing types are iterable by borrow. I have managed to do this using a for<'x> higher-ranked trait bound (HRTB) on &'x Self.
However, I also want to require that the IntoIter associated type implements ExactSizeIterator, but every way I try to describe this in the type system causes compilation issues which seem to stem from the use of the HRTB (which I am not fully confident I'm using correctly).
Here is the (simplified) code:
struct Thing<'thing>(&'thing ());
trait Trait<'thing>
where
for<'x> &'x Self: IntoIterator<Item = &'x Thing<'thing>>,
// Compiles fine until uncommenting this line:
//for<'x> <&'x Self as IntoIterator>::IntoIter: ExactSizeIterator
{ }
struct Bucket<'things> {
things: Vec<Thing<'things>>,
}
struct BucketRef<'a, 'things: 'a> {
bucket: &'a Bucket<'things>,
}
impl<'x, 'a, 'things: 'a> IntoIterator for &'x BucketRef<'a, 'things> {
type Item = &'x Thing<'things>;
type IntoIter = std::slice::Iter<'x, Thing<'things>>;
fn into_iter(self) -> Self::IntoIter {
self.bucket.things.iter()
}
}
impl<'a, 'things: 'a> Trait<'things> for BucketRef<'a, 'things> { }
fn foo<'a, 'things>(anchor: &BucketRef<'a, 'things>) {
println!("{}", ExactSizeIterator::len(&anchor.into_iter()));
}
As written this compiles fine, but when I try to further restrict the bounds on Trait via the commented line, I get the following compiler error:
error[E0277]: the trait bound `for<'x> <&'x anchor::BucketRef<'a, 'things> as std::iter::IntoIterator>::IntoIter: std::iter::ExactSizeIterator` is not satisfied
It seems to my not-a-compiler-writer mind that given rustc appears able to determine inside the function foo that all instances of &BucketRef are ExactSizeIterators, that it should be able to do similarly for the trait bound, but this is not borne out in reality.
Can anyone explain to me why this doesn't work and if there is a better way to express either the bound itself or the intent behind the bound?
active toolchain
----------------
stable-x86_64-unknown-linux-gnu (default)
rustc 1.43.0 (4fb7144ed 2020-04-20)
this maybe related to this compiler bug and will be fixed soon: https://github.com/rust-lang/rust/issues/56556
https://github.com/rust-lang/rust/pull/85499
workarounds:
https://github.com/Lucretiel/joinery/blob/master/src/join.rs#L174-L188
While trying to understand the Any trait better, I saw that it has an impl block for the trait itself. I don't understand the purpose of this construct, or even if it has a specific name.
I made a little experiment with both a "normal" trait method and a method defined in the impl block:
trait Foo {
fn foo_in_trait(&self) {
println!("in foo")
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("in impl")
}
}
impl Foo for u8 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let y = &42u8 as &dyn Foo;
y.foo_in_trait();
y.foo_in_impl(); // May cause an error, see below
}
Editor's note
In versions of Rust up to and including Rust 1.15.0, the line
y.foo_in_impl() causes the error:
error: borrowed value does not live long enough
--> src/main.rs:20:14
|
20 | let y = &42u8 as &Foo;
| ^^^^ does not live long enough
...
23 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
This error is no longer present in subsequent versions, but the
concepts explained in the answers are still valid.
From this limited experiment, it seems like methods defined in the impl block are more restrictive than methods defined in the trait block. It's likely that there's something extra that doing it this way unlocks, but I just don't know what it is yet! ^_^
The sections from The Rust Programming Language on traits and trait objects don't make any mention of this. Searching the Rust source itself, it seems like only Any and Error use this particular feature. I've not seen this used in the handful of crates where I have looked at the source code.
When you define a trait named Foo that can be made into an object, Rust also defines a trait object type named dyn Foo. In older versions of Rust, this type was only called Foo (see What does "dyn" mean in a type?). For backwards compatibility with these older versions, Foo still works to name the trait object type, although dyn syntax should be used for new code.
Trait objects have a lifetime parameter that designates the shortest of the implementor's lifetime parameters. To specify that lifetime, you write the type as dyn Foo + 'a.
When you write impl dyn Foo { (or just impl Foo { using the old syntax), you are not specifying that lifetime parameter, and it defaults to 'static. This note from the compiler on the y.foo_in_impl(); statement hints at that:
note: borrowed value must be valid for the static lifetime...
All we have to do to make this more permissive is to write a generic impl over any lifetime:
impl<'a> dyn Foo + 'a {
fn foo_in_impl(&self) { println!("in impl") }
}
Now, notice that the self argument on foo_in_impl is a borrowed pointer, which has a lifetime parameter of its own. The type of self, in its full form, looks like &'b (dyn Foo + 'a) (the parentheses are required due to operator precedence). A Box<u8> owns its u8 – it doesn't borrow anything –, so you can create a &(dyn Foo + 'static) out of it. On the other hand, &42u8 creates a &'b (dyn Foo + 'a) where 'a is not 'static, because 42u8 is put in a hidden variable on the stack, and the trait object borrows this variable. (That doesn't really make sense, though; u8 doesn't borrow anything, so its Foo implementation should always be compatible with dyn Foo + 'static... the fact that 42u8 is borrowed from the stack should affect 'b, not 'a.)
Another thing to note is that trait methods are polymorphic, even when they have a default implementation and they're not overridden, while inherent methods on a trait objects are monomorphic (there's only one function, no matter what's behind the trait). For example:
use std::any::type_name;
trait Foo {
fn foo_in_trait(&self)
where
Self: 'static,
{
println!("{}", type_name::<Self>());
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("{}", type_name::<Self>());
}
}
impl Foo for u8 {}
impl Foo for u16 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let x = Box::new(42u16) as Box<Foo>;
x.foo_in_trait();
x.foo_in_impl();
}
Sample output:
u8
dyn playground::Foo
u16
dyn playground::Foo
In the trait method, we get the type name of the underlying type (here, u8 or u16), so we can conclude that the type of &self will vary from one implementer to the other (it'll be &u8 for the u8 implementer and &u16 for the u16 implementer – not a trait object). However, in the inherent method, we get the type name of dyn Foo (+ 'static), so we can conclude that the type of &self is always &dyn Foo (a trait object).
I suspect that the reason is very simple: may be overridden or not?
A method implemented in a trait block can be overridden by implementors of the trait, it just provides a default.
On the other hand, a method implemented in an impl block cannot be overridden.
If this reasoning is right, then the error you get for y.foo_in_impl() is just a lack of polish: it should have worked. See Francis Gagné's more complete answer on the interaction with lifetimes.