I'm trying to implement common trait for a bunch of types created from binary data (read from a disk). Majority of trait methods could use default implementations and only conversions etc. would be needed to be implemented separately. I would like to use TryFrom<&[u8]> trait for conversions from binary data to my types but I don't know how to express (in the context of trait) that lifetime of &[u8] and lifetimes of values of my types created from it are not related. Here is minimal example of the problem.
use std::convert::TryFrom;
struct Foo;
// Value of Foo can be created from &[u8] but it doesn't borrow anything.
impl TryFrom<&[u8]> for Foo {
type Error = ();
fn try_from(v: &[u8]) -> Result<Self, ()> {
Ok(Foo)
}
}
trait Bar<'a>
where
Self: TryFrom<&'a [u8], Error = ()>, // `&` without an explicit lifetime name cannot be used here
{
fn baz() -> Self {
let vec = Vec::new();
Self::try_from(&vec).unwrap() // ERROR: vec does not live long enough (nothing is borrowed)
}
}
Alternative solution would be to make conversions as trait methods but it would be nicer to use common std traits. Is there a way to achieve this? (Or I could use const generics but I don't want to rely on nightly compiler.)
What you want are "higher ranked trait bounds" (HRTB, or simply hearty boy). They look like this: for<'a> T: 'a. This example just means: "for every possible lifetime 'a, T must ...". In your case:
trait Bar
where
Self: for<'a> TryFrom<&'a [u8], Error = ()>,
You can also specify that requirement as super trait bound directly instead of where clause:
trait Bar: for<'a> TryFrom<&'a [u8], Error = ()> { ... }
And yes, now it just means that all implementors of Bar have to implement TryFrom<&'a [u8], Error = ()> for all possible lifetimes. That's what you want.
Working Playground
Related
I am trying to implement a trait for a struct which in turn has functions that return traits.
I want this, because I do not want to bind the uer to a specific data structure.
However, trying to apply the compiler's correction suggestions, I fell deeper and deeper into a rabbit hole to no avail.
Here's a minimal example of what I'm trying to do:
trait WordsFilter {
fn starting_with(&self, chr: char) -> dyn Iterator<Item = String>;
}
struct WordsContainer {
words: Vec<String>,
}
impl WordsFilter for WordsContainer {
fn starting_with(&self, chr: char) -> dyn Iterator<Item = String>
{
self.words.iter().filter(|word| word.starts_with("a"))
}
}
fn main() {}
Which results in:
error[E0277]: the size for values of type `(dyn Iterator<Item = String> + 'static)` cannot be known at compilation time
--> .\traits.rs:10:40
|
10 | fn starting_with(&self, chr: char) -> dyn Iterator<Item = String>
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `(dyn Iterator<Item = String> + 'static)`
= note: the return type of a function must have a statically known size
error: aborting due to previous error
For more information about this error, try `rustc --explain E0277`.
I tried to apply the compiler's correction's step by step but they were just getting more.
TL;DR Use generics or associated types.
You cannot use bare dyn Trait types. They are Unsized which means that compiler cannot know their size at the compile time. Therefore you can only use them behind some reference (similarly to how you cannot use str or [T] and must use &str and &[T]). The easiest way is to use Box<dyn Trait>, like PitaJ earlier suggested. This requires of course one heap allocation, but allows a very nice API.
In normal functions and methods you can however use impl Trait trick to make compiler infer returned type. This is used like this
fn foo() -> impl Iterator<Item = ()> {
todo!()
}
Calling this function would return some concrete object that implements given trait. This however is currently not possible in Trait definitions. There is active development and hopefully in the future we will be able to use it. However now it's not the case. So following code will not compile!
// XXX: This doesn't currently work!
trait Foo {
fn foo() -> impl Iterator<Item = ()>;
}
There is luckily a simple solution that wouldn't require boxing trait objects. You can define an associated type and require each implementator to specify a concrete type. You can also require that this type must implement the Iterator trait.
trait WordsFilter {
type Iter: Iterator<Item = String>;
fn starting_with(&self, chr: char) -> Self::Iter;
}
This would make each implementation specify one concrete type of returned iterator. If you would like to be able to return multiple different iterators for a single type you can use generics.
trait WordsFilter<I>
where
I: Iterator<Item = String>
{
fn starting_with(&self, chr: char) -> I;
}
There are several things that need to be fixed to get this to work.
I'll first show the workng version then step through what had to change and why.
trait WordsFilter {
fn starting_with(&self, chr: char) -> Box<dyn Iterator<Item = String> + '_>;
}
struct WordsContainer {
words: Vec<String>,
}
impl WordsFilter for WordsContainer {
fn starting_with(&self, chr: char) -> Box<dyn Iterator<Item = String> + '_>
{
Box::new(self.words.iter().filter(|word| word.starts_with("a")).cloned())
}
}
fn main() {}
Boxing the iterator. Rust doesn't allow you to return naked dyn types. The compiler error tells you to Box it. So I did.
Clone after filter. Vec<X>::iter() returns an iterator with Item=&X so in this example we get Item=&String we want Item=String so we need to clone.
Add a lifetime. The problem was that without the lifetime checks I could write this...
let words_container = WordsContainer{...};
let it = words_container.starting_with();
drop(words_container)
it.next()
but it.next() is stepping through the internal vector that is inside words_container - which no longer exists.
Adding the '_ lifetime on the trait is saying the iterator is only valid for as long as the underlying container is. So now the above code would generate a compile error.
TL;RD. I would like to inherit a trait from From like this: pub trait VectorLike<'a, T: 'a + Clone>: From<&'a [T]> {} but in such a way that VectorLike doesn't have a lifetime parameter.
Loner version. I'm writing generic code that supports various vector kinds (e.g. standard Vec and SmallVec from the smallvec crate). In order for my functions to work with different vectors I'm making a trait that encompasses everything that is common to them. It goes like this:
pub trait VectorLike<T: Clone>:
ops::Deref<Target = [T]> +
IntoIterator<Item = T> +
FromIterator<T> +
{
fn pop(&mut self) -> Option<T>;
fn push(&mut self, value: T);
}
Everything above works fine.
However I run into problem when I try to support creating vector from slices like this: Vec::from(&[1, 2, 3][..]). In order to allow this Vec implements From<&'_ [T]> trait. So naturally I'm trying to add it as a parent trait for VectorLike:
pub trait VectorLike<T: Clone>:
// ...
From<&[T]> +
{
// ...
}
... and I get “expected named lifetime parameter” error. Rust compiler suggests to fix the code like this:
pub trait VectorLike<'a, T: 'a + Clone>:
// ...
From<&'a [T]> +
{
// ...
}
Now I have to specify explicit lifetime wherever VectorLike is used. This seems completely unnecessary:
This lifetime contains no valuable information: the vector does not inherit the lifetime, it copies all elements.
Lifetime specification of this sort is not required when Vec is used directly, e.g. this works: fn make_vector(elements: &[i32]) -> Vec<i32> { Vec::<i32>::from(elements) }.
I can workaround this limitation by adding a new function instead of implementing From: pub trait VectorLike<T: Clone>: /* ... */ { fn from_slice(s: &[T]) -> Self; }. This way my trait is functionally equivalent and can be used without lifetime specifiers.
So is there a way to remove the superfluous lifetime specifier here?
P.S. For now I'm using the “new function” option as a workaround, but it has drawbacks. For example, I find it confusing when the function has a different name, but giving it the same name leads to ambiguity which needs to be resolved with verbose constructs like this: <SmallVec::<[T; 8]> as From<&[T]>>::from(slice).
Every reference in Rust must have an associated lifetime.
The reason you usually don't have to write them out is because the compiler is very good at lifetime elision.
Whenever this doesn't work, you need to carefully consider where the lifetime you need is actually coming form.
In this case, the constraint you're trying to impose can be though of as
"for any lifetime 'a that you give me, VectorLike<T> implements From<&'a [T]>".
This is a higher-ranked trait bound that can be expressed using for<'a> syntax:
pub trait VectorLike<T: Clone>:
...
for<'a> From<&'a [T]>
{
...
}
Playground
I want trait implementations in Rust to be able to return arbitrary iterators (of specific item type) that may reference the original object with a lifetime 'a without having to explicitly mention 'a in the trait generics and everywhere where the trait is used or otherwise introducing significant trait bound bloat to user code. The only simple way I've figured to do this is that the trait has to be implemented for &'a MyStruct instead of MyStruct (this approach is used in some places in the standard library), but the significant drawback is that in generic code wrappers cannot “own” implementations of the trait (MyStruct) without exposing the lifetime in trait bounds all over the code. So nothing gained when ownership is needed.
Another way I figured out that should work (just done the simple test below so far) is to use higher-ranked lifetime bounds on a “base trait” that would implement the iterator-generation functions. In the code below Foo is the main trait, FooInterfaceGen is the iterator-generator trait that has its lifetime “hidden” through for <'a> when assumed as a super-trait of Foo. The FooInterface generated by FooInterfaceGen would be the trait for an appropriate type of iterator when modified to that application of the idea. However, it seems impossible to make additional trait bounds on the specific implementation FooInterfaceGen::Interface. The code below works, but when you uncomment the Asdf trait bound in the function footest, the compiler complains that
the trait `for<'a> Asdf` is not implemented for `<_ as FooInterfaceGen<'a>>::Interface
But I have implemented Asdf! It's as if the compiler is ignoring the 'a in the expression <T as FooInterfaceGen<'a>> and just applying for<'a> to the right-hand-side. Any ideas if this is a compiler bug, a known restriction, or of any ways around it?
trait FooInterface<'a> {
fn foo(&self) -> u32;
}
trait FooInterfaceGen<'a> {
type Interface : FooInterface<'a>;
fn gen(&'a self) -> Self::Interface;
}
trait Foo : for<'a> FooInterfaceGen<'a> { }
struct S2;
struct S1(S2);
impl<'a> FooInterfaceGen<'a> for S1 {
type Interface = &'a S2;
fn gen(&'a self) -> Self::Interface { &self.0 }
}
impl Foo for S1 { }
impl<'a> FooInterface<'a> for &'a S2 {
fn foo(&self) -> u32 { 42 }
}
trait Asdf {}
impl<'a> Asdf for &'a S2 {}
fn footest<T : Foo>(a : &T) -> u32
/* where for<'a> <T as FooInterfaceGen<'a>>::Interface : Asdf */ {
a.gen().foo()
}
fn main() {
let q = S1(S2);
println!("{}", footest(&q));
}
(Regarding some alternative implementations, maybe there's a technical reason for it, but otherwise I really don't understand the reason behind the significant trait bound bloat that Rust code easily introduces. Assuming a trait should in any reasonable situation automatically assume all the trait bound as well, also in generic code, not just specific code, without having to copy-paste an increasing number of where-clauses all over the code.)
The error seems to be a known compiler bug: https://github.com/rust-lang/rust/issues/89196
I've got this code snippet (playground):
struct TeddyBear {
fluffiness: u8,
}
trait Scruffy {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy>;
}
impl Scruffy for TeddyBear {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy> {
// do something about the TeddyBear's fluffiness
}
}
It doesn't compile. The error is:
the trait Scruffy cannot be made into an object
, along with the hint:
because method scruff_up's self parameter cannot be dispatched on.
I checked the "E0038" error description, but I haven't been able to figure out which category my error falls into.
I also read the "object-safety" entry in "The Rust Reference", and I believe this matches the "All associated functions must either be dispatchable from a trait object", but I'm not sure, partly because I'm not sure what "receiver" means in that context.
Can you please clarify for me what's the problem with this code and why it doesn't work?
The problem is when you pass it in as a reference, because the inner type may not be well-sized (e.g. a trait object, like if you passed in a Box<Fluffy>) the compiler doesn't have enough information to figure out how to call methods on it. If you restrict it to sized objects (like your TeddyBear) it should compile
trait Scruffy {
fn scruff_up(self: &mut Box<Self>) -> Box<dyn Scruffy> where Self: Sized;
}
A receiver is the self (&self, &mut self, self: &mut Box<Self> and so on).
Note that the list you cited from the reference lists both Box<Self> and &mut Self, but does not list &mut Box<Self> nor it says that combinations of these types are allowed.
This is, indeed, forbidden. As for the why, it is a little more complex.
In order for a type to be a valid receiver, it needs to hold the following condition:
Given any type Self that implements Trait and the receiver type Receiver, the receiver type should implement DispatchFromDyn<dyn Trait> for itself with all Self occurrences of Self replaced with dyn Trait.
For instance:
&self (has the type &Self) has to implement DispatchFromDyn<&dyn Trait>, which it does.
Box<Self> has to implement DispatchFromDyn<Box<dyn Trait>>, which it does.
But in order for &mut Box<Self> to be an object-safe receiver, it would need to impl DispatchFromDyn<&mut Box<dyn Trait>>. What you want is kind of blanket implementation DispatchFromDyn<&mut T> for &mut U where U: DispatchFromDyn<T>.
This impl will never exist. Because it is unsound (even ignoring coherence problems).
As explained in the code in rustc that calculates this:
The only case where the receiver is not dispatchable, but is still a valid receiver type (just not object-safe), is when there is more than one level of pointer indirection. E.g., self: &&Self, self: &Rc<Self>, self: Box<Box<Self>>. In these cases, there is no way, or at least no inexpensive way, to coerce the receiver from the version where Self = dyn Trait to the version where Self = T, where T is the unknown erased type contained by the trait object, because the object that needs to be coerced is behind a pointer.
The problem is inherent to how Rust handles dyn Trait.
dyn Trait is a fat pointer: it is actually two words sized. One is a pointer to the data, and the other is a pointer to the vtable.
When you call a method on dyn Trait, the compiler looks up in the vtable, find the method for the concrete type (which is unknown at compilation time, but known at runtime), and calls it.
This all may be very abstract without an example:
trait Trait {
fn foo(&self);
}
impl Trait for () {
fn foo(&self) {}
}
fn call_foo(v: &dyn Trait) {
v.foo();
}
fn create_dyn_trait(v: &impl Trait) {
let v: &dyn Trait = v;
call_foo(v);
}
The compiler generates code like:
trait Trait {
fn foo(&self);
}
impl Trait for () {
fn foo(&self) {}
}
struct TraitVTable {
foo: fn(*const ()),
}
static TRAIT_FOR_UNIT_VTABLE: TraitVTable = TraitVTable {
foo: unsafe { std::mem::transmute(<() as Trait>::foo) },
};
type DynTraitRef = (*const (), &'static TraitVTable);
impl Trait for dyn Trait {
fn foo(self: DynTraitRef) {
(self.1.foo)(self.0)
}
}
fn call_foo(v: DynTraitRef) {
v.foo();
}
fn create_dyn_trait(v: &impl Trait) {
let v: DynTraitRef = (v as *const (), &TRAIT_FOR_UNIT_VTABLE);
call_foo(v);
}
Now suppose that the pointer to the value is behind an indirection. I'll use Box<&self> because it's simple but demonstrates the concept best, but the concept applies to &mut Box<Self> too: they have the same layout. How will we write foo() for impl Trait for dyn Trait?
trait Trait {
fn foo(self: Box<&Self>);
}
impl Trait for () {
fn foo(self: Box<&Self>) {}
}
struct TraitVTable {
foo: fn(Box<*const ()>),
}
static TRAIT_FOR_UNIT_VTABLE: TraitVTable = TraitVTable {
foo: unsafe { std::mem::transmute(<() as Trait>::foo) },
};
type DynTraitRef = (*const (), &'static TraitVTable);
impl Trait for dyn Trait {
fn foo(self: Box<DynTraitRef>) {
let concrete_foo: fn(Box<*const ()>) = self.1.foo;
let data: *const () = self.0;
concrete_foo(data) // We need to wrap `data` in `Box`! Error.
}
}
You may think "then the compiler should just insert a call to Box::new()!" But besides Box not being the only one here (what with Rc, for example?) and we will need some trait to abstract over this behavior, Rust never performs any hard work implicitly. This is a design choice, and an important one (as opposed to e.g. C++, where an innocent-looking statement like auto v1 = v; can allocate and copy 10GB by a copy constructor). Converting a type to dyn Trait and back is done implicitly: the first one by a coercion, the second one when you call a method of the trait. Thus, the only thing that Rust does for that is attaching a VTable pointer in the first case, or discarding it in the second case. Even allowing only references (&&&Self, no need to call a method, just take the address of a temporary) exceeds that. And it can have severe implications in unexpected places, e.g. register allocation.
So, what to do? You can take &mut self or self: Box<Self>. Which one to choose depends on whether you need ownership (use Box) or not (use a reference). And anyway, &mut Box<Self> is not so useful (its only advantage over &mut T is that you can replace the box and not just its contents, but when you do that that's usually a mistake).
I'm trying to create a trait that captures the iter function in slice as well as VecDeque, BTreeMap and HashMap. I'd like the implementer of this trait to be able to specify and implement their own iterator type, but it looks like this iterator type must have a lifetime argument, and that cannot be given as an associated type.
In more detail, here's what I wish was possible in Rust:
trait RefIterable<T>
where for<'a> (T: 'a) => (Self::Iter<'a>: Iterator<Item = &'a T>)
{
type Iter; // Has kind (lifetime -> type)
fn refs<'a>(&'a self) -> Self::Iter<'a>
}
If this was possible, the implementation could look like this
impl RefIterable<T> for Vec<T> {
type Iter<'a> = std::slice::Iter<'a, T>; // This is not valid Rust code.
fn refs<'a>(&'a self) -> std::slice::Iter<'a, T> {
self.as_slice().iter()
}
}
I'm still relatively new to Rust, so I'm asking if there's already a way to do this that I'm not aware of, or if there's a nice workaround for this situation. I'd imagine that this situation is not very rare.
(Using Box<dyn 'a + Iterator<Item = &'a T>> is my current workaround, but that prevents some optimization from happening.)
Edit:
EvilTak's answer is probably the best thing we can do right now. The ability to combine all possible lifetimes together with the condition T: 'a into one unparametrized trait seems to be unsupported by Rust as of today.
Add the lifetime parameter to the trait instead, which allows you to use it in the associated type Iter's bound:
trait RefIterable<'a> {
type Item: 'a;
type Iter: Iterator<Item = &'a Self::Item>; // Has kind (lifetime -> type)
fn refs(&'a self) -> Self::Iter;
}
The Item: 'a bound is required to let the compiler know that the references (&'a Self::Item) do not outlive the type (Self::Item).
I have modified RefIterable to make it follow Iterator's convention of using an associated type to specify the type of the items that are iterated over for the same reason as the one behind Iterator's usage of an associated type.
Implementations are pretty straightforward:
impl<'a, T: 'a> RefIterable<'a> for Vec<T> {
type Item = T;
type Iter = std::slice::Iter<'a, T>;
fn refs(&'a self) -> std::slice::Iter<'a, T> {
self.as_slice().iter()
}
}
Playground