I have this code:
fn main() {
let p = Person;
let r = &p as &dyn Eatable;
Consumer::consume(r);
// Compile error
Consumer::consume_generic(r);
}
trait Eatable {}
struct Person;
impl Eatable for Person {}
struct Consumer;
impl Consumer {
fn consume(eatable: &dyn Eatable) {}
fn consume_generic<T: Eatable>(eatable: &T) {}
}
Error:
the size for values of type dyn Eatable cannot be known at
compilation time
I think it is strange. I have a method that literally takes a dyn Eatable and compiles fine, so that method knows somehow the size of Eatable. The generic method (consume_generic) will properly compile down for every used type for performance and the consume method will not.
So a few questions arise: why the compiler error? Are there things inside the body of the methods in which I can do something which I can not do in the other method? When should I prefer the one over the other?
Sidenote: I asked this question for the language Swift as well: Differences generic protocol type parameter vs direct protocol type. In Swift I get the same compile error but the underlying error is different: protocols/traits do not conform to themselves (because Swift protocols can holds initializers, static things etc. which makes it harder to generically reference them). I also tried it in Java, I believe the generic type is erased and it makes absolutely no difference.
The problem is not with the functions themselves, but with the trait bounds on types.
Every generic types in Rust has an implicit Sized bound: since this is correct in the majority of cases, it was decided not to force the developer to write this out every time. But, if you are using this type only behind some kind of reference, as you do here, you may want to lift this restriction by specifying T: ?Sized. If you add this, your code will compile fine:
impl Consumer {
fn consume(eatable: &dyn Eatable) {}
fn consume_generic<T: Eatable + ?Sized>(eatable: &T) {}
}
Playground as a proof
As for the other questions, the main difference is in static vs dynamic dispatch.
When you use the generic function (or the semantically equivalent impl Trait syntax), the function calls are dispatched statically. That is, for every type of argument you pass to the function, compiler generates the definition independently of others. This will likely result in more optimized code in most cases, but the drawbacks are possibly larger binary size and some limitations in API (e.g. you can't easily create a heterogeneous collection this way).
When you use dyn Trait syntax, you opt in for dynamic dispatch. The necessary data will be stored into the table attached to trait object, and the correct implementation for every trait method will be chosen at runtime. The consumer, however, needs to be compiled only once. This is usually slower, both due to the indirection and to the fact that individual optimizations are impossible, but more flexible.
As for the recommendations (note that this is an opinion, not the fact) - I'd say it's better to stick to generics whenever possible and only change it to trait objects if the goal is impossible to achieve otherwise.
Related
In the trait Extendable below, I'd like to make app generic where it currently uses the concrete type i32.
At first blush, you'd think to use a generic type but doing so while keeping Extendable object safe isn't easy.
trait Isoextender {
type Input;
type Output;
fn forward(&self, v: Self::Input) -> Self::Output;
fn backward(&self, v: Self::Output) -> Self::Input;
}
trait Extendable {
type Item;
fn app(
&self,
v: &dyn Isoextender<Input = Self::Item, Output = i32>,
) -> Box<dyn Extendable<Item = i32>>;
}
There's a (really neat) type erasure trick (link) I can use with std::Any but then I'm losing type information.
This is one of those things that feels like it should be possible from my understanding of how Rust works but which simply might not be. I do see there's an issue with size. Clearly rust needs to know the size of the associated type Item, but I don't see how to solve with with references/pointers in a way that's both object safe and keeps the type information.
Is it the case that:
I am missing something and it's actually possible?
This is not possible today, but it may become possible in the future?
This is not likely to be possible ever?
Update:
So I suppose a part of the issue here is that I feel like the following is a generic function that has only 1 possible implementation. You should only have to compile this once.
fn pointer_identity<T>(v: &T) -> &T {
v
}
It feels like I should be able to put this function into a vtable, but still somehow express that it is generic. There are a whole class of easily identifiable functions that effectively act this way. Interactions between trait objects often act this way. It's all pointers to functions where the types don't matter except to be carried forward.
I found some document ion that may answer my question.
Currently, the compiler has two concepts surrounding Generic code, only one of which seems to be surfaced in the type system (link).
It sounds like this may be possible some day, but not currently expressible with Rust.
Monomorphization
The compiler stamps out a different copy of the code of a generic function for each concrete type needed.
Polymorphization
In addition to MIR optimizations, rustc attempts to determine when fewer copies of functions are necessary and avoid making those copies - known as "polymorphization".
As a result of polymorphization, items collected during monomorphization cannot be assumed to be monomorphic.
It is intended that polymorphization be extended to more advanced cases, such as where only the size/alignment of a generic parameter are required.
A library I'm working on stores various data structures in a graph-like manner.
I'd like to let users store metadata ("annotations") in nodes, so they can retrieve them later. Currently, they have to create their own data structure which mirrors the library's, which is very inconvenient.
I'm placing very little constraints on what an annotation can be, because I do not know what the users will want to store in the future.
The rest of this question is about my current attempt at solving this use case, but I'm open to completely different implementations as well.
User annotations are represented with a trait:
pub trait Annotation {
fn some_important_method(&self)
}
This trait contains a few methods (all on &self) which are important for the domain, but these are always trivial to implement for users. The real data of an annotation implementation cannot be retrieved this way.
I can store a list of annotations this way:
pub struct Node {
// ...
annotations: Vec<Box<dyn Annotation>>,
}
I'd like to let the user retrieve whatever implementation they previously added to a list, something like this:
impl Node {
fn annotations_with_type<T>(&self) -> Vec<&T>
where
T: Annotation,
{
// ??
}
}
I originally aimed to convert dyn Annotation to dyn Any, then use downcast_ref, however trait upcasting coercion is unsable.
Another solution would be to require each Annotation implementation to store its TypeId, compare it with annotations_with_type's type parameter's TypeId, and std::mem::transmute the resulting &dyn Annotation to &T… but the documentation of transmute is quite scary and I honestly don't know whether that's one of the allowed cases in which it is safe. I definitely would have done some kind of void * in C.
Of course it's also possible that there's a third (safe) way to go through this. I'm open to suggestions.
What you are describing is commonly solved by TypeMaps, allowing a type to be associated with some data.
If you are open to using a library, you might consider looking into using an existing implementation, such as https://crates.io/crates/typemap_rev, to store data. For example:
struct MyAnnotation;
impl TypeMapKey for MyAnnotation {
type Value = String;
}
let mut map = TypeMap::new();
map.insert::<MyAnnotation>("Some Annotation");
If you are curious. It underlying uses a HashMap<TypeId, Box<(dyn Any + Send + Sync)>> to store the data. To retrieve data, it uses a downcast_ref on the Any type which is stable. This could also be a pattern to implement it yourself if needed.
You don't have to worry whether this is valid - because it doesn't compile (playground):
error[E0512]: cannot transmute between types of different sizes, or dependently-sized types
--> src/main.rs:7:18
|
7 | _ = unsafe { std::mem::transmute::<&dyn Annotation, &i32>(&*v) };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: `&dyn Annotation` (128 bits)
= note: target type: `&i32` (64 bits)
The error message should be clear, I hope: &dyn Trait is a fat pointer, and has size 2*size_of::<usize>(). &T, on the other hand, is a thin pointer (as long as T: Sized), of size of only one usize, and you cannot transmute between types of different sizes.
You can work around that with transmute_copy(), but it will just make things worse: it will work, but it is unsound and is not guaranteed to work in any way. It may become UB in future Rust versions. This is because the only guaranteed thing (as of now) for &dyn Trait references is:
Pointers to unsized types are sized. The size and alignment is guaranteed to be at least equal to the size and alignment of a pointer.
Nothing guarantees the order of the fields. It can be (data_ptr, vtable_ptr) (as it is now, and thus transmute_copy() works) or (vtable_ptr, data_ptr). Nothing is even guaranteed about the contents. It can not contain a data pointer at all (though I doubt somebody will ever do something like that). transmute_copy() copies the data from the beginning, meaning that for the code to work the data pointer should be there and should be first (which it is). For the code to be sound this needs to be guaranteed (which is not).
So what can we do? Let's check how Any does its magic:
// SAFETY: caller guarantees that T is the correct type
unsafe { &*(self as *const dyn Any as *const T) }
So it uses as for the conversion. Does it work? Certainly. And that means std can do that, because std can do things that are not guaranteed and relying on how things work in practice. But we shouldn't. So, is it guaranteed?
I don't have a firm answer, but I'm pretty sure the answer is no. I have found no authoritative source that guarantees the behavior of casts from unsized to sized pointers.
Edit: #CAD97 pointed on Zulip that the reference promises that *[const|mut] T as *[const|mut V] where V: Sized will be a pointer-to-pointer case, and that can be read as a guarantee this will work.
But I still feel fine with relying on that. Because, unlike the transmute_copy(), people are doing it. In production. And there is no better way in stable. So the chance it will become undefined behavior is very low. It is much more likely to be defined.
Does a guaranteed way even exist? Well, yes and no. Yes, but only using the unstable pointer metadata API:
#![feature(ptr_metadata)]
let v: &dyn Annotation;
let v = v as *const dyn Annotation;
let v: *const T = v.to_raw_parts().0.cast::<T>();
let v: &T = unsafe { &*v };
In conclusion, if you can use nightly features, I would prefer the pointer metadata API just to be extra safe. But in case you can't, I think the cast approach is fine.
Last point, there may be a crate that already does that. Prefer that, if it exists.
I want to abstract over a variety of different list data structures. My abstraction should be fairly flexible. I want a "base trait" (let's call it List) that represents the minimal interface required from all data structures. But there are optional capabilities the data structures could offer, which are independent from each other:
Some data structures allow mutation, while others only provide a read-only interface. This is something which is fairly common for Rust traits: the pair Trait and TraitMut (e.g. Index and IndexMut).
Some data structures provide a capability "foo".
Some data structures provide a capability "bar".
Initially, this seems easy: provide the traits List, ListMut, FooList and BarList where the latter three have List as super trait. Like this:
trait List {
fn num_elements(&self) -> usize;
}
trait ListMut: List {
fn clear(&mut self);
}
trait FooList: List {
fn num_foos(&self) -> usize;
}
trait BarList: List {
fn num_bars(&self) -> usize;
}
This works fine for the methods above. But the important part is that there are methods that require multiple capabilities. For example:
add_foo(&mut self): requires the mutability and the capability "foo"!
add_foo_and_bar(&mut self): requires mutability and the capabilities "foo" and "bar".
... and more: imagine there is a function for each combination of requirements.
Where should the methods with multiple requirements live?
One Trait per Combination of Capabilities
One way would be to additionally create a trait for each combination of optional requirements:
FooListMut
BarListMut
FooBarList
FooBarListMut
These traits would have appropriate super trait bounds and could house the methods with multiple requirements. There are two problems with this:
The number of traits would grow exponentially with the number of optional capabilities. Yes, there would only need to be as many traits as methods, but it can still lead to a very chaotic API with loads of traits where most traits only contain one/a small number of methods.
There is no way (I think) to force types that implement ListMut and FooList to also implement FooListMut. Thus, functions would probably need to add more bounds. This trait system would give implementors flexibility I might not want to give them.
where Self bounds on methods
One could also add where Self: Trait bounds to methods. For example:
trait FooList: List {
// ...
fn add_foo(&mut self)
where
Self: ListMut;
}
This also works, but has two important disadvantages:
Implementors of FooList that don't implement ListMut would need to dummy implement add_foo (usually with unreachable!()) because the Rust compiler still requires it.
It is not clear where to put the methods. add_foo could also live inside ListMut with the bound being where Self: FooList. This makes the trait API more confusing.
Define Capabilities via associated types/consts
In this solution, there would only be one trait. (Note that in the following code, dummy types are used. Ideally, this would be an associated const instead of type, but we cannot use consts in trait bounds yet, so dummy types it is.)
trait Bool {}
enum True {}
enum False {}
impl Bool for True {}
impl Bool for False {}
trait List {
type SupportsMut: Bool;
type SupportsFoo: Bool;
type SupportsBar: Bool;
fn add_foo(&mut self)
where
Self: List<SupportsMut = True, SupportsBar = True>;
// ...
}
This solves two problems: for one, we know that if Mut and Foo are supported, that we can use add_foo (in contrast to the first solution, where a data structure could implement ListMut and FooList but not FooListMut). Also, since all methods live in one trait, there it's not unclear anymore where a method should live.
But:
Implementors still might need to add a bunch of unreachable!() implementations as by the last solution.
It is more noisy to bound for certain capabilities. One could add trait ListMut: List<SupportsMut = True> (the same for foo and bar) as trait alias (with blanket impl) to make this a bit better, though.
Something else?
The three solutions so far are what I can think of. One could combine them somehow, of course. Or maybe there is a completely different solution even?
Are there clear advantages of one solution over the other solutions? Have they important semantic differences? Is one of these considered more idiomatic by the community? Have there been previous discussions about this? Which one should be preferred?
The following code uses a struct with generic type. While its implementation is only valid for the given trait bound, the struct can be defined with or without the same bound. The struct's fields are private so no other code could create an instance anyway.
trait Trait {
fn foo(&self);
}
struct Object<T: Trait> {
value: T,
}
impl<T: Trait> Object<T> {
fn bar(object: Object<T>) {
object.value.foo();
}
}
Should the trait bound for the structure should be omitted to conform to the DRY principle, or should it be given to clarify the dependency? Or are there circumstances one solution should be preferred over the other?
I believe that the existing answers are misleading. In most cases, you should not put a bound on a struct unless the struct literally will not compile without it.
tl;dr
Bounds on structs express the wrong thing for most people. They are infectious, redundant, sometimes nearsighted, and often confusing. Even when a bound feels right, you should usually leave it off until it's proven necessary.
(In this answer, anything I say about structs applies equally to enums.)
0. Bounds on structs have to be repeated everywhere the struct is
This is the most obvious, but (to me) least compelling reason to avoid writing bounds on structs. As of this writing (Rust 1.65), you have to repeat every struct's bounds on every impl that touches it, which is a good enough reason not to put bounds on structs for now. However, there is an accepted RFC (implied_bounds) which, when implemented and stabilized, will change this by inferring the redundant bounds. But even then, bounds on structs are still usually wrong:
1. Bounds on structs leak out of abstractions.
Your data structure is special. "Object<T> only makes sense if T is Trait," you say. And perhaps you are right. But the decision affects not just Object, but any other data structure that contains an Object<T>, even if it does not always contain an Object<T>. Consider a programmer who wants to wrap your Object in an enum:
enum MyThing<T> { // error[E0277]: the trait bound `T: Trait` is not satisfied
Wrapped(your::Object<T>),
Plain(T),
}
Within the downstream code this makes sense because MyThing::Wrapped is only used with Ts that do implement Thing, while Plain can be used with any type. But if your::Object<T> has a bound on T, this enum can't be compiled without that same bound, even if there are lots of uses for a Plain(T) that don't require such a bound. Not only does this not work, but even if adding the bound doesn't make it entirely useless, it also exposes the bound in the public API of any struct that happens to use MyThing.
Bounds on structs limit what other people can do with them. Bounds on code (impls and functions) do too, of course, but those constraints are (presumably) required by your own code, while bounds on structs are a preemptive strike against anyone downstream who might use your struct in an innovative way. This may be useful, but unnecessary bounds are particularly annoying for innovators because they constrain what can compile without usefully constraining what can actually run (more on that in a moment).
2. Bounds on structs are redundant with bounds on code.
So you don't think downstream innovation is possible? That doesn't mean the struct itself needs a bound. To make it impossible to construct an Object<T> without T: Trait, it is enough to put that bound on the impl that contains Object's constructor(s); if it's impossible to call a_method on an Object<T> without T: Trait you can say that on the impl that contains a_method, or perhaps on a_method itself. (Until implied_bounds is implemented, you have to, anyway, so you don't even have the weak justification of "saving keystrokes.")
Even and especially when you can't think of any way for downstream to use an un-bounded Object<T>, you should not forbid it a priori, because...
3. Bounds on structs mean something different to the type system than bounds on code.
A T: Trait bound on Object<T> means more than "all Object<T>s have to have T: Trait"; it actually means something like "the concept of Object<T> itself does not make sense unless T: Trait", which is a more abstract idea. Think about natural language: I've never seen a purple elephant, but I can easily name the concept of "purple elephant" despite the fact that it corresponds to no real-world animal. Types are a kind of language and it can make sense to refer to the idea of Elephant<Purple>, even when you don't know how to create one and you certainly have no use for one. Similarly, it can make sense to express the type Object<NotTrait> in the abstract even if you don't and can't have one in hand right now. Especially when NotTrait is a type parameter, which may not be known in this context to implement Trait but in some other context does.
Case study: Cell<T>
For one example of a struct that originally had a trait bound which was eventually removed, look no farther than Cell<T>, which originally had a T: Copy bound. In the RFC to remove the bound many people initially made the same kinds of arguments you may be thinking of right now, but the eventual consensus was that "Cell requires Copy" was always the wrong way to think about Cell. The RFC was merged, paving the way for innovations like Cell::as_slice_of_cells, which lets you do things you couldn't before in safe code, including temporarily opt-in to shared mutation. The point is that T: Copy was never a useful bound on Cell<T>, and it would have done no harm (and possibly some good) to leave it off from the beginning.
This kind of abstract constraint can be hard to wrap one's head around, which is probably one reason why it's so often misused. Which relates to my last point:
4. Unnecessary bounds invite unnecessary parameters (which are worse).
This does not apply to all cases of bounds on structs, but it is a common point of confusion. You may, for instance, have a struct with a type parameter that should implement a generic trait, but not know what parameter(s) the trait should take. In such cases it is tempting to use PhantomData to add a type parameter to the main struct, but this is usually a mistake, not least because PhantomData is hard to use correctly. Here are some examples of unnecessary parameters added because of unnecessary bounds: 1 2 3 4 5 In the majority of such cases, the correct solution is simply to remove the bound.
Exceptions to the rule
Okay, when do you need a bound on a struct? I can think of two possible reasons.
In Shepmaster's answer, the struct will simply not compile without a bound, because the Iterator implementation for I actually defines what the struct contains. One other way that a struct won't compile without a bound is when its implementation of Drop has to use the trait somehow. Drop can't have bounds that aren't on the struct, for soundness reasons, so you have to write them on the struct as well.
When you're writing unsafe code and you want it to rely on a bound (T: Send, for example), you might need to put that bound on the struct. unsafe code is special because it can rely on invariants that are guaranteed by non-unsafe code, so just putting the bound on the impl that contains the unsafe is not necessarily enough.
But in all other cases, unless you really know what you're doing, you should avoid bounds on structs entirely.
Trait bounds that apply to every instance of the struct should be applied to the struct:
struct IteratorThing<I>
where
I: Iterator,
{
a: I,
b: Option<I::Item>,
}
Trait bounds that only apply to certain instances should only be applied to the impl block they pertain to:
struct Pair<T> {
a: T,
b: T,
}
impl<T> Pair<T>
where
T: std::ops::Add<T, Output = T>,
{
fn sum(self) -> T {
self.a + self.b
}
}
impl<T> Pair<T>
where
T: std::ops::Mul<T, Output = T>,
{
fn product(self) -> T {
self.a * self.b
}
}
to conform to the DRY principle
The redundancy will be removed by RFC 2089:
Eliminate the need for “redundant” bounds on functions and impls where
those bounds can be inferred from the input types and other trait
bounds. For example, in this simple program, the impl would no longer
require a bound, because it can be inferred from the Foo<T> type:
struct Foo<T: Debug> { .. }
impl<T: Debug> Foo<T> {
// ^^^^^ this bound is redundant
...
}
It really depends on what the type is for. If it is only intended to hold values which implement the trait, then yes, it should have the trait bound e.g.
trait Child {
fn name(&self);
}
struct School<T: Child> {
pupil: T,
}
impl<T: Child> School<T> {
fn role_call(&self) -> bool {
// check everyone is here
}
}
In this example, only children are allowed in the school so we have the bound on the struct.
If the struct is intended to hold any value but you want to offer extra behaviour when the trait is implemented, then no, the bound shouldn't be on the struct e.g.
trait GoldCustomer {
fn get_store_points(&self) -> i32;
}
struct Store<T> {
customer: T,
}
impl<T: GoldCustomer> Store {
fn choose_reward(customer: T) {
// Do something with the store points
}
}
In this example, not all customers are gold customers and it doesn't make sense to have the bound on the struct.
I'm trying to extend the Grid struct from the piston-2dgraphics library. There's no method for getting the location on the window of a particular cell, so I implemented a trait to calculate that for me. Then, I wanted a method to calculate the neighbours of a particular cell on the grid, so I implemented another trait.
Something about this is ugly and feels unnecessary seeing as how I'll likely never use these traits for anything other than this specific grid structure. Is there another way in Rust to extend a type without having to implement traits each time?
As of Rust 1.67, no, there is no other way. It's not possible to define inherent methods on a type defined in another crate.
You can define your own trait with the methods you need, then implement that trait for an external type. This pattern is known as extension traits. The name of extension traits, by convention, ends with Ext, to indicate that this trait is not meant to be used as a generic bound or as a trait object. There are a few examples in the standard library.
trait DoubleExt {
fn double(&self) -> Self;
}
impl DoubleExt for i32 {
fn double(&self) -> Self {
*self * 2
}
}
fn main() {
let a = 42;
println!("{}", 42.double());
}
Other libraries can also export extension traits (example: byteorder). However, as for any other trait, you need to bring the trait's methods in scope with use SomethingExt;.
No. Currently, the only way to write new methods for a type that has been defined in another crate is through traits. However, this seems too cumbersome as you have to write both the trait definition and the implementation.
In my opinion, the way to go is to use free functions instead of methods. This would at least avoid the duplication caused by traits.