While trying to understand the Any trait better, I saw that it has an impl block for the trait itself. I don't understand the purpose of this construct, or even if it has a specific name.
I made a little experiment with both a "normal" trait method and a method defined in the impl block:
trait Foo {
fn foo_in_trait(&self) {
println!("in foo")
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("in impl")
}
}
impl Foo for u8 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let y = &42u8 as &dyn Foo;
y.foo_in_trait();
y.foo_in_impl(); // May cause an error, see below
}
Editor's note
In versions of Rust up to and including Rust 1.15.0, the line
y.foo_in_impl() causes the error:
error: borrowed value does not live long enough
--> src/main.rs:20:14
|
20 | let y = &42u8 as &Foo;
| ^^^^ does not live long enough
...
23 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
This error is no longer present in subsequent versions, but the
concepts explained in the answers are still valid.
From this limited experiment, it seems like methods defined in the impl block are more restrictive than methods defined in the trait block. It's likely that there's something extra that doing it this way unlocks, but I just don't know what it is yet! ^_^
The sections from The Rust Programming Language on traits and trait objects don't make any mention of this. Searching the Rust source itself, it seems like only Any and Error use this particular feature. I've not seen this used in the handful of crates where I have looked at the source code.
When you define a trait named Foo that can be made into an object, Rust also defines a trait object type named dyn Foo. In older versions of Rust, this type was only called Foo (see What does "dyn" mean in a type?). For backwards compatibility with these older versions, Foo still works to name the trait object type, although dyn syntax should be used for new code.
Trait objects have a lifetime parameter that designates the shortest of the implementor's lifetime parameters. To specify that lifetime, you write the type as dyn Foo + 'a.
When you write impl dyn Foo { (or just impl Foo { using the old syntax), you are not specifying that lifetime parameter, and it defaults to 'static. This note from the compiler on the y.foo_in_impl(); statement hints at that:
note: borrowed value must be valid for the static lifetime...
All we have to do to make this more permissive is to write a generic impl over any lifetime:
impl<'a> dyn Foo + 'a {
fn foo_in_impl(&self) { println!("in impl") }
}
Now, notice that the self argument on foo_in_impl is a borrowed pointer, which has a lifetime parameter of its own. The type of self, in its full form, looks like &'b (dyn Foo + 'a) (the parentheses are required due to operator precedence). A Box<u8> owns its u8 – it doesn't borrow anything –, so you can create a &(dyn Foo + 'static) out of it. On the other hand, &42u8 creates a &'b (dyn Foo + 'a) where 'a is not 'static, because 42u8 is put in a hidden variable on the stack, and the trait object borrows this variable. (That doesn't really make sense, though; u8 doesn't borrow anything, so its Foo implementation should always be compatible with dyn Foo + 'static... the fact that 42u8 is borrowed from the stack should affect 'b, not 'a.)
Another thing to note is that trait methods are polymorphic, even when they have a default implementation and they're not overridden, while inherent methods on a trait objects are monomorphic (there's only one function, no matter what's behind the trait). For example:
use std::any::type_name;
trait Foo {
fn foo_in_trait(&self)
where
Self: 'static,
{
println!("{}", type_name::<Self>());
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("{}", type_name::<Self>());
}
}
impl Foo for u8 {}
impl Foo for u16 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let x = Box::new(42u16) as Box<Foo>;
x.foo_in_trait();
x.foo_in_impl();
}
Sample output:
u8
dyn playground::Foo
u16
dyn playground::Foo
In the trait method, we get the type name of the underlying type (here, u8 or u16), so we can conclude that the type of &self will vary from one implementer to the other (it'll be &u8 for the u8 implementer and &u16 for the u16 implementer – not a trait object). However, in the inherent method, we get the type name of dyn Foo (+ 'static), so we can conclude that the type of &self is always &dyn Foo (a trait object).
I suspect that the reason is very simple: may be overridden or not?
A method implemented in a trait block can be overridden by implementors of the trait, it just provides a default.
On the other hand, a method implemented in an impl block cannot be overridden.
If this reasoning is right, then the error you get for y.foo_in_impl() is just a lack of polish: it should have worked. See Francis Gagné's more complete answer on the interaction with lifetimes.
Related
I want trait implementations in Rust to be able to return arbitrary iterators (of specific item type) that may reference the original object with a lifetime 'a without having to explicitly mention 'a in the trait generics and everywhere where the trait is used or otherwise introducing significant trait bound bloat to user code. The only simple way I've figured to do this is that the trait has to be implemented for &'a MyStruct instead of MyStruct (this approach is used in some places in the standard library), but the significant drawback is that in generic code wrappers cannot “own” implementations of the trait (MyStruct) without exposing the lifetime in trait bounds all over the code. So nothing gained when ownership is needed.
Another way I figured out that should work (just done the simple test below so far) is to use higher-ranked lifetime bounds on a “base trait” that would implement the iterator-generation functions. In the code below Foo is the main trait, FooInterfaceGen is the iterator-generator trait that has its lifetime “hidden” through for <'a> when assumed as a super-trait of Foo. The FooInterface generated by FooInterfaceGen would be the trait for an appropriate type of iterator when modified to that application of the idea. However, it seems impossible to make additional trait bounds on the specific implementation FooInterfaceGen::Interface. The code below works, but when you uncomment the Asdf trait bound in the function footest, the compiler complains that
the trait `for<'a> Asdf` is not implemented for `<_ as FooInterfaceGen<'a>>::Interface
But I have implemented Asdf! It's as if the compiler is ignoring the 'a in the expression <T as FooInterfaceGen<'a>> and just applying for<'a> to the right-hand-side. Any ideas if this is a compiler bug, a known restriction, or of any ways around it?
trait FooInterface<'a> {
fn foo(&self) -> u32;
}
trait FooInterfaceGen<'a> {
type Interface : FooInterface<'a>;
fn gen(&'a self) -> Self::Interface;
}
trait Foo : for<'a> FooInterfaceGen<'a> { }
struct S2;
struct S1(S2);
impl<'a> FooInterfaceGen<'a> for S1 {
type Interface = &'a S2;
fn gen(&'a self) -> Self::Interface { &self.0 }
}
impl Foo for S1 { }
impl<'a> FooInterface<'a> for &'a S2 {
fn foo(&self) -> u32 { 42 }
}
trait Asdf {}
impl<'a> Asdf for &'a S2 {}
fn footest<T : Foo>(a : &T) -> u32
/* where for<'a> <T as FooInterfaceGen<'a>>::Interface : Asdf */ {
a.gen().foo()
}
fn main() {
let q = S1(S2);
println!("{}", footest(&q));
}
(Regarding some alternative implementations, maybe there's a technical reason for it, but otherwise I really don't understand the reason behind the significant trait bound bloat that Rust code easily introduces. Assuming a trait should in any reasonable situation automatically assume all the trait bound as well, also in generic code, not just specific code, without having to copy-paste an increasing number of where-clauses all over the code.)
The error seems to be a known compiler bug: https://github.com/rust-lang/rust/issues/89196
I was writing some code where I want to use an iterator, or its reversed version depending on a flag, but the straightforward code gives an error
pub fn eggs<I,T>(iter:I)->Box<dyn Iterator<Item=T>>
where I:Iterator<Item=T>+DoubleEndedIterator
{
Box::new(iter.rev())
}
pub fn bacon<I,T>(iter:I, reverse:bool) -> Box<dyn Iterator<Item=T>>
where I:Iterator<Item=T>+DoubleEndedIterator
{
if reverse {
Box::new(iter.rev())
} else {
Box::new(iter)
}
}
fn main()
{
let pants:String = "pants".into();
eggs(pants.chars());
}
fails to compile:
error[E0310]: the parameter type `I` may not live long enough
--> src/main.rs:5:5
|
2 | pub fn eggs<I,T>(iter:I)->Box<dyn Iterator<Item=T>>
| - help: consider adding an explicit lifetime bound...: `I: 'static`
...
5 | Box::new(iter.rev())
| ^^^^^^^^^^^^^^^^^^^^ ...so that the type `Rev<I>` will meet its required lifetime bounds
With my limited understanding of Rust, I'm not sure where those lifetime bounds are coming from. There aren't any on the Iterator trait, or the Rev struct, and the parameter is being moved.
What is the proper way to declare these sorts of functions given that 'static isn't really an option.
rust playground
This doesn't have to do with .rev() at all, but with returning Box<dyn Iterator>:
// error[E0310]: the parameter type `I` may not live long enough
fn boxed_iter<I, T>(iter: I) -> Box<dyn Iterator<Item = T>>
// - help: consider adding an explicit lifetime bound...: `I: 'static`
where
I: Iterator<Item = T>,
{
Box::new(iter)
// ^^^^^^^^^^^^^^ ...so that the type `I` will meet its required lifetime bounds
}
The reason for this is that trait objects like Box<dyn Trait> have an implicit 'static lifetime if not specified. So when the compiler tries to cast Box<I> to Box<dyn Iterator>, it fails if I is does not also have a 'static lifetime. (There are some more specific rules if the trait contains lifetimes itself; you can read about those in more detail here.)
If you instead want a shorter lifetime, you need to specify it explicitly as Box<dyn 'a + Trait>. So for example:
fn boxed_iter<'a, I, T>(iter: I) -> Box<dyn 'a + Iterator<Item = T>>
where
I: 'a + Iterator<Item = T>,
{
Box::new(iter)
}
Frxstrem's answer is excellent. I just want to add that, if you know that the return value of your function has a specific concrete type, you can use the special impl trait syntax.
In the case of your eggs function, the return type is probably something like Rev<I>. That type, on its own, isn't very illuminating, but we know that such a type exists and the only thing we care about is that it's an iterator, so we can write
pub fn eggs<I,T>(iter:I) -> impl Iterator<Item=T> + DoubleEndedIterator
where I: Iterator<Item=T> + DoubleEndedIterator {
iter.rev()
}
Now the compiler still understands that there is a single concrete type and will act accordingly (no need to box the value or have dynamic dispatch), but we as the programmers still only have to care about the Iterator and DoubleEndedIterator aspects of it. Zero-cost abstraction at its finest.
Your bacon function can't benefit from this, as it could return either an I or a Rev<I> depending on input, so the dynamic dispatch is actually necessary. In that case, you'll need to follow Frxstrem's answer to correctly box your iterator.
I have this minimal example code:
use std::borrow::BorrowMut;
trait Foo {}
struct Bar;
impl Foo for Bar {}
fn main() {
let mut encryptor: Box<Foo> = Box::new(Bar);
encrypt(encryptor.borrow_mut());
}
fn encrypt(encryptor: &mut Foo) { }
but it fails with this error:
error: `encryptor` does not live long enough
--> src/main.rs:11:1
|
10 | encrypt(encryptor.borrow_mut());
| --------- borrow occurs here
11 | }
| ^ `encryptor` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
The kind people at #rustbeginners found that I have to dereference the box to get the contents, and then borrow the contents. Like this:
trait Foo {}
struct Bar;
impl Foo for Bar {}
fn main() {
let mut encryptor: Box<Foo> = Box::new(Bar);
encrypt(&mut *encryptor);
}
fn encrypt(encryptor: &mut Foo) { }
It works, but I don't understand it.
Why do I need to dereference first? What is the error trying to say? Normally it isn't an error that a value is dropped at the end of the function.
Apparently it's not just me who doesn't understand how this works; an issue has been filed.
Let's start with a change that allows the code to work:
fn encrypt(encryptor: &mut (Foo + 'static)) { }
The important difference is the addition of + 'static to the trait object - the parens are just needed for precedence.
The important thing to recognize is that there are two lifetimes present in &Foo:
a lifetime for the reference itself: &'a Foo
a lifetime that represents all the references inside the concrete value that the trait abstracts: &(Foo + 'b).
If I'm reading the RFCs correctly, this was introduced by RFC 192, and RFC 599 specified reasonable defaults for the lifetimes. In this case, the lifetimes should expand like:
fn encrypt(encryptor: &mut Foo) { }
fn encrypt<'a>(encryptor: &'a mut (Foo + 'a)) { }
On the other end of the pipe, we have a Box<Foo>. Expanded by the rules of the RFC, this becomes Box<Foo + 'static>. When we take a borrow of it, and try to pass it to the function, we have an equation to solve:
The lifetime inside the trait object is 'static.
The function takes a reference to a trait object.
The lifetime of the reference equals the lifetime of references inside the trait object.
Therefore, the reference to the trait object must be 'static. Uh oh!
The Box will be dropped at the end of the block so it certainly isn't static.
The fix with explicit lifetimes allows the lifetime of the reference to the trait object to differ from the lifetime of the references inside the trait object.
If you needed to support a trait object with internal references, an alternate is to do something like:
fn encrypt<'a>(encryptor: &mut (Foo + 'a)) { }
True credit for this explanation goes to nikomatsakis and his comment on GitHub, I just expanded it a bit.
References to wrapper types like &Rc<T> and &Box<T> are invariant in T (&Rc<T> is not a &Rc<U> even if T is a U). A concrete example of the issue (Rust Playground):
use std::rc::Rc;
use std::rc::Weak;
trait MyTrait {}
struct MyStruct {
}
impl MyTrait for MyStruct {}
fn foo(rc_trait: Weak<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
foo(Rc::downgrade(&a));
}
This code results in the following error:
<anon>:15:23: 15:25 error: mismatched types:
expected `&alloc::rc::Rc<MyTrait>`,
found `&alloc::rc::Rc<MyStruct>`
Similar example (with similar error) with Box<T> (Rust Playground):
trait MyTrait {}
struct MyStruct {
}
impl MyTrait for MyStruct {}
fn foo(rc_trait: &Box<MyTrait>) {}
fn main() {
let a = Box::new(MyStruct {});
foo(&a);
}
In these cases I could of course just annotate a with the desired type, but in many cases that won't be possible because the original type is needed as well. So what do I do then?
What you see here is not related to variance and subtyping at all.
First, the most informative read on subtyping in Rust is this chapter of Nomicon. You can find there that in Rust subtyping relationship (i.e. when you can pass a value of one type to a function or a variable which expects a variable of different type) is very limited. It can only be observed when you're working with lifetimes.
For example, the following piece of code shows how exactly &Box<T> is (co)variant:
fn test<'a>(x: &'a Box<&'a i32>) {}
fn main() {
static X: i32 = 12;
let xr: &'static i32 = &X;
let xb: Box<&'static i32> = Box::new(xr); // <---- start of box lifetime
let xbr: &Box<&'static i32> = &xb;
test(xbr); // Covariance in action: since 'static is longer than or the
// same as any 'a, &Box<&'static i32> can be passed to
// a function which expects &'a Box<&'a i32>
//
// Note that it is important that both "inner" and "outer"
// references in the function signature are defined with
// the same lifetime parameter, and thus in `test(xbr)` call
// 'a gets instantiated with the lifetime associated with
// the scope I've marked with <----, but nevertheless we are
// able to pass &'static i32 as &'a i32 because the
// aforementioned scope is less than 'static, therefore any
// shared reference type with 'static lifetime is a subtype of
// a reference type with the lifetime of that scope
} // <---- end of box lifetime
This program compiles, which means that both & and Box are covariant over their respective type and lifetime parameters.
Unlike most of "conventional" OOP languages which have classes/interfaces like C++ and Java, in Rust traits do not introduce subtyping relationship. Even though, say,
trait Show {
fn show(&self) -> String;
}
highly resembles
interface Show {
String show();
}
in some language like Java, they are quite different in semantics. In Rust bare trait, when used as a type, is never a supertype of any type which implements this trait:
impl Show for i32 { ... }
// the above does not mean that i32 <: Show
Show, while being a trait, indeed can be used in type position, but it denotes a special unsized type which can only be used to form trait objects. You cannot have values of the bare trait type, therefore it does not even make sense to talk about subtyping and variance with bare trait types.
Trait objects take form of &SomeTrait or &mut SomeTrait or SmartPointer<SomeTrait>, and they can be passed around and stored in variables and they are needed to abstract away the actual implementation of the trait. However, &T where T: SomeTrait is not a subtype of &SomeTrait, and these types do not participate in variance at all.
Trait objects and regular pointers have incompatible internal structure: &T is just a regular pointer to a concrete type T, while &SomeTrait is a fat pointer which contains a pointer to the original value of a type which implements SomeTrait and also a second pointer to a vtable for the implementation of SomeTrait of the aforementioned type.
The fact that passing &T as &SomeTrait or Rc<T> as Rc<SomeTrait> works happens because Rust does automatic coercion for references and smart pointers: it is able to construct a fat pointer &SomeTrait for a regular reference &T if it knows T; this is quite natural, I believe. For instance, your example with Rc::downgrade() works because Rc::downgrade() returns a value of type Weak<MyStruct> which gets coerced to Weak<MyTrait>.
However, constructing &Box<SomeTrait> out of &Box<T> if T: SomeTrait is much more complex: for one, the compiler would need to allocate a new temporary value because Box<T> and Box<SomeTrait> has different memory representations. If you have, say, Box<Box<T>>, getting Box<Box<SomeTrait>> out of it is even more complex, because it would need creating a new allocation on the heap to store Box<SomeTrait>. Thus, there are no automatic coercions for nested references and smart pointers, and again, this is not connected with subtyping and variance at all.
In the case of Rc::downgrade this is actually just a failure of the type inference in this particular case, and will work if it is done as a separate let:
fn foo(rc_trait: Weak<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
let b = Rc::downgrade(&a);
foo(b);
}
Playground
For Box<T> it is very likely you don't actually want a reference to the box as the argument, but a reference to the contents. In which case there is no invariance to deal with:
fn foo(rc_trait: &MyTrait) {}
fn main() {
let a = Box::new(MyStruct {});
foo(a.as_ref());
}
Playground
Similarly, for the case with Rc<T>, if you write a function that takes an Rc<T> you probably want a clone (i.e. a reference counted reference), and not a normal reference:
fn foo(rc_trait: Rc<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
foo(a.clone());
}
Playground
While trying to understand the Any trait better, I saw that it has an impl block for the trait itself. I don't understand the purpose of this construct, or even if it has a specific name.
I made a little experiment with both a "normal" trait method and a method defined in the impl block:
trait Foo {
fn foo_in_trait(&self) {
println!("in foo")
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("in impl")
}
}
impl Foo for u8 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let y = &42u8 as &dyn Foo;
y.foo_in_trait();
y.foo_in_impl(); // May cause an error, see below
}
Editor's note
In versions of Rust up to and including Rust 1.15.0, the line
y.foo_in_impl() causes the error:
error: borrowed value does not live long enough
--> src/main.rs:20:14
|
20 | let y = &42u8 as &Foo;
| ^^^^ does not live long enough
...
23 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
This error is no longer present in subsequent versions, but the
concepts explained in the answers are still valid.
From this limited experiment, it seems like methods defined in the impl block are more restrictive than methods defined in the trait block. It's likely that there's something extra that doing it this way unlocks, but I just don't know what it is yet! ^_^
The sections from The Rust Programming Language on traits and trait objects don't make any mention of this. Searching the Rust source itself, it seems like only Any and Error use this particular feature. I've not seen this used in the handful of crates where I have looked at the source code.
When you define a trait named Foo that can be made into an object, Rust also defines a trait object type named dyn Foo. In older versions of Rust, this type was only called Foo (see What does "dyn" mean in a type?). For backwards compatibility with these older versions, Foo still works to name the trait object type, although dyn syntax should be used for new code.
Trait objects have a lifetime parameter that designates the shortest of the implementor's lifetime parameters. To specify that lifetime, you write the type as dyn Foo + 'a.
When you write impl dyn Foo { (or just impl Foo { using the old syntax), you are not specifying that lifetime parameter, and it defaults to 'static. This note from the compiler on the y.foo_in_impl(); statement hints at that:
note: borrowed value must be valid for the static lifetime...
All we have to do to make this more permissive is to write a generic impl over any lifetime:
impl<'a> dyn Foo + 'a {
fn foo_in_impl(&self) { println!("in impl") }
}
Now, notice that the self argument on foo_in_impl is a borrowed pointer, which has a lifetime parameter of its own. The type of self, in its full form, looks like &'b (dyn Foo + 'a) (the parentheses are required due to operator precedence). A Box<u8> owns its u8 – it doesn't borrow anything –, so you can create a &(dyn Foo + 'static) out of it. On the other hand, &42u8 creates a &'b (dyn Foo + 'a) where 'a is not 'static, because 42u8 is put in a hidden variable on the stack, and the trait object borrows this variable. (That doesn't really make sense, though; u8 doesn't borrow anything, so its Foo implementation should always be compatible with dyn Foo + 'static... the fact that 42u8 is borrowed from the stack should affect 'b, not 'a.)
Another thing to note is that trait methods are polymorphic, even when they have a default implementation and they're not overridden, while inherent methods on a trait objects are monomorphic (there's only one function, no matter what's behind the trait). For example:
use std::any::type_name;
trait Foo {
fn foo_in_trait(&self)
where
Self: 'static,
{
println!("{}", type_name::<Self>());
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("{}", type_name::<Self>());
}
}
impl Foo for u8 {}
impl Foo for u16 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let x = Box::new(42u16) as Box<Foo>;
x.foo_in_trait();
x.foo_in_impl();
}
Sample output:
u8
dyn playground::Foo
u16
dyn playground::Foo
In the trait method, we get the type name of the underlying type (here, u8 or u16), so we can conclude that the type of &self will vary from one implementer to the other (it'll be &u8 for the u8 implementer and &u16 for the u16 implementer – not a trait object). However, in the inherent method, we get the type name of dyn Foo (+ 'static), so we can conclude that the type of &self is always &dyn Foo (a trait object).
I suspect that the reason is very simple: may be overridden or not?
A method implemented in a trait block can be overridden by implementors of the trait, it just provides a default.
On the other hand, a method implemented in an impl block cannot be overridden.
If this reasoning is right, then the error you get for y.foo_in_impl() is just a lack of polish: it should have worked. See Francis Gagné's more complete answer on the interaction with lifetimes.