On debugging (step-in) this Rust code:
use std::fs;
fn main() {
fs::create_dir("my_dir").unwrap();
}
I go to an impl I feel hard to understand:
// rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\src\rust\library\core\src\convert\mod.rs
// As lifts over &
#[stable(feature = "rust1", since = "1.0.0")]
impl<T: ?Sized, U: ?Sized> AsRef<U> for &T
where
T: AsRef<U>,
{
fn as_ref(&self) -> &U {
<T as AsRef<U>>::as_ref(*self)
}
}
Which then calls:
#[stable(feature = "rust1", since = "1.0.0")]
impl AsRef<Path> for str {
#[inline]
fn as_ref(&self) -> &Path {
Path::new(self)
}
}
What does this "As lifts over &" impl mean? Why it is needed?
This piece of code:
<T as AsRef<U>>
I've never seen this syntax before. What does it mean?
Seems like there is a lot of implicit conversions like this done by Rust compiler behind our daily Rust code. Is there any way to learn them besides debugging?
I assume As lifts over & simply means that if T can be taken as a reference to U, so does &T, because why not?
It also gives some convenience, e.g. if you happened to have a value of type &&&&&&T, you won't need to (******t).as_ref(), it's just t.as_ref(), because by using such implementation, as_ref goes through all levels of references up to the T object itself and takes it as a reference to U.
How can you learn it? Well, the top 3 most popular Rust textbooks have it in some way (not necessarily explicitly explaining you this particular case, but giving you enough knowledge to understand them implicitly. So I advice you to read one (and read it more carefully).
About the <T as AsRef<U>>, it is definitely covered in the books. It's needed to disambiguate calls to functions. For example, if some type implements two different traits, but both traits have the same method as_ref, then you'll need to fully qualify function of which trait you mean to call, for this you <T as AsRef<U>> and it's immediately clear which as_ref function you call.
Lets say you have 2 traits that share an item and a type that implements both. You can use this syntax to specify which implementation to use.
pub trait Foo {
const BAZ: usize;
}
pub trait Bar {
const BAZ: usize;
}
pub fn thing<T: Foo + Bar>() {
// This wont compile since it doesn't know which BAZ to use
println!("Unknown: {}", T::BAZ);
// You can use <a as b>::c to further specify which to use
println!("As Foo: {}", <T as Foo>::BAZ);
println!("As Bar: {}", <T as Bar>::BAZ);
}
You can read <A as B>::C as follows where C can be anything inside a trait (constant, function, type, etc.). It can also be used to make your code more verbose. If it is unclear which trait is being used, this can help make your code more explicit.
"Interpret type A as an instance of trait B for the purpose of retrieving C"
Related
I want trait implementations in Rust to be able to return arbitrary iterators (of specific item type) that may reference the original object with a lifetime 'a without having to explicitly mention 'a in the trait generics and everywhere where the trait is used or otherwise introducing significant trait bound bloat to user code. The only simple way I've figured to do this is that the trait has to be implemented for &'a MyStruct instead of MyStruct (this approach is used in some places in the standard library), but the significant drawback is that in generic code wrappers cannot “own” implementations of the trait (MyStruct) without exposing the lifetime in trait bounds all over the code. So nothing gained when ownership is needed.
Another way I figured out that should work (just done the simple test below so far) is to use higher-ranked lifetime bounds on a “base trait” that would implement the iterator-generation functions. In the code below Foo is the main trait, FooInterfaceGen is the iterator-generator trait that has its lifetime “hidden” through for <'a> when assumed as a super-trait of Foo. The FooInterface generated by FooInterfaceGen would be the trait for an appropriate type of iterator when modified to that application of the idea. However, it seems impossible to make additional trait bounds on the specific implementation FooInterfaceGen::Interface. The code below works, but when you uncomment the Asdf trait bound in the function footest, the compiler complains that
the trait `for<'a> Asdf` is not implemented for `<_ as FooInterfaceGen<'a>>::Interface
But I have implemented Asdf! It's as if the compiler is ignoring the 'a in the expression <T as FooInterfaceGen<'a>> and just applying for<'a> to the right-hand-side. Any ideas if this is a compiler bug, a known restriction, or of any ways around it?
trait FooInterface<'a> {
fn foo(&self) -> u32;
}
trait FooInterfaceGen<'a> {
type Interface : FooInterface<'a>;
fn gen(&'a self) -> Self::Interface;
}
trait Foo : for<'a> FooInterfaceGen<'a> { }
struct S2;
struct S1(S2);
impl<'a> FooInterfaceGen<'a> for S1 {
type Interface = &'a S2;
fn gen(&'a self) -> Self::Interface { &self.0 }
}
impl Foo for S1 { }
impl<'a> FooInterface<'a> for &'a S2 {
fn foo(&self) -> u32 { 42 }
}
trait Asdf {}
impl<'a> Asdf for &'a S2 {}
fn footest<T : Foo>(a : &T) -> u32
/* where for<'a> <T as FooInterfaceGen<'a>>::Interface : Asdf */ {
a.gen().foo()
}
fn main() {
let q = S1(S2);
println!("{}", footest(&q));
}
(Regarding some alternative implementations, maybe there's a technical reason for it, but otherwise I really don't understand the reason behind the significant trait bound bloat that Rust code easily introduces. Assuming a trait should in any reasonable situation automatically assume all the trait bound as well, also in generic code, not just specific code, without having to copy-paste an increasing number of where-clauses all over the code.)
The error seems to be a known compiler bug: https://github.com/rust-lang/rust/issues/89196
Suppose I have some custom collection of Foos:
struct Bar {}
struct Foo {
bar: Bar
}
struct SubList {
contents: Vec<Foo>,
}
and suppose I also have a SuperList which is a custom collection of SubLists:
struct SuperList {
contents: Vec<SubList>,
}
SubList and SuperList each provide a method bars:
impl SubList {
fn bars(&self) -> impl Iterator<Item = &Bar> + '_ {
self.contents.iter().map(|x| &x.bar)
}
}
impl SuperList {
fn bars(&self) -> impl Iterator<Item = &Bar> + '_ {
self.contents.iter().flat_map(|x| x.items())
}
}
I want to define a trait that provides a method items, and implement that trait on SubList and SuperList so that SubList::items is equivalent to SubList::bars and SuperList::items is equivalent to SuperList::bars, so that I can do this:
fn do_it<T: Buz<Bar>>(buz: &T) {
for item in buz.items() {
println!("yay!")
}
}
fn main() {
let foos = vec![Foo{ bar: Bar{} }];
let sublist = SubList{ contents: foos };
do_it(&sublist);
let superlist = SuperList{ contents: vec![sublist] };
do_it(&superlist);
}
I can do what I want with dynamic dispatch:
trait Buz<T> {
fn items(&self) -> Box<dyn Iterator<Item = &T> + '_>;
}
impl Buz<Bar> for SubList {
fn items(&self) -> Box<dyn Iterator<Item = &Bar> + '_> {
SubList::bars(self)
}
}
impl Buz<Bar> for SuperList {
fn items(&self) -> Box<dyn Iterator<Item = &Bar> + '_> {
SuperList::bars(self)
}
}
However, the following doesn't work:
trait Baz<T> {
fn items(&self) -> impl Iterator<Item = &T> + '_;
}
impl Baz<Bar> for SubList {
fn items(&self) -> impl Iterator<Item = &Bar> + '_ {
SubList::bars(self)
}
}
impl Baz<Bar> for SuperList {
fn items(&self) -> impl Iterator<Item = &Bar> + '_ {
SuperList::bars(self)
}
}
(error[E0562]: `impl Trait` not allowed outside of function and inherent method return types)
Here's a playground link to what I've tried so far
How can I define a trait Baz which provides an items method to abstract over the bars methods of SubList and SuperList without using dynamic dispatch?
Unfortunately, what you are trying to do is not really possible in Rust right now. Not by design, but simply because some relevant type level features are not implemented or stabilized yet.
Unless you have spare time, an interest in type level stuff and are willing to use nightly: just use boxed iterators. They are simple, they just work, and in most cases it will likely not even hurt performance in a meaningful way.
You're still reading? Ok let's talk about it.
As you intuitively tried, impl Trait in return type position would be the obvious solution here. But as you noticed, it doesn't work: error[E0562]: `impl Trait` not allowed outside of function and inherent method return types. Why is that? RFC 1522 says:
Initial limitations:
impl Trait may only be written within the return type of a freestanding or inherent-impl function, not in trait definitions [...] Eventually, we will want to allow the feature to be used within traits [...]
These initial limitations were put in place because the type level machinery to make this work was/is not in place yet:
One important usecase of abstract return types is to use them in trait methods.
However, there is an issue with this, namely that in combinations with generic trait methods, they are effectively equivalent to higher kinded types. Which is an issue because Rust's HKT story is not yet figured out, so any "accidental implementation" might cause unintended fallout.
The following explanation in the RFC is also worth reading.
That said, some uses of impl Trait in traits can be achieved already today: with associated types! Consider this:
trait Foo {
type Bar: Clone;
fn bar() -> Self::Bar;
}
struct A;
struct B;
impl Foo for A {
type Bar = u32;
fn bar() -> Self::Bar { 0 }
}
impl Foo for B {
type Bar = String;
fn bar() -> Self::Bar { "hello".into() }
}
This works and is "basically equivalent" to:
trait Foo {
fn bar() -> impl Clone;
}
Each impl block can choose a different return type as long as it implements a trait. So why then does impl Trait not simply desugar to an associated type? Well, let's try with your example:
trait Baz<T> {
type Iter: Iterator<Item = &Bar>;
fn items(&self) -> Self::Iter;
}
We get a missing lifetime specifier error:
4 | type Iter: Iterator<Item = &Bar>;
| ^ expected named lifetime parameter
Trying to add a lifetime parameter... we notice that we can't do that. What we need is to use the lifetime of &self here. But we can't get it at that point (in the associated type definition). This limitation is very unfortunate and is encountered in many situations (search term "streaming iterator"). The solution here are GATs: generic associated types. They allow us to write this:
trait Baz<T> {
// vvvv That's what GATs allow
type Iter<'s>: Iterator<Item = &'s Bar>;
fn items(&self) -> Self::Iter<'_>;
}
GATs are not fully implemented and certainly not stable yet. See the tracking issue.
But even with GATs, we cannot fully make your example work. That's because you use iterator types that are unnameable, due to using closures. So the impl Baz for ... blocks would not be able to provide the type Iter<'s> definition. Here we can use another feature that is not stable yet: impl Trait in type aliases!
impl Baz<Bar> for SubList {
type Iter<'s> = impl Iterator<Item = &'s Bar>;
fn items(&self) -> Self::Iter<'_> {
SubList::bars(self)
}
}
This actually works! (Again, on nightly, with these unstable features.) You can see your full, working example in this playground.
This type system work has been going on for a long time and it seems like it's slowly reaching a state of being usable. (Which I am very happy about ^_^). I expect that a few of the foundational features, or at least subsets of them, will be stabilized in the not-too-distant future. And once these are used in practice and we see how they work, more convenience features (like impl Trait in traits) will be reconsidered and stabilized. Or so I think.
Also worth noting that async fns in traits are also blocked by this, since they basically desugar into a method returning impl Future<Output = ...>. And those are also a highly requested feature.
In summary: these limitations have been a pain point for quite some time and they resurface in different practical situations (async, streaming iterator, your example, ...). I'm not involved in compiler development or language team discussions, but I kept an eye on this topic for a long time and I think we are getting close to finally resolving a lot of these issues. Certainly not in the next releases, but I see a decent chance we get some of this in the next year or two.
I have a collection of Trait, a function that iterates over it and does something, and then I would like to check the implementor type and if it is of type Foo then downcast it and call some Foo method.
Basically, something similar to Go's type-switch and interface conversion.
Searching around I found about the Any trait but it can only be implemented on 'static types.
To help demonstrate what I want:
let vec: Vec<Box<Trait>> = //
for e in vec.iter() {
e.trait_method();
// if typeof e == Foo {
// let f = e as Foo;
// f.foo_method();
//}
}
As you have noticed, downcasting only works with Any trait, and yes, it only supports 'static data. You can find a recent discussion on why it is so here. Basically, implementing reflection for references of arbitrary lifetimes is difficult.
It is also impossible (as of now, at least) to combine Any with your custom trait easily. However, a macro library for automatic implementation of Any for your trait has recently been created. You can also find some discussion on it here.
This isn't a Rust-specific problem, although the vocabulary may be a little different. The ideal way to solve a problem like this, not just with traits in Rust but in any language, is to add the desired behavior (foo_method in your example) to the abstract interface (Trait):
trait Trait {
fn trait_method(&self);
fn foo_method(&self) {} // does nothing by default
}
struct Foo;
impl Trait for Foo {
fn trait_method(&self) {
println!("In trait_method of Foo");
}
fn foo_method(&self) {
// override default behavior
println!("In foo_method");
}
}
struct Bar;
impl Trait for Bar {
fn trait_method(&self) {
println!("In trait_method of Bar");
}
}
fn main() {
let vec: Vec<Box<dyn Trait>> = vec![Box::new(Foo), Box::new(Bar)];
for e in &vec {
e.trait_method();
e.foo_method();
}
}
In this example, I have put a default implementation of foo_method in Trait which does nothing, so that you don't have to define it in every impl but only the one(s) where it applies. You should really attempt to make the above work before you resort to downcasting to a concrete type, which has serious drawbacks that all but erase the advantages of having trait objects in the first place.
That said, there are cases where downcasting may be necessary, and Rust does support it -- although the interface is a little clunky. You can downcast &Trait to &Foo by adding an intermediate upcast to &Any:
use std::any::Any;
trait Trait {
fn as_any(&self) -> &dyn Any;
}
struct Foo;
impl Trait for Foo {
fn as_any(&self) -> &dyn Any {
self
}
}
fn downcast<T: Trait + 'static>(this: &dyn Trait) -> Option<&T> {
this.as_any().downcast_ref()
}
as_any has to be a method in Trait because it needs access to the concrete type. Now you can attempt to call Foo methods on a Trait trait object like this (complete playground example):
if let Some(r) = downcast::<Foo>(&**e) {
r.foo_method();
}
To make this work, you have to specify what type you expect (::<Foo>) and use if let to handle what happens when the referenced object is not an instance of Foo. You can't downcast a trait object to a concrete type unless you know exactly what concrete type it is.
If you ever need to know the concrete type, trait objects are almost useless anyway! You probably should use an enum instead, so that you will get compile-time errors if you omit to handle a variant somewhere. Furthermore, you can't use Any with non-'static structs, so if any Foo might need to contain a reference, this design is a dead end. The best solution, if you can do it, is to add foo_method to the trait itself.
References to wrapper types like &Rc<T> and &Box<T> are invariant in T (&Rc<T> is not a &Rc<U> even if T is a U). A concrete example of the issue (Rust Playground):
use std::rc::Rc;
use std::rc::Weak;
trait MyTrait {}
struct MyStruct {
}
impl MyTrait for MyStruct {}
fn foo(rc_trait: Weak<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
foo(Rc::downgrade(&a));
}
This code results in the following error:
<anon>:15:23: 15:25 error: mismatched types:
expected `&alloc::rc::Rc<MyTrait>`,
found `&alloc::rc::Rc<MyStruct>`
Similar example (with similar error) with Box<T> (Rust Playground):
trait MyTrait {}
struct MyStruct {
}
impl MyTrait for MyStruct {}
fn foo(rc_trait: &Box<MyTrait>) {}
fn main() {
let a = Box::new(MyStruct {});
foo(&a);
}
In these cases I could of course just annotate a with the desired type, but in many cases that won't be possible because the original type is needed as well. So what do I do then?
What you see here is not related to variance and subtyping at all.
First, the most informative read on subtyping in Rust is this chapter of Nomicon. You can find there that in Rust subtyping relationship (i.e. when you can pass a value of one type to a function or a variable which expects a variable of different type) is very limited. It can only be observed when you're working with lifetimes.
For example, the following piece of code shows how exactly &Box<T> is (co)variant:
fn test<'a>(x: &'a Box<&'a i32>) {}
fn main() {
static X: i32 = 12;
let xr: &'static i32 = &X;
let xb: Box<&'static i32> = Box::new(xr); // <---- start of box lifetime
let xbr: &Box<&'static i32> = &xb;
test(xbr); // Covariance in action: since 'static is longer than or the
// same as any 'a, &Box<&'static i32> can be passed to
// a function which expects &'a Box<&'a i32>
//
// Note that it is important that both "inner" and "outer"
// references in the function signature are defined with
// the same lifetime parameter, and thus in `test(xbr)` call
// 'a gets instantiated with the lifetime associated with
// the scope I've marked with <----, but nevertheless we are
// able to pass &'static i32 as &'a i32 because the
// aforementioned scope is less than 'static, therefore any
// shared reference type with 'static lifetime is a subtype of
// a reference type with the lifetime of that scope
} // <---- end of box lifetime
This program compiles, which means that both & and Box are covariant over their respective type and lifetime parameters.
Unlike most of "conventional" OOP languages which have classes/interfaces like C++ and Java, in Rust traits do not introduce subtyping relationship. Even though, say,
trait Show {
fn show(&self) -> String;
}
highly resembles
interface Show {
String show();
}
in some language like Java, they are quite different in semantics. In Rust bare trait, when used as a type, is never a supertype of any type which implements this trait:
impl Show for i32 { ... }
// the above does not mean that i32 <: Show
Show, while being a trait, indeed can be used in type position, but it denotes a special unsized type which can only be used to form trait objects. You cannot have values of the bare trait type, therefore it does not even make sense to talk about subtyping and variance with bare trait types.
Trait objects take form of &SomeTrait or &mut SomeTrait or SmartPointer<SomeTrait>, and they can be passed around and stored in variables and they are needed to abstract away the actual implementation of the trait. However, &T where T: SomeTrait is not a subtype of &SomeTrait, and these types do not participate in variance at all.
Trait objects and regular pointers have incompatible internal structure: &T is just a regular pointer to a concrete type T, while &SomeTrait is a fat pointer which contains a pointer to the original value of a type which implements SomeTrait and also a second pointer to a vtable for the implementation of SomeTrait of the aforementioned type.
The fact that passing &T as &SomeTrait or Rc<T> as Rc<SomeTrait> works happens because Rust does automatic coercion for references and smart pointers: it is able to construct a fat pointer &SomeTrait for a regular reference &T if it knows T; this is quite natural, I believe. For instance, your example with Rc::downgrade() works because Rc::downgrade() returns a value of type Weak<MyStruct> which gets coerced to Weak<MyTrait>.
However, constructing &Box<SomeTrait> out of &Box<T> if T: SomeTrait is much more complex: for one, the compiler would need to allocate a new temporary value because Box<T> and Box<SomeTrait> has different memory representations. If you have, say, Box<Box<T>>, getting Box<Box<SomeTrait>> out of it is even more complex, because it would need creating a new allocation on the heap to store Box<SomeTrait>. Thus, there are no automatic coercions for nested references and smart pointers, and again, this is not connected with subtyping and variance at all.
In the case of Rc::downgrade this is actually just a failure of the type inference in this particular case, and will work if it is done as a separate let:
fn foo(rc_trait: Weak<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
let b = Rc::downgrade(&a);
foo(b);
}
Playground
For Box<T> it is very likely you don't actually want a reference to the box as the argument, but a reference to the contents. In which case there is no invariance to deal with:
fn foo(rc_trait: &MyTrait) {}
fn main() {
let a = Box::new(MyStruct {});
foo(a.as_ref());
}
Playground
Similarly, for the case with Rc<T>, if you write a function that takes an Rc<T> you probably want a clone (i.e. a reference counted reference), and not a normal reference:
fn foo(rc_trait: Rc<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
foo(a.clone());
}
Playground
I have a collection of Trait, a function that iterates over it and does something, and then I would like to check the implementor type and if it is of type Foo then downcast it and call some Foo method.
Basically, something similar to Go's type-switch and interface conversion.
Searching around I found about the Any trait but it can only be implemented on 'static types.
To help demonstrate what I want:
let vec: Vec<Box<Trait>> = //
for e in vec.iter() {
e.trait_method();
// if typeof e == Foo {
// let f = e as Foo;
// f.foo_method();
//}
}
As you have noticed, downcasting only works with Any trait, and yes, it only supports 'static data. You can find a recent discussion on why it is so here. Basically, implementing reflection for references of arbitrary lifetimes is difficult.
It is also impossible (as of now, at least) to combine Any with your custom trait easily. However, a macro library for automatic implementation of Any for your trait has recently been created. You can also find some discussion on it here.
This isn't a Rust-specific problem, although the vocabulary may be a little different. The ideal way to solve a problem like this, not just with traits in Rust but in any language, is to add the desired behavior (foo_method in your example) to the abstract interface (Trait):
trait Trait {
fn trait_method(&self);
fn foo_method(&self) {} // does nothing by default
}
struct Foo;
impl Trait for Foo {
fn trait_method(&self) {
println!("In trait_method of Foo");
}
fn foo_method(&self) {
// override default behavior
println!("In foo_method");
}
}
struct Bar;
impl Trait for Bar {
fn trait_method(&self) {
println!("In trait_method of Bar");
}
}
fn main() {
let vec: Vec<Box<dyn Trait>> = vec![Box::new(Foo), Box::new(Bar)];
for e in &vec {
e.trait_method();
e.foo_method();
}
}
In this example, I have put a default implementation of foo_method in Trait which does nothing, so that you don't have to define it in every impl but only the one(s) where it applies. You should really attempt to make the above work before you resort to downcasting to a concrete type, which has serious drawbacks that all but erase the advantages of having trait objects in the first place.
That said, there are cases where downcasting may be necessary, and Rust does support it -- although the interface is a little clunky. You can downcast &Trait to &Foo by adding an intermediate upcast to &Any:
use std::any::Any;
trait Trait {
fn as_any(&self) -> &dyn Any;
}
struct Foo;
impl Trait for Foo {
fn as_any(&self) -> &dyn Any {
self
}
}
fn downcast<T: Trait + 'static>(this: &dyn Trait) -> Option<&T> {
this.as_any().downcast_ref()
}
as_any has to be a method in Trait because it needs access to the concrete type. Now you can attempt to call Foo methods on a Trait trait object like this (complete playground example):
if let Some(r) = downcast::<Foo>(&**e) {
r.foo_method();
}
To make this work, you have to specify what type you expect (::<Foo>) and use if let to handle what happens when the referenced object is not an instance of Foo. You can't downcast a trait object to a concrete type unless you know exactly what concrete type it is.
If you ever need to know the concrete type, trait objects are almost useless anyway! You probably should use an enum instead, so that you will get compile-time errors if you omit to handle a variant somewhere. Furthermore, you can't use Any with non-'static structs, so if any Foo might need to contain a reference, this design is a dead end. The best solution, if you can do it, is to add foo_method to the trait itself.