So i have a struct.
pub struct Foo<TFn, TArg, TReturn> where TFn: Fn(TArg) -> TReturn { func: TFn }
This makes sence in my head being used to C# Generics, but why doesn't it work in rust?
I want the field 'func' to be of type Fn where the argument is of type 'TArg' and the return value is of type 'TReturn'.
The compiler is complaining that the paramter 'TArg' and 'TReturn' are never used, but they are helping to define the signature of the TFn value.
I tried removing the 'never used' parameters and just writing in a type in the constraint explicitly. That works fine.
#Aleksander Krauze's answer is right, but doesn't take into account one niche concern: variance. Consider this:
fn x<'s>(&'s str) -> String { // ...
x is a function that can deal with any &str that lives for 's. In particular this means that x can deal with any &str that lives longer than 's, a &'static str for example. This is true because &'static str is a subtype of &'s str. In Rust, T being a subtype of U (written T: U) means wherever I can accept a U, I can also accept a T, because a subtype can do anything its supertype can do and more (live longer, for example).
Subtyping comes up most often for lifetimes in Rust, where lifetime 'a is a subtype of lifetime 'b if 'a outlives (i.e. is longer than) 'b. So if I can accept a &'b T, I can also accept a &'a T as long as 'a: 'b. In other words, &'a T is a subtype of &'b T if 'a is a subtype of 'b. This is called covariance and is an instance of what is called variance.
However, functions are interesting in this regard. If I have types fn(&'a T) and fn(&'b T), fn(&'a T) is a subtype of fn(&'b T) if 'b is a subtype of 'a not the other way around. I.e. if I need a function that can deal with long lifetimes, then a function that can deal with shorter lifetimes will also do, because any argument I can pass it will be a subtype of the argument it expects. This is called contravariance and is a property we very much want for our functions.
Your type Foo is more or less a function so we'd like it to behave like one and be contravariant over its argument. But it isn't. It's covariant. That's because a struct in Rust inherits its variance from its fields. The type PhantomData<(TArg, TReturn)> is covariant over TArg and TReturn (because the type (TArg, TReturn) is) and so Foo will be covariant over TArg. To get it to behave like a function should, we can just mark it with the appropriate type: PhantomData<fn(TArg) -> TReturn>. This will be contravariant over TArg and covariant over TReturn (functions are covariant over their return type; I hope that follows from the explanations above).
I've written a little example (albeit an artificial one) to demonstrate how incorrect variance can break code that should work:
use std::marker::PhantomData;
pub struct Foo<TFn, TArg, TReturn>
{
func: TFn,
// this makes `Foo` covariant over `TArg`
_marker: PhantomData<(TArg, TReturn)>
}
impl<TFn, TArg, TReturn> Bar<TFn, TArg, TReturn>
where
TFn: Fn(TArg) -> TReturn,
{
// presumably this is how one might use a `Foo`
fn call(&self, arg: TArg) -> TReturn {
(self.func)(arg)
}
}
// `foo_factory` will return a `Foo` that is covariant over the lifetime `'a`
// of its argument
fn foo_factory<'a>(_: &'a str) -> Foo<fn(&'a str) -> String, &'a str, String> {
// We only care about the type signatures here
panic!()
}
fn main() {
let long_lifetime: &'static str = "hello";
// we make a `Foo` that is covariant over the `'static` lifetime
let mut foo = foo_factory(long_lifetime);
foo.call("world");
{
let short_lifetime = String::from("world");
// and because it's covariant, it can't accept a shorter lifetime
// than `'static`
// even though this should be perfectly fine, it won't compile
foo = foo_factory(&short_lifetime);
}
foo.call("world");
}
But if we fix the variance:
pub struct Foo<TFn, TArg, TReturn> {
func: TFn,
// `Foo` is now _contravariant_ over `TArg` and covariant over `TReturn`
_marker: PhantomData<fn(TArg) -> TReturn>,
}
The main function from above will now compile just fine as one would expect.
For more on variance in Rust and how it relates to data structures and the drop check, I recommend checking out the 'nomicon chapter on it and the one on PhantomData.
In rust your struct must use all of the generic types that it is generic over. And use mean, that they must appear in at least one type of the field. You can solve your problem with special type PhantomData. It is a marker type that is used to provide additional information to the compiler and is removed at compile time. The docs give you even example of how to use it to solve "unused type parameters" error. TLDR is here:
use std::marker::PhantomData;
pub struct Foo<TFn, TArg, TReturn>
where
TFn: Fn(TArg) -> TReturn
{
func: TFn,
_marker: PhantomData<(TArg, TRetur)> // This line tells the compiler that
// this struct should act like it owned
// type (Targ, TReturn)
}
And when you want to create an instance of your struct just put PhantomData as a value:
let s = Foo { func: f, _marker: PhantomData };
Related
When I try to store closures to a HashMap, I come across a lifetime bound requirement reported by the compiler. It seems like an inconsistent requirement.
struct NoBox<C: Fn() -> ()>(HashMap<String, C>);
impl<C> NoBox<C>
where
C: Fn() -> (),
{
fn new() -> NoBox<C> {
NoBox(HashMap::new())
}
fn add(&mut self, str: &str, closure: C) {
self.0.insert(str.to_string(), closure);
}
}
This is Ok. The compiler is happy with it. However, when I try to wrap the closure into a trait object and store it. The compiler imposes a 'static lifetime bound on it.
struct Boxed(HashMap<String, Box<dyn Fn() -> ()>>);
impl Boxed {
fn new() -> Boxed {
Boxed(HashMap::new())
}
fn add<C>(&mut self, str: &str, closure: C)
where
C: Fn() -> ()//add 'static here fix the error
{
self.0.insert(str.to_string(), Box::new(closure)); //error: type parameter C may not live long enough, consider adding 'static lifebound
}
}
According to the complain of the compiler, C may not live long enough. It makes sense to add a 'static bound to it.
But, why the first case without boxing doesn't have this requirement?
To my understanding, if C contains some reference to an early-dropped referent, then store it in NoBox would also cause the invalid-reference problem. For me, it seems like an inconsistency.
NoBox is not a problem because if the function contains a reference to the lifetime, the type will stay contain this lifetime because the function type needs to be specified explicitly.
For example, suppose we're storing a closure that captures something with lifetime 'a. Then the closure's struct will looks like (this is not how the compiler actually desugars closures but is enough for the example):
struct Closure<'a> {
captured: &'a i32,
}
And when specifying it in NoBox, the type will be NoBox<Closure<'a>>, and so we know it cannot outlive 'a. Note this type may never be actually explicitly specified - especially with closures - but the compiler's inferred type still have the lifetime in it.
With Boxed on the other hand, we erase this information, and thus may accidentally outlive 'a - because it does not appear on the type. So the compiler enforces it to be 'static, unless you explicitly specify otherwise:
struct Boxed<'a>(HashMap<String, Box<dyn Fn() + 'a>>);
Given a struct S implementing a trait T, why doesn't Box<S> implement Borrow<dyn T>?
The following code, that I would have expected to compile, doesn't:
trait T{}
struct S{}
impl T for S{}
fn f1(s: &S) -> &dyn T {
s
}
fn f2(s: &Box<S>) -> &dyn T {
std::borrow::Borrow::borrow(s)
}
Why does f1 compile while f2 doesn't? (The conversion from &S to &dyn T is done in the first case and not in the second).
This is to do with the way that type inference and type coercion work. The Borrow<B> trait's parameter is the type of the borrowed value, and the type checker needs to know what it is.
If you just write:
std::borrow::Borrow::borrow(s)
Then the type B in Borrow<B> will be inferred from the surrounding code. In your case it is inferred to be dyn T because that's the return value. However, dyn T is a completely different type from S, so it doesn't type-check.
Once the type checker knows that the value being returned is of type &S then it can coerce it to a &dyn T, but you need to give it that information:
fn f2(s: &Box<S>) -> &dyn T {
let s: &S = std::borrow::Borrow::borrow(s);
s
}
Or, more concisely:
fn f2(s: &Box<S>) -> &dyn T {
std::borrow::Borrow::<S>::borrow(s)
}
The reason why Sébastien Renauld's answer works is because Deref uses an associated type instead of a type parameter. The type-checker can easily infer the <S as Deref>::Target because there can only be one implementation of Deref per type and the associated Target type is uniquely determined. Borrow is different because Box<S> could implement Borrow<()>, Borrow<i32>, Borrow<Box<Option<Vec<bool>>>>,... so you have to be more explicit about which implementation you intend.
&Box<S> is not directly equal to Box<&S>, and this is why it does not compile directly.
You can relatively easily fix this by dereferencing, like so:
use std::ops::Deref;
trait T{}
struct S{}
impl T for S{}
fn f1(s : &S) -> &(dyn T) {
s
}
fn f2(s : &Box<S>) -> &(dyn T) {
s.deref()
}
(The trait Deref is there for slightly easier readability)
The call to deref() operates over &self, so having &Box<S> is sufficient to call it. It simply returns &S, and since that implements T the types check out.
While trying to understand the Any trait better, I saw that it has an impl block for the trait itself. I don't understand the purpose of this construct, or even if it has a specific name.
I made a little experiment with both a "normal" trait method and a method defined in the impl block:
trait Foo {
fn foo_in_trait(&self) {
println!("in foo")
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("in impl")
}
}
impl Foo for u8 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let y = &42u8 as &dyn Foo;
y.foo_in_trait();
y.foo_in_impl(); // May cause an error, see below
}
Editor's note
In versions of Rust up to and including Rust 1.15.0, the line
y.foo_in_impl() causes the error:
error: borrowed value does not live long enough
--> src/main.rs:20:14
|
20 | let y = &42u8 as &Foo;
| ^^^^ does not live long enough
...
23 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
This error is no longer present in subsequent versions, but the
concepts explained in the answers are still valid.
From this limited experiment, it seems like methods defined in the impl block are more restrictive than methods defined in the trait block. It's likely that there's something extra that doing it this way unlocks, but I just don't know what it is yet! ^_^
The sections from The Rust Programming Language on traits and trait objects don't make any mention of this. Searching the Rust source itself, it seems like only Any and Error use this particular feature. I've not seen this used in the handful of crates where I have looked at the source code.
When you define a trait named Foo that can be made into an object, Rust also defines a trait object type named dyn Foo. In older versions of Rust, this type was only called Foo (see What does "dyn" mean in a type?). For backwards compatibility with these older versions, Foo still works to name the trait object type, although dyn syntax should be used for new code.
Trait objects have a lifetime parameter that designates the shortest of the implementor's lifetime parameters. To specify that lifetime, you write the type as dyn Foo + 'a.
When you write impl dyn Foo { (or just impl Foo { using the old syntax), you are not specifying that lifetime parameter, and it defaults to 'static. This note from the compiler on the y.foo_in_impl(); statement hints at that:
note: borrowed value must be valid for the static lifetime...
All we have to do to make this more permissive is to write a generic impl over any lifetime:
impl<'a> dyn Foo + 'a {
fn foo_in_impl(&self) { println!("in impl") }
}
Now, notice that the self argument on foo_in_impl is a borrowed pointer, which has a lifetime parameter of its own. The type of self, in its full form, looks like &'b (dyn Foo + 'a) (the parentheses are required due to operator precedence). A Box<u8> owns its u8 – it doesn't borrow anything –, so you can create a &(dyn Foo + 'static) out of it. On the other hand, &42u8 creates a &'b (dyn Foo + 'a) where 'a is not 'static, because 42u8 is put in a hidden variable on the stack, and the trait object borrows this variable. (That doesn't really make sense, though; u8 doesn't borrow anything, so its Foo implementation should always be compatible with dyn Foo + 'static... the fact that 42u8 is borrowed from the stack should affect 'b, not 'a.)
Another thing to note is that trait methods are polymorphic, even when they have a default implementation and they're not overridden, while inherent methods on a trait objects are monomorphic (there's only one function, no matter what's behind the trait). For example:
use std::any::type_name;
trait Foo {
fn foo_in_trait(&self)
where
Self: 'static,
{
println!("{}", type_name::<Self>());
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("{}", type_name::<Self>());
}
}
impl Foo for u8 {}
impl Foo for u16 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let x = Box::new(42u16) as Box<Foo>;
x.foo_in_trait();
x.foo_in_impl();
}
Sample output:
u8
dyn playground::Foo
u16
dyn playground::Foo
In the trait method, we get the type name of the underlying type (here, u8 or u16), so we can conclude that the type of &self will vary from one implementer to the other (it'll be &u8 for the u8 implementer and &u16 for the u16 implementer – not a trait object). However, in the inherent method, we get the type name of dyn Foo (+ 'static), so we can conclude that the type of &self is always &dyn Foo (a trait object).
I suspect that the reason is very simple: may be overridden or not?
A method implemented in a trait block can be overridden by implementors of the trait, it just provides a default.
On the other hand, a method implemented in an impl block cannot be overridden.
If this reasoning is right, then the error you get for y.foo_in_impl() is just a lack of polish: it should have worked. See Francis Gagné's more complete answer on the interaction with lifetimes.
References to wrapper types like &Rc<T> and &Box<T> are invariant in T (&Rc<T> is not a &Rc<U> even if T is a U). A concrete example of the issue (Rust Playground):
use std::rc::Rc;
use std::rc::Weak;
trait MyTrait {}
struct MyStruct {
}
impl MyTrait for MyStruct {}
fn foo(rc_trait: Weak<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
foo(Rc::downgrade(&a));
}
This code results in the following error:
<anon>:15:23: 15:25 error: mismatched types:
expected `&alloc::rc::Rc<MyTrait>`,
found `&alloc::rc::Rc<MyStruct>`
Similar example (with similar error) with Box<T> (Rust Playground):
trait MyTrait {}
struct MyStruct {
}
impl MyTrait for MyStruct {}
fn foo(rc_trait: &Box<MyTrait>) {}
fn main() {
let a = Box::new(MyStruct {});
foo(&a);
}
In these cases I could of course just annotate a with the desired type, but in many cases that won't be possible because the original type is needed as well. So what do I do then?
What you see here is not related to variance and subtyping at all.
First, the most informative read on subtyping in Rust is this chapter of Nomicon. You can find there that in Rust subtyping relationship (i.e. when you can pass a value of one type to a function or a variable which expects a variable of different type) is very limited. It can only be observed when you're working with lifetimes.
For example, the following piece of code shows how exactly &Box<T> is (co)variant:
fn test<'a>(x: &'a Box<&'a i32>) {}
fn main() {
static X: i32 = 12;
let xr: &'static i32 = &X;
let xb: Box<&'static i32> = Box::new(xr); // <---- start of box lifetime
let xbr: &Box<&'static i32> = &xb;
test(xbr); // Covariance in action: since 'static is longer than or the
// same as any 'a, &Box<&'static i32> can be passed to
// a function which expects &'a Box<&'a i32>
//
// Note that it is important that both "inner" and "outer"
// references in the function signature are defined with
// the same lifetime parameter, and thus in `test(xbr)` call
// 'a gets instantiated with the lifetime associated with
// the scope I've marked with <----, but nevertheless we are
// able to pass &'static i32 as &'a i32 because the
// aforementioned scope is less than 'static, therefore any
// shared reference type with 'static lifetime is a subtype of
// a reference type with the lifetime of that scope
} // <---- end of box lifetime
This program compiles, which means that both & and Box are covariant over their respective type and lifetime parameters.
Unlike most of "conventional" OOP languages which have classes/interfaces like C++ and Java, in Rust traits do not introduce subtyping relationship. Even though, say,
trait Show {
fn show(&self) -> String;
}
highly resembles
interface Show {
String show();
}
in some language like Java, they are quite different in semantics. In Rust bare trait, when used as a type, is never a supertype of any type which implements this trait:
impl Show for i32 { ... }
// the above does not mean that i32 <: Show
Show, while being a trait, indeed can be used in type position, but it denotes a special unsized type which can only be used to form trait objects. You cannot have values of the bare trait type, therefore it does not even make sense to talk about subtyping and variance with bare trait types.
Trait objects take form of &SomeTrait or &mut SomeTrait or SmartPointer<SomeTrait>, and they can be passed around and stored in variables and they are needed to abstract away the actual implementation of the trait. However, &T where T: SomeTrait is not a subtype of &SomeTrait, and these types do not participate in variance at all.
Trait objects and regular pointers have incompatible internal structure: &T is just a regular pointer to a concrete type T, while &SomeTrait is a fat pointer which contains a pointer to the original value of a type which implements SomeTrait and also a second pointer to a vtable for the implementation of SomeTrait of the aforementioned type.
The fact that passing &T as &SomeTrait or Rc<T> as Rc<SomeTrait> works happens because Rust does automatic coercion for references and smart pointers: it is able to construct a fat pointer &SomeTrait for a regular reference &T if it knows T; this is quite natural, I believe. For instance, your example with Rc::downgrade() works because Rc::downgrade() returns a value of type Weak<MyStruct> which gets coerced to Weak<MyTrait>.
However, constructing &Box<SomeTrait> out of &Box<T> if T: SomeTrait is much more complex: for one, the compiler would need to allocate a new temporary value because Box<T> and Box<SomeTrait> has different memory representations. If you have, say, Box<Box<T>>, getting Box<Box<SomeTrait>> out of it is even more complex, because it would need creating a new allocation on the heap to store Box<SomeTrait>. Thus, there are no automatic coercions for nested references and smart pointers, and again, this is not connected with subtyping and variance at all.
In the case of Rc::downgrade this is actually just a failure of the type inference in this particular case, and will work if it is done as a separate let:
fn foo(rc_trait: Weak<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
let b = Rc::downgrade(&a);
foo(b);
}
Playground
For Box<T> it is very likely you don't actually want a reference to the box as the argument, but a reference to the contents. In which case there is no invariance to deal with:
fn foo(rc_trait: &MyTrait) {}
fn main() {
let a = Box::new(MyStruct {});
foo(a.as_ref());
}
Playground
Similarly, for the case with Rc<T>, if you write a function that takes an Rc<T> you probably want a clone (i.e. a reference counted reference), and not a normal reference:
fn foo(rc_trait: Rc<MyTrait>) {}
fn main() {
let a = Rc::new(MyStruct {});
foo(a.clone());
}
Playground
While trying to understand the Any trait better, I saw that it has an impl block for the trait itself. I don't understand the purpose of this construct, or even if it has a specific name.
I made a little experiment with both a "normal" trait method and a method defined in the impl block:
trait Foo {
fn foo_in_trait(&self) {
println!("in foo")
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("in impl")
}
}
impl Foo for u8 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let y = &42u8 as &dyn Foo;
y.foo_in_trait();
y.foo_in_impl(); // May cause an error, see below
}
Editor's note
In versions of Rust up to and including Rust 1.15.0, the line
y.foo_in_impl() causes the error:
error: borrowed value does not live long enough
--> src/main.rs:20:14
|
20 | let y = &42u8 as &Foo;
| ^^^^ does not live long enough
...
23 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
This error is no longer present in subsequent versions, but the
concepts explained in the answers are still valid.
From this limited experiment, it seems like methods defined in the impl block are more restrictive than methods defined in the trait block. It's likely that there's something extra that doing it this way unlocks, but I just don't know what it is yet! ^_^
The sections from The Rust Programming Language on traits and trait objects don't make any mention of this. Searching the Rust source itself, it seems like only Any and Error use this particular feature. I've not seen this used in the handful of crates where I have looked at the source code.
When you define a trait named Foo that can be made into an object, Rust also defines a trait object type named dyn Foo. In older versions of Rust, this type was only called Foo (see What does "dyn" mean in a type?). For backwards compatibility with these older versions, Foo still works to name the trait object type, although dyn syntax should be used for new code.
Trait objects have a lifetime parameter that designates the shortest of the implementor's lifetime parameters. To specify that lifetime, you write the type as dyn Foo + 'a.
When you write impl dyn Foo { (or just impl Foo { using the old syntax), you are not specifying that lifetime parameter, and it defaults to 'static. This note from the compiler on the y.foo_in_impl(); statement hints at that:
note: borrowed value must be valid for the static lifetime...
All we have to do to make this more permissive is to write a generic impl over any lifetime:
impl<'a> dyn Foo + 'a {
fn foo_in_impl(&self) { println!("in impl") }
}
Now, notice that the self argument on foo_in_impl is a borrowed pointer, which has a lifetime parameter of its own. The type of self, in its full form, looks like &'b (dyn Foo + 'a) (the parentheses are required due to operator precedence). A Box<u8> owns its u8 – it doesn't borrow anything –, so you can create a &(dyn Foo + 'static) out of it. On the other hand, &42u8 creates a &'b (dyn Foo + 'a) where 'a is not 'static, because 42u8 is put in a hidden variable on the stack, and the trait object borrows this variable. (That doesn't really make sense, though; u8 doesn't borrow anything, so its Foo implementation should always be compatible with dyn Foo + 'static... the fact that 42u8 is borrowed from the stack should affect 'b, not 'a.)
Another thing to note is that trait methods are polymorphic, even when they have a default implementation and they're not overridden, while inherent methods on a trait objects are monomorphic (there's only one function, no matter what's behind the trait). For example:
use std::any::type_name;
trait Foo {
fn foo_in_trait(&self)
where
Self: 'static,
{
println!("{}", type_name::<Self>());
}
}
impl dyn Foo {
fn foo_in_impl(&self) {
println!("{}", type_name::<Self>());
}
}
impl Foo for u8 {}
impl Foo for u16 {}
fn main() {
let x = Box::new(42u8) as Box<dyn Foo>;
x.foo_in_trait();
x.foo_in_impl();
let x = Box::new(42u16) as Box<Foo>;
x.foo_in_trait();
x.foo_in_impl();
}
Sample output:
u8
dyn playground::Foo
u16
dyn playground::Foo
In the trait method, we get the type name of the underlying type (here, u8 or u16), so we can conclude that the type of &self will vary from one implementer to the other (it'll be &u8 for the u8 implementer and &u16 for the u16 implementer – not a trait object). However, in the inherent method, we get the type name of dyn Foo (+ 'static), so we can conclude that the type of &self is always &dyn Foo (a trait object).
I suspect that the reason is very simple: may be overridden or not?
A method implemented in a trait block can be overridden by implementors of the trait, it just provides a default.
On the other hand, a method implemented in an impl block cannot be overridden.
If this reasoning is right, then the error you get for y.foo_in_impl() is just a lack of polish: it should have worked. See Francis Gagné's more complete answer on the interaction with lifetimes.