Basically, I want to implement pipeline-operator (binary operator for function application) on Box<T> using Operator overloading in Rust.
I understand Box<T> is a structure
https://doc.rust-lang.org/src/alloc/boxed.rs.html#175-178
and the below is the code I found here for struct Wrapped<T>(T) that has the pipeline-operator as Shr >>.
https://stackoverflow.com/a/17111630
use std::ops::Shr;
struct Wrapped<T>(T);
impl<A, B, F> Shr<F> for Wrapped<A>
where
F: FnOnce(A) -> B,
{
type Output = Wrapped<B>;
fn shr(self, f: F) -> Wrapped<B> {
Wrapped(f(self.0))
}
}
fn main() {
let string = Wrapped(1) >> (|x| x + 1) >> (|x| 2 * x) >> (|x: i32| x.to_string());
println!("{}", string.0);
}
// prints `4`
Here I want Box<T> instead of Wrapped<T> for the pipeline-operator.
I also found a code here to overload Add::add for Vector
https://stackoverflow.com/a/28005283
impl<'a, 'b> Add<&'b Vector> for &'a Vector {
type Output = Vector;
fn add(self, other: &'b Vector) -> Vector {
Vector {
x: self.x + other.x,
y: self.y + other.y,
}
}
}
Therefore, I suppose there must be a way to implement pipeline-operator (binary operator for function application) on Box<T>.
So far, I did
use std::ops::Shr;
impl<A, B, F> Shr<F> for Box<A>
where
F: FnOnce(A) -> B,
{
type Output = Box<B>;
fn shr(self, f: F) -> Box<B> {
Box::new(f(&self))
}
}
etc. but the code doesn't work.
error[E0210]: type parameter `A` must be used as the type parameter for some local type (e.g., `MyStruct<A>`)
--> src/main.rs:6:10
|
6 | impl<A, B, F> Shr<F> for Box<A>
| ^ type parameter `A` must be used as the type parameter for some local type
|
= note: implementing a foreign trait is only possible if at least one of the types for which it is implemented is local
= note: only traits defined in the current crate can be implemented for a type parameter
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=5eb74e35e2b5c23b0ab9ba020f1a7b54
Can you fix the code? Thanks.
Related
question and my goal
My goal is to let this (playground) compile. Below is the core of the code. This is attempt to implement FlatMap for all F<F<A>>, such as Option<Option<i32>>.
trait HKT1 {
type Unwrapped;
type Wrapped<T>;
}
trait FlatMap: HKT1 + Sized {
fn flat_map<B, F>(self, f: F) -> Self::Wrapped<B>
where
F: FnOnce(Self::Unwrapped) -> Self::Wrapped<B>;
// Below cannot compile
fn flatten<A>(ffa: Self::Wrapped<Self::Wrapped<A>>) -> Self::Wrapped<A>
// How to set generic bound correctly?
where
Self::Wrapped<Self::Wrapped<A>>: FlatMap
{
ffa.flat_map(|x| x)
}
}
error[E0308]: mismatched types
--> src/main.rs:15:26
|
15 | ffa.flat_map(|x| x)
| ^ expected HKT1::Wrapped, found HKT1::Unwrapped
|
= note: expected associated type `<<Self as HKT1>::Wrapped<<Self as HKT1>::Wrapped<A>> as HKT1>::Wrapped<_>`
found associated type `<<Self as HKT1>::Wrapped<<Self as HKT1>::Wrapped<A>> as HKT1>::Unwrapped`
error[E0308]: mismatched types
--> src/main.rs:15:9
|
6 | trait FlatMap: HKT1 + Sized {
| --------------------------- this type parameter
...
11 | fn flatten<A>(ffa: Self::Wrapped<Self::Wrapped<A>>) -> Self::Wrapped<A>
| ---------------- expected `<Self as HKT1>::Wrapped<A>` because of return type
...
15 | ffa.flat_map(|x| x)
| ^^^^^^^^^^^^^^^^^^^ expected type parameter `Self`, found associated type
|
= note: expected associated type `<Self as HKT1>::Wrapped<A>`
found associated type `<<Self as HKT1>::Wrapped<<Self as HKT1>::Wrapped<A>> as HKT1>::Wrapped<_>`
= note: you might be missing a type parameter or trait bound
HKT1 is a way to emulate F<_> and allow all traits that are based on it to access F and _ separately. I learned from this blog.
what I am trying to do
I'm trying to build up a functional programming crate which is kind of translating Scala lib cats to Rust. (Current work here). To implement the higher kinded type, I follow this blog and define the HKT1 trait. It is quite elegant until I want to implement flatten(ffa: F<F<A>>) -> F<A> but cannot figure out how to set the generic bound correctly.
It seems easy to implement if I leave it blank in the trait. For example:
trait FlatMap: HKT1 + Sized {
fn flat_map<B, F>(self, f: F) -> Self::Wrapped<B>
where
F: FnOnce(Self::Unwrapped) -> Self::Wrapped<B>;
fn flatten<A>(ffa: Self::Wrapped<Self::Wrapped<A>>) -> Self::Wrapped<A>;
}
struct MyF<T>(pub T);
impl<A> HKT1 for MyF<A> {
type Unwrapped = A;
type Wrapped<T> = MyF<T>;
}
impl<T> FlatMap for MyF<T> {
fn flat_map<B, F>(self, f: F) -> MyF<B>
where
F: FnOnce(Self::Unwrapped) -> MyF<B>,
{
f(self.0)
}
fn flatten<A>(ffa: MyF<MyF<A>>) -> MyF<A> {
ffa.flat_map(|x| x)
}
}
But I would like to have a default implementation through flat_map. How can I achieve this?
A solution via EqT
I just learned how to achieve this from the implementation of crate functional. (Well, the implementation is a little different from above)
The key idea is to add a trait EqT, whose purpose is to assert two types are equal. (playground)
trait HKT1 {
type Unwrapped;
type Wrapped<T>;
}
trait FlatMap: HKT1 + Sized {
fn flat_map<B, F>(self, f: F) -> Self::Wrapped<B>
where
F: FnOnce(Self::Unwrapped) -> Self::Wrapped<B>;
fn flatten<A>(self) -> Self::Wrapped<A>
where
Self::Unwrapped: EqT<Self::Wrapped<A>>,
{
self.flat_map(|x| x.cast())
}
}
trait EqT<T> {
fn cast(self) -> T;
}
impl<T> EqT<T> for T {
fn cast(self) -> T {
self
}
}
Then we can easily implement and use flatten:
struct MyF<T>(pub T);
impl<A> HKT1 for MyF<A> {
type Unwrapped = A;
type Wrapped<T> = MyF<T>;
}
impl<T> FlatMap for MyF<T> {
fn flat_map<B, F>(self, f: F) -> MyF<B>
where
F: FnOnce(Self::Unwrapped) -> MyF<B>,
{
f(self.0)
}
}
let ffa = MyF(MyF(1));
let fa = ffa.flatten();
println!("{:?}", fa); // Expect MyF(1)
I'm trying to wrap an async function in a struct. For example:
use std::future::Future;
struct X;
struct Y;
async fn f(x: &X) -> Y {
Y
}
struct MyStruct<F, Fut>(F)
where
F: Fn(&X) -> Fut,
Fut: Future<Output = Y>;
fn main() {
MyStruct(f);
}
The compiler complains about this with the following (unhelpful) error:
error[E0308]: mismatched types
--> src/main.rs:16:5
|
16 | MyStruct(f);
| ^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&X) -> impl Future {f} as FnOnce<(&X,)>>::Output`
found associated type `<for<'_> fn(&X) -> impl Future {f} as FnOnce<(&X,)>>::Output`
Is something like this actually possible? As I understand it, f desugars to something like:
fn f<'a>(x: &'a X) -> impl Future<Output = Y> + 'a {
Y
}
so I'd need to somehow express in MyStruct that Fut has the same lifetime as x.
I actually don't know much about async.
However, when it comes to trait-bounds, typically, fewer trait-bound are better. In other words, only declare those trait-bounds that you really need.
In the case of a struct, as long as you don't need an associated type within your struct, you are mostly good without any bounds. This is pretty much to what #George Glavan wrote in his answer.
When you add methods to your struct, you are more likely to use traits and thus require trait-bounds. Sometimes it is useful to combine the trait-bound of multiple functions by declaring it on the impl-block itself. Tho, this has some restrictions. You should also consider whether each function really needs all these constraints.
For instance, consider the following code:
struct X;
struct Y;
struct MyStruct<F>(F);
impl<F> MyStruct<F> {
pub fn new(f: F) -> Self {
MyStruct(f)
}
pub fn invoke<'a, Fut>(&self) -> Fut
where
F: Fn(&'a X) -> Fut,
{
(self.0)(&X)
}
}
I added a new and a invoke function. The former doesn't require any traits, thus it doesn't have trait-bounds. The latter only calls the function, so it bounds F by Fn. And this is good enough, because in the end, the caller must already know what the return type is, i.e. whether it is some Future or not.
However, there are a few cases where one really needs additional trait-bounds, which involves additional generics such as for a function return type. In such a case, you can declare additional (phantom) generics on a struct, e.g.:
use std::future::Future;
use std::marker::PhantomData;
struct X;
struct Y;
struct MyStruct<F, Fut> {
func: F,
_fut: PhantomData<Fut>,
}
impl<'a, F, Fut> MyStruct<F, Fut>
where
F: Fn(&'a X) -> Fut,
Fut: Future<Output = Y> + Send + Sync + 'a,
{
pub fn new(f: F) -> Self {
MyStruct {
func: f,
_fut: PhantomData,
}
}
pub fn invoke(&self) {
(self.func)(&X);
}
}
Notice, that in this example the trait-bounds apply to both function new and invoke, and both are over-restricted. Still, you don't need to over-restrict the struct itself.
I want to write a generic function count_calls which calls a function f which takes a function pointer (lambda) where count_calls counts how often function f called the given lambda function.
I struggle with the approach (Playground).
fn count_calls<S, F>(s: S, f: F) -> u32
where
S: Clone,
F: Sized + FnMut(Fn() -> S) -> (),
{
let mut counter: u32 = 0;
f(|| {
counter += 1;
s.clone()
});
counter
}
#[cfg(test)]
mod stackoverflow {
use super::*;
fn f(p: fn() -> i32) {
p();
p();
}
#[test]
fn test() {
let counts = count_calls(3, f);
assert_eq!(counts, 2);
}
}
Here I get the error:
error[E0277]: the size for values of type `(dyn std::ops::Fn() -> S + 'static)` cannot be known at compilation time
--> src/lib.rs:1:1
|
1 | / fn count_calls<S, F>(s: S, f: F) -> u32
2 | | where
3 | | S: Clone,
4 | | F: Sized + FnMut(Fn() -> S) -> (),
... |
12 | | counter
13 | | }
| |_^ doesn't have a size known at compile-time
|
= help: within `((dyn std::ops::Fn() -> S + 'static),)`, the trait `std::marker::Sized` is not implemented for `(dyn std::ops::Fn() -> S + 'static)`
= note: to learn more, visit <https://doc.rust-lang.org/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait>
= note: required because it appears within the type `((dyn std::ops::Fn() -> S + 'static),)`
= note: required by `std::ops::FnMut`
Does someone know how to fix this?
[Edit]
I think using Box<Fn()->S> might be a solution. But I would prefer a stack only solution, if possible.
The error "the size for values of type (dyn std::ops::Fn() -> S + 'static) cannot be known at compilation time" is caused by your trait bound for F:
F: Sized + FnMut(Fn() -> S) -> ()
This is equivalent to F: Sized + FnMut(dyn Fn() -> S). This means that the closure F would take a trait object (dyn Fn() -> S) by value. But trait objects are unsized and cannot be passed by value (yet).
One solution would be to pass the trait object by reference or in a Box. The answer by rodrigo explains and discusses these solutions.
Can we avoid trait objects and dynamic dispatch?
Not properly, I think.
Non solutions
One idea would be to add another type parameter to count_calls:
fn count_calls<S, F, G>(s: S, f: F) -> u32
where
S: Clone,
F: Sized + FnMut(G),
G: Fn() -> S,
However, this doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:9:7
|
9 | f(|| {
| _______^
10 | | counter += 1;
11 | | s.clone()
12 | | });
| |_____^ expected type parameter, found closure
|
= note: expected type `G`
found type `[closure#src/lib.rs:9:7: 12:6 counter:_, s:_]`
The problem here is that type arguments of count_calls are chosen by the caller of count_calls. But we actually want G to always be the type of our own closure. So that doesn't work.
What we want is a generic closure (where we can choose it's type parameters). Something similar is possible, but restricted to lifetime parameters. It's called HRTBs and looks like F: for<'a> Fn(&'a u32). But it doesn't help here because we need a type parameter and for<T> doesn't exist (yet?).
Sub-optimal, nightly solution
One solution would be to not use a closure, but a type with a known name which implements FnMut. Sadly, you can't implement the Fn* traits for your own type on stable yet. On nightly, this works.
struct CallCounter<S> {
counter: u32,
s: S,
}
impl<S: Clone> FnOnce<()> for CallCounter<S> {
type Output = S;
extern "rust-call" fn call_once(self, _: ()) -> Self::Output {
// No point in incrementing the counter here
self.s
}
}
impl<S: Clone> FnMut<()> for CallCounter<S> {
extern "rust-call" fn call_mut(&mut self, _: ()) -> Self::Output {
self.counter += 1;
self.s.clone()
}
}
fn count_calls<S, F>(s: S, mut f: F) -> u32
where
S: Clone,
F: Sized + FnMut(&mut CallCounter<S>), // <----
{
let mut counter = CallCounter {
counter: 0,
s,
};
f(&mut counter); // <-------
counter.counter
}
Unfortunately, now you have this strange type in your public interface (which should be implementation detail).
Apart from that, I can't think of any real solution (only other super verbose solutions with plenty of disadvantages). The developments in the type system corner (in particular in regards to GATs and HKTs) could solve this properly in the future. However, I think there are still a few different features lacking; in particular, I don't think that GATs as proposed would already solve this.
So if this is a real life problem which needs to be solved right now, I would:
step back and rethink the problem in a bigger scope to maybe avoid this Rust limitation, or
just use dynamic dispatch.
This is the simplest code that I managed to get working (playground):
fn count_calls<S, F>(s: S, mut f: F) -> u32
where
S: Clone,
F: FnMut(&mut dyn FnMut() -> S) -> (),
{
let mut counter: u32 = 0;
f(&mut || {
counter += 1;
s.clone()
});
counter
}
#[cfg(test)]
mod stackoverflow {
use super::*;
fn f(p: &mut dyn FnMut() -> i32) {
p();
p();
}
#[test]
fn test() {
let counts = count_calls(3, f);
assert_eq!(counts, 2);
}
}
The key change is that the function argument for F is changed from Fn() -> S into &mut dyn FnMut() -> S. You need a reference because you are using dynamic dispatching. Also you need FnMut because you are capturing counter and changing it inside, and a Fn will not allow it.
Note that you cannot use Box<FnMut() -> S. It will not allow capturing a reference to counter because boxed functions must be 'static.
If you find that changing your Fn to FnMut is undesirable (because you are changing your public API) you could go back to F: FnMut(&mut dyn Fn() -> S) -> () by defining counter as a Cell<u32>:
fn count_calls<S, F>(s: S, mut f: F) -> u32
where
S: Clone,
F: FnMut(&dyn Fn() -> S) -> (),
{
let counter: Cell<u32> = Cell::new(0);
f(&|| {
counter.set(counter.get() + 1);
s.clone()
});
counter.into_inner()
}
The compiler allows me to write blanket implementation a function like this:
trait Invoke {
type S;
type E;
fn fun(&mut self) -> Result<Self::S, Self::E>;
}
impl<F, S, E> Invoke for F
where
F: Fn() -> Result<S, E>,
{
type S = S;
type E = E;
fn fun(&mut self) -> Result<S, E> {
self()
}
}
but it starts complaining when I try to add a function parameter:
trait Invoke {
type A;
type S;
type E;
fn fun(&mut self, arg: Self::A) -> Result<Self::S, Self::E>;
}
impl<F, A, S, E> Invoke for F
where
F: Fn(A) -> Result<S, E>,
{
type A = A;
type S = S;
type E = E;
fn fun(&mut self, arg: A) -> Result<S, E> {
self(arg)
}
}
error[E0207]: the type parameter `A` is not constrained by the impl trait, self type, or predicates
--> src/lib.rs:9:9
|
9 | impl<F, A, S, E> Invoke for F
| ^ unconstrained type parameter
error[E0207]: the type parameter `S` is not constrained by the impl trait, self type, or predicates
--> src/lib.rs:9:12
|
9 | impl<F, A, S, E> Invoke for F
| ^ unconstrained type parameter
error[E0207]: the type parameter `E` is not constrained by the impl trait, self type, or predicates
--> src/lib.rs:9:15
|
9 | impl<F, A, S, E> Invoke for F
| ^ unconstrained type parameter
I cannot understand why these two cases are different. Isn't A a part of constraint signature?
I realized I can rewrite it like the Fn trait declaration, but I still do not get the idea:
trait Invoke<A> {
type S;
type E;
fn fun(&mut self, arg: A) -> Result<Self::S, Self::E>;
}
impl<F, A, S, E> Invoke<A> for F
where
F: Fn(A) -> Result<S, E>,
{
type S = S;
type E = E;
fn fun(&mut self, arg: A) -> Result<S, E> {
self(arg)
}
}
Type parameters represent "input" types, while associated types represent "output" types.
Rust allows you to implement multiple instances of a generic trait so long as the combination of type parameters are unique. For example, a single struct Foo could implement PartialEq<Foo> and PartialEq<Bar> together.
In contrast, associated types are assigned by the trait implementation. For example, the Add trait has a type parameter, RHS, and an associated type, Output. For each combination of Self (the type on which the trait is implemented) and RHS, the associated type Output is fixed.
The main reason for using associated types is to reduce the number of type parameters on traits, especially where uses of that trait might have to define a type parameter just to properly bound that trait. However, associated types are not always appropriate; that's why we still have type parameters!
The Fn(Args) -> Output syntax for the Fn trait (and its friends FnMut and FnOnce) hides the underlying implementation of these traits. Here's your first impl again with the unstable "low-level" syntax:
#![feature(unboxed_closures)]
impl<F, S, E> Invoke for F
where
F: Fn<(), Output = Result<S, E>>,
{
type S = S;
type E = E;
fn fun(&mut self) -> Result<S, E> {
self()
}
}
As you can see, the function's result type is an associated type, named Output. Output = Result<S, E> is a predicate, so that satisfies one of the compiler's conditions for type parameters on impl blocks.
Now, here's your second impl with the unstable syntax:
#![feature(unboxed_closures)]
impl<F, A, S, E> Invoke for F
where
F: Fn<(A,), Output = Result<S, E>>,
{
type A = A;
type S = S;
type E = E;
fn fun(&mut self, arg: A) -> Result<S, E> {
self(arg)
}
}
Here, A is used in Fn's type parameter.
Why is this not valid? In theory1, a single type could have multiple implementations of Fn<Args> with different values of Args. Which implementation should the compiler select in that case? You can only choose one, because A is not passed as a type parameter to Invoke, and thus F can only have a single implementation of Invoke.
1 In practice, you need to use a nightly compiler to do this, because implementing Fn, FnMut or FnOnce directly is an unstable feature. On a stable versions, the compiler will only generate up to one implementation of each of these traits for functions and closures. Also, you could have the same issue with any other trait that has type parameters, even on a stable compiler.
See also:
Why can't I add a blanket impl on a trait with a type parameter?
Rust issue 25041: type parameter not constrained when using closure bounds
I am trying to write a function that can descent into any kind of Value and inform a Delegate about the similarities it observes. The idea is to make this work across all kinds of Json/Yaml/YouNameIt values, generically.
Here is an MVCE to trigger the issue (playground link):
pub trait Value: PartialEq<Self> {
type Item;
type Key;
fn items<'a>(&'a self) -> Option<Box<Iterator<Item = (Self::Key, &'a Self::Item)> + 'a>>;
}
pub trait Delegate<'a, V> {
fn something(&mut self, _v: &'a V) {}
}
pub fn diff<'a, V, D>(l: &'a V, d: &'a mut D)
where V: Value,
<V as Value>::Item: Value,
D: Delegate<'a, V>
{
d.something(l);
let v = l.items().unwrap().next().unwrap();
d.something(v.1);
}
struct Recorder;
impl<'a, V> Delegate<'a, V> for Recorder {}
#[derive(PartialEq)]
struct RecursiveValue;
impl Value for RecursiveValue {
type Key = usize;
type Item = RecursiveValue;
fn items<'a>(&'a self) -> Option<Box<Iterator<Item = (Self::Key, &'a Self::Item)> + 'a>> {
None
}
}
fn main() {
let v = RecursiveValue;
let mut r = Recorder;
diff(&v, &mut r);
}
When attempting to compile the code, the following error is produced:
error[E0308]: mismatched types
--> <anon>:19:17
|
19 | d.something(v.1);
| ^^^ expected type parameter, found associated type
|
= note: expected type `&'a V`
= note: found type `&<V as Value>::Item`
I'm trying to say that the associated Item type is of type V too. Is there a way to make such an algorithm work generically?
The answer lies at the very bottom of the Associated Types chapter of the Rust Book.
When using a generic type in a bound, as in V: Value, it is possible to constrain one or several of its associated types to specific types by using the Generic<AssociatedType = SpecificType> syntax.
In your case, it means constraining V to Value<Item = V>. This should also obsolete any reason to further constrain V::Item since the bounds to V are naturally available.
I do encourage you to read the book to help you learn Rust, or at least skim it to know what's available there and be able to refer to it when you have a difficulty.
Here's a further reduced example:
pub trait Value {
type Item;
fn items(&self) -> &Self::Item;
}
pub trait Delegate<V> {
fn something(&mut self, v: &V);
}
pub fn diff<V, D>(l: &V, d: &mut D)
where V: Value,
V::Item: Value,
D: Delegate<V>
{
let v = l.items();
d.something(v);
}
fn main() {}
The important thing to focus on are the restrictions on the generics of diff:
pub fn diff<V, D>(l: &V, d: &mut D)
where V: Value,
V::Item: Value,
D: Delegate<V>
In words, this says:
V can be any type so long as it implements the Value trait.
V::Item can be any type so long as it implements the Value trait.
D can be any type so long as it implements the Delegate<V> trait.
Nowhere in that list of requirements was listed "V and V::Item must be the same". In fact, it's a feature that they are not required to be the same.
In this reduction, another solution would be to say D: Delegate<V::Item>. However, that wouldn't apply to the slightly larger reproduction:
pub fn diff<V, D>(l: &V, d: &mut D)
where V: Value,
V::Item: Value,
D: Delegate<V::Item>
{
d.something(l);
let v = l.items();
d.something(v);
}
As Matthieu M. has pointed out, you want to specify the associated type of the trait:
pub fn diff<V, D>(l: &V, d: &mut D)
where V: Value<Item = V>,
D: Delegate<V>
For further reading, check out Requiring implementation of Mul in generic function.