Type annotation for a closure - rust

I have this code that I use in several places:
iterable.iter().map(|elem| f(elem, &mut acc))
where f is a function like fn(T, &mut S) -> T, taking an input, returning an output, and using the acc: &mut S to store some intermediate state (e.g. a HashMap). I wanted to refactor the code and create something like
struct MyIterator {
iter: ???
}
impl MyIterator {
fn new(iterable: Vec<T>, acc: &mut S) -> Self {
Iterator {
iter: iterable.iter().map(|elem| f(elem, &mut acc))
}
}
}
But I don't know what should be the type signature in ???. Rust complains that closure cannot contain a captured variable.
Example
I didn't want to overwhelm the question with unnecessary details, but I'll add an example as requested. I want MyIterator to follow the Iterator trait. Let's say that when iterating, at each self.next() step I want to post-process the results produced self.iter.next(). For example, f returns Result<T, Error> and I want an Err to stop the iterator while keeping the error message.
struct MyIterator {
iter: ???
err: Result<(), ()>
}
impl MyItertor {
fn new(iterable: Vec<T>, acc: &mut S) -> Self {
MyIterator {
iter: iterable
.iter()
.map(|elem| f(elem, &mut ACC)),
err: Ok(()),
}
}
fn err(&self) -> Result<(), ()> {
self.err
}
}
impl Iterator for MyItertor {
type Item = T;
fn next(&mut self) -> Option<Self::Item> {
match self.iter.next() {
Ok(result) => result,
Err(err) => {
self.err = Err(err);
None
},
}
}
}
As for the types, it can be anything, making the code work for a trivial case like below will help me move along.
type T = i64;
type S = i64;
fn f(x: &T, acc: &mut S) -> Result<T, ()> {
let y = x + *acc;
*acc = y;
if *acc > 100 {
return Err(())
}
Ok(y)
}

First, you can make MyIterator generic over the type of the wrapped iterator:
struct MyIterator<I> {
iter: I,
err: Result<(), ()>,
}
Then, you can declare a free-standing my_iter function to serve as the constructor for a MyIterator. It returns a MyIterator parameterized over an anonymous iterator type via the impl Trait syntax:
fn my_iter<'a>(
iterable: &'a Vec<T>,
acc: &'a mut S,
) -> MyIterator<impl Iterator<Item = Result<T, ()>> + 'a> {
MyIterator {
iter: iterable.iter().map(|elem| f(elem, acc)),
err: Ok(()),
}
}
impl<I> MyIterator<I> {
fn err(&self) -> Result<(), ()> {
self.err
}
}
Note that the associated type Iterator::Item in the return type must be equal to the return type of f. Also, my_iter is not placed in the MyIterator impl only because the function is not constrained by the type parameter I of the impl.
Iterator can then be implemented for MyIterator objects wrapping the type of iterator you need (i.e. the same type of iterator as created in my_iter):
impl<T, I: Iterator<Item=Result<T, ()>>> Iterator for MyIterator<I> {
type Item = T;
fn next(&mut self) -> Option<Self::Item> {
match self.iter.next() {
Some(Ok(result)) => Some(result),
Some(Err(err)) => {
self.err = Err(err);
None
},
None => None,
}
}
}
Playground

Related

What does this line of code in Rust mean?

I was going through rust-by-practice and came across a problem. I have no idea what a particular line of code does. This is the exact piece of code for context:
fn example1() {
// `T: Trait` is the commonly used way.
// `T: Fn(u32) -> u32` specifies that we can only pass a closure to `T`.
struct Cacher<T: Fn(u32) -> u32> {
calculation: T,
value: Option<u32>,
}
impl<T: Fn(u32) -> u32> Cacher<T> {
fn new(calculation: T) -> Cacher<T> {
Cacher {
calculation,
value: None,
}
}
fn value(&mut self, arg: u32) -> u32 {
match self.value {
Some(v) => v,
None => {
let v = (self.calculation)(arg); // This exact line of code
self.value = Some(v);
v
},
}
}
}
let mut cacher = Cacher::new(|x| x+1);
assert_eq!(cacher.value(10), __);
assert_eq!(cacher.value(15), __);
}
The calculation field of Cacher is a closure (type T where T: Fn(u32) -> u32), and that line is calling it with arg as an argument, then assigning the resulting u32 to v. Without the parentheses, let v = self.calculation(arg) will look for a method named calculation, which doesn't exist. The parenthesis force a different order of operations which produces a different result.

Return only an owned type from a trait method that can accept an owned or borrowed value as input

I want to have a trait that can be implemented for T and &T but has methods that always return T.
What I would like to do is something like this
use std::borrow::ToOwned;
trait Foo<X: ToOwned> {
fn f(&self, x: X) -> f64;
fn g(&self) -> X::Owned;
}
struct Float(f64);
impl Foo<f64> for Float {
fn f(&self, x: f64) -> f64 {
x + self.0
}
fn g(&self) -> f64 {
self.0 * 2.0
}
}
struct List(Vec<f64>);
impl Foo<&Vec<f64>> for List {
fn f(&self, x: &Vec<f64>) -> f64 {
x.iter().sum()
}
// Error here - `&Vec<f64>` return type expected
fn g(&self) -> Vec<f64> {
self.0.iter().map(|&x| 2.0 * x).collect()
}
}
fn main() {
let float = Float(2.0);
println!("{} {}", float.f(3.0), float.g());
let list = List(vec![0.0, 1.0, 2.0]);
println!("{} {:?}", list.f(&vec![1.0, 2.0]), list.g());
}
I know that one option is to have a trait that defines the output type like so
trait FooReturn {
type Output;
}
trait Foo<X: FooReturn> {
fn f(&self, x: X) -> f64;
fn g(&self) -> X::Output;
}
then implement the trait for all relevant types, but I was wondering if there was a more standard/robust way to do this.
This is how you would do it once specialization is complete. Meanwhile, I couldn't even get a simple working example to compile on 1.55.0-nightly.
#![feature(specialization)]
trait MaybeOwned {
type Owned;
}
default impl<X> MaybeOwned for X {
type Owned = X;
}
impl<'a, X> MaybeOwned for &'a X {
type Owned = X;
}
trait Foo<X: MaybeOwned> {
fn f(&self, x: &X) -> f64;
fn g(&self) -> <X as MaybeOwned>::Owned;
}

How to use dynamic dispatch with a method which takes an iterator as a parameter?

I am writing a command line application in rust for processing audio from a sensor. I would like the user to be able to choose an algorithm or filter to apply from several options. I was hoping to use dynamic dispatch to switch out a struct which implements my filter trait at runtime. However, this is not allowed by the compiler, because one of the trait methods takes a generic parameter.
How could I implement this same functionality, without causing any compiler troubles? I know that an easy solution is to change the parameter of the process method to an array or a vector, but this is my last resort, as I would much prefer to take an iterator or an IntoIterator, as it is more general, and suits my specific needs.
Here is some code which demonstrates the problem.
trait SensorFilter {
fn process(&self, sig: &mut impl Iterator<Item = f32>) -> Vec<f32>;
}
struct Alg1 {
mul: f32,
}
struct Alg2 {
add: f32,
}
impl SensorFilter for Alg1 {
fn process(&self, sig: &mut impl Iterator<Item = f32>) -> Vec<f32> {
sig.map(|x| x * self.mul).collect()
}
}
impl SensorFilter for Alg2 {
fn process(&self, sig: &mut impl Iterator<Item = f32>) -> Vec<f32> {
sig.map(|x| x * self.add).collect()
}
}
enum AlgChoice {
Alg1,
Alg2
}
fn main() {
let choice = AlgChoice::Alg1; // user chooses via command-line.
let mut sig = vec![0.,1.,2.,3.,4.,5.,6.].into_iter(); // iterator gets data from sensor.
// This doesn't work, because my trait cannot be made into an object.
let alg: &dyn SensorFilter = match choice {
AlgChoice::Alg1 => Alg1{mul:0.3},
_ => Alg2{add:1.2},
};
let result = alg.process(&mut sig);
println!("{:?}",result);
}
Thanks :)
The trick here is to change your generic function parameter to a generic trait parameter:
// Make the generic param into a type argument w/ constraints
trait SensorFilter<I> where I: Iterator<Item = f32> {
fn process(&self, sig: &mut I) -> Vec<f32>;
}
struct Alg1 {
mul: f32,
}
struct Alg2 {
add: f32,
}
// Implement trait for all I that match the iterator constraint
impl<I: Iterator<Item = f32>> SensorFilter<I> for Alg1 {
fn process(&self, sig: &mut I) -> Vec<f32> {
sig.map(|x| x * self.mul).collect()
}
}
impl<I: Iterator<Item = f32>> SensorFilter<I> for Alg2 {
fn process(&self, sig: &mut I) -> Vec<f32> {
sig.map(|x| x * self.add).collect()
}
}
enum AlgChoice {
Alg1,
Alg2
}
fn main() {
let choice = AlgChoice::Alg1; // user chooses via command-line.
let mut sig = vec![0.,1.,2.,3.,4.,5.,6.].into_iter(); // iterator gets data from sensor.
// Specify the type argument of your trait.
let alg: &dyn SensorFilter<std::vec::IntoIter<f32>> = match choice {
AlgChoice::Alg1 => &Alg1{mul:0.3},
_ => &Alg2{add:1.2},
};
let result = alg.process(&mut sig);
println!("{:?}",result);
}
The simplest way to make SensorFilter object safe is to simply change process to accept dyn Iterator instead of impl Iterator:
trait SensorFilter {
fn process(&self, sig: &mut dyn Iterator<Item = f32>) -> Vec<f32>;
}
If you couldn't do this, for example because Iterator were actually non-object-safe, you could instead extract the common, non-object-safe part into a second trait, and implement it automatically for everything that is SensorFilter:
// This trait is object-safe.
trait SensorFilter {
fn filter(&self, x: f32) -> f32;
}
// This trait will not be object-safe because it uses generics.
trait Process {
fn process<I: IntoIterator<Item = f32>>(&self, sig: I) -> Vec<f32>;
}
// The `?Sized` bound allows you to call `.process()` on `dyn SensorFilter`.
impl<T: ?Sized + SensorFilter> Process for T {
fn process<I: IntoIterator<Item = f32>>(&self, sig: I) -> Vec<f32> {
sig.into_iter().map(|x| self.filter(x)).collect()
}
}
// ...
impl SensorFilter for Alg1 {
fn filter(&self, x: f32) -> f32 {
x * self.mul
}
}
impl SensorFilter for Alg2 {
fn filter(&self, x: f32) -> f32 {
x * self.add
}
}
Playground
Note that instead of Iterator I used IntoIterator, which is strictly more general.
A variation on this idea, when you can't easily remove the genericity from SensorFilter, is to use double dispatch: write SensorFilter to use dyn Iterator instead of impl Iterator, and then write a convenience trait that just wraps it with the specific type:
trait SensorFilter {
fn process_dyn(&self, sig: &mut dyn Iterator<Item = f32>) -> Vec<f32>;
}
trait Process {
fn process<I: IntoIterator<Item = f32>>(&self, sig: I) -> Vec<f32>;
}
impl<T: ?Sized + SensorFilter> Process for T {
fn process<I: IntoIterator<Item = f32>>(&self, sig: I) -> Vec<f32> {
self.process_dyn(&mut sig.into_iter())
}
}

How can I create an iterator of &T from either a &Vec<T> or Vec<&T>?

I have an enum with two variants. Either it contains a reference to a Vec of Strings or it contains a Vec of references to Strings:
enum Foo<'a> {
Owned(&'a Vec<String>),
Refs(Vec<&'a String>),
}
I want to iterate over references to the Strings in this enum.
I tried to implement a method on Foo, but don't know how to make it return the right iterator:
impl<'a> Foo<'a> {
fn get_items(&self) -> Iter<'a, String> {
match self {
Foo::Owned(v) => v.into_iter(),
Foo::Refs(v) => /* what to put here? */,
}
}
}
fn main() {
let test: Vec<String> = vec!["a".to_owned(), "b".to_owned()];
let foo = Foo::Owned(&test);
for item in foo.get_items() {
// item should be of type &String here
println!("{:?}", item);
}
}
playground
What is an idiomatic method to achieve this abstraction over &Vec<T> and Vec<&T>? get_items may also return something different, as long as it implements the IntoIterator trait so that I can use it in the for loop.
You can't just use the std::slice::Iter type for this.
If you don't want to copy the strings or vector, you'll have to implement your own iterator, for example:
struct FooIter<'a, 'b> {
idx: usize,
foo: &'b Foo<'a>,
}
impl<'a, 'b> Iterator for FooIter<'a, 'b> {
type Item = &'a String;
fn next(&mut self) -> Option<Self::Item> {
self.idx += 1;
match self.foo {
Foo::Owned(v) => v.get(self.idx - 1),
Foo::Refs(v) => v.get(self.idx - 1).map(|s| *s),
}
}
}
impl<'a, 'b> Foo<'a> {
fn get_items(&'b self) -> FooIter<'a, 'b> {
FooIter { idx: 0, foo: self }
}
}
fn main() {
let test: Vec<String> = vec!["a".to_owned(), "b".to_owned()];
let foo = Foo::Owned(&test);
for item in foo.get_items() {
println!("{:?}", item);
}
let a = "a".to_string();
let b = "b".to_string();
let test: Vec<&String> = vec![&a, &b];
let foo = Foo::Refs(test);
for item in foo.get_items() {
println!("{:?}", item);
}
}
There is a handy crate, auto_enums, which can generate a type for you so a function can have multiple return types, as long as they implement the same trait. It's similar to the code in Denys Séguret's answer except it's all done for you by the auto_enum macro:
use auto_enums::auto_enum;
impl<'a> Foo<'a> {
#[auto_enum(Iterator)]
fn get_items(&self) -> impl Iterator<Item = &String> {
match self {
Foo::Owned(v) => v.iter(),
Foo::Refs(v) => v.iter().copied(),
}
}
}
Add the dependency by adding this in your Cargo.toml:
[dependencies]
auto_enums = "0.6.3"
If you don't want to implement your own iterator, you need dynamic dispatch for this, because you want to return different iterators depending on the enum variant.
We need a trait object (&dyn Trait, &mut dyn Trait or Box<dyn Trait>) to use dynamic dispatch:
impl<'a> Foo<'a> {
fn get_items(&'a self) -> Box<dyn Iterator<Item = &String> + 'a> {
match self {
Foo::Owned(v) => Box::new(v.into_iter()),
Foo::Refs(v) => Box::new(v.iter().copied()),
}
}
}
.copied() converts the Iterator<Item = &&String> into an Iterator<Item = &String>, so this doesn't actually copy anything :)
A few things you should know first:
You are most definitely going to have two different iterators, because they're different base types you're iterating over. Therefore I'm going to use a Box<dyn Iterator<Item = &'a _>>, but feel free to use an enum if this causes a quantifiable performance drop.
You need to introduce self's lifetime here, because what if we return an iterator whose lifetime is 'a, but 'a > 'self? Therefore we make a new lifetime (Which I'll call 'b.).
Now it's just a matter of wrangling with the reference layers:
Here's the implementation using the original types:
enum Foo<'a> {
Owned(&'a Vec<String>),
Refs(Vec<&'a String>)
}
impl<'a> Foo<'a> {
fn get_items<'b>(&'b self) -> Box<dyn Iterator<Item = &'a String> + 'b> {
match self {
Foo::Owned(v) => //v: &'a Vec<String>
Box::new(
v.iter() //Iterator<Item = &'a String> -- Good!
),
Foo::Refs(v) => //v: Vec<&'a String>
Box::new(
v.iter() //Iterator<Item = &'b &'a String> -- Bad!
.map(|x| *x) //Iterator<Item = &'a String> -- Good!
),
}
}
}
These types aren't really rust-like (Or more formally, idiomatic), so here's that version using slices and strs:
enum Foo<'a> {
Owned(&'a [String]),
Refs(Vec<&'a str>)
}
impl<'a> Foo<'a> {
fn get_items<'b>(&'b self) -> Box<dyn Iterator<Item = &'a str> + 'b> {
match self {
Foo::Owned(v) =>
Box::new(
v.into_iter()
.map(|x| &**x) //&'a String -> &'a str
),
Foo::Refs(v) =>
Box::new(
v.iter()
.map(|x| *x) //&'b &'a str -> &'a str
)/* what to put here? */,
}
}
}
Playground
Ideally you would want:
fn get_items(&self) -> impl Iterator<Item = &String> {
match self {
Foo::Owned(v) => v.into_iter(),
Foo::Refs(v) => v.iter().copied(),
}
}
The call to copied is here to convert an Iterator<Item = &&String> into the Iterator<Item = &String> we want. This doesn't work because the two match arms have different types:
error[E0308]: match arms have incompatible types
--> src/main.rs:12:30
|
10 | / match self {
11 | | Foo::Owned(v) => v.into_iter(),
| | ------------- this is found to be of type `std::slice::Iter<'_, std::string::String>`
12 | | Foo::Refs(v) => v.iter().copied(),
| | ^^^^^^^^^^^^^^^^^ expected struct `std::slice::Iter`, found struct `std::iter::Copied`
13 | | }
| |_________- `match` arms have incompatible types
|
= note: expected type `std::slice::Iter<'_, std::string::String>`
found type `std::iter::Copied<std::slice::Iter<'_, &std::string::String>>`
You can fix this error thanks to the itertools or either crates, which contain a handy adapter called Either (*) that allows you to choose dynamically between two iterators:
fn get_items(&self) -> impl Iterator<Item = &String> {
match self {
Foo::Owned(v) => Either::Left(v.into_iter()),
Foo::Refs(v) => Either::Right(v.iter().copied()),
}
}
playground

Strange behavior of HRTBs

I have this code:
use std::fmt::Debug;
struct S<A>
where
for<'a> A: Debug + 'a,
{
f: Box<Fn(A) -> i32>,
}
impl<A> S<A>
where
for<'a> A: Debug + 'a,
{
fn call(&self, a: A) {
println!("Return {:?}", (self.f)(a));
}
}
fn create<A>(f: Box<Fn(A) -> i32>) -> S<A>
where
for<'a> A: Debug + 'a,
{
S::<A> { f }
}
fn helper() {
let x = create::<&i32>(Box::new(|x: &i32| *x * 2));
let arg = 333;
x.call(&arg);
}
fn main() {
let x = helper();
}
It's failed to compile:
error[E0310]: the parameter type `A` may not live long enough
In code 2, I changed Fn(A) -> i32 to Fn(&A) -> i32, the code works.
...
f: Box<Fn(&A) -> i32>,
...
Since A is argument of Fn trait, it's a type that has Higher-Rank lifetime. It shouldn't be affected by the lifetime of struct S<A> .
But why can't code 1 be compiled?
How can I workaround it for borrow or non-borrow type A?
There is no easy way to make helper work in current Rust, even if you remove all the for<'a> A: Debug + 'a, bounds (which only further restricts what types A can be, whereas you want to allow more).
This is as simple as I can make your example:
struct S<A> {
f: Box<Fn(A) -> i32>,
}
impl<A> S<A> {
fn call(&self, a: A) {
println!("Return {:?}", (self.f)(a));
}
}
fn create<A>(f: Box<Fn(A) -> i32>) -> S<A> {
S { f }
}
fn helper() {
let x = create(Box::new(|x: &i32| *x * 2));
let arg = 333;
x.call(&arg);
}
fn main() {
helper();
}
The reason it doesn't work is that A "comes from the outside", and Rust can't infer that you want for<'a> S<&'a A>, it can't even talk about such a type.
Note that if let arg = 333; is placed above let x, this example does compile (because it infers a reference to arg specifically, not a for<'a>).
The closest you can get today is with an associated type on a trait with a lifetime parameter, e.g.:
// Emulating `type Type<'a>` by moving `'a` to the trait.
trait Apply<'a> {
type Type;
}
struct Plain<T>(std::marker::PhantomData<T>);
impl<'a, T> Apply<'a> for Plain<T> {
type Type = T;
}
struct Ref<T: ?Sized>(std::marker::PhantomData<T>);
impl<'a, T: ?Sized + 'a> Apply<'a> for Ref<T> {
type Type = &'a T;
}
struct S<A: for<'a> Apply<'a>> {
f: Box<for<'a> Fn(<A as Apply<'a>>::Type) -> i32>,
}
impl<A: for<'a> Apply<'a>> S<A> {
fn call<'a>(&self, a: <A as Apply<'a>>::Type) {
println!("Return {:?}", (self.f)(a));
}
}
fn create<A: for<'a> Apply<'a>>(
f: Box<for<'a> Fn(<A as Apply<'a>>::Type) -> i32>,
) -> S<A> {
S { f }
}
fn helper() {
let x = create::<Ref<i32>>(Box::new(|x: &i32| *x * 2));
let arg = 333;
x.call(&arg);
}
fn main() {
helper();
}
However, it turns out that this encoding hits https://github.com/rust-lang/rust/issues/52812, so it's not actually usable at the moment (and I'm not aware of an workaround).

Resources