How can I put an async function into a map in Rust? - rust

I cannot handle async functions when writing an async router for hyper.
This code:
use std::collections::HashMap;
use std::future::Future;
type BoxedResult<T> = Result<T, Box<dyn std::error::Error + Send + Sync>>;
type CalcFn = Box<dyn Fn(i32, i32) -> dyn Future<Output = BoxedResult<i32>>>;
async fn add(a: i32, b: i32) -> BoxedResult<i32> {
Ok(a + b)
}
async fn sub(a: i32, b: i32) -> BoxedResult<i32> {
Ok(a - b)
}
fn main() {
let mut map: HashMap<&str, CalcFn> = Default::default();
map.insert("add", Box::new(add));
map.insert("sub", Box::new(sub));
println!("map size: {}", map.len());
}
Generates the following compiler error:
error[E0271]: type mismatch resolving `<fn(i32, i32) -> impl std::future::Future {add} as std::ops::FnOnce<(i32, i32)>>::Output == dyn std::future::Future<Output = std::result::Result<i32, std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>>`
--> src/main.rs:17:23
|
17 | map.insert("add", Box::new(add));
| ^^^^^^^^^^^^^ expected opaque type, found trait std::future::Future
|
= note: expected type `impl std::future::Future`
found type `dyn std::future::Future<Output = std::result::Result<i32, std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>>`
= note: required for the cast to the object type `dyn std::ops::Fn(i32, i32) -> dyn std::future::Future<Output = std::result::Result<i32, std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>>`
error[E0271]: type mismatch resolving `<fn(i32, i32) -> impl std::future::Future {sub} as std::ops::FnOnce<(i32, i32)>>::Output == dyn std::future::Future<Output = std::result::Result<i32, std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>>`
--> src/main.rs:18:23
|
18 | map.insert("sub", Box::new(sub));
| ^^^^^^^^^^^^^ expected opaque type, found trait std::future::Future
|
= note: expected type `impl std::future::Future`
found type `dyn std::future::Future<Output = std::result::Result<i32, std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>>`
= note: required for the cast to the object type `dyn std::ops::Fn(i32, i32) -> dyn std::future::Future<Output = std::result::Result<i32, std::boxed::Box<dyn std::error::Error + std::marker::Send + std::marker::Sync>>>`
It seems there is a conflict between impl Future and dyn Future, but I have no idea how to handle it.

This happens because impl Future is a concrete unique type while dyn Future is an abstract type. HashMap expects the abstract type since it can only hold instances of a single type.
If we can box the return type of the async functions, we will able to add these futures into a HashMap.
First we need to change the type of CalcFn:
type CalcFn = Box<dyn Fn(i32, i32) -> Pin<Box<dyn Future<Output = i32>>>>;
Then this can do the trick:
let mut map: HashMap<&str, CalcFn> = Default::default();
map.insert("add", Box::new(|a, b| Box::pin(add(a, b))));
map.insert("sub", Box::new(|a, b| Box::pin(sub(a, b))));
println!("map size: {}", map.len());
//map.get("add").unwrap()(2, 3).await
This complete example
simplified Future's Item type, using an i32 instead of a Result. Please also check the full code for your case.
You can also use types from the futures crate like LocalBoxFuture and BoxFuture created by the FutureExt::boxed and FutureExt::boxed_local methods respectively:
use futures::future::{FutureExt, LocalBoxFuture}; // 0.3.5
use std::collections::HashMap;
type BoxedResult<T> = Result<T, Box<dyn std::error::Error + Send + Sync>>;
type CalcFn = Box<dyn Fn(i32, i32) -> LocalBoxFuture<'static, BoxedResult<i32>>>;
async fn add(a: i32, b: i32) -> BoxedResult<i32> {
Ok(a + b)
}
async fn sub(a: i32, b: i32) -> BoxedResult<i32> {
Ok(a - b)
}
async fn example() {
let mut map: HashMap<&str, CalcFn> = Default::default();
map.insert("add", Box::new(|a, b| add(a, b).boxed()));
map.insert("sub", Box::new(|a, b| sub(a, b).boxed()));
println!("map size: {}", map.len());
//map.get("add").unwrap()(2, 3).await
}

Related

How to specify lifetimes for `One type is more general then the other` compile error?

I have the following setup where I am trying to create a Dummy future which is intended to modify the values in-place inside the future obtained from f: FnMut(&mut i32) -> Fut. I am passing the generic function fun to substitute it, but I am getting different lifetimes.
#![allow(unused_variables)]
pub struct Dummy<Fut, F> {
f: F,
futures: futures::stream::FuturesUnordered<Fut>
}
impl<Fut, F> Dummy<Fut, F>
where
F: FnMut(&mut i32) -> Fut,
Fut: futures::Future<Output = ()>,
{
fn new(f: F) -> Self {
Self {
f,
futures: futures::stream::FuturesUnordered::new()
}
}
}
impl<Fut, F> futures::Future for Dummy<Fut, F>
where
F: FnMut(&mut i32) -> Fut,
Fut: futures::Future<Output = ()>
{
type Output = ();
fn poll(self: core::pin::Pin<&mut Self>, cx: &mut core::task::Context<'_>) -> core::task::Poll<()> {
// logic emitted
core::task::Poll::Ready(())
}
}
fn fun<'a>(f: &'a mut i32) -> impl futures::Future<Output = ()> + 'a {
async move {
*f += 1;
}
}
#[tokio::main]
async fn main() {
Dummy::new(fun).await;
}
playground link
which gives the compile-error:
error[E0308]: mismatched types
--> src/main.rs:45:20
|
45 | Dummy::new(fun).await;
| ^^^^^^ one type is more general than the other
|
= note: expected trait `for<'r> FnOnce<(&'r mut i32,)>`
found trait `for<'a> FnOnce<(&'a mut i32,)>`
= note: the lifetime requirement is introduced here
error[E0308]: mismatched types
--> src/main.rs:45:20
|
45 | Dummy::new(fun).await;
| ^^^^^^ one type is more general than the other
|
= note: expected trait `for<'r> FnOnce<(&'r mut i32,)>`
found trait `for<'a> FnOnce<(&'a mut i32,)>`
For more information about this error, try `rustc --explain E0308`.
error: could not compile `playground` due to 4 previous errors
Is it possible to annotate the impl in a way that removes these mismatched types error? I could annotate everything with 'static, as I have done here: playground link. But I was trying to avoid this solution since I want to mutually borrow dependent data in the future poll method.
The signature of fun() connects the lifetime of the reference with the lifetime of the future it returns. You need to specify the same kind of connection on the Future implementation of Dummy:
impl<'a, Fut, F> futures::Future for Dummy<Fut, F>
where
F: FnMut(&'a mut i32) -> Fut,
Fut: futures::Future<Output = ()> + 'a,
Playground

Parsing Traces with Opaque Errors in Rust

How does one read and understand Rust traces with Opaque errors?
Example Error
For example, consider the following typically long error. How do you infer the problem from the error output?
error[E0599]: the method `next` exists for struct `Pin<&mut impl futures::Future<Output = AsyncStream<Result<u32, Box<dyn StdError>>, Opaque(DefId(0:51 ~ armadillo_rust[2000]::query_all::{opaque#0}::{opaque#0}), ['_#6r])>>>`, but its trait bounds were not satisfied
--> src/main.rs:198:31
|
198 | while let Some(x) = ticks.next().await {
| ^^^^ method cannot be called on `Pin<&mut impl futures::Future<Output = AsyncStream<Result<u32, Box<dyn StdError>>, Opaque(DefId(0:51 ~ armadillo_rust[2000]::query_all::{opaque#0}::{opaque#0}), ['_#6r])>>>` due to unsatisfied trait bounds
|
::: ../toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/pin.rs:408:1
|
408 | pub struct Pin<P> {
| -----------------
| |
| doesn't satisfy `_: StreamExt`
| doesn't satisfy `_: futures::Stream`
|
= note: the following trait bounds were not satisfied:
`Pin<&mut impl futures::Future<Output = AsyncStream<Result<u32, Box<dyn StdError>>, Opaque(DefId(0:51 ~ armadillo_rust[2000]::query_all::{opaque#0}::{opaque#0}), ['_#6r])>>>: futures::Stream`
which is required by `Pin<&mut impl futures::Future<Output = AsyncStream<Result<u32, Box<dyn StdError>>, Opaque(DefId(0:51 ~ armadillo_rust[2000]::query_all::{opaque#0}::{opaque#0}), ['_#6r])>>>: StreamExt`
`impl futures::Future<Output = AsyncStream<Result<u32, Box<dyn StdError>>, Opaque(DefId(0:51 ~ armadillo_rust[2000]::query_all::{opaque#0}::{opaque#0}), ['_#6r])>>: futures::Stream`
which is required by `impl futures::Future<Output = AsyncStream<Result<u32, Box<dyn StdError>>, Opaque(DefId(0:51 ~ armadillo_rust[2000]::query_all::{opaque#0}::{opaque#0}), ['_#6r])>>: StreamExt`
Minimal, Working Example:
use futures::StreamExt;
use async_stream::AsyncStream;
async fn query_all<'a>(date: &'a str) -> AsyncStream<Result<u32, Box<dyn std::error::Error + 'a>>, impl futures::Future + 'a> {
todo!();
}
#[tokio::main]
async fn main() {
let ticks = query_all("2022-06-14");
futures::pin_mut!(ticks);
while let Some(x) = ticks.next().await {
println!("{:?}", x);
}
}
Naive Solution
The error seems to be saying two things:
A pinned stream doesn't satisfy necessary bounds
The return type of query_all fails to implement certain traits.
However, confusingly, the exact same types work in this minimal example:
#[tokio::main]
async fn main() {
let ticks = stream! {
yield Ok(0);
yield Err("");
};
futures::pin_mut!(ticks);
while let Some(x) = ticks.next().await {
println!("{:?}", x);
}
}
How am I misreading this and what is Rust really trying to say?

Wrap an async function

The AWS Lambda Runtime for rust requires handlers be like async fn handler(event: Value, cx: Context) -> Result<Value, Error>. It handles the errors you return internally, but I don't like how it logs them. I want to create a function that I can pass a handler function and it returns me a wrapped version, which just calls the original, inspects its results, maybe logs, and returns the results. This has proved ... difficult.
I've managed to do it with a macro, but I can't work out how to make a real function do it. Any help would be much appreciated. Code:
use anyhow::{Result};
use lambda_runtime::{handler_fn, Context, Error};
use serde_json::{Value};
// handler
async fn handler(event: Value, cx: Context) -> Result<Value> {
Ok(())
}
// non-generic version (works)
async fn wrapper(event: Value, cx: Context) -> Result<Value> {
let res = handler(event, cx).await;
if let Err(e) = &res {
log::error!("got error from handler: {:?}", &e);
}
res
}
// macro (works)
macro_rules! wrap_handler {
($f:ident) => {{
|e: Value, c: Context| async {
let res = $f(e, c).await;
if let Err(e) = &res {
log::error!("got error from handler: {:?}", &e);
}
res
}
}};
}
// attempt at generic wrapper-creator fn
// should take the function to wrap, and return a Fn wrapping it
use std::future::Future;
use std::pin::Pin;
pub fn make_wrapper<F, FO>(
f: F,
) -> Box<dyn Fn(Value, Context) -> Pin<Box<dyn Future<Output = Result<Value>> + 'static>> + 'static>
where
F: Fn(Value, Context) -> FO + 'static,
FO: Future<Output = Result<Value>> + 'static,
{
use std::sync::Arc;
let af = Arc::new(f);
Box::new(move |v: Value, c: Context| {
let my_f = af.clone();
Box::pin(async move {
//
my_f(v, c).await
})
})
}
// main
#[tokio::main]
async fn main() -> Result<(), Error> {
// use the non-generic version
let func = handler_fn(wrapper);
// OR
// use the macro
let func = handler_fn(wrap_handler!(handler));
// OR
// use the generic wrapper function
let func = handler_fn(make_wrapper(handler));
lambda_runtime::run(func).await?;
Ok(())
}
The error with my latest attempt (the fn in the code above) is:
error[E0277]: `dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>` cannot be sent between threads safely
--> src/bin/classify-partition.rs:126:25
|
126 | lambda_runtime::run(func).await?;
| ^^^^ `dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>` cannot be sent between threads safely
|
::: /home/miles/.cargo/registry/src/github.com-1ecc6299db9ec823/lambda_runtime-0.3.0/src/lib.rs:294:24
|
294 | F: Handler<A, B> + Send + Sync + 'static,
| ---- required by this bound in `run`
|
= help: the trait `Send` is not implemented for `dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>`
= note: required because of the requirements on the impl of `Send` for `Unique<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>`
= note: required because it appears within the type `Box<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>`
= note: required because it appears within the type `HandlerFn<Box<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>>`
error[E0277]: `dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>` cannot be shared between threads safely
--> src/bin/classify-partition.rs:126:25
|
126 | lambda_runtime::run(func).await?;
| ^^^^ `dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>` cannot be shared between threads safely
|
::: /home/miles/.cargo/registry/src/github.com-1ecc6299db9ec823/lambda_runtime-0.3.0/src/lib.rs:294:31
|
294 | F: Handler<A, B> + Send + Sync + 'static,
| ---- required by this bound in `run`
|
= help: the trait `Sync` is not implemented for `dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>`
= note: required because of the requirements on the impl of `Sync` for `Unique<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>`
= note: required because it appears within the type `Box<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>`
= note: required because it appears within the type `HandlerFn<Box<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>>`
error[E0277]: `(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)` cannot be sent between threads safely
--> src/bin/classify-partition.rs:126:25
|
126 | lambda_runtime::run(func).await?;
| ^^^^ `(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)` cannot be sent between threads safely
|
= help: the trait `Send` is not implemented for `(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)`
= note: required because of the requirements on the impl of `Send` for `Unique<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>`
= note: required because it appears within the type `Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>`
= note: required because it appears within the type `Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>`
= note: required because of the requirements on the impl of `Handler<Value, Value>` for `HandlerFn<Box<dyn Fn(Value, lambda_runtime::Context) -> Pin<Box<(dyn Future<Output = Result<Value, anyhow::Error>> + 'static)>>>>`
error: aborting due to 3 previous errors
For more information about this error, try `rustc --explain E0277`.
error: could not compile `enrichment-service-functions`
To learn more, run the command again with --verbose.

How to indicate that the lifetime of an async function's return value is the same as a parameter?

We can match a normal function with non-static parameter like this:
fn processor(data: &i32) -> &i32 {
data
}
fn process<'b>(data: &'b i32, processor: impl 'static + for<'a> Fn(&'a i32) -> &'a i32) -> &'b i32 {
processor(data)
}
fn main() {
let data = 1;
println!("data: {}", process(&data, processor));
}
Since async functions return anonymous futures, we cannot indicate that the lifetime of anonymous future is the same as the parameter:
use std::future::Future;
async fn processor(data: &i32) -> &i32 {
data
}
async fn process<'b, F>(data: &'b i32, processor: impl 'static + Fn(&i32) -> F) -> &'b i32
where
F: 'b + Future<Output = &'b i32>,
{
processor(data).await
}
async fn _main() {
let data = 1;
println!("data: {}", process(&data, processor).await);
}
The compiler will complain:
error[E0271]: type mismatch resolving `for<'r> <for<'_> fn(&i32) -> impl std::future::Future {processor} as std::ops::FnOnce<(&'r i32,)>>::Output == _`
--> src/lib.rs:16:26
|
7 | async fn process<'b, F>(data: &'b i32, processor: impl 'static + Fn(&i32) -> F) -> &'b i32
| ------- - required by this bound in `process`
...
16 | println!("data: {}", process(&data, processor).await);
| ^^^^^^^ expected bound lifetime parameter, found concrete lifetime
How can I match it?
You need to state that:
the closure accepts a reference with the same lifetime as the parameter.
the returned future returns a reference with the same lifetime as the parameter.
the returned future captures a reference with the same lifetime as the parameter.
async fn process<'b, F, Fut>(data: &'b i32, processor: F) -> &'b i32
where
F: Fn(&'b i32) -> Fut,
// ^^ [1]
F: 'static,
Fut: Future<Output = &'b i32> + 'b,
// ^^ [2] ^^ [3]
{
processor(data).await
}

How to use closure instead of trait?

I want to register Gen::my_g as a callback. The simple way is to implement trait Foo for Gen, but I do not want to implement trait Foo for Gen.
In other words, I want to comment out the code marked as B!!! and uncomment the code marked as A!!!.
This is not my code; I can not modify this:
struct S1;
struct TT;
trait MyRes {}
trait Foo {
fn g<'a>(&self, ecx: &'a mut S1, tt: &[TT]) -> Box<MyRes + 'a>;
}
impl<F> Foo for F
where
F: for<'a> Fn(&'a mut S1, &[TT]) -> Box<MyRes + 'a>,
{
fn g<'a>(&self, ecx: &'a mut S1, tt: &[TT]) -> Box<MyRes + 'a> {
(*self)(ecx, tt)
}
}
fn f1<F>(name: &str, extension: F)
where
F: Foo + 'static,
{
}
This is my code:
struct Gen {}
impl Gen {
fn register(self) {
// f1("aaa", move |ecx, tt| self.my_g(ecx, tt));//A!!!
f1("aaa", self); //B!!!
}
fn my_g<'a>(&self, ecx: &'a mut S1, tt: &[TT]) -> Box<MyRes + 'a> {
unimplemented!();
}
}
impl Foo for Gen {
fn g<'a>(&self, ecx: &'a mut S1, tt: &[TT]) -> Box<MyRes + 'a> {
self.my_g(ecx, tt)
}
}
If I uncomment //A!!!, the compiler tells me something that I do not understand:
error[E0271]: type mismatch resolving `for<'a, 'r> <[closure#src/main.rs:29:19: 29:52 self:_] as std::ops::FnOnce<(&'a mut S1, &'r [TT])>>::Output == std::boxed::Box<MyRes + 'a>`
--> src/main.rs:29:9
|
29 | f1("aaa", move |ecx, tt| self.my_g(ecx, tt)); //A!!!
| ^^ expected bound lifetime parameter, found concrete lifetime
|
= note: concrete lifetime that was found is lifetime '_#12r
= note: required because of the requirements on the impl of `Foo` for `[closure#src/main.rs:29:19: 29:52 self:_]`
= note: required by `f1`
error[E0281]: type mismatch: `[closure#src/main.rs:29:19: 29:52 self:_]` implements the trait `std::ops::Fn<(&mut S1, &[TT])>`, but the trait `for<'a, 'r> std::ops::Fn<(&'a mut S1, &'r [TT])>` is required
--> src/main.rs:29:9
|
29 | f1("aaa", move |ecx, tt| self.my_g(ecx, tt)); //A!!!
| ^^ --------------------------------- implements `std::ops::Fn<(&mut S1, &[TT])>`
| |
| requires `for<'a, 'r> std::ops::Fn<(&'a mut S1, &'r [TT])>`
| expected concrete lifetime, found bound lifetime parameter
|
= note: required because of the requirements on the impl of `Foo` for `[closure#src/main.rs:29:19: 29:52 self:_]`
= note: required by `f1`
This is a known issue:
Confusing type error due to strange inferred type for a closure argument
The Rust compiler currently cannot infer that a closure is valid for any lifetime (which is what the type of Foo::g requires). It will infer any concrete lifetime, but does not generalize beyond that.

Resources