I have a struct Task which is defined as below:
pub struct Task<F>
where
F: Future<Output = ()>,
{
pub name: &'static str,
pub should_run_late: bool,
pub time: Time,
runner: fn() -> F,
}
impl<F> Task<F>
where
F: Future<Output = ()>,
{
pub async fn run(self) {
(self.runner)().await;
}
pub fn set_Runner(&mut self, func: fn() -> F)
{
self.runner = func;
}
}
and call it this way
tokio::spawn(async move {
task.run().await;
});
I'm putting Tasks in a vector in some struct like this:
some struct {
pub tasks: HashMap<String, HashMap<String, Vec<Task<dyn Future<Output = ()>>>>>
}
The problem is in the above which says it doesn't implement Sized trait and when then wrapped it with Box:
HashMap<String, HashMap<String, Vec<Task<Box<dyn Future<Output = ()>>>>>>
It says to use Box::pin.
It really makes everything confusing.
I'm fairly new in rust, and what ever I try I can't make it.
Changing the type of the F parameter in the Vec<Task<F>> won't help, since as written, F must match the actual return type of the runner function. You need to handle the function “dynamically” too, not just the future. Once you do, the code can also be simplified. Here's a new definition of Task:
pub struct Task {
pub name: &'static str,
pub should_run_late: bool,
pub time: Time,
runner: Box<dyn FnOnce() -> Pin<Box<dyn Future<Output = ()> + Send>> + Send>,
}
Notice that the stored function returns Pin<Box<dyn Future>>, so we need to wrap the function when it's stored by set_runner. (This is going to match the Box::pin() that you noticed being asked for.) Actually running the task is the same:
impl Task {
pub async fn run(self) {
(self.runner)().await;
}
pub fn set_runner<Func, Fut>(&mut self, func: Func)
where
Func: Send + 'static + FnOnce() -> Fut,
Fut: Send + 'static + Future<Output = ()>,
{
self.runner = Box::new(move || Box::pin(func()));
}
}
I generalized the func parameter so it will accept closures and not just function pointers — since we're boxing the function anyway, this is free.
Compilable sample on Rust Playground
Now, this code can be a bit messy to read. There are a couple of libraries providing trait aliases which can help you out by removing the boilerplate. The futures library contains futures::future::BoxFuture which tidies up storing a boxed future:
use futures::future::BoxFuture;
pub struct Task {
...
runner: Box<dyn FnOnce() -> BoxFuture<'static, ()> + Send>,
}
For the input to set_runner, the two lines of trait bounds are just saying "Func is an async function which takes no arguments and returns ()". We can simplify that (somewhat) using async_fn_traits:
use async_fn_traits::AsyncFnOnce0;
impl Task {
...
pub fn set_runner<F>(&mut self, func: F)
where
F: AsyncFnOnce0<Output = ()> + Send + 'static,
F::OutputFuture: Send + 'static,
{
self.runner = Box::new(move || Box::pin(func()));
}
}
In situations that don't involve storing the future this would be even more of an improvement (because we would not need to write out the Send + 'static bounds), but at least here we avoid needing the separate Fut type parameter which can't actually vary independently.
Using these aliases is entirely optional — it's up to you which version of the code you think is clearer. For what it's worth, futures is quite commonly used for other future-related utilities, and async_fn_traits is considerably more obscure.
Related
I'm trying to write a multi-thread TCP server that can handle multiple connections at the same time with tokio.
I want to structure it in an event-driven way, where it's possible to attach one or more closures to a specific event (like new connection, message received, client disconnected etc).
For example:
server.on_message(|msg: String, stream: &mut TcpStream| {
async move {
println!("Recieved {:?}", msg);
stream.write_all(b"Hello\n").await;
}
}).await;
Every connection will receive its own thread which should have read access to a Vec of callbacks.
pub async fn run(&mut self) {
let listener = TcpListener::bind("127.0.0.1:9090").await.unwrap();
loop {
let (mut socket, _) = listener.accept().await.unwrap();
let cb = self.on_message.clone();
tokio::spawn(async move {
Self::process(socket, cb).await;
});
}
}
Unfortunately, my understanding of Rust is still very rudimentary and I run in circles with:
finding the right type to be stored in Vec
finding the right type for function argument that takes closure callback
Whenever I feel like I make progress in one place then I realise the other got messed up. This is the best I could do but it still doesn't work.
type Callback<T> = dyn Fn(T, &mut TcpStream) -> Pin<Box<dyn Future<Output=()> + Send>> + Send + 'static;
unsafe impl<T> Send for TcpStreamCallbackList<T> {}
unsafe impl<T> Sync for TcpStreamCallbackList<T> {}
impl<T> TcpStreamCallbackList<T> {
pub fn new() -> Self {
Self { callbacks: Vec::new() }
}
pub fn push<G: Send + 'static>(&mut self, mut fun: impl Fn(T, &mut TcpStream) -> G + Send + 'static) where G: Future<Output=()> {
self.callbacks.push(Arc::new(Box::new(move |val:T, stream: &mut TcpStream| Box::pin(fun(val, stream)))));
}
pub async fn call(&self, val: T, stream: &mut TcpStream) where T: Clone {
for cb in self.callbacks.iter() {
let _cb = cb.clone();
_cb(val.clone(), stream).await; // B O O M
}
}
}
The above code doesn't compile until I remove .await on the Future returned by callback in the call function (which defeats the purpose).
error[E0277]: `dyn for<'a> Fn(String, &'a mut tokio::net::TcpStream) -> Pin<Box<dyn futures::Future<Output = ()> + std::marker::Send>> + std::marker::Send` cannot be shared between threads safely
--> src/main.rs:94:26
From what I understand the problem is that the retuned Future is not Send.
note: required by a bound in `tokio::spawn`
--> /Users/lukasz/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.25.0/src/task/spawn.rs:163:21
|
163 | T: Future + Send + 'static,
| ^^^^ required by this bound in `tokio::spawn`
I have no idea if my type makes sense and is thread-safe. I also don't know why the compiler thinks the return type is not Send. I'm really stuck here and would appreciate any help.
I've put together one that is slightly simpler but can be spawned (playground). The key is that Callback needs to be Sync in order for &self to be Send. I tried to use the trick mentioned in this comment, but it doesn't appear to work here, nor does making call take &mut self. I wrote more about Send and Sync on that answer.
use std::future::Future;
use std::pin::Pin;
type CallbackFuture<O> = Pin<Box<dyn Future<Output = O> + Send>>;
type Callback<T> = dyn (Fn(T) -> CallbackFuture<()>) + Send + Sync;
pub struct CallbackList<T> {
list: Vec<Box<Callback<T>>>,
}
impl<T> CallbackList<T> {
pub fn new() -> Self {
Self { list: Vec::new() }
}
pub fn push<F>(&mut self, f: F)
where
F: Fn(T) -> CallbackFuture<()>,
F: Send + Sync + 'static,
{
self.list.push(Box::new(f))
}
pub async fn call(&self, t: T)
where
T: Clone,
{
for f in &self.list {
f(t.clone()).await;
}
}
}
#[tokio::main]
async fn main() {
let mut calls = CallbackList::new();
calls.push(|i| {
Box::pin(async move {
println!("{i}");
})
});
calls.push(|i| {
Box::pin(async move {
println!("{}", i + 1);
})
});
let handle = tokio::spawn(async move {
calls.call(34).await;
});
handle.await.unwrap();
}
I have removed as many trait bounds, 'statics, and wrappers as possible, but you may need to add some back depending on what you do with it. Right now it takes T, but it should be possible to separate that into T and &mut TcpStream. If you update your question with a main function that uses all the elements, I can change mine to match. If all else fails, you can use (_, Arc<Mutex<TcpStream>>) as T.
Can I propagate the Send trait of function parameters to its return type, so that the return type is impl Send if and only if the parameters are?
Details:
An async function has a nice feature. Its returned Future is automatically Send if it can be. In the following example, the async function will create a Future that is Send, if the inputs to the function are Send.
struct MyStruct;
impl MyStruct {
// This async fn returns an `impl Future<Output=T> + Send` if `T` is Send.
// Otherwise, it returns an `impl Future<Output=T>` without `Send`.
async fn func<T>(&self, t: T) -> T {
t
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func(4u64));
// And the following correctly fails
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
Now, I want to move such a function into a trait, which requires using async-trait (which is some codegen that effectively writes my async fn as a function returning Pin<Box<dyn Future>>) or doing something similar manually. Is there a way to write this in a way to retain this auto-Send behavior where the returned Future is made Send if T is Send? The following example implements it as two separate functions:
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func_send<T: 'static + Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + Send>> {
Box::pin(async{t})
}
fn func_not_send<T: 'static>(&self, t: T) -> Pin<Box<dyn Future<Output = T>>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func_send(4u64));
// And the following correctly fails
// assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
But actually, I don't want them to be separate. I want them to be one function similar to how async fn does it automatically. Something along the lines of
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func<T: 'static + ?Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + ?Send>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This should
assert_is_send(MyStruct.func(4u64));
// And this should fail
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
Is something like this possible in Rust? I'm ok with writing the async-trait magic manually and modifying it instead of using the async-trait crate if that is a way to make it work.
Some ideas I had but they haven't really borne fruit yet:
Use min-specialization to specialize on Send? But doesn't seem like that feature is going to be stabilized anytime soon so maybe not the best option.
Return a custom MyFuture type instead of just impl Future and somehow impl Send for MyFuture where T: Send? Would probably be difficult though since I would have to be able to name that Future and async code usually produces impl Future types that cannot be named.
Writing a procedural macro that adds + Send to the return type if it recognizes that the input type is Send. Actually, can procedural macros detect if a certain type implements Send? My guess would be it's not possible since they just work on token streams.
(2) is the only way that could work.
There are two ways to make it work:
Write the future manually, without the help of async and .await. But that means writing the future manually:
enum ConditionalSendFut<T> {
Start { t: T },
Done,
}
impl<T> Unpin for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, _context: &mut Context<'_>) -> Poll<Self::Output> {
match &mut *self {
Self::Start { .. } => {
let t = match std::mem::replace(&mut *self, Self::Done) {
Self::Start { t } => t,
_ => unreachable!(),
};
Poll::Ready(t)
}
Self::Done => Poll::Pending,
}
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut::Start { t }
}
}
Playground.
Store a Pin<Box<dyn Future<Output = T>>> and conditionally impl Send on the future. But this requires unsafe code and manually ensuring that you don't hold other non-Send types across .await points:
struct ConditionalSendFut<T>(Pin<Box<dyn Future<Output = T>>>);
// SAFETY: The only non-`Send` type we're holding across an `.await`
// point is `T`.
unsafe impl<T: Send> Send for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().poll(context)
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut(Box::pin(async { t }))
}
}
Playground.
(1) cannot work with traits, as each impl will have a different future. This leaves us with (2) only. I would not recommend it, but it is possible.
It is very likely that when async fns in traits will be stable there will be a mechanism to that (what is talked about currently is to impl them conditionally and use bounds on use sites to require them) but currently there is no such thing, even on the nightly implementation of async fns in traits.
Part 1: What should be the signature of a function returning an async function?
pub async fn some_async_func(arg: &str) {}
// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
some_async_func
}
Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.
I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.
Returning a function
Returning the actual function pointer requires heap allocation and a wrapper:
use std::future::Future;
use std::pin::Pin;
pub async fn some_async_func(arg: &str) {}
pub fn some_async_func_wrapper<'a>(arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
Box::pin(some_async_func(arg))
}
pub fn higher_order_func<'a>(action: &str)
-> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
some_async_func_wrapper
}
Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a, but this would require much more guesswork from the compiler and currently is not supported.
If you are OK with Fn instead of fn, you can get rid of the wrapper:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
|arg: &'a str| {
Box::pin(some_async_func(arg))
}
}
To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
if action.starts_with("one") {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_one(arg))
})
} else {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_two(arg))
})
}
}
Alternative: returning a future
To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> impl Future<Output=()> + 'a
{
some_async_func(arg)
}
It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func, no user code is getting executed. The function call returns a Future: the actual function body will be executed only when someone awaits the returned future.
You can use the new function almost the same way as the previous function:
// With higher order function returning function pointer
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func(action)(arg).await;
}
// With higher order function returning future
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func_future(action, arg).await;
}
Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.
If you wanted to be able to call different async functions based on action value, you need boxing again:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
if action.starts_with("one") {
Box::pin(some_async_func_one(arg))
} else {
Box::pin(some_async_func_two(arg))
}
}
Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.
Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()>. But nested impl trait is not supported. However, you can emulate that using a trait.
The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32, for example, it could look like:
trait AsyncFn: Fn(u32) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
F: Fn(u32) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we would take impl AsyncFn. Trying to apply that naively to &str doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:16:27
|
16 | fn higher_order_func() -> impl AsyncFn {
| ^^^^^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, i.e. for a signature async fn foo<'a>(arg: &'a str), the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a. There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB:
trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
F: Fn(Arg) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we specify the type as:
fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
some_async_func
}
In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:
pub async fn some_async_func(arg: &str) {}
macro_rules! higher_order_func {
($action: expr) => {
some_async_func
};
}
fn main() {
let future = higher_order_func!("action")("arg");
}
Part 1: What should be the signature of a function returning an async function?
pub async fn some_async_func(arg: &str) {}
// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
some_async_func
}
Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.
I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.
Returning a function
Returning the actual function pointer requires heap allocation and a wrapper:
use std::future::Future;
use std::pin::Pin;
pub async fn some_async_func(arg: &str) {}
pub fn some_async_func_wrapper<'a>(arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
Box::pin(some_async_func(arg))
}
pub fn higher_order_func<'a>(action: &str)
-> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
some_async_func_wrapper
}
Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a, but this would require much more guesswork from the compiler and currently is not supported.
If you are OK with Fn instead of fn, you can get rid of the wrapper:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
|arg: &'a str| {
Box::pin(some_async_func(arg))
}
}
To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
if action.starts_with("one") {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_one(arg))
})
} else {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_two(arg))
})
}
}
Alternative: returning a future
To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> impl Future<Output=()> + 'a
{
some_async_func(arg)
}
It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func, no user code is getting executed. The function call returns a Future: the actual function body will be executed only when someone awaits the returned future.
You can use the new function almost the same way as the previous function:
// With higher order function returning function pointer
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func(action)(arg).await;
}
// With higher order function returning future
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func_future(action, arg).await;
}
Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.
If you wanted to be able to call different async functions based on action value, you need boxing again:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
if action.starts_with("one") {
Box::pin(some_async_func_one(arg))
} else {
Box::pin(some_async_func_two(arg))
}
}
Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.
Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()>. But nested impl trait is not supported. However, you can emulate that using a trait.
The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32, for example, it could look like:
trait AsyncFn: Fn(u32) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
F: Fn(u32) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we would take impl AsyncFn. Trying to apply that naively to &str doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:16:27
|
16 | fn higher_order_func() -> impl AsyncFn {
| ^^^^^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, i.e. for a signature async fn foo<'a>(arg: &'a str), the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a. There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB:
trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
F: Fn(Arg) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we specify the type as:
fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
some_async_func
}
In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:
pub async fn some_async_func(arg: &str) {}
macro_rules! higher_order_func {
($action: expr) => {
some_async_func
};
}
fn main() {
let future = higher_order_func!("action")("arg");
}
Part 1: What should be the signature of a function returning an async function?
pub async fn some_async_func(arg: &str) {}
// What should be sig here?
pub fn higher_order_func(action: &str) -> ???
{
some_async_func
}
Part 2: What should be the sig, if based on the action parameter, higher_order_func had to return either async_func1 or async_func2.
I am also interested in learning the performance tradeoffs if there are multiple solutions. Please note that I'd like to return the function itself as an fn pointer or an Fn* trait, and not the result of invoking it.
Returning a function
Returning the actual function pointer requires heap allocation and a wrapper:
use std::future::Future;
use std::pin::Pin;
pub async fn some_async_func(arg: &str) {}
pub fn some_async_func_wrapper<'a>(arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
Box::pin(some_async_func(arg))
}
pub fn higher_order_func<'a>(action: &str)
-> fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
some_async_func_wrapper
}
Why boxing? higher_order_func needs to have a concrete return type, which is a function pointer. The pointed function needs to also have a concrete return type, which is impossible for async function since it returns opaque type. In theory, it could be possible to write return type as fn(&'a str) -> impl Future<Output=()> + 'a, but this would require much more guesswork from the compiler and currently is not supported.
If you are OK with Fn instead of fn, you can get rid of the wrapper:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> impl Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>
{
|arg: &'a str| {
Box::pin(some_async_func(arg))
}
}
To return a different function based on action value, you will need to box the closure itself, which is one more heap allocation:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func<'a>(action: &str)
-> Box<dyn Fn(&'a str) -> Pin<Box<dyn Future<Output=()> + 'a>>>
{
if action.starts_with("one") {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_one(arg))
})
} else {
Box::new(|arg: &'a str| {
Box::pin(some_async_func_two(arg))
})
}
}
Alternative: returning a future
To simplify things, consider returning a future itself instead of a function pointer. This is virtually the same, but much nicer and does not require heap allocation:
pub async fn some_async_func(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> impl Future<Output=()> + 'a
{
some_async_func(arg)
}
It might look like, when higher_order_func_future is called, some_async_func is getting executed - but this is not the case. Because of the way async functions work, when you call some_async_func, no user code is getting executed. The function call returns a Future: the actual function body will be executed only when someone awaits the returned future.
You can use the new function almost the same way as the previous function:
// With higher order function returning function pointer
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func(action)(arg).await;
}
// With higher order function returning future
async fn my_function() {
let action = "one";
let arg = "hello";
higher_order_func_future(action, arg).await;
}
Notice, once more, that in both cases the actual some_async_func body is executed only when the future is awaited.
If you wanted to be able to call different async functions based on action value, you need boxing again:
pub async fn some_async_func_one(arg: &str) {}
pub async fn some_async_func_two(arg: &str) {}
pub fn higher_order_func_future<'a>(action: &str, arg: &'a str)
-> Pin<Box<dyn Future<Output=()> + 'a>>
{
if action.starts_with("one") {
Box::pin(some_async_func_one(arg))
} else {
Box::pin(some_async_func_two(arg))
}
}
Still, this is just one heap allocation, so I strongly advise returning a future. The only scenario that I can imagine where the previous solution is better is when you want to save the boxed closure somewhere and use it many times. In this case, excessive allocation happens only once, and you spare some CPU time by dispatching the call based on action only once - when you make the closure.
Ideally, what you'd want is a nested impl trait: -> impl Fn(&str) -> impl Future<Output = ()>. But nested impl trait is not supported. However, you can emulate that using a trait.
The idea is to define a trait that will abstract over the notion of "function returning a future". If our function would take a u32, for example, it could look like:
trait AsyncFn: Fn(u32) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<F, Fut> AsyncFn for F
where
F: Fn(u32) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we would take impl AsyncFn. Trying to apply that naively to &str doesn't work:
error[E0308]: mismatched types
--> src/lib.rs:16:27
|
16 | fn higher_order_func() -> impl AsyncFn {
| ^^^^^^^^^^^^ one type is more general than the other
|
= note: expected associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
found associated type `<for<'_> fn(&str) -> impl Future<Output = ()> {some_async_func} as FnOnce<(&str,)>>::Output`
The error may look very strange, but it arises from the fact that async fn returns a future bound by the lifetime of all of its argument, i.e. for a signature async fn foo<'a>(arg: &'a str), the future is not impl Future<Output = ()> but impl Future<Output = ()> + 'a. There is a way to capture this relationship in our trait, we just need to make it generic over the argument and use HRTB:
trait AsyncFn<Arg>: Fn(Arg) -> Self::Future {
type Future: Future<Output = ()>;
}
impl<Arg, F, Fut> AsyncFn<Arg> for F
where
F: Fn(Arg) -> Fut,
Fut: Future<Output = ()>,
{
type Future = Fut;
}
And then we specify the type as:
fn higher_order_func() -> impl for<'a> AsyncFn<&'a str> {
some_async_func
}
In addition to the great accepted answer, depending on your use case it's also possible to "fake" the higher order function and avoid any heap allocations by using a simple macro to expand the wrapper code in-place instead:
pub async fn some_async_func(arg: &str) {}
macro_rules! higher_order_func {
($action: expr) => {
some_async_func
};
}
fn main() {
let future = higher_order_func!("action")("arg");
}