I have a use case where I need to specify a dummy Future to make a type concrete. The use case is I have a struct which takes a generic parameter, which is a Future:
let thing = Thing<Fut>::some_method().await?;
some_method is a method on Thing which does not use the Fut at all, but the compiler complains because of the compiler limitations such that the type can't be inferred and is necessary because of the async block.
Since I need a concrete sized thing for Fut, I was hoping there was something in futures_util that I could use. Right now, this is my very bad approach:
use futures::Future;
use std::pin::Pin;
struct Thing<Fut>
where
for<'b> Fut: Future<Output = ()> + Send + 'b,
{
callback: dyn Fn() -> Fut + Send + Sync,
}
impl<Fut> Thing<Fut> where for<'b> Fut: Future<Output = ()> + Send + 'b {
async fn some_method(a: i32) -> Result<(), Box<dyn std::error::Error>> {
println!("I am some_method with value {}", a);
Ok(())
}
}
struct Dummy {
}
impl Future for Dummy {
type Output = ();
fn poll(
self: Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Self::Output> {
unimplemented!()
}
}
#[tokio::main]
async fn main() {
// this fails
Thing::some_method(3).await.unwrap();
// this works
//Thing::<Dummy>::some_method(3).await.unwrap();
}
Is there something I can leverage here to make such a dummy value available?
Here is a minimal playground example:
playground link
The error:
error[E0698]: type inside `async` block must be known in this context
--> src/main.rs:33:5
|
33 | Thing::some_method(3).await.unwrap();
| ^^^^^^^^^^^^^^^^^^ cannot infer type for type parameter `Fut`
|
note: the type is part of the `async` block because of this `await`
--> src/main.rs:33:26
|
33 | Thing::some_method(3).await.unwrap();
| ^^^^^^
If you uncomment the success line, it will remove the error.
You can use std::future::Ready (or probably Pending alternatively). This is the type returned by the std::future::ready function.
use std::future::Ready;
type Dummy = Ready<()>;
#[tokio::main]
async fn main() {
Thing::<Dummy>::some_method(3).await.unwrap();
}
But if this function has no dependence on Fut, then I'd question why it's an associated function on Thing in the first place. Maybe it should be a free-standing function instead.
Related
Can I propagate the Send trait of function parameters to its return type, so that the return type is impl Send if and only if the parameters are?
Details:
An async function has a nice feature. Its returned Future is automatically Send if it can be. In the following example, the async function will create a Future that is Send, if the inputs to the function are Send.
struct MyStruct;
impl MyStruct {
// This async fn returns an `impl Future<Output=T> + Send` if `T` is Send.
// Otherwise, it returns an `impl Future<Output=T>` without `Send`.
async fn func<T>(&self, t: T) -> T {
t
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func(4u64));
// And the following correctly fails
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
Now, I want to move such a function into a trait, which requires using async-trait (which is some codegen that effectively writes my async fn as a function returning Pin<Box<dyn Future>>) or doing something similar manually. Is there a way to write this in a way to retain this auto-Send behavior where the returned Future is made Send if T is Send? The following example implements it as two separate functions:
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func_send<T: 'static + Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + Send>> {
Box::pin(async{t})
}
fn func_not_send<T: 'static>(&self, t: T) -> Pin<Box<dyn Future<Output = T>>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func_send(4u64));
// And the following correctly fails
// assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
But actually, I don't want them to be separate. I want them to be one function similar to how async fn does it automatically. Something along the lines of
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func<T: 'static + ?Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + ?Send>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This should
assert_is_send(MyStruct.func(4u64));
// And this should fail
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
Is something like this possible in Rust? I'm ok with writing the async-trait magic manually and modifying it instead of using the async-trait crate if that is a way to make it work.
Some ideas I had but they haven't really borne fruit yet:
Use min-specialization to specialize on Send? But doesn't seem like that feature is going to be stabilized anytime soon so maybe not the best option.
Return a custom MyFuture type instead of just impl Future and somehow impl Send for MyFuture where T: Send? Would probably be difficult though since I would have to be able to name that Future and async code usually produces impl Future types that cannot be named.
Writing a procedural macro that adds + Send to the return type if it recognizes that the input type is Send. Actually, can procedural macros detect if a certain type implements Send? My guess would be it's not possible since they just work on token streams.
(2) is the only way that could work.
There are two ways to make it work:
Write the future manually, without the help of async and .await. But that means writing the future manually:
enum ConditionalSendFut<T> {
Start { t: T },
Done,
}
impl<T> Unpin for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, _context: &mut Context<'_>) -> Poll<Self::Output> {
match &mut *self {
Self::Start { .. } => {
let t = match std::mem::replace(&mut *self, Self::Done) {
Self::Start { t } => t,
_ => unreachable!(),
};
Poll::Ready(t)
}
Self::Done => Poll::Pending,
}
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut::Start { t }
}
}
Playground.
Store a Pin<Box<dyn Future<Output = T>>> and conditionally impl Send on the future. But this requires unsafe code and manually ensuring that you don't hold other non-Send types across .await points:
struct ConditionalSendFut<T>(Pin<Box<dyn Future<Output = T>>>);
// SAFETY: The only non-`Send` type we're holding across an `.await`
// point is `T`.
unsafe impl<T: Send> Send for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().poll(context)
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut(Box::pin(async { t }))
}
}
Playground.
(1) cannot work with traits, as each impl will have a different future. This leaves us with (2) only. I would not recommend it, but it is possible.
It is very likely that when async fns in traits will be stable there will be a mechanism to that (what is talked about currently is to impl them conditionally and use bounds on use sites to require them) but currently there is no such thing, even on the nightly implementation of async fns in traits.
Filtering via async predicate, the "easy" way
One way would be to join_all!() the Futures that compute the filters on every item. And then filters synchronously based on those:
let arr = vec![...]
let filters = join_all!(arr.iter().map(|it| async { predicate(it).await })
let filtered = arr.enumerate().filter(|index, item| filters[index]).collect::<Vec<_>>();
However, exploring Rust, there's a cleaner way via futures::stream::iter iterators:
let filtered = futures::stream::iter(vec![...])
.filter(|item| async { predicate(item).await })
.collect::<Vec<_>>
.await
All good up to now.
Configurable filter: trouble begins
What if we want to use a functional API to make the predicate easily configurable?
In that case, our calls will look like:
let filtered = futures::stream::iter(vec![...])
.filter(by_length(4)) // neat!
.collect::<Vec<_>>
.await
And the predicate:
fn by_length(min_length: usize) -> impl FnMut(&i32) -> Future<Output = bool> {
|n| async { query_length(n).await > min_length }
}
async fn query_length(n: &i32) -> usize {
// pretend we're making a network request to fetch `len`...
// for easy reproducibility's sake this will work here
n.to_string().len()
}
Unfortunately compiler is not happy anymore: it complains the Future needs dyn keyword. And, after adding dyn, it complains it's not Sized, as in this minimal reproduction:
use futures::future::Future;
#[tokio::main]
async fn main() {
let arr = vec![10, 100, 1000];
let filtered = futures::stream::iter(arr.into_iter())
.filter(by_length(3))
.collect::<Vec<_>>()
.await;
println!("{:?}", filtered); // should print [100, 1000]
}
fn by_length(min_length: usize) -> impl FnMut(&i32) -> Future<Output = bool> {
|n| async { query_length(n).await > min_length }
}
// yeah it doesn't need to be async in this case, but let's pretend
async fn query_length(n: &i32) -> usize {
n.to_string().len()
}
The error:
Compiling playground v0.0.1 (/playground)
error[E0277]: the size for values of type `(dyn futures::Future<Output = bool> + 'static)` cannot be known at compilation time
--> src/main.rs:16:9
|
16 | |n| async { query_length(n).await > min_length }
| ^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `(dyn futures::Future<Output = bool> + 'static)`
= note: the return type of a function must have a statically known size
Questions
So how can the filter predicate be made configurable?
From what I gather, the dyn keyword requested by the compiler here is to make it explicit it will rely automatic dispatch, incurring in additional overhead.
However we only need a single typed case here, for which we should be able to generate inline-able machine code. How can Rust be instructed to do that?
While you can't return an impl Future from an impl FnMut, you can return a boxed future, i.e. a dyn Future which must be boxed because it's in return position. After a bit of borrow checker tetris, we arrive to this:
fn by_length(min_length: usize) -> impl FnMut(&i32) -> Pin<Box<dyn Future<Output = bool>>> {
move |&n| Box::pin(async move { query_length(&n).await > min_length })
}
Playground
However we only need a single typed case here, for which we should be able to generate inline-able machine code. How can Rust be instructed to do that?
I don't think that's currently possible, at least not if you're invoking an async fn like query_length(). Consider a manually written implementation:
fn by_length(min_length: usize) -> impl FnMut(&i32) -> ByLength {
move |&n| ByLength { n }
}
Now, how do we define ByLength? It must implement Future, and its poll() simply transmits the result of polling query_length(n). But the future returned by query_length(n) could suspend multiple times, so ByLength must store the future so it can poll it as many times as needed - for example:
struct ByLength {
n: i32,
query_fut: Option<???>,
}
impl Future for ByLength {
type Output = usize;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<usize> {
if self.query_fut.is_none() {
self.query_fut = Some(query_length(self.n));
}
self.query_fut.unwrap().poll(cx)
}
}
But now the problem becomes apparent: there is no type to substitute for ??? because query_length() is an async function which returns a future of an anonymous type. Making ByLength generic doesn't work because then we're back at the problem that a closure can't return a generic type that it provides. The signature we'd like would require higher-kinded types:
fn by_length(min_length: usize) -> impl for<T> FnMut(&i32) -> ByLength<T> {
move |&n| ByLength { n }
}
...but if we had that, we could just use query_length() directly:
fn by_length(min_length: usize) -> impl for<T: Future<Output = usize>> FnMut(&i32) -> T {
move |&n| async move { by_length(&n) }
}
I am trying to create a async function which takes a function pointer as a parameter. It does some stuff, calls the function, awaits on the result, then does some more stuff:
use std::future::Future;
async fn run_another_async_fn<F, Fut>(f: F)
where
Fut: Future<Output = ()>,
F: FnOnce(&mut i32) -> Fut,
{
let mut i = 42;
println!("running function");
f(&mut i).await;
println!("ran function");
}
async fn foo(i: &mut i32) {}
async fn bar() {
run_another_async_fn(foo);
}
[view on Rust Playground]
Unfortunately this fails to compile:
error[E0308]: mismatched types
--> src/lib.rs:17:5
|
17 | run_another_async_fn(foo);
| ^^^^^^^^^^^^^^^^^^^^ lifetime mismatch
|
= note: expected associated type `<for<'_> fn(&mut i32) -> impl Future {foo} as FnOnce<(&mut i32,)>>::Output`
found associated type `<for<'_> fn(&mut i32) -> impl Future {foo} as FnOnce<(&mut i32,)>>::Output`
= note: the required lifetime does not necessarily outlive the empty lifetime
note: the lifetime requirement is introduced here
--> src/lib.rs:6:28
|
6 | F: FnOnce(&mut i32) -> Fut,
| ^^^
Firstly, it seems the compiler found exactly what it expected but it's complaining anyway?
Secondly, what's "the empty lifetime"? I guess it must mean the '_, does that have some special significance?
Finally, what's the way to get this to compile?
The issue is that there is no way to specify the same lifetime for F and Fut in the where clause.
Luckily (if you don't mind heap allocating the future) there is an easy workaround. You can use the already existing futures::future::BoxFuture; which looks like:
pub type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
With its help you can specify the same lifetime parameter for both the borrow and as a trait bound for the future:
where for<'a> F: FnOnce(&'a mut i32) -> BoxFuture<'a, ()>,
You also have to add an adapter function which will have the correct return type - i.e. BoxFuture<'_, T> instead of impl Future:
fn asd(i: &mut i32) -> BoxFuture<'_, ()> {
foo(i).boxed()
}
or use a closure:
run_another_async_fn(|i| foo(i).boxed());
As a result your code would look like:
use futures::future::BoxFuture;
use futures::FutureExt;
use std::future::Future;
async fn run_another_async_fn<F>(f: F)
where
for<'a> F: FnOnce(&'a mut i32) -> BoxFuture<'a, ()>,
{
let mut i = 42;
println!("running function");
f(&mut i).await;
println!("ran function");
}
fn asd(i: &mut i32) -> BoxFuture<'_, ()> {
foo(i).boxed()
}
async fn foo<'a>(i: &'a mut i32) {
// no-op
}
async fn bar() {
run_another_async_fn(asd);
run_another_async_fn(|i| foo(i).boxed());
}
I'm trying to do a simple future example. I followed https://blog.logrocket.com/a-practical-guide-to-async-in-rust/
So I did:
use futures::Future;
type Result<T> = std::result::Result<T, Box<dyn std::error::Error + Send + Sync>>;
pub async fn receive_fut(
f: &mut dyn FnMut(&[u8])
) -> impl Future<Output = Result<Vec<u8>>> {
futures::future::ok(Ok(Vec::new())).await
}
but I get
error[E0277]: `std::result::Result<std::result::Result<Vec<_>, _>, _>` is not a future
--> src/lib.rs:6:10
|
6 | ) -> impl Future<Output = Result<Vec<u8>>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `std::result::Result<std::result::Result<Vec<_>, _>, _>` is not a future
|
= help: the trait `futures::Future` is not implemented for `std::result::Result<std::result::Result<Vec<_>, _>, _>`
error: aborting due to previous error
Why std::result::Result should implement future, if I used futures::future::ok to turn it into a future result?
This is answered in the article:
Async functions differ in one important way: all your return types are “wrapped” into a Future.
You might read the documentation about Futures in Rust and think your async function needs to look like this:
async fn our_async_program() -> impl Future<Output = Result<String>> {
future::ok("Hello world".to_string()).await
}
This is wrong! If you’re doing this, you’re overthinking it. An async function already wraps the return type, so you can write functions the way you’re used to.
This is what you actually want:
async fn our_async_program() -> Result<String> {
future::ok("Hello world".to_string()).await
}
So you'd want to drop the impl Future<... part of the return type:
pub async fn receive_fut(
f: &mut dyn FnMut(&[u8])
) -> Result<Vec<u8>> {
futures::future::ok(Ok(Vec::new())).await
}
And then realize that future::ok already yields a Result::Ok, so you don't need to wrap it in Ok() again.
pub async fn receive_fut(
f: &mut dyn FnMut(&[u8])
) -> Result<Vec<u8>> {
futures::future::ok(Vec::new()).await
}
use futures::{future, Future};
fn test() -> Box<dyn Future<Output = bool>> {
Box::new(future::ok::<bool>(true))
}
async fn async_fn() -> bool {
let result: bool = test().await;
return result;
}
fn main(){
async_fn();
println!("Hello!");
}
Playground
Error:
error[E0277]: the trait bound `dyn core::future::future::Future<Output = bool>: std::marker::Unpin` is not satisfied
--> src/main.rs:8:24
|
8 | let result: bool = test().await;
| ^^^^^^^^^^^^ the trait `std::marker::Unpin` is not implemented for `dyn core::future::future::Future<Output = bool>`
|
= note: required because of the requirements on the impl of `core::future::future::Future` for `std::boxed::Box<dyn core::future::future::Future<Output = bool>>`
According to the implementation:
impl<F> Future for Box<F>
where
F: Unpin + Future + ?Sized,
Boxed futures only implement the Future trait when the future inside the Box implements Unpin.
Since your function doesn't guarantee that the returned future implements Unpin, your return value will be considered to not implement Future. You'll not able to await it because your type is basically not a Future.
The solution from #Stargateur, adding an explicit type boundary to the signature, works (Playground):
fn test() -> Box<dyn Future<Output = Result<bool, ()>> + Unpin>
If you are using futures-rs, there is a helper type BoxFuture. You can use BoxedFuture without explicitly stating Unpin:
use futures::future::BoxFuture;
fn test() -> BoxFuture<'static, Result<bool, ()>> {
Box::pin(async { Ok(true) })
}
Playground
When it comes to Box and future, it almost always make sense to use Box::pin instead of Box::new:
use std::pin::Pin;
use futures::{future, Future};
fn test() -> Pin<Box<dyn Future<Output = Result<bool, ()>>>> {
Box::pin(future::ok(true))
}
async fn async_fn() -> bool {
test().await.unwrap()
}
The reason is quite interesting. Pin has a blanket implementation for Unpin:
impl<P> Unpin for Pin<P> where
P: Unpin,
And the Box<T> inside it is unconditionally Unpin:
impl<T> Unpin for Box<T> where
T: ?Sized,
So a Pin<Box<dyn Future>> is a unpinned Future. Everything works out, but why Box itself doesn't? This is one place where Deref gets in the way:
impl<T: ?Sized> Deref for Box<T> {
type Target = T;
}
await expects an unpinned Future, and the Box<dyn Future> you created with Box::new does contain a Future. So it is auto-dereferenced and the Unpin is lost unless you explicitly state it that way with Box<dyn Future + Unpin>.
Edit: #ÖmerErden is right about why Box<dyn Future> wouldn't work.