Is there a way to add code to a Future/async fn in Rust such that it will always run when the execution of the Future is terminating, whether this is because the Future is being dropped or because it has finished normally?
I found this implemented by the future-utils crate called Finally but it relies on an older version of the Future trait before it was stabilized in the standard library. I was bamboozled thinking it was the futures-utils crate, which IS up to date, but doesn't have this functionality.
Regardless, here's an implementation that will call a function when a future is dropped:
[dependencies]
pin-project = "1.0.12"
//! on_drop.rs
use std::future::Future;
use std::mem::ManuallyDrop;
use std::pin::Pin;
use std::task::{Poll, Context};
use pin_project::{pin_project, pinned_drop};
#[pin_project(PinnedDrop)]
pub struct OnDrop<Fut, F> where F: FnOnce() {
#[pin]
future: Fut,
drop_fn: ManuallyDrop<F>,
}
impl <Fut, F> OnDrop<Fut, F>
where F: FnOnce()
{
pub fn new(future: Fut, drop_fn: F) -> Self {
Self {
future,
drop_fn: ManuallyDrop::new(drop_fn),
}
}
}
impl<Fut, F> Future for OnDrop<Fut, F>
where
F: FnOnce(),
Fut: Future,
{
type Output = Fut::Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
this.future.poll(cx)
}
}
#[pinned_drop]
impl<Fut, F> PinnedDrop for OnDrop<Fut, F>
where F: FnOnce()
{
fn drop(self: Pin<&mut Self>) {
let mut this = self.project();
// SAFETY: we always construct `drop_fn` with a value and this is the
// only place we take the value out of it, when the value is being
// dropped.
let drop_fn = unsafe { ManuallyDrop::take(&mut this.drop_fn) };
drop_fn();
}
}
pub trait FutureOnDropExt {
fn on_drop<F: FnOnce()>(self, drop_fn: F) -> OnDrop<Self, F>
where
Self: Sized;
}
impl<Fut> FutureOnDropExt for Fut where Fut: Future {
fn on_drop<F: FnOnce()>(self, drop_fn: F) -> OnDrop<Self, F>
where
Self: Sized
{
OnDrop::new(self, drop_fn)
}
}
And can be used like:
//! main.rs
mod on_drop;
use on_drop::FutureOnDropExt;
fn main() {
async { println!("this doesn't print since we don't poll") }
.on_drop(|| println!("this does print when dropped"));
}
this does print when dropped
I took a quick look to see if anyone had suggested this for the futures crate but didn't see anything relevant. If you have a genuine use-case for this, I'd suggest proposing such an adapter. Either it could be added in the future, or sparking the idea may find there is a flaw in the technique or a better solution to achieve your goals.
Related
Can I propagate the Send trait of function parameters to its return type, so that the return type is impl Send if and only if the parameters are?
Details:
An async function has a nice feature. Its returned Future is automatically Send if it can be. In the following example, the async function will create a Future that is Send, if the inputs to the function are Send.
struct MyStruct;
impl MyStruct {
// This async fn returns an `impl Future<Output=T> + Send` if `T` is Send.
// Otherwise, it returns an `impl Future<Output=T>` without `Send`.
async fn func<T>(&self, t: T) -> T {
t
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func(4u64));
// And the following correctly fails
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
Now, I want to move such a function into a trait, which requires using async-trait (which is some codegen that effectively writes my async fn as a function returning Pin<Box<dyn Future>>) or doing something similar manually. Is there a way to write this in a way to retain this auto-Send behavior where the returned Future is made Send if T is Send? The following example implements it as two separate functions:
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func_send<T: 'static + Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + Send>> {
Box::pin(async{t})
}
fn func_not_send<T: 'static>(&self, t: T) -> Pin<Box<dyn Future<Output = T>>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func_send(4u64));
// And the following correctly fails
// assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
But actually, I don't want them to be separate. I want them to be one function similar to how async fn does it automatically. Something along the lines of
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func<T: 'static + ?Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + ?Send>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This should
assert_is_send(MyStruct.func(4u64));
// And this should fail
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
Is something like this possible in Rust? I'm ok with writing the async-trait magic manually and modifying it instead of using the async-trait crate if that is a way to make it work.
Some ideas I had but they haven't really borne fruit yet:
Use min-specialization to specialize on Send? But doesn't seem like that feature is going to be stabilized anytime soon so maybe not the best option.
Return a custom MyFuture type instead of just impl Future and somehow impl Send for MyFuture where T: Send? Would probably be difficult though since I would have to be able to name that Future and async code usually produces impl Future types that cannot be named.
Writing a procedural macro that adds + Send to the return type if it recognizes that the input type is Send. Actually, can procedural macros detect if a certain type implements Send? My guess would be it's not possible since they just work on token streams.
(2) is the only way that could work.
There are two ways to make it work:
Write the future manually, without the help of async and .await. But that means writing the future manually:
enum ConditionalSendFut<T> {
Start { t: T },
Done,
}
impl<T> Unpin for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, _context: &mut Context<'_>) -> Poll<Self::Output> {
match &mut *self {
Self::Start { .. } => {
let t = match std::mem::replace(&mut *self, Self::Done) {
Self::Start { t } => t,
_ => unreachable!(),
};
Poll::Ready(t)
}
Self::Done => Poll::Pending,
}
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut::Start { t }
}
}
Playground.
Store a Pin<Box<dyn Future<Output = T>>> and conditionally impl Send on the future. But this requires unsafe code and manually ensuring that you don't hold other non-Send types across .await points:
struct ConditionalSendFut<T>(Pin<Box<dyn Future<Output = T>>>);
// SAFETY: The only non-`Send` type we're holding across an `.await`
// point is `T`.
unsafe impl<T: Send> Send for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().poll(context)
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut(Box::pin(async { t }))
}
}
Playground.
(1) cannot work with traits, as each impl will have a different future. This leaves us with (2) only. I would not recommend it, but it is possible.
It is very likely that when async fns in traits will be stable there will be a mechanism to that (what is talked about currently is to impl them conditionally and use bounds on use sites to require them) but currently there is no such thing, even on the nightly implementation of async fns in traits.
I'm looking to implement a wrapper struct for any stream that returns a certain type, to cut down on the dynamic keywords littering my application. I've come across BoxStream, but have no idea how to make use of it in Stream::poll_next. Here's what I have so far:
use std::pin::Pin;
use std::task::{Context, Poll};
use futures::prelude::stream::BoxStream;
use futures::Stream;
pub struct Row;
pub struct RowCollection<'a> {
stream: BoxStream<'a, Row>,
}
impl RowCollection<'_> {
pub fn new<'a>(stream: BoxStream<Row>) -> RowCollection {
RowCollection { stream }
}
}
impl Stream for RowCollection<'_> {
type Item = Row;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// I have no idea what to put here, but it needs to get the information from self.stream and return appropriate value
}
}
Dependencies:
futures = "0.3"
Since Box implements Unpin, then BoxStream implements Unpin, and so will RowCollection.
Because of this, you can make use of Pin::get_mut which will give you a &mut RowCollection. From that, you can get a &mut BoxStream. You can re-pin that via Pin::new and then call poll_next on it. This is called pin-projection.
impl Stream for RowCollection<'_> {
type Item = Row;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
Pin::new(&mut self.get_mut().stream).poll_next(cx)
}
}
See also:
No method named `poll` found for a type that implements `Future`
Following code does not compile because the element pulled from the vector does not implement the Pin<> type.
The error is in ele.poll() call
#[pin_project]
pub struct FifoCompletions<T, F>
where
F: Future<Output = Result<T, Error>>
{
#[pin]
pending: VecDeque<F>,
}
impl <T, F> FifoCompletions<T, F>
where
F: Future<Output = Result<T, Error>>
{
pub fn push(&mut self, f: F) {
self.pending.push_back(f);
}
}
impl <T, F> Future for FifoCompletions<T, F>
where
F: Future<Output = Result<T, Error>>
{
type Output = Result<T, Error>;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
while !this.pending.is_empty() {
match this.pending.front_mut() {
None => unreachable!(),
Some(ele) => {
let output = ready!(ele.poll(cx));
}
}
}
Poll::Pending
}
}
Error message is
no method named `poll` found for mutable reference `&mut F` in the current scope
method not found in `&mut F`
help: items from traits can only be used if the type parameter is bounded by the traitrustc(E0599)
sm.rs(66, 45): method not found in `&mut F`
Here's the literal answer to your question:
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
pub struct Completions<F>
where
F: Future<Output = ()>,
{
pending: Vec<F>,
}
impl<F> Future for Completions<F>
where
F: Future<Output = ()>,
{
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
// I copied this from Stack Overflow without reading the prose
// that describes why this is or is not safe.
// ... It's probably not safe.
let first = unsafe { self.map_unchecked_mut(|this| &mut this.pending[0]) };
first.poll(cx)
}
}
However, this is very likely to be unsafe and introduce undefined behavior. The problem is that the requirements for pinning are complex and nuanced. Specifically, once pinned, a value may never be moved in memory, including when it is dropped. From the docs, emphasis mine:
This can be tricky, as witnessed by VecDeque<T>: the destructor of VecDeque<T> can fail to call drop on all elements if one of the destructors panics. This violates the Drop guarantee, because it can lead to elements being deallocated without their destructor being called. (VecDeque<T> has no pinning projections, so this does not cause unsoundness.)
I checked with taiki-e, the author of the pin-project crate. A lightly-edited quote:
Operations such as Vec(Deque)'s push, insert, remove, etc. can move elements. If you want to pin elements, these methods should not be called without Unpin after they have been pinned. The pin API actually blocks this.
If the destructor moves elements, it is unsound to pin the element returned by accessors like get_mut, front_mut without Unpin.
Make the element Unpin or use a collection that can handle !Unpin futures like FutureUnordered instead of VecDeque.
Applied directly, the safest way to do this is:
use pin_project::pin_project; // 1.0
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
#[pin_project]
pub struct Completions<F>
where
F: Unpin,
F: Future<Output = ()>,
{
#[pin]
pending: Vec<F>,
}
impl<F> Future for Completions<F>
where
F: Unpin,
F: Future<Output = ()>,
{
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
let pending = this.pending.get_mut();
let first = Pin::new(&mut pending[0]);
first.poll(cx)
}
}
See also:
No method named `poll` found for a type that implements `Future`
When is it safe to move a member value out of a pinned future?
I can't figure out how to provide a Stream where I await async functions to get the data needed for the values of the stream.
I've tried to implement the the Stream trait directly, but I run into issues because I'd like to use async things like awaiting, the compiler does not want me to call async functions.
I assume that I'm missing some background on what the goal of Stream is and I'm just attacking this incorrectly and perhaps I shouldn't be looking at Stream at all, but I don't know where else to turn. I've seen the other functions in the stream module that could be useful, but I'm unsure how I could store any state and use these functions.
As a slightly simplified version of my actual goal, I want to provide a stream of 64-byte Vecs from an AsyncRead object (i.e. TCP stream), but also store a little state inside whatever logic ends up producing values for the stream, in this example, a counter.
pub struct Receiver<T>
where
T: AsyncRead + Unpin,
{
readme: T,
num: u64,
}
// ..code for a simple `new() -> Self` function..
impl<T> Stream for Receiver<T>
where
T: AsyncRead + Unpin,
{
type Item = Result<Vec<u8>, io::Error>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let mut buf: [u8; 64] = [0; 64];
match self.readme.read_exact(&mut buf).await {
Ok(()) => {
self.num += 1;
Poll::Ready(Some(Ok(buf.to_vec())))
}
Err(e) => Poll::Ready(Some(Err(e))),
}
}
}
This fails to build, saying
error[E0728]: `await` is only allowed inside `async` functions and blocks
I'm using rustc 1.36.0-nightly (d35181ad8 2019-05-20) and my Cargo.toml looks like this:
[dependencies]
futures-preview = { version = "0.3.0-alpha.16", features = ["compat", "io-compat"] }
pin-utils = "0.1.0-alpha.4"
Answer copy/pasted from the reddit post by user Matthias247:
It's unfortunately not possible at the moment - Streams have to be implemented by hand and can not utilize async fn. Whether it's possible to change this in the future is unclear.
You can work around it by defining a different Stream trait which makes use of Futures like:
trait Stream<T> {
type NextFuture: Future<Output=T>;
fn next(&mut self) -> Self::NextFuture;
}
This article and this futures-rs issue have more information around it.
You can do it with gen-stream crate:
#![feature(generators, generator_trait, gen_future)]
use {
futures::prelude::*,
gen_stream::{gen_await, GenTryStream},
pin_utils::unsafe_pinned,
std::{
io,
marker::PhantomData,
pin::Pin,
sync::{
atomic::{AtomicU64, Ordering},
Arc,
},
task::{Context, Poll},
},
};
pub type Inner = Pin<Box<dyn Stream<Item = Result<Vec<u8>, io::Error>> + Send>>;
pub struct Receiver<T> {
inner: Inner,
pub num: Arc<AtomicU64>,
_marker: PhantomData<T>,
}
impl<T> Receiver<T> {
unsafe_pinned!(inner: Inner);
}
impl<T> From<T> for Receiver<T>
where
T: AsyncRead + Unpin + Send + 'static,
{
fn from(mut readme: T) -> Self {
let num = Arc::new(AtomicU64::new(0));
Self {
inner: Box::pin(GenTryStream::from({
let num = num.clone();
static move || loop {
let mut buf: [u8; 64] = [0; 64];
match gen_await!(readme.read_exact(&mut buf)) {
Ok(()) => {
num.fetch_add(1, Ordering::Relaxed);
yield Poll::Ready(buf.to_vec())
}
Err(e) => return Err(e),
}
}
})),
num,
_marker: PhantomData,
}
}
}
impl<T> Stream for Receiver<T>
where
T: AsyncRead + Unpin,
{
type Item = Result<Vec<u8>, io::Error>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.inner().poll_next(cx)
}
}
I'm working with Tokio doing some UDP stuff.
I want to record the amount of time my UDP probe future takes to resolve. I came up with the following function, time_future(), to wrap a future and give me the result and a duration. The function seems very naive and I think Rust has the power to express the concept much more cleanly.
My working code (Playground):
extern crate futures; // 0.1.25
extern crate tokio; // 0.1.11
use std::time::{Duration, Instant};
use futures::future::{lazy, ok};
use futures::Future;
use tokio::runtime::current_thread::Runtime;
use tokio::timer::Delay;
struct TimedFutureResult<T, E> {
elapsed: Duration,
result: Result<T, E>,
}
impl<T, E> TimedFutureResult<T, E> {
pub fn elapsed_ms(&self) -> i64 {
return (self.elapsed.as_secs() * 1000 + (self.elapsed.subsec_nanos() / 1000000) as u64)
as i64;
}
}
fn time_future<F: Future>(f: F) -> impl Future<Item = TimedFutureResult<F::Item, F::Error>> {
lazy(|| {
let start = Instant::now();
f.then(move |result| {
ok::<TimedFutureResult<F::Item, F::Error>, ()>(TimedFutureResult {
elapsed: start.elapsed(),
result: result,
})
})
})
}
fn main() {
let when = Instant::now() + Duration::from_millis(100);
let f = time_future(Delay::new(when)).then(|r| match r {
Ok(r) => {
println!("resolved in {}ms", r.elapsed_ms());
r.result
}
_ => unreachable!(),
});
let mut runtime = Runtime::new().unwrap();
runtime.block_on(f).unwrap();
}
How can I improve this and make it more idiomatic? Can I somehow get the interface to work similarly to inspect() or then()?
Delay::new(when)
.timed(|res, elapsed| println!("{}ms!", elapsed))
.and_then(...);
I tried creating a Timed trait and implementing it for Future but I didn't feel at all confident in how I was going about it. The types just really threw me for a loop.
Am I at least barking up the right tree?
Shepmaster's answer is great. However, the version of futures they used is out-of-date and incompatible with stdlib futures. Here's my rewrite to use stdlib futures.
use pin_project::pin_project;
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::{Duration, Instant};
use tokio::time; // 0.2
/// A wrapper around a Future which adds timing data.
#[pin_project]
struct Timed<Fut, F>
where
Fut: Future,
F: FnMut(&Fut::Output, Duration),
{
#[pin]
inner: Fut,
f: F,
start: Option<Instant>,
}
impl<Fut, F> Future for Timed<Fut, F>
where
Fut: Future,
F: FnMut(&Fut::Output, Duration),
{
type Output = Fut::Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output> {
let this = self.project();
let start = this.start.get_or_insert_with(Instant::now);
match this.inner.poll(cx) {
// If the inner future is still pending, this wrapper is still pending.
Poll::Pending => Poll::Pending,
// If the inner future is done, measure the elapsed time and finish this wrapper future.
Poll::Ready(v) => {
let elapsed = start.elapsed();
(this.f)(&v, elapsed);
Poll::Ready(v)
}
}
}
}
trait TimedExt: Sized + Future {
fn timed<F>(self, f: F) -> Timed<Self, F>
where
F: FnMut(&Self::Output, Duration),
{
Timed {
inner: self,
f,
start: None,
}
}
}
// All futures can use the `.timed` method defined above
impl<F: Future> TimedExt for F {}
#[tokio::main]
async fn main() {
let when = Instant::now() + Duration::from_millis(100);
let fut = time::delay_until(when.into())
.timed(|res, elapsed| println!("{:?} elapsed, got {:?}", elapsed, res));
fut.await;
}
The act of writing the future is easy enough, and adding a chainable method is the same technique as that shown in How can I add new methods to Iterator?.
The only really tricky aspect is deciding when the time starts — is it when the future is created or when it is first polled?
I chose to use when it's first polled, as that seems more useful:
extern crate futures; // 0.1.25
extern crate tokio; // 0.1.11
use std::time::{Duration, Instant};
use futures::{try_ready, Async, Future, Poll};
use tokio::{runtime::current_thread::Runtime, timer::Delay};
struct Timed<Fut, F>
where
Fut: Future,
F: FnMut(&Fut::Item, Duration),
{
inner: Fut,
f: F,
start: Option<Instant>,
}
impl<Fut, F> Future for Timed<Fut, F>
where
Fut: Future,
F: FnMut(&Fut::Item, Duration),
{
type Item = Fut::Item;
type Error = Fut::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
let start = self.start.get_or_insert_with(Instant::now);
let v = try_ready!(self.inner.poll());
let elapsed = start.elapsed();
(self.f)(&v, elapsed);
Ok(Async::Ready(v))
}
}
trait TimedExt: Sized + Future {
fn timed<F>(self, f: F) -> Timed<Self, F>
where
F: FnMut(&Self::Item, Duration),
{
Timed {
inner: self,
f,
start: None,
}
}
}
impl<F: Future> TimedExt for F {}
fn main() {
let when = Instant::now() + Duration::from_millis(100);
let f = Delay::new(when).timed(|res, elapsed| println!("{:?} elapsed, got {:?}", elapsed, res));
let mut runtime = Runtime::new().unwrap();
runtime.block_on(f).unwrap();
}