How do I pin project an element of a vector? - rust

Following code does not compile because the element pulled from the vector does not implement the Pin<> type.
The error is in ele.poll() call
#[pin_project]
pub struct FifoCompletions<T, F>
where
F: Future<Output = Result<T, Error>>
{
#[pin]
pending: VecDeque<F>,
}
impl <T, F> FifoCompletions<T, F>
where
F: Future<Output = Result<T, Error>>
{
pub fn push(&mut self, f: F) {
self.pending.push_back(f);
}
}
impl <T, F> Future for FifoCompletions<T, F>
where
F: Future<Output = Result<T, Error>>
{
type Output = Result<T, Error>;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
while !this.pending.is_empty() {
match this.pending.front_mut() {
None => unreachable!(),
Some(ele) => {
let output = ready!(ele.poll(cx));
}
}
}
Poll::Pending
}
}
Error message is
no method named `poll` found for mutable reference `&mut F` in the current scope
method not found in `&mut F`
help: items from traits can only be used if the type parameter is bounded by the traitrustc(E0599)
sm.rs(66, 45): method not found in `&mut F`

Here's the literal answer to your question:
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
pub struct Completions<F>
where
F: Future<Output = ()>,
{
pending: Vec<F>,
}
impl<F> Future for Completions<F>
where
F: Future<Output = ()>,
{
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
// I copied this from Stack Overflow without reading the prose
// that describes why this is or is not safe.
// ... It's probably not safe.
let first = unsafe { self.map_unchecked_mut(|this| &mut this.pending[0]) };
first.poll(cx)
}
}
However, this is very likely to be unsafe and introduce undefined behavior. The problem is that the requirements for pinning are complex and nuanced. Specifically, once pinned, a value may never be moved in memory, including when it is dropped. From the docs, emphasis mine:
This can be tricky, as witnessed by VecDeque<T>: the destructor of VecDeque<T> can fail to call drop on all elements if one of the destructors panics. This violates the Drop guarantee, because it can lead to elements being deallocated without their destructor being called. (VecDeque<T> has no pinning projections, so this does not cause unsoundness.)
I checked with taiki-e, the author of the pin-project crate. A lightly-edited quote:
Operations such as Vec(Deque)'s push, insert, remove, etc. can move elements. If you want to pin elements, these methods should not be called without Unpin after they have been pinned. The pin API actually blocks this.
If the destructor moves elements, it is unsound to pin the element returned by accessors like get_mut, front_mut without Unpin.
Make the element Unpin or use a collection that can handle !Unpin futures like FutureUnordered instead of VecDeque.
Applied directly, the safest way to do this is:
use pin_project::pin_project; // 1.0
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
#[pin_project]
pub struct Completions<F>
where
F: Unpin,
F: Future<Output = ()>,
{
#[pin]
pending: Vec<F>,
}
impl<F> Future for Completions<F>
where
F: Unpin,
F: Future<Output = ()>,
{
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
let pending = this.pending.get_mut();
let first = Pin::new(&mut pending[0]);
first.poll(cx)
}
}
See also:
No method named `poll` found for a type that implements `Future`
When is it safe to move a member value out of a pinned future?

Related

Is there a way to add a cleanup routine to a Future?

Is there a way to add code to a Future/async fn in Rust such that it will always run when the execution of the Future is terminating, whether this is because the Future is being dropped or because it has finished normally?
I found this implemented by the future-utils crate called Finally but it relies on an older version of the Future trait before it was stabilized in the standard library. I was bamboozled thinking it was the futures-utils crate, which IS up to date, but doesn't have this functionality.
Regardless, here's an implementation that will call a function when a future is dropped:
[dependencies]
pin-project = "1.0.12"
//! on_drop.rs
use std::future::Future;
use std::mem::ManuallyDrop;
use std::pin::Pin;
use std::task::{Poll, Context};
use pin_project::{pin_project, pinned_drop};
#[pin_project(PinnedDrop)]
pub struct OnDrop<Fut, F> where F: FnOnce() {
#[pin]
future: Fut,
drop_fn: ManuallyDrop<F>,
}
impl <Fut, F> OnDrop<Fut, F>
where F: FnOnce()
{
pub fn new(future: Fut, drop_fn: F) -> Self {
Self {
future,
drop_fn: ManuallyDrop::new(drop_fn),
}
}
}
impl<Fut, F> Future for OnDrop<Fut, F>
where
F: FnOnce(),
Fut: Future,
{
type Output = Fut::Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
this.future.poll(cx)
}
}
#[pinned_drop]
impl<Fut, F> PinnedDrop for OnDrop<Fut, F>
where F: FnOnce()
{
fn drop(self: Pin<&mut Self>) {
let mut this = self.project();
// SAFETY: we always construct `drop_fn` with a value and this is the
// only place we take the value out of it, when the value is being
// dropped.
let drop_fn = unsafe { ManuallyDrop::take(&mut this.drop_fn) };
drop_fn();
}
}
pub trait FutureOnDropExt {
fn on_drop<F: FnOnce()>(self, drop_fn: F) -> OnDrop<Self, F>
where
Self: Sized;
}
impl<Fut> FutureOnDropExt for Fut where Fut: Future {
fn on_drop<F: FnOnce()>(self, drop_fn: F) -> OnDrop<Self, F>
where
Self: Sized
{
OnDrop::new(self, drop_fn)
}
}
And can be used like:
//! main.rs
mod on_drop;
use on_drop::FutureOnDropExt;
fn main() {
async { println!("this doesn't print since we don't poll") }
.on_drop(|| println!("this does print when dropped"));
}
this does print when dropped
I took a quick look to see if anyone had suggested this for the futures crate but didn't see anything relevant. If you have a genuine use-case for this, I'd suggest proposing such an adapter. Either it could be added in the future, or sparking the idea may find there is a flaw in the technique or a better solution to achieve your goals.

How can I wrap a dynamically typed stream for API convenience?

I'm looking to implement a wrapper struct for any stream that returns a certain type, to cut down on the dynamic keywords littering my application. I've come across BoxStream, but have no idea how to make use of it in Stream::poll_next. Here's what I have so far:
use std::pin::Pin;
use std::task::{Context, Poll};
use futures::prelude::stream::BoxStream;
use futures::Stream;
pub struct Row;
pub struct RowCollection<'a> {
stream: BoxStream<'a, Row>,
}
impl RowCollection<'_> {
pub fn new<'a>(stream: BoxStream<Row>) -> RowCollection {
RowCollection { stream }
}
}
impl Stream for RowCollection<'_> {
type Item = Row;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// I have no idea what to put here, but it needs to get the information from self.stream and return appropriate value
}
}
Dependencies:
futures = "0.3"
Since Box implements Unpin, then BoxStream implements Unpin, and so will RowCollection.
Because of this, you can make use of Pin::get_mut which will give you a &mut RowCollection. From that, you can get a &mut BoxStream. You can re-pin that via Pin::new and then call poll_next on it. This is called pin-projection.
impl Stream for RowCollection<'_> {
type Item = Row;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
Pin::new(&mut self.get_mut().stream).poll_next(cx)
}
}
See also:
No method named `poll` found for a type that implements `Future`

Writing an async function that calls another async function

I am trying to create a async function which takes a function pointer as a parameter. It does some stuff, calls the function, awaits on the result, then does some more stuff:
use std::future::Future;
async fn run_another_async_fn<F, Fut>(f: F)
where
Fut: Future<Output = ()>,
F: FnOnce(&mut i32) -> Fut,
{
let mut i = 42;
println!("running function");
f(&mut i).await;
println!("ran function");
}
async fn foo(i: &mut i32) {}
async fn bar() {
run_another_async_fn(foo);
}
[view on Rust Playground]
Unfortunately this fails to compile:
error[E0308]: mismatched types
--> src/lib.rs:17:5
|
17 | run_another_async_fn(foo);
| ^^^^^^^^^^^^^^^^^^^^ lifetime mismatch
|
= note: expected associated type `<for<'_> fn(&mut i32) -> impl Future {foo} as FnOnce<(&mut i32,)>>::Output`
found associated type `<for<'_> fn(&mut i32) -> impl Future {foo} as FnOnce<(&mut i32,)>>::Output`
= note: the required lifetime does not necessarily outlive the empty lifetime
note: the lifetime requirement is introduced here
--> src/lib.rs:6:28
|
6 | F: FnOnce(&mut i32) -> Fut,
| ^^^
Firstly, it seems the compiler found exactly what it expected but it's complaining anyway?
Secondly, what's "the empty lifetime"? I guess it must mean the '_, does that have some special significance?
Finally, what's the way to get this to compile?
The issue is that there is no way to specify the same lifetime for F and Fut in the where clause.
Luckily (if you don't mind heap allocating the future) there is an easy workaround. You can use the already existing futures::future::BoxFuture; which looks like:
pub type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
With its help you can specify the same lifetime parameter for both the borrow and as a trait bound for the future:
where for<'a> F: FnOnce(&'a mut i32) -> BoxFuture<'a, ()>,
You also have to add an adapter function which will have the correct return type - i.e. BoxFuture<'_, T> instead of impl Future:
fn asd(i: &mut i32) -> BoxFuture<'_, ()> {
foo(i).boxed()
}
or use a closure:
run_another_async_fn(|i| foo(i).boxed());
As a result your code would look like:
use futures::future::BoxFuture;
use futures::FutureExt;
use std::future::Future;
async fn run_another_async_fn<F>(f: F)
where
for<'a> F: FnOnce(&'a mut i32) -> BoxFuture<'a, ()>,
{
let mut i = 42;
println!("running function");
f(&mut i).await;
println!("ran function");
}
fn asd(i: &mut i32) -> BoxFuture<'_, ()> {
foo(i).boxed()
}
async fn foo<'a>(i: &'a mut i32) {
// no-op
}
async fn bar() {
run_another_async_fn(asd);
run_another_async_fn(|i| foo(i).boxed());
}

how to pin RefCell contents?

I have a struct MyAsyncStream and tokio::io::AsyncWrite implementation for it:
impl<S: AsyncRead + AsyncWrite + Unpin> AsyncWrite for MyAsyncStream<S> {
fn poll_write(mut self: Pin<&mut Self>, cx: &mut Context, buf: &[u8]) -> Poll<Result<usize>> {
....
}
I also have MyAsyncStreamWrapper:
struct MyAsyncStreamWrapper{inner: MyAsyncStream}
Now I want to implement AnotherTrait trait for MyAsyncStreamWrapper, with the following method:
impl AnotherTrait for MyAsyncStreamWrapper {
fn poll_send_to<B>(self: Pin<&Self>, cx: &mut Context<'_>, buf: &[u8], addr: B,) -> Poll<Result<usize, Self::Error>> {
Pin::new(&mut self.inner).poll_write(cx, buf)
}
....
}
In this method implementation, I want to call poll_write on the inner. But, unfortunately they are different on self mutability: Pin<&mut Self> vs Pin<&Self>. As expected it does not compile.
Is there an idiomatic "workaround" for such case? My idea is to wrap inner into Mutex so I could have mutable MyAsyncStream in the non-mutable context:
MyAsyncStreamWrapper{inner: Mutex<RefCell<MyAsyncStream>>}
...
fn poll_send_to<B>(mut self: Pin<&Self>, cx: &mut Context<'_>, buf: &[u8], addr: B,) -> Poll<Result<usize, Self::Error>> {
let rc = self.stream.lock().unwrap();
let ref mut inner = rc.borrow();
let pin = Pin::new(inner);
pin.poll_write(cx, buf);
}
...
But, unfortunately, it also does not compile, with the following error:
pin.poll_write(cx, buf);
^^^^^^^^^^ method not found in `std::pin::Pin<&mut std::cell::RefMut<'_, MyAsyncStream>>`
What is the right way to go?
Dereference then re-borrow seems to work:
use std::cell::RefCell;
use std::pin::Pin;
use std::sync::Mutex;
fn main() {
let a = Mutex::new(RefCell::new(42));
let rc = a.lock().unwrap();
let mut inner = rc.borrow_mut();
let pinned = Pin::new(&mut *inner);
print_type_name(pinned);
}
fn print_type_name<T>(_: T) {
println!("{}", std::any::type_name::<T>());
}
It outputs that the type is core::pin::Pin<&mut i32>.
That being said, using blocking synchronization primitives like Mutex in asynchronous context is probably not a good idea. If possible it is better to let poll_send_to take a Pin<&mut Self> parameter.

Creating a stream of values while calling async fns?

I can't figure out how to provide a Stream where I await async functions to get the data needed for the values of the stream.
I've tried to implement the the Stream trait directly, but I run into issues because I'd like to use async things like awaiting, the compiler does not want me to call async functions.
I assume that I'm missing some background on what the goal of Stream is and I'm just attacking this incorrectly and perhaps I shouldn't be looking at Stream at all, but I don't know where else to turn. I've seen the other functions in the stream module that could be useful, but I'm unsure how I could store any state and use these functions.
As a slightly simplified version of my actual goal, I want to provide a stream of 64-byte Vecs from an AsyncRead object (i.e. TCP stream), but also store a little state inside whatever logic ends up producing values for the stream, in this example, a counter.
pub struct Receiver<T>
where
T: AsyncRead + Unpin,
{
readme: T,
num: u64,
}
// ..code for a simple `new() -> Self` function..
impl<T> Stream for Receiver<T>
where
T: AsyncRead + Unpin,
{
type Item = Result<Vec<u8>, io::Error>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let mut buf: [u8; 64] = [0; 64];
match self.readme.read_exact(&mut buf).await {
Ok(()) => {
self.num += 1;
Poll::Ready(Some(Ok(buf.to_vec())))
}
Err(e) => Poll::Ready(Some(Err(e))),
}
}
}
This fails to build, saying
error[E0728]: `await` is only allowed inside `async` functions and blocks
I'm using rustc 1.36.0-nightly (d35181ad8 2019-05-20) and my Cargo.toml looks like this:
[dependencies]
futures-preview = { version = "0.3.0-alpha.16", features = ["compat", "io-compat"] }
pin-utils = "0.1.0-alpha.4"
Answer copy/pasted from the reddit post by user Matthias247:
It's unfortunately not possible at the moment - Streams have to be implemented by hand and can not utilize async fn. Whether it's possible to change this in the future is unclear.
You can work around it by defining a different Stream trait which makes use of Futures like:
trait Stream<T> {
type NextFuture: Future<Output=T>;
fn next(&mut self) -> Self::NextFuture;
}
This article and this futures-rs issue have more information around it.
You can do it with gen-stream crate:
#![feature(generators, generator_trait, gen_future)]
use {
futures::prelude::*,
gen_stream::{gen_await, GenTryStream},
pin_utils::unsafe_pinned,
std::{
io,
marker::PhantomData,
pin::Pin,
sync::{
atomic::{AtomicU64, Ordering},
Arc,
},
task::{Context, Poll},
},
};
pub type Inner = Pin<Box<dyn Stream<Item = Result<Vec<u8>, io::Error>> + Send>>;
pub struct Receiver<T> {
inner: Inner,
pub num: Arc<AtomicU64>,
_marker: PhantomData<T>,
}
impl<T> Receiver<T> {
unsafe_pinned!(inner: Inner);
}
impl<T> From<T> for Receiver<T>
where
T: AsyncRead + Unpin + Send + 'static,
{
fn from(mut readme: T) -> Self {
let num = Arc::new(AtomicU64::new(0));
Self {
inner: Box::pin(GenTryStream::from({
let num = num.clone();
static move || loop {
let mut buf: [u8; 64] = [0; 64];
match gen_await!(readme.read_exact(&mut buf)) {
Ok(()) => {
num.fetch_add(1, Ordering::Relaxed);
yield Poll::Ready(buf.to_vec())
}
Err(e) => return Err(e),
}
}
})),
num,
_marker: PhantomData,
}
}
}
impl<T> Stream for Receiver<T>
where
T: AsyncRead + Unpin,
{
type Item = Result<Vec<u8>, io::Error>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.inner().poll_next(cx)
}
}

Resources