I'm willing to impl a simple Future counter, but smth goes decidedly wrong about it. Without any use of Context the following programme would just block forever;
use std::future::Future;
use std::task::{Poll, Context};
use std::pin::Pin;
use futures::executor::block_on;
struct MyStruct {
counter: u32
}
impl Future for MyStruct {
type Output = String;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
if self.counter == 5 {
Poll::Ready(self.counter.to_string())
} else {
unsafe {
Pin::get_unchecked_mut(self).counter += 1;
}
// cx.waker().wake();
Poll::Pending
}
}
}
fn main() {
let ms = MyStruct{counter: 0};
block_on(ms);
}
I imagine that I have to postpone somehow a call of Waker, but it's not that straightforward. So I wonder, how to wake it in the most simple form?
You can call .wake_by_ref(). Waking the waker will tell the executor that is handling the Future to be .poll()'d again:
impl Future for MyStruct {
type Output = String;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
if self.counter == 5 {
Poll::Ready(self.counter.to_string())
} else {
unsafe {
Pin::get_unchecked_mut(self).counter += 1;
}
cx.waker().wake_by_ref();
Poll::Pending
}
}
}
So your implementation will always ask to be immediately re-polled. I'll note that this is logically no different than simply returning Poll::Ready("5".to_string()) on the first invocation since there's no asynchronous work being done here. Because of that, there's no reason for it to be a Future at all.
Related
I want to add sleep to avoid busy loop in Future::poll(), expect the finally effect is that runtime call poll() every second, the fifth second will return Poll::Ready(()).
According to my understanding, poll() is be called in first, this.sleeper.poll(cx) will return Poll::Pending immediately, cx.waker().wake_by_ref() will make current task to execute again immediately.
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use pin_project_lite::pin_project;
use std::time::{Duration, Instant};
pin_project! {
struct Delay {
when: Instant,
#[pin]
sleeper: tokio::time::Sleep,
}
}
impl Delay {
pub fn new(when: Instant) -> Self {
Self {
when,
sleeper: tokio::time::sleep(Duration::from_secs(1))
}
}
}
impl Future for Delay {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
if self.when < Instant::now() {
Poll::Ready(())
} else {
let this = self.project();
// I want to add sleep to avoid busy loop.
this.sleeper.poll(cx);
// here will make CPU busy loop, uninterrupted call poll()
cx.waker().wake_by_ref();
// expect five "1", but will get many "1"
print!("1");
Poll::Pending
}
}
}
#[tokio::main]
async fn main() {
let when = Instant::now() + Duration::from_secs(5);
let delay1 = Delay::new(when);
delay1.await;
println!("Hello, world!");
}
Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=030fbade7a2f1e5df50c4774fc06fdf1
There are two problems here. First and simplest, when your future is delegating to another future in poll(), you do not need to call the waker at all — the other future will do that. In this case, the Sleeper takes care of getting the waker called at the right time, causing your future to be polled again.
Second, you're ignoring the return value of the sleeper.poll(). If it returns Poll::Ready, you need to make sure you don't poll it again — either do something else, or replace self.sleeper with a new sleep(). Polling a future again after it returned Poll::Ready is not meaningful, and has consequences left up to the future (often a panic).
finally, I resoved this problem by changing Sleep to Interval and call poll_tick().
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use pin_project_lite::pin_project;
use std::time::{Duration, Instant};
use tokio::time::Sleep;
pin_project! {
struct Delay {
when: Instant,
#[pin]
sleeper: tokio::time::Interval,
}
}
impl Delay {
pub fn new(when: Instant) -> Self {
Self {
when,
sleeper: tokio::time::interval(Duration::from_secs(1))
}
}
}
impl Future for Delay {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
println!("0");
if self.when < Instant::now() {
println!("1");
Poll::Ready(())
} else {
println!("2");
let this = unsafe {self.get_unchecked_mut()};
//let this = self.project();
this.sleeper.poll_tick(cx);
Poll::Pending
}
}
}
#[tokio::main]
async fn main() {
let when = Instant::now() + Duration::from_secs(5);
let delay1 = Delay::new(when);
delay1.await;
}
I'm trying to create a DelayedValue future that resolves to a value after a certain time period has elapsed. To do this I simply wanted to wrap the Sleep future from tokio crate. But I get errors relating to Pin and no matter what I do I can't seem to call the poll method on the underlying Sleep member.
For reference here is a full program which fails to compile but should illustrate what I want:
use futures::task::{Context, Poll};
use futures::Future;
use std::pin::Pin;
use tokio::time::{sleep, Sleep, Duration};
struct DelayedValue<T> {
value: T,
sleep: Sleep,
}
impl<T> Future for DelayedValue<T> {
type Output = T;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match &mut self.sleep.poll(cx) {
Poll::Ready(()) => Poll::Ready(self.value),
x => x,
}
}
}
#[tokio::main]
async fn main() {
let dv = DelayedValue {
value: 10_u8,
sleep: sleep(Duration::from_millis(5000)),
};
println!("waiting for delayed value");
let v = dv.await;
println!("delayed value: {}", v);
}
There is also a playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=d573d8dcbbef5c99314d98cacc3d6c92
The easiest way is to use pin-project or pin-project-lite:
pin_project_lite::pin_project! {
struct DelayedValue<T> {
value: Option<T>,
#[pin]
sleep: Sleep,
}
}
impl<T> Future for DelayedValue<T> {
type Output = T;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let this = self.project();
match this.sleep.poll(cx) {
Poll::Ready(()) => Poll::Ready(this.value.take().unwrap()),
Poll::Pending => Poll::Pending,
}
}
}
For reference, since I only needed this for this struct I opted to not use pin-project. Instead I implemented it myself for the field I needed:
#[derive(Debug)]
pub struct DelayedValue<T: Copy> {
value: T,
sleep: Sleep,
}
impl<T: Copy> DelayedValue<T> {
pub fn new(value: T, sleep: Sleep) -> DelayedValue<T> {
DelayedValue {value, sleep}
}
}
impl<T: Copy> Future for DelayedValue<T> {
type Output = T;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let x = self.value;
let s = unsafe { self.map_unchecked_mut(|s| &mut s.sleep) };
match &mut s.poll(cx) {
Poll::Ready(()) => Poll::Ready(x),
Poll::Pending => Poll::Pending,
}
}
}
The easiest is probably to just map the result of a Sleep future instead of implementing a full new struct:
use futures::FutureExt;
use tokio::time::{ Duration, sleep };
#[tokio::main]
async fn main() {
let dv = sleep (Duration::from_millis (5000)).map (|_| { 10_u8 });
println!("waiting for delayed value");
let v = dv.await;
println!("delayed value: {}", v);
}
Playground
I want to sleep 1 second in poll(), but it gets stuck in Pending.
From what I understand, I pass it &mut cx and it should wake up the current task after 1 second.
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::{Duration};
struct MyDelay {}
impl Future for MyDelay {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
println!("poll");
let sleep = tokio::time::sleep(Duration::from_secs(1));
tokio::pin!(sleep);
sleep.poll(cx)
}
}
#[tokio::main]
async fn main() {
let delay = MyDelay{};
let a = delay.await;
dbg!(a);
}
Play url: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=ed6c09df100f1140ddc85b2ae600ab93
Each time you poll you create a new Sleep future, and it starts from scratch.
Instead, you should store the Sleep inside your future. It is easy with pin-project-lite:
pin_project_lite::pin_project! {
struct MyDelay {
#[pin]
sleep: tokio::time::Sleep,
}
}
impl MyDelay {
fn new() -> Self {
Self {
sleep: tokio::time::sleep(Duration::from_secs(1)),
}
}
}
impl Future for MyDelay {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
println!("poll");
let this = self.project();
this.sleep.poll(cx)
}
}
Playground.
Note that the number of times this will be polled is not guaranteed, and may not be constant.
So I'm getting a Response from the reqwest crate and passing it to a HttpResponseBuilder from the actix_web create. However I've tried and failed to understand how to implement the Stream trait from the futures create on a custom struct to act as a middleman and copy the contents down to a file.
I've tried doing this so far, but I'm not sure what to put inside that poll_next function to make it all work.
struct FileCache {
stream: Box<dyn futures::Stream<Item = reqwest::Result<bytes::Bytes>>>,
}
impl FileCache {
fn new(stream: Box<dyn futures::Stream<Item = reqwest::Result<bytes::Bytes>>>) -> Self {
FileCache { stream }
}
}
impl Stream for FileCache {
type Item = reqwest::Result<bytes::Bytes>;
fn poll_next(
self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<Self::Item>> {
}
}
This is possible but requires you to understand what pinning is and how to use it safely.
Basically, we just need to proxy to self.stream.poll_next(), but this method accepts Pin<&mut Self> (as you can see in your own implementation). Storing the box as Pin<Box<T>> instead of Box<T> will give us a way to obtain this Pin relatively easily, without requiring unsafe. Making this change is straightforward, since there is a From implementation allowing conversion of Box<T> to Pin<Box<T>> directly:
use std::pin::Pin;
use std::task::{Context, Poll};
use futures::Stream;
struct FileCache {
stream: Pin<Box<dyn Stream<Item = reqwest::Result<bytes::Bytes>>>>,
}
impl FileCache {
fn new(stream: Box<dyn Stream<Item = reqwest::Result<bytes::Bytes>>>) -> FileCache {
FileCache { stream: stream.into() }
}
}
Now we have to figure out how to go from Pin<&mut FileCache> to Pin<&mut dyn Stream<...>>. The correct incantation here is self.get_mut().stream.as_mut():
impl Stream for FileCache {
type Item = reqwest::Result<bytes::Bytes>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
match self.get_mut().stream.as_mut().poll_next(cx) {
Poll::Pending => Poll::Pending,
Poll::Ready(v) => {
// Do what you need to do with v here.
Poll::Ready(v)
}
}
}
}
The catch is that poll_next isn't async and so you can't asynchronously wait for whatever you're doing with v. bytes::Bytes is atomically-refcounted, though, so you could clone the inner bytes::Bytes value and spawn a separate task on your executor, which is probably what you want to do anyway so that whoever is waiting for FileCache doesn't have to wait for that task to complete before using the data. So you'd do something like:
Poll::Ready(v) => {
if let Some(Ok(ref bytes)) = &v {
let bytes = bytes.clone();
spawn_new_task(async move {
// Do something with bytes
});
}
Poll::Ready(v)
}
Where spawn_new_task() is the function your executor provides, e.g. tokio::spawn().
Now that we can see what we're doing here, we can simplify this down and eliminate the match by pushing Poll::Ready into our pattern, and unconditionally returning whatever the inner poll_next() call did:
impl Stream for FileCache {
type Item = reqwest::Result<bytes::Bytes>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let r = self.get_mut().stream.as_mut().poll_next(cx);
if let Poll::Ready(Some(Ok(ref bytes))) = &r {
let bytes = bytes.clone();
spawn_new_task(async move {
// Do something with bytes
});
}
r
}
}
I'm trying to implement Async read for a struct that has a futures::lock::Mutex:
pub struct SmolSocket<'a> {
stack: Arc<futures::lock::Mutex<SmolStackWithDevice<'a>>>,
}
impl<'a> AsyncRead for SmolSocket<'a> {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>
) -> Poll<std::io::Result<()>> {
block_on(self.stack).read(...)
}
}
The problem is that, since poll_read is not async, I cannot call await. But I also don't want to, as it'd block. I could call try_lock to try and if not, I'd register a Waker to be called by SmolSocket in the future.
Since I cannot do that either because it's not async, is there a version of block_on that does the same as try_lock for futures::lock::Mutex outside of async?
You probably mean to poll the MutexLockFuture instead, this can for example be done with the core::task::ready! macro, which desugars as following:
let num = match fut.poll(cx) {
Poll::Ready(t) => t,
Poll::Pending => return Poll::Pending,
};
To poll a future, you also need to pin it (ensure it doesn't get moved). This can be done on the stack with tokio::pin!, or Pin::new if the type is already Unpin (MutexLockFuture is), or by moving onto the heap with Box::pin.
Below is a runnable example.
⚠️ KEEP READING TO SEE WHY YOU DON'T WANT TO DO THIS!
#![feature(ready_macro)]
use core::{
future::Future,
pin::Pin,
task::{ready, Context, Poll},
};
use std::sync::Arc;
use tokio::io::{AsyncRead, AsyncReadExt};
pub struct SmolStackWithDevice<'a> {
counter: usize,
data: &'a [u8],
}
impl<'a> AsyncRead for SmolStackWithDevice<'a> {
fn poll_read(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>,
) -> Poll<std::io::Result<()>> {
if self.counter % 2 == 0 {
self.counter += 1;
cx.waker().wake_by_ref();
println!("read nothing");
return Poll::Pending;
}
buf.put_slice(&[self.data[self.counter / 2]]);
self.counter += 1;
println!("read something");
Poll::Ready(Ok(()))
}
}
pub struct SmolSocket<'a> {
stack: Arc<futures::lock::Mutex<SmolStackWithDevice<'a>>>,
}
impl<'a> AsyncRead for SmolSocket<'a> {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>,
) -> Poll<std::io::Result<()>> {
let mut lock_fut = self.stack.lock();
let pinned_lock_fut = Pin::new(&mut lock_fut);
let mut guard = ready!(pinned_lock_fut.poll(cx));
println!("acquired lock");
let pinned_inner = Pin::new(&mut *guard);
pinned_inner.poll_read(cx, buf)
}
}
#[tokio::main(flavor = "current_thread")]
async fn main() {
let data = b"HORSE";
let mut buf = [0; 5];
let mut s = SmolSocket {
stack: Arc::new(
SmolStackWithDevice {
counter: 0,
data: &data[..],
}
.into(),
),
};
s.read_exact(&mut buf).await.unwrap();
println!("{}", String::from_utf8_lossy(&buf));
}
Look at it go! (in Rust Playground)
⚠️ KEEP READING TO SEE WHY YOU DON'T WANT TO DO THIS!
So, what is the problem?
Well, as you can see from the output, whenever we succeed in acquiring the lock, but the underlying source is not ready to read, or only gives us a small read, we drop the lock, and on the next poll we will have to acquire it again.
This is a good point to remember that async flavors of Mutex are only recommended over std or parking_lot when it is expected that the Guard from a successful locking will be held across an await, or explicitly stored in a Future data structure.
We are not doing that here, we are only ever exercising the fast path equivalent to Mutex::try_lock, because whenever the lock is not immediately available, we drop the MutexLockFuture instead of waiting to be waked to poll it again.
However, storing the lock in the data structure would make it easy to accidentally deadlock. So a good design might be creating an awkward-to-store(borrowing) AsyncRead adapter that wraps the lock:
pub struct SmolSocket<'a> {
stack: Arc<futures::lock::Mutex<SmolStackWithDevice<'a>>>,
}
impl<'a> SmolSocket<'a> {
fn read(&'a self) -> Reader<'a> {
Reader::Locking(self.stack.lock())
}
}
pub enum Reader<'a> {
Locking(futures::lock::MutexLockFuture<'a, SmolStackWithDevice<'a>>),
Locked(futures::lock::MutexGuard<'a, SmolStackWithDevice<'a>>),
}
impl<'a> AsyncRead for Reader<'a> {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>,
) -> Poll<std::io::Result<()>> {
let this = self.get_mut();
match this {
Reader::Locking(f) => {
*this = Reader::Locked(ready!(Pin::new(f).poll(cx)));
println!("acquired lock");
Pin::new(this).poll_read(cx, buf)
}
Reader::Locked(l) => Pin::new(&mut **l).poll_read(cx, buf),
}
}
}
#[tokio::main(flavor = "current_thread")]
async fn main() {
let data = b"HORSE";
let mut buf = [0; 5];
let s = SmolSocket {
stack: Arc::new(
SmolStackWithDevice {
counter: 0,
data: &data[..],
}
.into(),
),
};
s.read().read_exact(&mut buf).await.unwrap();
println!("{}", String::from_utf8_lossy(&buf));
}
Look at it go! (executable Playground link)
This works out, because both the LockFuture and our SmolStackWithDevice are Unpin (non-self-referential) and so we don't have to guarantee we aren't moving them.
In a general case, for example if your SmolStackWithDevice is not Unpin, you'd have to project the Pin like this:
unsafe {
let this = self.get_unchecked_mut();
match this {
Reader::Locking(f) => {
*this = Reader::Locked(ready!(Pin::new_unchecked(f).poll(cx)));
println!("acquired lock");
Pin::new_unchecked(this).poll_read(cx, buf)
}
Reader::Locked(l) => Pin::new_unchecked(&mut **l).poll_read(cx, buf),
}
}
Not sure how to encapsulate the unsafety, pin_project isn't enough here, as we also need to dereference the guard.
But this only acquires the lock once, and drops it when the Reader is dropped, so, great success.
You can also see that it doesn't deadlock if you do
let mut r1 = s.read();
let mut r2 = s.read();
r1.read_exact(&mut buf[..3]).await.unwrap();
drop(r1);
r2.read_exact(&mut buf[3..]).await.unwrap();
println!("{}", String::from_utf8_lossy(&buf));
This is only possible because we deferred locking until polling.