I am developing Rust Tokio library for ISO-TP. CAN protocol, which lets you send larger messages. The program is aimed towards linux only.
For this, I am using Tokio structure AsyncFd. When the write is called, I create the Future and then poll it. The problem is when I do two consecutive writes, one after the other.
socket_tx1.write_packet(packet.clone())?.await?;
socket_tx1.write_packet(packet.clone())?.await?;
The first write will end successfully, however second will end with
std::io::ErrorKind::WouldBlock
Which is OK and expected. The buffer is full and we should wait until it's clear and ready for the next write. The poll does not guarantee, that if it returns OK, the following write will be successful.
The problem is that I don't know how to handle this behavior correctly.
I tried the following implementations:
impl Future for IsoTpWriteFuture {
type Output = io::Result<()>;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
loop {
let guard = ready!(self.socket.0.poll_write_ready(cx))?;
match self.socket.0.get_ref().0.write(&self.packet) {
Err(err) if err.kind() == io::ErrorKind::WouldBlock => continue,
Ok(_) => return Poll::Ready(Ok(())),
Err(err) => return Poll::Ready(Err(err))
}
}
}
}
This one works, but after I get WouldBlock, this loop results in busy waiting, which I would like to avoid. Since Socket is ready from poll perspective, write is immediately called, Wouldblock is again returned, and routine spins sometime before resolving the write.
The second implementation is more correct, from my perspective, but it doesn't work right now, and I am not sure how to make it work.
impl Future for IsoTpWriteFuture {
type Output = io::Result<()>;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
loop {
let guard = ready!(self.socket.0.poll_write_ready(cx))?;
match guard.try_io(|inner|inner.get_ref().0.write(&self.packet)) {
Err(err) => continue,
Ok(_) => return Poll::Ready(Ok(())),
}
}
}
}
This doesn't work since once try_io() encounters WouldBlock, it will clear readiness from guard. And since poll is edge triggered, this will hang at poll_write_ready and won't make progress.
Is it possible to poll for change after the write returns WouldBlock? Or is the bussy waiting approach unavoidable?
In an asynchronous system, after you get a WouldBlock, you need to return control to the asynchronous runtime (tokio in this case), as well as tell it what it needs to wait on (the file descriptor being ready).
You say you are using Tokio's AsyncFd. It offers several methods, such as poll_read_ready, to check on the fd's readiness and register it with tokio if it needs to wait.
Related
I am trying to create a future polling for inputs from the crossterm crate, which does not provide an asynchronous API, as far as I know.
At first I tried to do something like the following :
use crossterm::event::poll as crossterm_poll;
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;
use tokio::time::{sleep, timeout};
struct Polled {}
impl Polled {
pub fn new() -> Polled {
Polled {}
}
}
impl Future for Polled {
type Output = bool;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
// If there are events pending, it returns "Ok(true)", else it returns instantly
let poll_status = crossterm_poll(Duration::from_secs(0));
if poll_status.is_ok() && poll_status.unwrap() {
return Poll::Ready(true);
}
Poll::Pending
}
}
pub async fn poll(d: Duration) -> Result<bool, ()> {
let polled = Polled::new();
match timeout(d, polled).await {
Ok(b) => Ok(b),
Err(_) => Err(()),
}
}
It technically works but obivously the program started using 100% CPU all the time since the executor always try to poll the future in case there's something new. Thus I wanted to add some asynchronous equivalent to sleep, that would delay the next time the executor tries to poll the Future, so I tried adding the following (right before returning Poll::Pending), which obviously did not work since sleep_future::poll() just returns Pending :
let mut sleep_future = sleep(Duration::from_millis(50));
tokio::pin!(sleep_future);
sleep_future.poll(cx);
cx.waker().wake_by_ref();
The fact that poll is not async forbids the use of async functions, and I'm starting to wonder if what I want to do is actually feasible, or if I'm not diving in my first problem the wrong way.
Is finding a way to do some async sleep the good way to go ?
If not, what is it ? Am I missing something in the asynchronous paradigm ?
Or is it just sometimes impossible to wrap some synchronous logic into a Future if the crate does not give you the necessary tools to do so ?
Thanks in advance anyway !
EDIT : I found a way to do what I want using an async block :
pub async fn poll(d: Duration) -> Result<bool, ()> {
let mdr = async {
loop {
let a = crossterm_poll(Duration::from_secs(0));
if a.is_ok() && a.unwrap() {
break;
}
sleep(Duration::from_millis(50)).await;
}
true
};
match timeout(d, mdr).await {
Ok(b) => Ok(b),
Err(_) => Err(()),
}
}
Is it the idiomatic way to do so ? Or did I miss something more elegant ?
Yes, using an async block is a good way to compose futures, like your custom poller and tokio's sleep.
However, if you did want to write your own Future which also invokes tokio's sleep, here's what you would need to do differently:
Don't call wake_by_ref() immediately — the sleep future will take care of that when its time comes, and that's how you avoid spinning (using 100% CPU).
You must construct the sleep() future once when you intend to sleep (not every time you're polled), then store it in your future (this will require pin-projection) and poll the same future again the next time you're polled. That's how you ensure you wait the intended amount of time and not shorter.
Async blocks are usually a much easier way to get the same result.
In async/await, what's the most idiomatic/efficient way to take an input buffer and, for each item, route it to an existing Future or a new one based on a condition? For example, you're running a server and each incoming packet might either be the start of a new interaction with a client or a continuation of an existing one. You can tell the difference by something in the packet, such as a UUID identifying the interaction.
I'm trying to do this and it's working pretty well for the most part, but every time a new client interaction starts, the latency between the initial request and response increases and I can't figure out why. I'm trying it using these two structs that I made:
pub struct InputBuffer<T> {
queue_send: SyncSender<(T, Instant)>,
queue_recv: Arc<Mutex<Receiver<(T, Instant)>>>,
waker_send: SyncSender<Waker>,
waker_recv: Arc<Mutex<Receiver<Waker>>>,
}
pub struct InputBufferNext<'a, T, F: Fn(&T) -> bool>(pub InputBuffer<T>, pub &'a F);
The relevant functions on InputBuffer are below. Every time we add a new item to the queue using push, we wake all the interested wakers. The choose function takes a Fn(&T) -> bool and returns the next item that satisfies this condition. The choose_and_then function repeatedly chooses items off the queue based on a given condition and runs a given function on them. This is what I'm using to start new client interactions. I've got an Arc<Mutex<HashSet<u64>>> storing UUIDs for active client interactions and this is shared with a closure that returns whether or not the UUID is new and this is the F parameter. If it's not already in the set, then the G parameter is a function that starts a new interaction. (In my use case, the key is more complicated than a u64, but this is adapted for simplicity).
impl<T> InputBuffer<T> {
pub fn push(&self, t:T) {
self.queue_send.send((t, Instant::now())).unwrap();
if let Ok(waker_recv) = self.waker_recv.lock() {
while let Ok(waker) = waker_recv.try_recv() {
waker.wake();
}
}
}
pub async fn choose<F: Fn(&T) -> bool>(self, choose: &F) -> T {
InputBufferNext::<T, F>(Clone::clone(&self), choose).await
}
pub async fn choose_and_then<F: Fn(&T) -> bool, G: Fn(T)>(self, choose: F, and_then: G) {
loop {
and_then(self.clone().choose(&choose).await);
}
}
}
The type that actually implements Future is InputBufferNext and it does it the way shown below. Each time we poll it, we receive items until we find one that matches our condition, then put back the ones that don't. There's a timeout implemented to make sure the buffer can't just accumulate unclaimed items indefinitely. If we don't find what we're looking for, we send our waker to the InputBuffer.
impl<'a, T, F: Fn(&T) -> bool> std::future::Future for InputBufferNext<'a, T, F> {
type Output = T;
fn poll(self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output> {
let mut put_back = vec![];
if let Ok(queue_recv) = self.0.queue_recv.lock() {
let mut accepted: Option<T> = None;
'inner: while let Ok((x, t0)) = queue_recv.try_recv() {
if (self.1)(&x) {
accepted = Some(x);
break 'inner;
} else if t0.elapsed() < INPUT_BUFFER_TIMEOUT {
put_back.push((x, t0));
}
}
for x_t0 in put_back {
self.0.queue_send.send(x_t0).unwrap();
}
if let Some(x) = accepted {
return Poll::Ready(x);
}
}
// We made it all the way through what's available and didn't find what we were looking for
// Let us know when we should try again
self.0.waker_send.send(cx.waker().clone()).unwrap();
Poll::Pending
}
}
Since InputBuffer can be cloned (I had to implement Clone by hand because it didn't like the derive macro, but it's made up of all cloneable types), I just clone it and give it to each async fn that gets spawned with each new interaction. When an interaction wants its next item, it calls choose with a function that picks out its UUID and gets back a Future.
Like I said, this implementation seems to be mostly working, but there's some kind of latency accumulating somewhere. It takes longer and longer for the server to respond to initial requests. I've checked the amount of time that each poll call takes along with the number of calls per second and those both seem to be stable, despite the growing latency. I've also checked the size of the queue of wakers and it seems steady at a low, reasonable number. The waker queue also gets emptied every time a new item is added. The timeout also ensures that the number of items in the queue stays limited.
I'm using a timeout around 100ms and seeing the latency grow to unacceptable levels after 80+ seconds. My use cases starts a new client interaction about 30-50 times per second, to give a rough idea. Each interaction also ends after a short time (<1 sec), so we're not just accumulating active interactions indefinitely. I'm using the ThreadPool executor from futures.
Any suggestions for how I could improve my implementation or abandon it completely in favor of a more highly-optimized library or different design pattern would be much appreciated. Thanks!
Rust's future uses poll_read (poll_read) to poll for available data:
fn poll_read(
&mut self,
cx: &mut Context,
buf: &mut [u8]
) -> Result<Async<usize>, Error>
Obviously, when called, the poll_read function fills buf with data and returns Poll::Ready(n) where n is the quantity written to the buffer. If there's no data available at the moment, it returns Poll::Pending. The future caller can poll_read again, or not. In the case where it does not poll again, it is giving the poll_read function the chance to tell when it has to be polled. It does that by calling cx.waker().wake() when there is available data.
So, for example, I can implement poll_read for my struct:
fn poll_read(
&mut self,
cx: &mut Context,
buf: &mut [u8]
) -> Result<Async<usize>, Error> {
let my_buffer = get_buffer_from_internet();
if my_buffer.len() <= buf.capacity() {
buf.put_slice(my_buffer);
return Poll::Ready(my_buffer.len());
} else {
//we can only fill buf partially
buf.put_slice(my_buffer);
//let's store the remaining in our struct
self.remaining.store(&my_buffer[buf.len()..]);
//How can I make the future caller know it needs to call poll_read again to read the remaining data? Can I call cx.waker().wake() while inside the poll_read?
return Poll::Ready(buf.len());
}
}
you see that in the case where there was no sufficient space on buf, I copied just the needed data and then stored the remaining on our struct. So we can enhance this poll_read by making it check if there is remaining data to be written, before getting new data from the internet. But as you see in my comment: How can I make the future caller know it needs to call poll_read again to read the remaining data? Can I call cx.waker().wake() while inside the poll_read?
You do not need to do anything. The documentation reads:
If no data is available for reading, the method returns Poll::Pending and arranges for the current task to receive a notification when the object becomes readable or is closed.
It does not need to arrange to be woken up if it returns data. The implementation for Cursor<T>, for example, just defers to io::Read and ignores the Context entirely, regardless whether the buffer is big enough or not.
The poller should know that there still may be data to read until poll_read returns Poll::Ready(Ok(0)).
I am making my own channel implementation, but std::task::Context doesn't make it clear how the waker was generated.
My fake code:
struct MyAtomicWaker {
lock: SpinLock,
is_waked: AtomicBool,
waker: std::task::Waker,
}
struct WeakAtomicWaker (Weak<MyAtomicWaker>)
impl MyAtomicWaker {
fn is_waked(&self) -> bool {}
fn weak(self: Arc<MyAtomicWaker>) -> WeakAtomicWaker;
fn cancel(&self) {} // nullify WeakAtomicWaker, means the waker is not waked by a future
}
impl WeakAtomicWaker {
fn wake(self) {} // upgrade to arc and can wake only once when waker not cancelled
}
struct ReceiveFuture<T> {
waker: Option<Arc<MyAtomicWaker>>,
}
impl<T> Drop for ReceiveFuture<T> {
fn drop(&mut self) {
if let Some(waker) = self.waker.take() { waker.cancel(); }
}
}
impl<T> Future for ReceiveFuture<T> {
type Output = Result<(), SendError<T>>;
fn poll(self: Pin<&mut Self>, ctx: &mut Context) -> Poll<Self::Output> {
let _self = self.get_mut();
if _self.waker.is_none() {
let my_waker = _self.reg_waker(ctx.waker().clone()); // wrap the waker with Arc, store it inside _self, and send the weak ref to other side of channel
_self.waker.replace(my_waker);
}
// do some polling
match _self.recv.try_recv() {
Ok(item)=>{
if let Some(waker) = _self.waker.take() {
waker.cancel();
}
return Poll::Ready(item); //canncel my waker and ready
},
Err(TryRecvError)=>{
if let Some(waker) = _self.waker.as_ref() {
if waker.is_wake() { // the waker is triggered but it's a false alarm, should make a new one.
let my_waker = self.reg_waker(ctx.waker().clone());
_self.waker.replace(my_waker);
} else { // the waker has not trigger, we do not have to make a new one ?
}
}
return Poll::Pending;
},
Err(...)
}
}
}
Is it necessary to register a new waker every time poll() is called? In my code, there's a lot of timeouts and looping selects due to the combination of different futures.
I have a little experiment that works on the playground, but I'm not sure whether it will always work fine for both Tokio and async-std in various settings.
In my production code, I register a new waker and cancel the old waker in every poll() call. I don't know whether it is safe to only register a waker the first time and reuse it on the next polls.
Given the following order:
f.reg_waker(waker1)
f.poll() gets Poll::Pending
combined future (or future::select) wakeup due to other future selecting, but waker1 has not woken up.
f.poll() gets Poll::Pending
some outsider call waker1.wake();
will waker1.wake() guarantee to wake up f after that?
I'm asking this because:
I have a Stream that multiplexes multiple receiving channels
My MPMC and MPSC channel implementations are lockless. Some channels inside a multiplex selection may be used as a close notification channel and seldom gets message. When I'm polling it a lot (say a million times), it will lead to a million waker thrown to the other side (which looks like a memory leak). Canceling previous wakers produced in the same future without lock is more complex logic than an implementation with lock
For these reasons, I have a waker canceling solution that leads to fairness problem, which needs to be avoided as much as possible
I'm not interested in what the book states, or what the API laws declare; I'm only interested in how the low level is implemented. Show some code why this works or why this does not work would be helpful. I code to implement product; if necessary I will stick to a specified dependency or do some hacking in order to get the job done until I have a better way.
Yes, it is required to re-set the waker each time. Future::poll states (emphasis mine):
Note that on multiple calls to poll, only the Waker from the Context passed to the most recent call should be scheduled to receive a wakeup.
See also:
Why do I not get a wakeup for multiple futures when they use the same underlying socket?
Is it valid to wake a Rust future while it's being polled?
Given the fact that "a waker can be waked in parallel of Future::poll.
Counter-evidence: Presuming every time a waker must be clone() and re-registered in order for this future to wake up properly, this will make previous waker invalid, so it's not possible for "concurrent wake up from different thread, e.g. future::select block". The conclusion is not true, so it counter-proves this statement:
"A waker is always valid from the time starting from ctx.waker().clone() until waker.wake()". This serves my motive. waker is not needed to reset every time, if it's not used to woken yet.
In addition to investigate tokio waker implementation, all RawWaker produce by ctx.waker.clone() is just a ref_count to a manual drop memory entry on the heap, if wakers clone outside prevent the ref_count dropping zero, the real waker entry always exists.
In a rust async function is there any way to get access to the current Context without writing an explicit implementation of a Future?
Before actually answering the question, it is useful to remember what a Context is; whenever you are writing an implementation of a Future that depends on outside resources (say, I/O), you do not want to busy-wait anything. As a result, you'll most likely have implementations of Future where you'll return Pending and then wake it up. Context (and Waker) exist for that purpose.
However, this is what they are: low-level, implementation details. If you are using a Future already as opposed to writing a low-level implementation of one, the Waker will most likely be contained somewhere, but not directly accessible to you.
As a result of this, a Waker directly leaking is an implementation detail leak 99.9% of the time and not actually recommended. A Waker being used as part of a bigger struct is perfectly fine, however, but this is where you'll need to implement your own Future from scratch. There is no other valid use case for this, and in normal terms, you should never need direct access to a Waker.
Due to the limitations of the playground, I sadly cannot show you a live example of when it is useful to get this Waker; however, such a future setup may be used in such a situation: let's assume we're building the front door of a house. We have a doorbell and a door, and we want to be notified when somebody rings the doorbell. However, we don't want to have to wait at the door for visitors.
We therefore make two objects: a FrontDoor and a Doorbell, and we give the option to wire() the Doorbell to connect the two.
pub struct FrontDoor {
doorbell: Arc<RwLock<Doorbell>>
}
impl FrontDoor {
pub fn new() -> FrontDoor {
FrontDoor {
doorbell: Arc::new(RwLock::new(Doorbell {
waker: None,
visitor: false
}))
}
}
pub fn wire(&self) -> Arc<RwLock<Doorbell>> {
self.doorbell.clone() // We retrieve the bell
}
}
impl Future for FrontDoor {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output> {
self.doorbell.read().map(|guard| {
match guard.visitor {
true => Poll::Ready(()),
false => Poll::Pending
}
}).unwrap_or(Poll::Pending)
}
}
pub struct Doorbell {
waker: Option<Waker>,
pub visitor: bool
}
impl Doorbell {
pub fn ring(&mut self) {
self.visitor = true;
self.waker.as_ref().map(|waker| waker.wake_by_ref());
}
}
Our FrontDoor implements Future, which means we can just throw it on an executor of your choice; waker is contained in the Doorbell object and allows us to "ring" and wake up our future.