Recently, I am reading source code in the repo parity-bridges-common. There are some rs files full of incomprehensibly asynchronous syntax that I am not familiar with. Especially about FusedFuture. Example file:
sync_loop.rs
message_loop.rs
According to the futures::future::fuse, what I understand is that we can transfer a Future into FusedFuture and then poll it, again and again, for many times. Besides this, I find a function named terminated() in FuseFurture and example below.
Creates a new Fuse-wrapped future which is already terminated.
This can be useful in combination with looping and the select! macro, which bypasses terminated futures.
let (sender, mut stream) = mpsc::unbounded();
// Send a few messages into the stream
sender.unbounded_send(()).unwrap();
sender.unbounded_send(()).unwrap();
drop(sender);
// Use `Fuse::terminated()` to create an already-terminated future
// which may be instantiated later.
// bear: We must terminated() here in order to use select!?
let foo_printer = Fuse::terminated();
pin_mut!(foo_printer);
loop {
select! {
_ = foo_printer => {},
() = stream.select_next_some() => {
if !foo_printer.is_terminated() {
println!("Foo is already being printed!");
} else {
// bear: here we reset the foo_printer pointed value?
foo_printer.set(async {
// do some other async operations
println!("Printing foo from `foo_printer` future");
}.fuse());
}
},
complete => break, // `foo_printer` is terminated and the stream is done
}
}
I cannot understand when to use this function and how to combine it with select!. Can anyone help me understand it more easily? Or are there better docs or examples about this usage?
Some posts i found, maybe useful:
Why doesn’t tokio::select! require FusedFuture?
What is the difference between futures::select! and tokio::select?
Related
I want to let my modem-AT-command-writing-thread only write to the modem's /dev/ttyUSB3 when the modem-AT-command-reading-thread has seen an "OK" or an "ERROR".
This initially sounds like a job for a Mutex<()>, but I have an additional requirement: If the modem-AT-command-reading-thread does not see an "OK" or "ERROR" within three seconds, the writing thread should just get on with sending the next AT command. i.e. If the reading thread gets nothing, the writing thread should still send one of its AT commands every three seconds. (Modems' AT command interfaces are often not nicely behaved.)
At the moment, I have a workaround using mpsc::channel:
Set-up:
let (sender, receiver) = channel::<()>();
modem-AT-command-reading-thread:
if line.starts_with("OK") || line.contains("ERROR") {
debug!("Sending go-ahead to writing_thread.");
sender.send(()).unwrap();
}
modem-AT-command-writing-thread:
/* This receive is just a way of blocking until the modem is ready. */
match receiver.recv_timeout(Duration::from_secs(3)) {
Ok(_) => {
debug!("Received go-ahead from reading thread.");
/*
* Empty the channel, in case the modem was too effusive. We don't want
* to "bank" earlier OK/ERRORs to allow multiple AT commands to be sent in
* quick succession.
*/
while let Ok(_) = receiver.try_recv() {}
}
Err(err) => match err {
RecvTimeoutError::Timeout => {
debug!("Timed-out waiting for go-ahead from reading thread.");
}
RecvTimeoutError::Disconnected => break 'outer
},
}
I cannot find a Mutex::lock_with_timeout().
How can I implement this properly, using a Mutex<()> or similar?
You can use parking_lot's Mutex, it has try_lock_for().
My requirement is very simple, which is a very reasonable requirement in many programs. It is to send a specified message to my Channel after a specified time.
I've checked tokio for topics related to delay, interval or timeout, but none of them seem that straightforward to implement.
What I've come up with now is to spawn an asynchronous task, then wait or sleep for a certain amount of time, and finally send the message.
But, obviously, spawning an asynchronous task is a relatively heavy operation. Is there a better solution?
async fn my_handler(sender: mpsc::Sender<i32>, dur: Duration) {
tokio::spawn(async {
time::sleep(dur).await;
sender.send(0).await;
}
}
You could try adding a second channel and a continuously running task that buffers messages until the time they are to be received. Implementing this is more involved than it sounds, I hope I'm handling cancellations right here:
fn make_timed_channel<T: Ord + Send + Sync + 'static>() -> (Sender<(Instant, T)>, Receiver<T>) {
// Ord is an unnecessary requirement arising from me stuffing both the Instant and the T into the Binary heap
// You could drop this requirement by using the priority_queue crate instead
let (sender1, receiver1) = mpsc::channel::<(Instant, T)>(42);
let (sender2, receiver2) = mpsc::channel::<T>(42);
let mut receiver1 = Some(receiver1);
tokio::spawn(async move {
let mut buf = std::collections::BinaryHeap::<Reverse<(Instant, T)>>::new();
loop {
// Pretend we're a bounded channel or exit if the upstream closed
if buf.len() >= 42 || receiver1.is_none() {
match buf.pop() {
Some(Reverse((time, element))) => {
sleep_until(time).await;
if sender2.send(element).await.is_err() {
break;
}
}
None => break,
}
}
// We have some deadline to send a message at
else if let Some(Reverse((then, _))) = buf.peek() {
if let Ok(recv) = timeout_at(*then, receiver1.as_mut().unwrap().recv()).await {
match recv {
Some(recv) => buf.push(Reverse(recv)),
None => receiver1 = None,
}
} else {
if sender2.send(buf.pop().unwrap().0 .1).await.is_err() {
break;
}
}
}
// We're empty, wait around
else {
match receiver1.as_mut().unwrap().recv().await {
Some(recv) => buf.push(Reverse(recv)),
None => receiver1 = None,
}
}
}
});
(sender1, receiver2)
}
Playground
Whether this is more efficient than spawning tasks, you'd have to benchmark. (I doubt it. Tokio iirc has some much fancier solution than a BinaryHeap for waiting for waking up at the next timeout, e.g.)
One optimization you could make if you don't need a Receiver<T> but just something that .poll().await can be called on: You could drop the second channel and maintain the BinaryHeap inside a custom receiver.
I'm trying to understand how to implement polling multiple futures with different types. For context, I'm calling an API that will return something like:
[{"type": "source_a", "id": 123}, {"type": "source_b", "id": 234}, ...]
Each type in the API response requires a call to another API, with each API returning different data types. The code I've written works something like this:
async fn get_data(sources: Vec<Source>) -> Data {
let mut data = Default::default();
for source in sources {
if source.kind == "source_a" {
let source_data = get_source_a(source).await;
process_source_a(source_data, &mut data);
} else if source.kind == "source_b" {
...
}
}
data
}
This won't run concurrently, it will simply fetch sources one at a time and process them. How can I rewrite this so that each source is fetched concurrently and then processed once data is available? Speaking Rustily, I think what I want is to execute a closure that mutably borrows data when the future is ready. Should I be looking at something like an Arc<RefCell<Data>>?
To process the futures in parallel, you need to await something like join_all, which will run them concurrently and return when they are all done. For this to work, you have to resolve two issues:
join_all requires futures of the same type, so you need to box them or otherwise unify them.
data needs to be accessed by multiple async blocks, so it needs to be protected by Arc and Mutex.
The first issue can be solved simply by spawning the async fns as tasks, which has the added advantage of potentially running them in parallel (in addition to them being run concurrently). The example below uses tokio::spawn, but it would be almost exactly the same for async_std. Since there is no reproducible example, I can't test the code, but it could look like this:
async fn get_data(sources: Vec<Source>) -> Data {
let data = Arc::new(Mutex::new(Data::default()));
let mut tasks = vec![];
for source in sources {
if source.kind == "source_a" {
let data = Arc::clone(&data);
tasks.push(tokio::task::spawn(async move {
let source_data = get_source_a(source).await;
process_source_a(source_data, &mut data.lock().unwrap());
}));
} else if source.kind == "source_b" {
// ...
}
}
// Wait for all sources to finish and propagate the panic if any.
// With async_std this wouldn't require the `for_each()`.
futures::future::join_all(tasks)
.await
.for_each(|x| x.unwrap());
// As all tasks are done, there should be no references to `data` at
// this point, so we can extract it out of the Arc<Mutex<_>> wrapping.
data.try_unwrap().unwrap().into_inner()
}
Just to explain, I send a command and an id to the other thread, the other thread executes the thing, and return the command and id as a tuple. Because multiple functions might be called at nearly the same time and they are io operations, they might be returned in a different order than they came. So, I have this function to wait for the right return value and put the rest into a map(the variable called results) so that the other functions calling can retrieve theirs as well. However, I think if there is a case of two functions calling this one at the same time, they might be racing to retrieve the value at the channels. I just think there should be a more elegant way of doing this.
For now I have this function to wait for an async execution in another thread:
async fn wait_for(&mut self, id : u64) -> (u64, io::Result<SomeResult>) {
loop {
let temp = self.channels.1.recv().await.unwrap();
if temp.0 == id {
return temp
} else {
self.results.insert(temp.0, temp.1);
}
if let Some(res) = self.results.remove(&id) {
return (id, res)
}
}
}
P.S. I also had a thought of having another thread do the checking and return the results in order, but that pretty much defeats the purpose of async/await (as far as I know).
Is it possible, if a task sends to a and an other (at the same time) sends to b, that tokio::select! on a and b drops one of the value by cancelling the remaining future? Or is it guaranteed to be received at the next loop iteration?
use tokio::sync::mpsc::Receiver;
async fn foo(mut a: Receiver<()>, mut b: Receiver<()>) {
loop {
tokio::select!{
_ = a.recv() => {
println!("A!");
}
_ = b.recv() => {
println!("B!");
}
}
}
}
My mind can't get around what is really happening behind the async magic in that case.
It doesn't appear to be guaranteed in the documentation anywhere, but is likely to work for reading directly from a channel because of the way rusts poll based architecture works. A select is equivalent to polling each of futures in a random order, until one of them is ready, or if none are, then waiting until a waker is signaled and then repeating the process. A message is only removed from the channel when returned by a successful poll. A successful poll stops the select, so the rest of the channels will not be touched. Thus they will be polled the next time the loop occurs and then return the message.
However, this is a dangerous approach, because if the receiver is replaced with something that returns a future that does anything more complex than a direct read, where it could potentially suspend after the read, then you could lose messages when that happens. As such it should probably be treated as if it wouldn't work. A safer approach would be to store the futures in mutable variables that you update when they fire:
use tokio::sync::mpsc::Receiver;
async fn foo(mut a: Receiver<()>, mut b: Receiver<()>) {
let mut a_fut = a.recv();
let mut b_fut = b.recv();
loop {
tokio::select!{
_ = a_fut => {
println!("A!");
a_fut = a.recv();
}
_ = b_fut => {
println!("B!");
b_fut = b.recv();
}
}
}
}