Re-building setInterval in JS:
pub async fn set_interval<T, F>(interval: Duration, do_something: T)
where
T: (Fn() -> F) + Send + Sync + 'static,
F: Future + Send,
{
let forever = task::spawn(async move {
let mut interval = time::interval(interval);
loop {
interval.tick().await;
do_something().await;
}
});
forever.await;
}
This works, but I want it to be .awaitable until the first execution ends.
I.e. Instead of doing:
do_something().await
set_interval(Duration::from_secs(1), do_something)
I want to:
set_interval(Duration::from_secs(1), do_something).await
Note that this is different behaviour from above in that it runs the do_something task immediately, but it's intended.
My solution:
pub async fn set_interval<T, F>(interval: Duration, do_something: T)
where
T: (Fn() -> F) + Send + Sync + 'static,
F: Future + Send,
{
do_something().await;
task::spawn(async move {
let forever = task::spawn(async move {
let mut interval = time::interval(interval);
loop {
interval.tick().await;
do_something().await;
}
});
forever.await;
});
}
This works, but I have some doubts
Is it a waste of resources to task::spawn twice here? Any way it can be done only once?
Is it a mistake to discard the outer task::spawn as soon as the function ends? Will it keep getting run indefinitely, even though it's not tracked anymore, just fired and forgotten?
To answer your two questions:
Don't worry about spawning a task too many: On your average hardware, you should be able to spawn a few 100,000 of them per second. It is unnecessary in this case, though, because:
Tasks get "detached" if they are dropped, so the futures inside them continue to be polled by your runtime. This also means that you don't need forever.await;
The following seems to work:
pub async fn set_interval<T, F>(interval: std::time::Duration, do_something: T)
where
T: (Fn() -> F) + Send + Sync + 'static,
F: std::future::Future + Send,
{
// The interval time alignment is decided at construction time.
// For all calls to be evenly spaced, the interval must be constructed first.
let mut interval = tokio::time::interval(interval);
// The first tick happens without delay.
// Whether to tick before the first do_something or after doesn't matter.
interval.tick().await;
do_something().await;
tokio::task::spawn(async move {
loop {
interval.tick().await;
do_something().await;
}
});
}
Playground
Related
I have read here that futures in Rust do nothing unless they are awaited. However, I tried a more complex example and it is a little unclear why I get a message printed by the 2nd print in this example because task::spawn gives me a JoinHanlde on which I do not do any .await.
Meanwhile, I tried the same example, but with an await above the 2nd print, and now I get printed only the message in the 1st print.
If I wait for all the futures at the end, I get printed both messages, which I understood. My question is why the behaviour in the previous 2 cases.
use futures::stream::{FuturesUnordered, StreamExt};
use futures::TryStreamExt;
use rand::prelude::*;
use std::collections::VecDeque;
use std::sync::Arc;
use tokio::sync::Semaphore;
use tokio::task::JoinHandle;
use tokio::{task, time};
fn candidates() -> Vec<i32> {
Vec::from([2, 2])
}
async fn produce_event(nanos: u64) -> i32 {
println!("waiting {}", nanos);
time::sleep(time::Duration::from_nanos(nanos)).await;
1
}
async fn f(seconds: i64, semaphore: &Arc<Semaphore>) {
let mut futures = vec![];
for (i, j) in (0..1).enumerate() {
for (i, event) in candidates().into_iter().enumerate() {
let permit = Arc::clone(semaphore).acquire_owned().await;
let secs = 500;
futures.push(task::spawn(async move {
let _permit = permit;
produce_event(500); // 2nd example has an .await here
println!("Event produced at {}", seconds);
}));
}
}
}
#[tokio::main()]
async fn main() {
let semaphore = Arc::new(Semaphore::new(45000));
for _ in 0..1 {
let mut futures: FuturesUnordered<_> = (0..2).map(|moment| f(moment, &semaphore)).collect();
while let Some(item) = futures.next().await {
let () = item;
}
}
}
However, I tried a more complex example and it is a little unclear why I get a message printed by the 2nd print in this example because task::spawn gives me a JoinHanlde on which I do not do any .await.
You're spawning tasks. A task is a separate thread of execution which can execute concurrently to the current task, and can be scheduled in parallel.
All the JoinHandle does there is wait for that task to end, it doesn't control the task running.
Meanwhile, I tried the same example, but with an await above the 2nd print, and now I get printed only the message in the 1st print.
You spawn a bunch of tasks and make them sleep. Since you don't wait for them to terminate (don't join them) nor is there any sort of sleep in their parent task, once all the tasks have been spawned the loops terminate, you reach the end of the main function and the program terminates.
At this point all the tasks are still sleeping.
I have a task (downloading something from the Web) that runs regularly with pauses 10 min between runs.
If my program notices that the data is outdated, then it should run the download task immediately unless it is already running. If the download task happened out-of-time, the next task should be after 10 min since the out-of-time task so all future tasks and pauses are shifted later in time.
How do I do this with Tokio?
I made a library to run a sequence of tasks, but trying to use it for my problem failed.
mod tasks_with_regular_pauses;
use std::future::Future;
use std::pin::Pin;
use std::sync::Arc;
use tokio::spawn;
use tokio::sync::mpsc::{channel, Receiver, Sender};
use tokio::sync::Mutex;
use tokio::task::JoinHandle;
use tokio_interruptible_future::{
interruptible, interruptible_sendable, interruptible_straight, InterruptError,
};
pub type TaskItem = Pin<Box<dyn Future<Output = ()> + Send>>;
/// Execute futures from a stream of futures in order in a Tokio task. Not tested code.
pub struct TaskQueue {
tx: Sender<TaskItem>,
rx: Arc<Mutex<Receiver<TaskItem>>>,
}
impl TaskQueue {
pub fn new() -> Self {
let (tx, rx) = channel(1);
Self {
tx,
rx: Arc::new(Mutex::new(rx)),
}
}
async fn _task(this: Arc<Mutex<Self>>) {
// let mut rx = ReceiverStream::new(rx);
loop {
let this2 = this.clone();
let fut = {
// block to shorten locks lifetime
let obj = this2.lock().await;
let rx = obj.rx.clone();
let mut rx = rx.lock().await;
rx.recv().await
};
if let Some(fut) = fut {
fut.await;
} else {
break;
}
}
}
pub fn spawn(
this: Arc<Mutex<Self>>,
notify_interrupt: async_channel::Receiver<()>,
) -> JoinHandle<Result<(), InterruptError>> {
spawn(interruptible_straight(notify_interrupt, async move {
Self::_task(this).await;
Ok(())
}))
}
pub async fn push_task(&self, fut: TaskItem) {
let _ = self.tx.send(fut).await;
}
}
I'd recommend using select! instead of interruptible futures to detect one of 3 conditions in your loop:
download task is finished
the data is outdated signal
data expired timeout signal
"The data is outdated" signal can be conveyed using a dedicated channel.
select! allows waiting for futures (like downloading and timeouts), and reading from channels at the same time. See the tutorial for examples of that.
Solution sketch:
loop {
// it is time to download
let download_future = ...; // make your URL request
let download_result = download_future.await;
// if the outdated signal is generated while download
// was in progress, ignore the signal by draining the receiver
while outdated_data_signal_receiver.try_recv().is_ok() {}
// send results upstream for processing
download_results_sender.send(download_result);
// wait to re-download
select! {
// after a 10 min pause
_ = sleep(Duration::from_minutes(10)) => break,
// or by an external signal
_ = outdated_data_signal_receiver.recv() => break,
}
}
This logic can be simplified further by the timeout primitive:
loop {
// it is time to download
let download_future = ...; // make your URL request
let download_result = download_future.await;
// if the outdated signal is generated while download
// was in progress, ignore the signal by draining the receiver
while outdated_data_signal_receiver.try_recv().is_ok() {}
// send results upstream for processing
download_results_sender.send(download_result);
// re-download by a signal, or timeout (whichever comes first)
_ = timeout(Duration::from_minutes(10), outdated_data_signal_receiver.recv()).await;
}
struct ThreadHolder{
state: ???
thread: ???
}
impl ThreadHolder {
fn launch(&mut self) {
self.thread = ???
// in thread change self.state
}
}
#[test]
fn test() {
let mut th = ThreadHolder{...};
th.launch();
// thread will be destroy as soon as th go out of scope
}
I think there is something to deal with lifetime, but I don't know how to write it.
What you want is so simple that you don't even need it to be mutable in any way, and then it becomes trivial to share it across threads, unless you want to reset it. You said you need to leave a thread, for one reason or another, therefore I'll assume that you don't care about this.
You instead can poll it every tick (most games run in ticks so I don't think there will be any issue implementing that).
I will provide example that uses sleep, so it's not most accurate thing, it is painfully obvious on the last subsecond duration, but I am not trying to do your work for you anyway, there's enough resources on internet that can help you deal with it.
Here it goes:
use std::{
sync::Arc,
thread::{self, Result},
time::{Duration, Instant},
};
struct Timer {
end: Instant,
}
impl Timer {
fn new(duration: Duration) -> Self {
// this code is valid for now, but might break in the future
// future so distant, that you really don't need to care unless
// you let your players draw for eternity
let end = Instant::now().checked_add(duration).unwrap();
Timer { end }
}
fn left(&self) -> Duration {
self.end.saturating_duration_since(Instant::now())
}
// more usable than above with fractional value being accounted for
fn secs_left(&self) -> u64 {
let span = self.left();
span.as_secs() + if span.subsec_millis() > 0 { 1 } else { 0 }
}
}
fn main() -> Result<()> {
let timer = Timer::new(Duration::from_secs(10));
let timer_main = Arc::new(timer);
let timer = timer_main.clone();
let t = thread::spawn(move || loop {
let seconds_left = timer.secs_left();
println!("[Worker] Seconds left: {}", seconds_left);
if seconds_left == 0 {
break;
}
thread::sleep(Duration::from_secs(1));
});
loop {
let seconds_left = timer_main.secs_left();
println!("[Main] Seconds left: {}", seconds_left);
if seconds_left == 5 {
println!("[Main] 5 seconds left, waiting for worker thread to finish work.");
break;
}
thread::sleep(Duration::from_secs(1));
}
t.join()?;
println!("[Main] worker thread finished work, shutting down!");
Ok(())
}
By the way, this kind of implementation wouldn't be any different in any other language, so please don't blame Rust for it. It's not the easiest language, but it provides more than enough tools to build anything you want from scratch as long as you put effort into it.
Goodluck :)
I think I got it work
use std::sync::{Arc, Mutex};
use std::thread::{sleep, spawn, JoinHandle};
use std::time::Duration;
struct Timer {
pub(crate) time: Arc<Mutex<u32>>,
jh_ticker: Option<JoinHandle<()>>,
}
impl Timer {
fn new<T>(i: T, duration: Duration) -> Self
where
T: Iterator<Item = u32> + Send + 'static,
{
let time = Arc::new(Mutex::new(0));
let arc_time = time.clone();
let jh_ticker = Some(spawn(move || {
for item in i {
let mut mg = arc_time.lock().unwrap();
*mg = item;
drop(mg); // needed, otherwise this thread will always hold lock
sleep(duration);
}
}));
Timer { time, jh_ticker }
}
}
impl Drop for Timer {
fn drop(&mut self) {
self.jh_ticker.take().unwrap().join();
}
}
#[test]
fn test_timer() {
let t = Timer::new(0..=10, Duration::from_secs(1));
let a = t.time.clone();
for _ in 0..100 {
let b = *a.lock().unwrap();
println!("{}", b);
sleep(Duration::from_millis(100));
}
}
I want to print "Hello" once a second.
Quoting the doc:
Futures use a poll based model. The consumer of a future repeatedly calls the poll function. The future then attempts to complete. If the future is able to complete, it returns Async::Ready(value). If the future is unable to complete due to being blocked on an internal resource (such as a TCP socket), it returns Async::NotReady.
My poll function returns NotReady if Delays return is NotReady, but nothing is printed to stdout.
use futures::{Async, Future, Stream}; // 0.1.25
use std::time::{Duration, Instant};
use tokio::timer::Delay; // 0.1.15
struct SomeStream;
impl Stream for SomeStream {
type Item = String;
type Error = ();
fn poll(&mut self) -> Result<Async<Option<Self::Item>>, Self::Error> {
let when = Instant::now() + Duration::from_millis(1000);
let mut task = Delay::new(when).map_err(|e| eprintln!("{:?}", e));
match task.poll() {
Ok(Async::Ready(value)) => {}
Ok(Async::NotReady) => return Ok(Async::NotReady),
Err(err) => return Err(()),
}
Ok(Async::Ready(Some("Hello".to_string())))
}
}
fn main() {
let s = SomeStream;
let future = s
.for_each(|item| {
println!("{:?}", item);
Ok(())
})
.map_err(|e| {});
tokio::run(future);
}
The main issue here is that state management is missing. You are creating a new Delay future every time the stream is polled, rather than holding on to it until it's resolved.
This would lead to never seeing any items coming out of the stream, since these futures are only being polled once, likely yielding NotReady each time.
You need to keep track of the delay future in your type SomeStream. In this case, one can use an option, so as to also identify whether we need to create a new delay.
#[derive(Debug, Default)]
struct SomeStream {
delay: Option<Delay>,
}
The subsequent code for SomeStream::poll, with better error handling and more idiomatic constructs, would become something like this:
impl Stream for SomeStream {
type Item = String;
type Error = Box<dyn std::error::Error + Send + Sync>; // generic error
fn poll(&mut self) -> Result<Async<Option<Self::Item>>, Self::Error> {
let delay = self.delay.get_or_insert_with(|| {
let when = Instant::now() + Duration::from_millis(1000);
Delay::new(when)
});
match delay.poll() {
Ok(Async::Ready(value)) => {
self.delay = None;
Ok(Async::Ready(Some("Hello".to_string())))
},
Ok(Async::NotReady) => Ok(Async::NotReady),
Err(err) => Err(err.into()),
}
}
}
Or, even better, using the try_ready! macro, which makes the return of errors and NotReady signals with less boilerplate.
fn poll(&mut self) -> Result<Async<Option<Self::Item>>, Self::Error> {
let delay = self.delay.get_or_insert_with(|| {
let when = Instant::now() + Duration::from_millis(1000);
Delay::new(when)
});
try_ready!(delay.poll());
// tick!
self.delay = None;
Ok(Async::Ready(Some("Hello".to_string())))
}
(Playground)
I've got some Rust code that currently looks like this
fn read_stdin(mut tx: mpsc::Sender<String>) {
loop {
// read from stdin and send value over tx.
}
}
fn sleep_for(n: u64) -> impl Future<Item = (), Error = ()> {
thread::sleep(time::Duration::from_millis(n));
println!("[{}] slept for {} ms", Local::now().format("%T%.3f"), n);
future::ok(())
}
fn main() {
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(move || read_stdin(stdin_tx));
let server = stdin_rx
.map(|data| data.trim().parse::<u64>().unwrap_or(0))
.for_each(|n| tokio::spawn(sleep_for(n * 100)));
tokio::run(server);
}
It uses tokio and futures, with the aim of running some "cpu heavy" work (emulated by the sleep_for function) and then outputting some stuff to stdout.
When I run it, things seems to work fine and I get this output
2
[00:00:00.800] slept for 200 ms
10
1
[00:00:01.800] slept for 1000 ms
[00:00:01.900] slept for 100 ms
The first output with the value 2 is exactly as expected, and I see the timestamp printed after 200ms. But for the next inputs, it becomes clear that the sleep_for function is being executed sequentially, and not concurrently.
The output that I want to see is
2
[00:00:00.800] slept for 200 ms
10
1
[00:00:00.900] slept for 100 ms
[00:00:01.900] slept for 1000 ms
It seems that to get the output I'm looking for I want to execute sleep_for(10) and sleep_for(1) concurrently. How would I go about doing this in Rust with futures and tokio?
(Note: the actual values of the timestamps aren't important I'm using them more to show the ordering of execution within the program)
Found a solution with the use of the futures-timer crate.
use chrono::Local;
use futures::{future, sync::mpsc, Future, Sink, Stream};
use futures_timer::Delay;
use std::{io::stdin, thread, time::Duration};
fn read_stdin(mut tx: mpsc::Sender<String>) {
let stdin = stdin();
loop {
let mut buf = String::new();
stdin.read_line(&mut buf).unwrap();
tx = tx.send(buf).wait().unwrap()
}
}
fn main() {
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(move || read_stdin(stdin_tx));
let server = stdin_rx
.map(|data| data.trim().parse::<u64>().unwrap_or(0) * 100)
.for_each(|delay| {
println!("[{}] {} ms -> start", Local::now().format("%T%.3f"), delay);
tokio::spawn({
Delay::new(Duration::from_millis(delay))
.and_then(move |_| {
println!("[{}] {} ms -> done", Local::now().format("%T%.3f"), delay);
future::ok(())
})
.map_err(|e| panic!(e))
})
});
tokio::run(server);
}
The issue is that the rather letting the future to become parked and then notifying the current task, the code presented in the question was just sleeping the thread and so no progress could be made.
Update: Now I've just come across tokio-timer which seems like the standard way of doing this.