I am using a crate called timer and I'm trying to put a timer "guard" into a hashmap to store it so that I can either drop it before it ends or make sure I don't create a duplicate.
The problem is that the timer does not trigger when inserted into a hashmap through a function, but works fine if the body of the function is moved to main instead.
extern crate chrono;
extern crate timer;
use std::collections::HashMap;
use std::thread;
use std::time::Duration;
fn insert_to_guard_map(guard_map: &mut HashMap<i32, timer::Guard>) {
let timer = timer::Timer::new();
let guard = timer.schedule_with_delay(chrono::Duration::seconds(2), || {
println!("Called after 2s.");
});
guard_map.insert(42, guard);
}
fn main() {
let mut guard_map = HashMap::new();
insert_to_guard_map(&mut guard_map);
thread::sleep(Duration::from_secs(4));
}
The documentation of Timer and Guard mention that if either the Timer or the Guard is dropped, then the scheduled execution is ultimately cancelled.
So in that case, either insert both the timer and guard into the HashMap.
use timer::{Guard, Timer};
fn insert_to_guard_map(guard_map: &mut HashMap<i32, (Timer, Guard)>) {
let timer = Timer::new();
let guard = timer.schedule_with_delay(chrono::Duration::seconds(2), || {
println!("Called after 2s.");
});
guard_map.insert(42, (timer, guard));
}
Alternatively (and likely better), pass a reference of Timer to insert_to_guard_map().
use timer::{Guard, Timer};
fn insert_to_guard_map(guard_map: &mut HashMap<i32, Guard>, timer: &Timer) {
let guard = timer.schedule_with_delay(chrono::Duration::seconds(2), || {
println!("Called after 2s.");
});
guard_map.insert(42, guard);
}
fn main() {
let timer = Timer::new();
let mut guard_map = HashMap::new();
insert_to_guard_map(&mut guard_map, &timer);
thread::sleep(Duration::from_secs(4));
}
Related
I want to build a function that takes a HashMap reference as an argument. This HashMap should be shared between threads for read only access. The code example is very simple:
I insert some value into the HashMap, pass it to the function and want antoher thread to read that value. I get an Error that the borrowed value does not live long enough at line let exit_code = test(&m);. Why is this not working?
use std::thread;
use std::collections::HashMap;
use std::sync::{Arc, RwLock };
fn main(){
let mut m: HashMap<u32, f64> = HashMap::new();
m.insert(0, 0.1);
let exit_code = test(&m);
std::process::exit(exit_code);
}
fn test(m: &'static HashMap<u32, f64>) -> i32{
let map_lock = Arc::new(RwLock::new(m));
let read_thread = thread::spawn(move || {
if let Ok(r_guard) = map_lock.read(){
println!("{:?}", r_guard.get(&0).unwrap());
}
});
read_thread.join().unwrap();
return 0;
}
if I don't put the 'static in the function signature for the HashMap argument, Arc::new(RwLock::new(m)); doesn't work. How can I sovlve this problem?
A reference is not safe to share unless is 'static meaning that something will live for the extent of the program. Otherwise the compiler is not able to track the liveliness of the shared element.
You should wrap it outside of the function, and take ownership of an Arc:
use std::thread;
use std::collections::HashMap;
use std::sync::{Arc, RwLock };
fn main(){
let mut map = HashMap::new();
map.insert(0, 0.1);
let m = Arc::new(RwLock::new(map));
let exit_code = test(m);
std::process::exit(exit_code);
}
fn test(map_lock: Arc<RwLock<HashMap<u32, f64>>>) -> i32 {
let read_thread = thread::spawn(move || {
if let Ok(r_guard) = map_lock.read(){
println!("{:?}", r_guard.get(&0).unwrap());
}
});
read_thread.join().unwrap();
return 0;
}
Playground
I have a Vec of futures which I want to execute concurrently (but not necessarily in parallel). Basically, I'm looking for some kind of select function that is similar to tokio::select! but takes a collection of futures, or, conversely, a function that is similar to futures::join_all but returns once the first future is done.
An additional requirement is that once a future finished I might want to add a new future to the Vec.
With such a function, my code would roughly look like this:
use std::future::Future;
use std::time::Duration;
use tokio::time::sleep;
async fn wait(millis: u64) -> u64 {
sleep(Duration::from_millis(millis)).await;
millis
}
// This pseudo-implementation simply removes the last
// future and awaits it. I'm looking for something that
// instead polls all futures until one is finished, then
// removes that future from the Vec and returns it.
async fn select<F, O>(futures: &mut Vec<F>) -> O
where
F: Future<Output=O>
{
let future = futures.pop().unwrap();
future.await
}
#[tokio::main]
async fn main() {
let mut futures = vec![
wait(500),
wait(300),
wait(100),
wait(200),
];
while !futures.is_empty() {
let finished = select(&mut futures).await;
println!("Waited {}ms", finished);
if some_condition() {
futures.push(wait(200));
}
}
}
This is exactly what futures::stream::FuturesUnordered is for (which I've found by looking through the source of StreamExt::for_each_concurrent):
use futures::{stream::FuturesUnordered, StreamExt};
use std::time::Duration;
use tokio::time::{sleep, Instant};
async fn wait(millis: u64) -> u64 {
sleep(Duration::from_millis(millis)).await;
millis
}
#[tokio::main]
async fn main() {
let mut futures = FuturesUnordered::new();
futures.push(wait(500));
futures.push(wait(300));
futures.push(wait(100));
futures.push(wait(200));
let start_time = Instant::now();
let mut num_added = 0;
while let Some(wait_time) = futures.next().await {
println!("Waited {}ms", wait_time);
if num_added < 3 {
num_added += 1;
futures.push(wait(200));
}
}
println!("Completed all work in {}ms", start_time.elapsed().as_millis());
}
(playground)
Here's a working prototype based on streams and StreamExt::for_each_concurrent, as Martin Gallagher has suggested in a comment:
use std::time::Duration;
use tokio::sync::RwLock;
use tokio::time::sleep;
use futures::stream::{self, StreamExt};
use futures::{channel::mpsc, sink::SinkExt};
async fn wait(millis: u64) -> u64 {
sleep(Duration::from_millis(millis)).await;
millis
}
#[tokio::main]
async fn main() {
let (mut sink, futures_stream) = mpsc::unbounded();
let start_futures = vec![wait(500), wait(300), wait(100), wait(200)];
let num_futures = RwLock::new(start_futures.len());
sink.send_all(&mut stream::iter(start_futures.into_iter().map(Ok)))
.await
.unwrap();
let sink_lock = RwLock::new(sink);
futures_stream
.for_each_concurrent(None, |fut| async {
let wait_time = fut.await;
println!("Waited {}", wait_time);
if some_condition() {
println!("Adding new future");
let mut sink = sink_lock.write().await;
sink.send(wait(100)).await.unwrap();
} else {
let mut num_futures = num_futures.write().await;
*num_futures -= 1;
}
let num_futures = num_futures.read().await;
if *num_futures <= 0 {
// Close the sink to exit the for_each_concurrent
sink_lock.write().await.close().await.unwrap();
}
})
.await;
}
While this approach works it has the drawback that we need to maintain a separate counter of remaining futures so that we can close the sink -- there's no Vec of futures for which we can check whether it's empty. Closing the sink requires another lock.
Given that I'm fairly new to Rust I wouldn't be surprised if this approach could be made more elegant.
I am trying to use hyper to grab the content of an HTML page and would like to synchronously return the output of a future. I realized I could have picked a better example since synchronous HTTP requests already exist, but I am more interested in understanding whether we could return a value from an async calculation.
extern crate futures;
extern crate hyper;
extern crate hyper_tls;
extern crate tokio;
use futures::{future, Future, Stream};
use hyper::Client;
use hyper::Uri;
use hyper_tls::HttpsConnector;
use std::str;
fn scrap() -> Result<String, String> {
let scraped_content = future::lazy(|| {
let https = HttpsConnector::new(4).unwrap();
let client = Client::builder().build::<_, hyper::Body>(https);
client
.get("https://hyper.rs".parse::<Uri>().unwrap())
.and_then(|res| {
res.into_body().concat2().and_then(|body| {
let s_body: String = str::from_utf8(&body).unwrap().to_string();
futures::future::ok(s_body)
})
}).map_err(|err| format!("Error scraping web page: {:?}", &err))
});
scraped_content.wait()
}
fn read() {
let scraped_content = future::lazy(|| {
let https = HttpsConnector::new(4).unwrap();
let client = Client::builder().build::<_, hyper::Body>(https);
client
.get("https://hyper.rs".parse::<Uri>().unwrap())
.and_then(|res| {
res.into_body().concat2().and_then(|body| {
let s_body: String = str::from_utf8(&body).unwrap().to_string();
println!("Reading body: {}", s_body);
Ok(())
})
}).map_err(|err| {
println!("Error reading webpage: {:?}", &err);
})
});
tokio::run(scraped_content);
}
fn main() {
read();
let content = scrap();
println!("Content = {:?}", &content);
}
The example compiles and the call to read() succeeds, but the call to scrap() panics with the following error message:
Content = Err("Error scraping web page: Error { kind: Execute, cause: None }")
I understand that I failed to launch the task properly before calling .wait() on the future but I couldn't find how to properly do it, assuming it's even possible.
Standard library futures
Let's use this as our minimal, reproducible example:
async fn example() -> i32 {
42
}
Call executor::block_on:
use futures::executor; // 0.3.1
fn main() {
let v = executor::block_on(example());
println!("{}", v);
}
Tokio
Use the tokio::main attribute on any function (not just main!) to convert it from an asynchronous function to a synchronous one:
use tokio; // 0.3.5
#[tokio::main]
async fn main() {
let v = example().await;
println!("{}", v);
}
tokio::main is a macro that transforms this
#[tokio::main]
async fn main() {}
Into this:
fn main() {
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap()
.block_on(async { {} })
}
This uses Runtime::block_on under the hood, so you can also write this as:
use tokio::runtime::Runtime; // 0.3.5
fn main() {
let v = Runtime::new().unwrap().block_on(example());
println!("{}", v);
}
For tests, you can use tokio::test.
async-std
Use the async_std::main attribute on the main function to convert it from an asynchronous function to a synchronous one:
use async_std; // 1.6.5, features = ["attributes"]
#[async_std::main]
async fn main() {
let v = example().await;
println!("{}", v);
}
For tests, you can use async_std::test.
Futures 0.1
Let's use this as our minimal, reproducible example:
use futures::{future, Future}; // 0.1.27
fn example() -> impl Future<Item = i32, Error = ()> {
future::ok(42)
}
For simple cases, you only need to call wait:
fn main() {
let s = example().wait();
println!("{:?}", s);
}
However, this comes with a pretty severe warning:
This method is not appropriate to call on event loops or similar I/O situations because it will prevent the event loop from making progress (this blocks the thread). This method should only be called when it's guaranteed that the blocking work associated with this future will be completed by another thread.
Tokio
If you are using Tokio 0.1, you should use Tokio's Runtime::block_on:
use tokio; // 0.1.21
fn main() {
let mut runtime = tokio::runtime::Runtime::new().expect("Unable to create a runtime");
let s = runtime.block_on(example());
println!("{:?}", s);
}
If you peek in the implementation of block_on, it actually sends the future's result down a channel and then calls wait on that channel! This is fine because Tokio guarantees to run the future to completion.
See also:
How can I efficiently extract the first element of a futures::Stream in a blocking manner?
As this is the top result that come up in search engines by the query "How to call async from sync in Rust", I decided to share my solution here. I think it might be useful.
As #Shepmaster mentioned, back in version 0.1 futures crate had beautiful method .wait() that could be used to call an async function from a sync one. This must-have method, however, was removed from later versions of the crate.
Luckily, it's not that hard to re-implement it:
trait Block {
fn wait(self) -> <Self as futures::Future>::Output
where Self: Sized, Self: futures::Future
{
futures::executor::block_on(self)
}
}
impl<F,T> Block for F
where F: futures::Future<Output = T>
{}
After that, you can just do following:
async fn example() -> i32 {
42
}
fn main() {
let s = example().wait();
println!("{:?}", s);
}
Beware that this comes with all the caveats of original .wait() explained in the #Shepmaster's answer.
This works for me using tokio:
tokio::runtime::Runtime::new()?.block_on(fooAsyncFunction())?;
I have a struct that sends messages to a channel as well as updating some of its own fields. How do I implement a monitoring thread that looks (read only) at its internal fields periodically?
I can write it using a Arc<Mutex<T>> wrapper, but I feel it is not that efficient as A::x could have been i32 which is stored and updated on the stack. Is there any better way to do it without the locks?
use std::sync::{Arc, Mutex};
use std::sync::mpsc::{channel, Sender};
use std::{thread, time};
struct A {
x: Arc<Mutex<i32>>,
y: Sender<i32>,
}
impl A {
fn do_some_loop(&mut self) {
let sleep_time = time::Duration::from_millis(200);
// This is a long running thread.
for x in 1..1000000 {
*self.x.lock().unwrap() = x;
self.y.send(x);
thread::sleep(sleep_time);
}
}
}
fn test() {
let (sender, recever) = channel();
let x = Arc::new(Mutex::new(1));
let mut a = A { x: x.clone(), y: sender };
thread::spawn(move || {
// Monitor every 10 secs.
let sleep_time = time::Duration::from_millis(10000);
loop {
thread::sleep(sleep_time);
println!("{}", *x.lock().unwrap());
}
});
a.do_some_loop();
}
I am a beginner in Rust.
I have a long running IO-bound process that I want to spawn and monitor via a REST API. I chose Iron for that, following this tutorial . Monitoring means getting its progress and its final result.
When I spawn it, I give it an id and map that id to a resource that I can GET to get the progress. I don't have to be exact with the progress; I can report the progress from 5 seconds ago.
My first attempt was to have a channel via which I send request for progress and receive the status. I got stuck where to store the receiver, as in my understanding it belongs to one thread only. I wanted to put it in the context of the request, but that won't work as there are different threads handling subsequent requests.
What would be the idiomatic way to do this in Rust?
I have a sample project.
Later edit:
Here is a self contained example which follows the sample principle as the answer, namely a map where each thread updates its progress:
extern crate iron;
extern crate router;
extern crate rustc_serialize;
use iron::prelude::*;
use iron::status;
use router::Router;
use rustc_serialize::json;
use std::io::Read;
use std::sync::{Mutex, Arc};
use std::thread;
use std::time::Duration;
use std::collections::HashMap;
#[derive(Debug, Clone, RustcEncodable, RustcDecodable)]
pub struct Status {
pub progress: u64,
pub context: String
}
#[derive(RustcEncodable, RustcDecodable)]
struct StartTask {
id: u64
}
fn start_process(status: Arc<Mutex<HashMap<u64, Status>>>, task_id: u64) {
let c = status.clone();
thread::spawn(move || {
for i in 1..100 {
{
let m = &mut c.lock().unwrap();
m.insert(task_id, Status{ progress: i, context: "in progress".to_string()});
}
thread::sleep(Duration::from_secs(1));
}
let m = &mut c.lock().unwrap();
m.insert(task_id, Status{ progress: 100, context: "done".to_string()});
});
}
fn main() {
let status: Arc<Mutex<HashMap<u64, Status>>> = Arc::new(Mutex::new(HashMap::new()));
let status_clone: Arc<Mutex<HashMap<u64, Status>>> = status.clone();
let mut router = Router::new();
router.get("/:taskId", move |r: &mut Request| task_status(r, &status.lock().unwrap()));
router.post("/start", move |r: &mut Request|
start_task(r, status_clone.clone()));
fn task_status(req: &mut Request, statuses: & HashMap<u64,Status>) -> IronResult<Response> {
let ref task_id = req.extensions.get::<Router>().unwrap().find("taskId").unwrap_or("/").parse::<u64>().unwrap();
let payload = json::encode(&statuses.get(&task_id)).unwrap();
Ok(Response::with((status::Ok, payload)))
}
// Receive a message by POST and play it back.
fn start_task(request: &mut Request, statuses: Arc<Mutex<HashMap<u64, Status>>>) -> IronResult<Response> {
let mut payload = String::new();
request.body.read_to_string(&mut payload).unwrap();
let task_start_request: StartTask = json::decode(&payload).unwrap();
start_process(statuses, task_start_request.id);
Ok(Response::with((status::Ok, json::encode(&task_start_request).unwrap())))
}
Iron::new(router).http("localhost:3000").unwrap();
}
One possibility is to use a global HashMap that associate each worker id with the progress (and result). Here is simple example (without the rest stuff)
#[macro_use]
extern crate lazy_static;
use std::sync::Mutex;
use std::collections::HashMap;
use std::thread;
use std::time::Duration;
lazy_static! {
static ref PROGRESS: Mutex<HashMap<usize, usize>> = Mutex::new(HashMap::new());
}
fn set_progress(id: usize, progress: usize) {
// insert replaces the old value if there was one.
PROGRESS.lock().unwrap().insert(id, progress);
}
fn get_progress(id: usize) -> Option<usize> {
PROGRESS.lock().unwrap().get(&id).cloned()
}
fn work(id: usize) {
println!("Creating {}", id);
set_progress(id, 0);
for i in 0..100 {
set_progress(id, i + 1);
// simulates work
thread::sleep(Duration::new(0, 50_000_000));
}
}
fn monitor(id: usize) {
loop {
if let Some(p) = get_progress(id) {
if p == 100 {
println!("Done {}", id);
// to avoid leaks, remove id from PROGRESS.
// maybe save that the task ends in a data base.
return
} else {
println!("Progress {}: {}", id, p);
}
}
thread::sleep(Duration::new(1, 0));
}
}
fn main() {
let w = thread::spawn(|| work(1));
let m = thread::spawn(|| monitor(1));
w.join().unwrap();
m.join().unwrap();
}
You need to register one channel per request thread, because if cloning Receivers were possible the responses might/will end up with the wrong thread if two request are running at the same time.
Instead of having your thread create a channel for answering requests, use a future. A future allows you to have a handle to an object, where the object doesn't exist yet. You can change the input channel to receive a Promise, which you then fulfill, no output channel necessary.