I have a stuct that looks like this:
pub struct Coordinator {
map_tasks: Mutex<Vec<Task>>,
reduce_tasks: Mutex<Vec<Task>>,
}
And I have a function that is called with a parameter to determine which of the two vectors to modify. The function takes a type and an id to find in the vector and modify. My original approach is this:
impl Coordinator {
pub fn update_status(&self, task_type: TaskType, task_id: u8) {
if task_type == TaskType::Map {
let tasks = &mut self.map_tasks.lock().unwrap();
for task in tasks.iter_mut() {
if task.task_id == task_id {
task.status = TaskStatus::Success;
}
}
} else {
let tasks = &mut self.reduce_tasks.lock().unwrap();
for task in tasks.iter_mut() {
if task.task_id == task_id {
task.status = TaskStatus::Success;
}
}
}
}
}
I would like to shorten the code since a lot of it is duplicated. I've tried this, but it doesnt work with the error temporary value dropped while borrowed, consider using a "let" binding to create a longer lived value
impl Coordinator {
pub fn update_status(&self, task_type: TaskType, task_id: u8) {
let tasks;
if task_type == TaskType::Map {
tasks = &mut self.map_tasks.lock().unwrap();
} else {
tasks = &mut self.reduce_tasks.lock().unwrap();
}
for task in tasks.iter_mut() {
if task.task_id == task_id {
task.status = TaskStatus::Success;
}
}
}
}
How could I fix this?
You don't need to take a mutable reference into the guard, you can just store the guard directly. Also, if..else is an expression, you can use it as a ternary:
impl Coordinator {
pub fn update_status(&self, task_type: TaskType, task_id: u8) {
let mut tasks = if task_type == TaskType::Map {
self.map_tasks.lock().unwrap()
} else {
self.reduce_tasks.lock().unwrap()
};
for task in tasks.iter_mut() {
if task.task_id == task_id {
task.status = TaskStatus::Success;
}
}
}
}
Related
I have a polling function that will forever poll and always do pending. Inside the polling function poll_event_loop I want to control timings when the context should be polled again and the function to be called again in x seconds depending on some conditions. I could do this using another thread which calls the waker.wake_by_ref function. But this feels like a cheat. How could I do this without other threads.
poll_fn(|cx| self.poll_event_loop(cx)).await
// function will never be Ready, always pending, polling sould be fast so not just sleeping x seconds inside
fn poll_event_loop(&mut self, cx: &mut Context) -> Poll<anyhow::Result<()>> {
while some_codeandfunc() { /*....*/ }
// guarantee another poll_fn in 1 sec
if condition {
context_callback(cx, 1000);
}
// guarantee another poll_fn in 2 sec
if condition {
context_callback(cx, 2000);
}
Poll::Pending
}
fn context_callback(context: &mut Context, millisec: u64) {
let mut future = Box::pin(tokio::time::sleep(Durationtk::from_millis(millisec)));
//let cb = future.as_mut().poll(context);
future.poll_unpin(context);
}
// ugly way to auto poll the function every x seconds
fn spawn_qeueu_thread(waker: &Waker, rx: &Receiver<String>) -> Option<JoinHandle<()>> {
debug!("doing spawning thread");
//self.thread_spawned = true;
let waker = waker.clone();
let rx2 = rx.clone();
let spawn = tokio::spawn(async move {
loop {
tokio::time::sleep(Durationtk::from_millis(WAKEUPINTERVAL)).await;
debug!("doing other thread wakebyref");
waker.wake_by_ref();
let try_result = rx2.try_recv();
match try_result {
Err(_) => {}
Ok(_msg) => break,
}
}
debug!("ending spawned thread");
});
return Some(spawn);
//self.threadhandle = Some(spawn);
}
After some experimenting, I found a working solution that lets you call the polling function on multiple desired times, working example:
use chrono::Utc;
use futures::FutureExt;
use futures::future::poll_fn;
use tokio::time::Sleep;
use std::pin::Pin;
use std::task::Context;
use std::task::Poll;
use tokio::time::Duration as Durationtk;
pub struct ControllerModule {
vec: Vec<Pin<Box<Sleep>>>,
i: i64,
}
impl ControllerModule {
fn new() -> Self {
let vec = vec![];
let i = 0;
Self { vec, i }
}
async fn start(&mut self) {
poll_fn(|cx| self.poll_event_loop(cx)).await;
print!("worked");
}
fn poll_event_loop(&mut self, context: &mut Context) -> Poll<anyhow::Result<()>> {
self.i += 1;
if self.i % 3 == 0 {
let mut sleep = Box::pin(tokio::time::sleep(Durationtk::from_millis(5000)));
sleep.poll_unpin(context);
self.vec.push(sleep);
} else if self.i % 3 == 1 {
let mut sleep = Box::pin(tokio::time::sleep(Durationtk::from_millis(4000)));
sleep.poll_unpin(context);
self.vec.push(sleep);
} else {
context.waker().wake_by_ref();
}
self.vec.retain(|e| !e.is_elapsed());
Poll::Pending
}
}
#[tokio::main]
async fn main() {
let mut i = ControllerModule::new();
i.start().await
}
I am just a beginner in Rust and so far I have managed to obey borrow-checker's warnings in a async-heavy tokio app until now.
Basically I have a struct which has a HashMap of games. I want to insert a created game into the hashmap and at the same time pass it to a game_loop that I spawn with tokio::spawn. Game loop will update the game but I also want to be able to retrieve the game from the hashmap to execute some functions to check its state etc.
I've tried wrapping it in Arc and Mutexes and whatnot. At the moment I just clone it to the game_loop but - as smarter people probably know - that will only pass a clone of the original and the entity in the hashmap wont update.
GameManager
pub struct GameManager {
games: HashMap<Uuid, Game>,
}
impl GameManager {
fn find_or_create_game(&mut self, user_options: &GameOptions) -> Uuid {
for g in self.games.values() {
println!("game id {:?}", g.id);
println!("game players {:?}", g.state.get_players());
if g.allows_joining() && g.matches_player_options(user_options) {
println!("Joining existing game");
return g.id;
}
}
let rng = ::rand::rngs::StdRng::from_seed(OsRng.gen());
let mut game = Game::new(Some(user_options.clone()), rng);
let game_id = game.id;
let (game_sender, game_receiver) = mpsc::unbounded_channel::<GameEvent>();
let broadcast = self.broadcast.clone();
self.game_channels.insert(game_id, game_sender.clone());
self.games.insert(game_id, game);
tokio::spawn(game_loop(game, broadcast, game_receiver));
return game_id;
}
}
game_loop
pub async fn game_loop(
mut game: Game,
broadcast: UnboundedSender<ServerEvent>,
mut receiver: UnboundedReceiver<GameEvent>,
) -> Result<(), io::Error> {
let dur = std::time::Duration::from_secs_f64(1.0 / game.state.options.fps as f64);
let mut interval = tokio::time::interval(dur);
loop {
interval.tick().await;
while let Some(is_event) = unconstrained(receiver.recv()).now_or_never() {
if let Some(event) = is_event {
handle_game_event(event, &mut game, &broadcast);
}
}
if game.has_ended() {
break;
} else {
game.tick();
let _ = broadcast.send(ServerEvent::Tick(game.get_tick()));
}
}
Ok(())
}
Okay yeah, I am a dummy. Thanks #Peterrabbit though for giving me the motivation to try using Arc again. Had to wrap it in a mutex but all together, I am just happy that it now works.
So now it is:
pub struct GameManager {
games: HashMap<Uuid, Arc<Mutex<Game>>>,
}
impl GameManager {
fn find_or_create_game(&mut self, user_options: &GameOptions) -> Uuid {
for g in self.games.values() {
if g.allows_joining() && g.matches_player_options(user_options) {
println!("Joining existing game");
return g.id;
}
}
let rng = ::rand::rngs::StdRng::from_seed(OsRng.gen());
let mut game = Arc::new(Mutex::new(Game::new(Some(user_options.clone()), rng)));
let game_id = game.lock().await.id;
let (game_sender, game_receiver) = mpsc::unbounded_channel::<GameEvent>();
let broadcast = self.broadcast.clone();
self.game_channels.insert(game_id, game_sender.clone());
self.games.insert(game_id, game.clone());
tokio::spawn(game_loop(game.clone(), broadcast, game_receiver));
return game_id;
}
}
pub async fn game_loop(
mut game: Arc<Mutex<Game>>,
broadcast: UnboundedSender<ServerEvent>,
mut receiver: UnboundedReceiver<GameEvent>,
) -> Result<(), io::Error> {
let dur = std::time::Duration::from_secs_f64(1.0 / game.lock().await.state.options.fps as f64);
let mut interval = tokio::time::interval(dur);
loop {
interval.tick().await;
while let Some(is_event) = unconstrained(receiver.recv()).now_or_never() {
if let Some(event) = is_event {
handle_game_event(event, &mut game, &broadcast).await;
}
}
if game.lock().await.has_ended() {
break;
} else if game.lock().await.is_running() {
handle_game_event(GameEvent::Tick(), &mut game, &broadcast).await;
}
}
Ok(())
}
I've failed to get this code past the borrow-checker:
use std::sync::Arc;
use std::thread::{sleep, spawn};
use std::time::Duration;
#[derive(Debug, Clone)]
struct State {
count: u64,
not_copyable: Vec<u8>,
}
fn bar(thread_num: u8, arc_state: Arc<State>) {
let state = arc_state.clone();
loop {
sleep(Duration::from_millis(1000));
println!("thread_num: {}, state.count: {}", thread_num, state.count);
}
}
fn main() -> std::io::Result<()> {
let mut state = State {
count: 0,
not_copyable: vec![],
};
let arc_state = Arc::new(state);
for i in 0..2 {
spawn(move || {
bar(i, arc_state.clone());
});
}
loop {
sleep(Duration::from_millis(300));
state.count += 1;
}
}
I'm probably trying the wrong thing.
I want one (main) thread which can update state and many threads which can read state.
How should I do this in Rust?
I have read the Rust book on shared state, but that uses mutexes which seem overly complex for a single writer / multiple reader situation.
In C I would achieve this with a generous sprinkling of _Atomic.
Atomics are indeed a proper way, there are plenty of those in std (link. Your example needs 2 fixes.
Arc must be cloned before moving into the closure, so your loop becomes:
for i in 0..2 {
let arc_state = arc_state.clone();
spawn(move || { bar(i, arc_state); });
}
Using AtomicU64 is fairly straight forward, though you need explicitly use newtype methods with specified Ordering (Playground):
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use std::thread::{sleep, spawn};
use std::time::Duration;
#[derive(Debug)]
struct State {
count: AtomicU64,
not_copyable: Vec<u8>,
}
fn bar(thread_num: u8, arc_state: Arc<State>) {
let state = arc_state.clone();
loop {
sleep(Duration::from_millis(1000));
println!(
"thread_num: {}, state.count: {}",
thread_num,
state.count.load(Ordering::Relaxed)
);
}
}
fn main() -> std::io::Result<()> {
let state = State {
count: AtomicU64::new(0),
not_copyable: vec![],
};
let arc_state = Arc::new(state);
for i in 0..2 {
let arc_state = arc_state.clone();
spawn(move || {
bar(i, arc_state);
});
}
loop {
sleep(Duration::from_millis(300));
// you can't use `state` here, because it moved
arc_state.count.fetch_add(1, Ordering::Relaxed);
}
}
I implemented the future and made a request of it, but it blocked my curl and the log shows that poll was only invoked once.
Did I implement anything wrong?
use failure::{format_err, Error};
use futures::{future, Async};
use hyper::rt::Future;
use hyper::service::{service_fn, service_fn_ok};
use hyper::{Body, Method, Request, Response, Server, StatusCode};
use log::{debug, error, info};
use std::{
sync::{Arc, Mutex},
task::Waker,
thread,
};
pub struct TimerFuture {
shared_state: Arc<Mutex<SharedState>>,
}
struct SharedState {
completed: bool,
resp: String,
}
impl Future for TimerFuture {
type Item = Response<Body>;
type Error = hyper::Error;
fn poll(&mut self) -> futures::Poll<Response<Body>, hyper::Error> {
let mut shared_state = self.shared_state.lock().unwrap();
if shared_state.completed {
return Ok(Async::Ready(Response::new(Body::from(
shared_state.resp.clone(),
))));
} else {
return Ok(Async::NotReady);
}
}
}
impl TimerFuture {
pub fn new(instance: String) -> Self {
let shared_state = Arc::new(Mutex::new(SharedState {
completed: false,
resp: String::new(),
}));
let thread_shared_state = shared_state.clone();
thread::spawn(move || {
let res = match request_health(instance) {
Ok(status) => status.clone(),
Err(err) => {
error!("{:?}", err);
format!("{}", err)
}
};
let mut shared_state = thread_shared_state.lock().unwrap();
shared_state.completed = true;
shared_state.resp = res;
});
TimerFuture { shared_state }
}
}
fn request_health(instance_name: String) -> Result<String, Error> {
std::thread::sleep(std::time::Duration::from_secs(1));
Ok("health".to_string())
}
type BoxFut = Box<dyn Future<Item = Response<Body>, Error = hyper::Error> + Send>;
fn serve_health(req: Request<Body>) -> BoxFut {
let mut response = Response::new(Body::empty());
let path = req.uri().path().to_owned();
match (req.method(), path) {
(&Method::GET, path) => {
return Box::new(TimerFuture::new(path.clone()));
}
_ => *response.status_mut() = StatusCode::NOT_FOUND,
}
Box::new(future::ok(response))
}
fn main() {
let endpoint_addr = "0.0.0.0:8080";
match std::thread::spawn(move || {
let addr = endpoint_addr.parse().unwrap();
info!("Server is running on {}", addr);
hyper::rt::run(
Server::bind(&addr)
.serve(move || service_fn(serve_health))
.map_err(|e| eprintln!("server error: {}", e)),
);
})
.join()
{
Ok(e) => e,
Err(e) => println!("{:?}", e),
}
}
After compile and run this code, a server with port 8080 is running. Call the server with curl and it will block:
curl 127.0.0.1:8080/my-health-scope
Did I implement anything wrong?
Yes, you did not read and follow the documentation for the method you are implementing (emphasis mine):
When a future is not ready yet, the Async::NotReady value will be returned. In this situation the future will also register interest of the current task in the value being produced. This is done by calling task::park to retrieve a handle to the current Task. When the future is then ready to make progress (e.g. it should be polled again) the unpark method is called on the Task.
As a minimal, reproducible example, let's use this:
use futures::{future::Future, Async};
use std::{
mem,
sync::{Arc, Mutex},
thread,
time::Duration,
};
pub struct Timer {
data: Arc<Mutex<String>>,
}
impl Timer {
pub fn new(instance: String) -> Self {
let data = Arc::new(Mutex::new(String::new()));
thread::spawn({
let data = data.clone();
move || {
thread::sleep(Duration::from_secs(1));
*data.lock().unwrap() = instance;
}
});
Timer { data }
}
}
impl Future for Timer {
type Item = String;
type Error = ();
fn poll(&mut self) -> futures::Poll<Self::Item, Self::Error> {
let mut data = self.data.lock().unwrap();
eprintln!("poll was called");
if data.is_empty() {
Ok(Async::NotReady)
} else {
let data = mem::replace(&mut *data, String::new());
Ok(Async::Ready(data))
}
}
}
fn main() {
let v = Timer::new("Some text".into()).wait();
println!("{:?}", v);
}
It only prints out "poll was called" once.
You can call task::current (previously task::park) in the implementation of Future::poll, save the resulting value, then use the value with Task::notify (previously Task::unpark) whenever the future may be polled again:
use futures::{
future::Future,
task::{self, Task},
Async,
};
use std::{
mem,
sync::{Arc, Mutex},
thread,
time::Duration,
};
pub struct Timer {
data: Arc<Mutex<(String, Option<Task>)>>,
}
impl Timer {
pub fn new(instance: String) -> Self {
let data = Arc::new(Mutex::new((String::new(), None)));
let me = Timer { data };
thread::spawn({
let data = me.data.clone();
move || {
thread::sleep(Duration::from_secs(1));
let mut data = data.lock().unwrap();
data.0 = instance;
if let Some(task) = data.1.take() {
task.notify();
}
}
});
me
}
}
impl Future for Timer {
type Item = String;
type Error = ();
fn poll(&mut self) -> futures::Poll<Self::Item, Self::Error> {
let mut data = self.data.lock().unwrap();
eprintln!("poll was called");
if data.0.is_empty() {
let v = task::current();
data.1 = Some(v);
Ok(Async::NotReady)
} else {
let data = mem::replace(&mut data.0, String::new());
Ok(Async::Ready(data))
}
}
}
fn main() {
let v = Timer::new("Some text".into()).wait();
println!("{:?}", v);
}
See also:
Why does Future::select choose the future with a longer sleep period first?
Why is `Future::poll` not called repeatedly after returning `NotReady`?
What is the best approach to encapsulate blocking I/O in future-rs?
The following code works, but it doesn't look nice as the definition of is_empty is too far away from the usage.
fn remove(&mut self, index: I, primary_key: &Rc<K>) {
let is_empty;
{
let ks = self.data.get_mut(&index).unwrap();
ks.remove(primary_key);
is_empty = ks.is_empty();
}
// I have to wrap `ks` in an inner scope so that we can borrow `data` mutably.
if is_empty {
self.data.remove(&index);
}
}
Do we have some ways to drop the variables in condition before entering the if branches, e.g.
if {ks.is_empty()} {
self.data.remove(&index);
}
Whenever you have a double look-up of a key, you need to think Entry API.
With the entry API, you get a handle to a key-value pair and can:
read the key,
read/modify the value,
remove the entry entirely (getting the key and value back).
It's extremely powerful.
In this case:
use std::collections::HashMap;
use std::collections::hash_map::Entry;
fn remove(hm: &mut HashMap<i32, String>, index: i32) {
if let Entry::Occupied(o) = hm.entry(index) {
if o.get().is_empty() {
o.remove_entry();
}
}
}
fn main() {
let mut hm = HashMap::new();
hm.insert(1, String::from(""));
remove(&mut hm, 1);
println!("{:?}", hm);
}
I did this in the end:
match self.data.entry(index) {
Occupied(mut occupied) => {
let is_empty = {
let ks = occupied.get_mut();
ks.remove(primary_key);
ks.is_empty()
};
if is_empty {
occupied.remove();
}
},
Vacant(_) => unreachable!()
}