I am trying to read and write to my serial port on Linux to communicate with a microcontroller and I'm trying to do so in Rust.
My normal pattern when developing in say C++ or Python is to have two threads: one which sends requests out over serial periodically and one which reads bytes out of the buffer and handles them.
In Rust, I'm running into trouble with the borrow checker while using the serial crate. This makes sense to me why this is, but I'm unsure what designing for an asynchronous communication interface would look like in Rust. Here's a snippet of my source:
let mut port = serial::open(&device_path.as_os_str()).unwrap();
let request_temperature: Vec<u8> = vec![0xAA];
thread::spawn(|| {
let mut buffer: Vec<u8> = Vec::new();
loop {
let _bytes_read = port.read(&mut buffer);
// process data
thread::sleep(Duration::from_millis(100));
}
});
loop {
port.write(&request_temperature);
thread::sleep(Duration::from_millis(1000));
}
How can I emulate this functionality where I have two threads holding onto a mutable resource in Rust? I know that since this specific example could be done in a single thread, but I'm thinking for an eventual larger program this would end up being multiple threads.
You can wrap your port in a Arc and a Mutex, then you can write something like:
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
struct Port;
impl Port {
pub fn read(&mut self, _v: &mut Vec<u8>) {
println!("READING...");
}
pub fn write(&mut self, _v: &Vec<u8>) {
println!("WRITING...");
}
}
pub fn main() {
let mut port = Arc::new(Mutex::new(Port));
let p2 = port.clone();
let handle = thread::spawn(move || {
let mut buffer: Vec<u8> = Vec::new();
for j in 0..100 {
let _bytes_read = p2.lock().unwrap().read(&mut buffer);
thread::sleep(Duration::from_millis(10));
}
});
let request_temperature: Vec<u8> = vec![0xAA];
for i in 0..10 {
port.lock().unwrap().write(&request_temperature);
thread::sleep(Duration::from_millis(100));
}
handle.join();
}
So that this will run on a test machine, I've replaced the serial port with a stub class, reduced the sleeps and replaced the infinite loop with some finite loops.
While this works, you'll probably actually want proper communication between the threads at some stage, at which point you'll want to look at std::sync::mpsc::channel
Related
I am writing a program that pings a set of targets 100 times, and stores each RTT value returned from the ping into a vector, thus giving me a set of RTT values for each target. Say I have n targets, I would like all of the pinging to be done concurrently. The rust code looks like this:
let mut sample_rtts_map = HashMap::new();
for addr in targets.to_vec() {
let mut sampleRTTvalues: Vec<f32> = vec![];
//sample_rtts_map.insert(addr, sampleRTTvalues);
thread::spawn(move || {
while sampleRTTvalues.len() < 100 {
let sampleRTT = ping(addr);
sampleRTTvalues.push(sampleRTT);
// thread::sleep(Duration::from_millis(5000));
}
});
}
The hashmap is used to tell which vector of values belongs to which target. The problem is, how do I retrieve the updated sampleRTTvalues from each thread after the thread is done executing? I would like something like:
let (name, sampleRTTvalues) = thread::spawn(...)
The name, being the name of the thread, and sampleRTTvalues being the vector. However, since I'm creating threads in a for loop, each thread is being instantiated the same way, so how I differentiate them?
Is there some better way to do this? I've looked into schedulers, future, etc., but it seems my case can just be done with simple threads.
I go the desired behavior with the following code:
use std::thread;
use std::sync::mpsc;
use std::collections::HashMap;
use rand::Rng;
use std::net::{Ipv4Addr,Ipv6Addr,IpAddr};
const RTT_ONE: IpAddr = IpAddr::V4(Ipv4Addr::new(127,0,0,1));
const RTT_TWO: IpAddr = IpAddr::V6(Ipv6Addr::new(0,0,0,0,0,0,0,1));
const RTT_THREE: IpAddr = IpAddr::V4(Ipv4Addr::new(127,0,1,1));//idk how ip adresses work, forgive if this in invalid but you get the idea
fn ping(address: IpAddr) -> f32 {
rand::thread_rng().gen_range(5.0..107.0)
}
fn main() {
let targets = [RTT_ONE,RTT_TWO,RTT_THREE];
let mut sample_rtts_map: HashMap<IpAddr,Vec<f32>> = HashMap::new();
for addr in targets.into_iter() {
let (sample_values,moved_values) = mpsc::channel();
let mut sampleRTTvalues: Vec<f32> = vec![];
thread::spawn(move || {
while sampleRTTvalues.len() < 100 {
let sampleRTT = ping(addr);
sampleRTTvalues.push(sampleRTT);
//thread::sleep(Duration::from_millis(5000));
}
});
sample_rtts_map.insert(addr,moved_values.recv().unwrap());
}
}
note that the use rand::Rng can be removed when implementing, as it is only so the example works. what this does is pass data from the spawned thread to the main thread, and in the method used it waits until the data is ready before adding it to the hash map. If this is problematic (takes a long time, etc.) then you can use try_recv instead of recv which will add an error / option type that will return a recoverable error if the value is ready when unwrapped, or return the value if it's ready
You can use a std::sync::mpsc channel to collect your data:
use std::collections::HashMap;
use std::sync::mpsc::channel;
use std::thread;
fn ping(_: &str) -> f32 { 0.0 }
fn main() {
let targets = ["a", "b"]; // just for example
let mut sample_rtts_map = HashMap::new();
let (tx, rx) = channel();
for addr in targets {
let tx = tx.clone();
thread::spawn(move || {
for _ in 0..100 {
let sampleRTT = ping(addr);
tx.send((addr, sampleRTT));
}
});
}
drop(tx);
// exit loop when all thread's tx have dropped
while let Ok((addr, sampleRTT)) = rx.recv() {
sample_rtts_map.entry(addr).or_insert(vec![]).push(sampleRTT);
}
println!("sample_rtts_map: {:?}", sample_rtts_map);
}
This will run all pinging threads simultaneously, and collect data in main thread synchronously, so that we can avoid using locks. Do not forget to drop sender in main thread after cloning to all pinging threads, or the main thread will hang forever.
In my browser application, two closures access data stored in a Rc<RefCell<T>>. One closure mutably borrows the data, while the other immutably borrows it. The two closures are invoked independently of one another, and this will occasionally result in a BorrowError or BorrowMutError.
Here is my attempt at an MWE, though it uses a future to artificially inflate the likelihood of the error occurring:
use std::cell::RefCell;
use std::future::Future;
use std::pin::Pin;
use std::rc::Rc;
use std::task::{Context, Poll, Waker};
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsValue;
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(js_namespace = console)]
pub fn log(s: &str);
#[wasm_bindgen(js_name = setTimeout)]
fn set_timeout(closure: &Closure<dyn FnMut()>, millis: u32) -> i32;
#[wasm_bindgen(js_name = setInterval)]
fn set_interval(closure: &Closure<dyn FnMut()>, millis: u32) -> i32;
}
pub struct Counter(u32);
#[wasm_bindgen(start)]
pub async fn main() -> Result<(), JsValue> {
console_error_panic_hook::set_once();
let counter = Rc::new(RefCell::new(Counter(0)));
let counter_clone = counter.clone();
let log_closure = Closure::wrap(Box::new(move || {
let c = counter_clone.borrow();
log(&c.0.to_string());
}) as Box<dyn FnMut()>);
set_interval(&log_closure, 1000);
log_closure.forget();
let counter_clone = counter.clone();
let increment_closure = Closure::wrap(Box::new(move || {
let counter_clone = counter_clone.clone();
wasm_bindgen_futures::spawn_local(async move {
let mut c = counter_clone.borrow_mut();
// In reality this future would be replaced by some other
// time-consuming operation manipulating the borrowed data
SleepFuture::new(5000).await;
c.0 += 1;
});
}) as Box<dyn FnMut()>);
set_timeout(&increment_closure, 3000);
increment_closure.forget();
Ok(())
}
struct SleepSharedState {
waker: Option<Waker>,
completed: bool,
closure: Option<Closure<dyn FnMut()>>,
}
struct SleepFuture {
shared_state: Rc<RefCell<SleepSharedState>>,
}
impl Future for SleepFuture {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let mut shared_state = self.shared_state.borrow_mut();
if shared_state.completed {
Poll::Ready(())
} else {
shared_state.waker = Some(cx.waker().clone());
Poll::Pending
}
}
}
impl SleepFuture {
fn new(duration: u32) -> Self {
let shared_state = Rc::new(RefCell::new(SleepSharedState {
waker: None,
completed: false,
closure: None,
}));
let state_clone = shared_state.clone();
let closure = Closure::wrap(Box::new(move || {
let mut state = state_clone.borrow_mut();
state.completed = true;
if let Some(waker) = state.waker.take() {
waker.wake();
}
}) as Box<dyn FnMut()>);
set_timeout(&closure, duration);
shared_state.borrow_mut().closure = Some(closure);
SleepFuture { shared_state }
}
}
panicked at 'already mutably borrowed: BorrowError'
The error makes sense, but how should I go about resolving it?
My current solution is to have the closures use try_borrow or try_borrow_mut, and if unsuccessful, use setTimeout for an arbitrary amount of time before attempting to borrow again.
Think about this problem independently of Rust's borrow semantics. You have a long-running operation that's updating some shared state.
How would you do it if you were using threads? You would put the shared state behind a lock. RefCell is like a lock except that you can't block on unlocking it — but you can emulate blocking by using some kind of message-passing to wake up the reader.
How would you do it if you were using pure JavaScript? You don't automatically have anything like RefCell, so either:
The state can be safely read while the operation is still ongoing (in a concurrency-not-parallelism sense): in this case, emulate that by not holding a single RefMut (result of borrow_mut()) alive across an await boundary.
The state is not safe to be read: you'd either write something lock-like as described above, or perhaps arrange so that it's only written once when the operation is done, and until then, the long-running operation has its own private state not shared with the rest of the application (so there can be no BorrowError conflicts).
Think about what your application actually needs and pick a suitable solution. Implementing any of these solutions will most likely involve having additional interior-mutable objects used for communication.
I'm creating a multithreaded application using mpsc to share memory between my threads:
use std::thread;
use std::sync::mpsc::{Sender, Receiver};
#[derive(Debug)]
struct Msg {
pub content: Vec<i16>,
/* ... */
}
#[derive(Debug)]
struct MsgBack {
pub content: Vec<i16>,
pub new_content: Vec<i16>,
/* ... */
}
fn child(rx: mpsc::Receiver<Msg>, tx: mpsc::Sender<MsgBack>) {
let message = rx.recv().unwrap();
let new_content = message.content.iter().map(|x| -x).collect();
tx.send(MsgBack { // The memory is moved/copied
content: message.content,
new_content: new_content,
});
}
fn main() {
let (tx, rx): (Sender<Msg>, Receiver<Msg>) = mpsc::channel();
let (tx_back, rx_back): (Sender<MsgBack>, Receiver<MsgBack>) = mpsc::channel();
thread::spawn(move || {
child(rx, tx_back);
});
let message = Msg {
content: (0..100).map(|x| x).collect(), // Dummy initialisation
};
println!("{:#?}", message);
tx.send(message).unwrap(); // The memory is moved/copied
let answer = rx_back.recv().unwrap();
println!("{:#?}", answer);
}
I did some profiling and I saw that sending the data is responsible for 1/3 of the execution time in my real program (which sends more than just a Vec).
I want to keep this code structure but avoid moves/copies when sending a message to save a lot of time.
Move of a Vec doesn't move its content, but only the 3-word "header". Therefore, unless your MsgBack contains a lot of other fields or large fixed-size arrays inline, it should be rather cheap to move.
In general, you can put things in Box to allocate them on the heap, so then the Box<T> itself is only pointer-sized. "Moving" of the Box doesn't move any data, only copies the pointer.
If your actual iterators are more complex than your example and don't have useful size_hint, you may be seeing memcpy from .collect() reallocating the vector as it grows. You can avoid that by pre-allocating required size:
Instead of:
let dst = iterator.collect();
use:
let mut dst = Vec::with_capacity(required_size);
dst.extend(iterator);
I am a beginner in Rust.
I have a long running IO-bound process that I want to spawn and monitor via a REST API. I chose Iron for that, following this tutorial . Monitoring means getting its progress and its final result.
When I spawn it, I give it an id and map that id to a resource that I can GET to get the progress. I don't have to be exact with the progress; I can report the progress from 5 seconds ago.
My first attempt was to have a channel via which I send request for progress and receive the status. I got stuck where to store the receiver, as in my understanding it belongs to one thread only. I wanted to put it in the context of the request, but that won't work as there are different threads handling subsequent requests.
What would be the idiomatic way to do this in Rust?
I have a sample project.
Later edit:
Here is a self contained example which follows the sample principle as the answer, namely a map where each thread updates its progress:
extern crate iron;
extern crate router;
extern crate rustc_serialize;
use iron::prelude::*;
use iron::status;
use router::Router;
use rustc_serialize::json;
use std::io::Read;
use std::sync::{Mutex, Arc};
use std::thread;
use std::time::Duration;
use std::collections::HashMap;
#[derive(Debug, Clone, RustcEncodable, RustcDecodable)]
pub struct Status {
pub progress: u64,
pub context: String
}
#[derive(RustcEncodable, RustcDecodable)]
struct StartTask {
id: u64
}
fn start_process(status: Arc<Mutex<HashMap<u64, Status>>>, task_id: u64) {
let c = status.clone();
thread::spawn(move || {
for i in 1..100 {
{
let m = &mut c.lock().unwrap();
m.insert(task_id, Status{ progress: i, context: "in progress".to_string()});
}
thread::sleep(Duration::from_secs(1));
}
let m = &mut c.lock().unwrap();
m.insert(task_id, Status{ progress: 100, context: "done".to_string()});
});
}
fn main() {
let status: Arc<Mutex<HashMap<u64, Status>>> = Arc::new(Mutex::new(HashMap::new()));
let status_clone: Arc<Mutex<HashMap<u64, Status>>> = status.clone();
let mut router = Router::new();
router.get("/:taskId", move |r: &mut Request| task_status(r, &status.lock().unwrap()));
router.post("/start", move |r: &mut Request|
start_task(r, status_clone.clone()));
fn task_status(req: &mut Request, statuses: & HashMap<u64,Status>) -> IronResult<Response> {
let ref task_id = req.extensions.get::<Router>().unwrap().find("taskId").unwrap_or("/").parse::<u64>().unwrap();
let payload = json::encode(&statuses.get(&task_id)).unwrap();
Ok(Response::with((status::Ok, payload)))
}
// Receive a message by POST and play it back.
fn start_task(request: &mut Request, statuses: Arc<Mutex<HashMap<u64, Status>>>) -> IronResult<Response> {
let mut payload = String::new();
request.body.read_to_string(&mut payload).unwrap();
let task_start_request: StartTask = json::decode(&payload).unwrap();
start_process(statuses, task_start_request.id);
Ok(Response::with((status::Ok, json::encode(&task_start_request).unwrap())))
}
Iron::new(router).http("localhost:3000").unwrap();
}
One possibility is to use a global HashMap that associate each worker id with the progress (and result). Here is simple example (without the rest stuff)
#[macro_use]
extern crate lazy_static;
use std::sync::Mutex;
use std::collections::HashMap;
use std::thread;
use std::time::Duration;
lazy_static! {
static ref PROGRESS: Mutex<HashMap<usize, usize>> = Mutex::new(HashMap::new());
}
fn set_progress(id: usize, progress: usize) {
// insert replaces the old value if there was one.
PROGRESS.lock().unwrap().insert(id, progress);
}
fn get_progress(id: usize) -> Option<usize> {
PROGRESS.lock().unwrap().get(&id).cloned()
}
fn work(id: usize) {
println!("Creating {}", id);
set_progress(id, 0);
for i in 0..100 {
set_progress(id, i + 1);
// simulates work
thread::sleep(Duration::new(0, 50_000_000));
}
}
fn monitor(id: usize) {
loop {
if let Some(p) = get_progress(id) {
if p == 100 {
println!("Done {}", id);
// to avoid leaks, remove id from PROGRESS.
// maybe save that the task ends in a data base.
return
} else {
println!("Progress {}: {}", id, p);
}
}
thread::sleep(Duration::new(1, 0));
}
}
fn main() {
let w = thread::spawn(|| work(1));
let m = thread::spawn(|| monitor(1));
w.join().unwrap();
m.join().unwrap();
}
You need to register one channel per request thread, because if cloning Receivers were possible the responses might/will end up with the wrong thread if two request are running at the same time.
Instead of having your thread create a channel for answering requests, use a future. A future allows you to have a handle to an object, where the object doesn't exist yet. You can change the input channel to receive a Promise, which you then fulfill, no output channel necessary.
I'm trying to share a RwLock amongst several threads without using scoped threads but I can't figure out how to get the lifetimes correct. I assume that this is possible (what's the point of RwLocks otherwise?) but I can't find any examples of it.
Here is a toy example of what I'm trying to accomplish. Any advice would be appreciated.
rust playpen for this code
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(&mut stuff);
println!("done");
}
fn helper(stuff: &mut Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}
&mut references are not safe to send to a non-scoped thread, because the thread may still run after the referenced data has been deallocated. Furthermore, after helper returns, the main thread would still be able to mutate stuff, and the spawned thread would also be able to mutate stuff indirectly, which is not allowed in Rust (there can only be one mutable alias for a variable).
Instead, the RwLock should own the data, rather than borrow it. This means helper should receive a Stuff rather than a &mut Stuff.
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(stuff);
println!("done");
}
fn helper(stuff: Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}