I want to build a function that takes a HashMap reference as an argument. This HashMap should be shared between threads for read only access. The code example is very simple:
I insert some value into the HashMap, pass it to the function and want antoher thread to read that value. I get an Error that the borrowed value does not live long enough at line let exit_code = test(&m);. Why is this not working?
use std::thread;
use std::collections::HashMap;
use std::sync::{Arc, RwLock };
fn main(){
let mut m: HashMap<u32, f64> = HashMap::new();
m.insert(0, 0.1);
let exit_code = test(&m);
std::process::exit(exit_code);
}
fn test(m: &'static HashMap<u32, f64>) -> i32{
let map_lock = Arc::new(RwLock::new(m));
let read_thread = thread::spawn(move || {
if let Ok(r_guard) = map_lock.read(){
println!("{:?}", r_guard.get(&0).unwrap());
}
});
read_thread.join().unwrap();
return 0;
}
if I don't put the 'static in the function signature for the HashMap argument, Arc::new(RwLock::new(m)); doesn't work. How can I sovlve this problem?
A reference is not safe to share unless is 'static meaning that something will live for the extent of the program. Otherwise the compiler is not able to track the liveliness of the shared element.
You should wrap it outside of the function, and take ownership of an Arc:
use std::thread;
use std::collections::HashMap;
use std::sync::{Arc, RwLock };
fn main(){
let mut map = HashMap::new();
map.insert(0, 0.1);
let m = Arc::new(RwLock::new(map));
let exit_code = test(m);
std::process::exit(exit_code);
}
fn test(map_lock: Arc<RwLock<HashMap<u32, f64>>>) -> i32 {
let read_thread = thread::spawn(move || {
if let Ok(r_guard) = map_lock.read(){
println!("{:?}", r_guard.get(&0).unwrap());
}
});
read_thread.join().unwrap();
return 0;
}
Playground
Related
I'm trying to use ndarray as an asynchronous process to do linear algebra and such.
I used Rust's tokio and ndarray to create the following code.
use std::sync::{Arc, Mutex};
use ndarray::prelude::*;
use futures::future::join_all;
fn print_type_of<T>(_: &T) {
println!("{}", std::any::type_name::<T>())
}
#[tokio::main]
async fn main() {
let db = Arc::new(Mutex::new(array![0,0,0,0,0,0,0,0]));
let mut handels = vec![];
for i in 0..8 {
let db = db.clone();
let unchange_array = unchange_array.clone();
handels.push(tokio::spawn(async move{
print(i, db).await;
}));
}
join_all(handels).await;
let array = Arc::try_unwrap(db).unwrap();
let array = array.lock().unwrap();
print_type_of(&array); // -> std::sync::mutex::MutexGuard<ndarray::ArrayBase<ndarray::data_repr::OwnedRepr<u32>, ndarray::dimension::dim::Dim<[usize; 1]>>>
}
async fn print(i: u32, db: Arc<Mutex<Array1<u32>>>) {
let unchange = unchange.to_owned();
let mut tmp = 0;
// time-consuming process
for k in 0..100000000 {
tmp = k;
}
tmp += i;
let mut db = db.lock().unwrap();
db.fill(i);
println!("{:?}", unchange);
print_type_of(&db);
}
I would like to change the data std::sync::mutex::MutexGuard<ndarray::ArrayBase<OwnedRepr<u32>, Dim<[usize; 1]>>>
to ndarray::ArrayBase<OwnedRepr<u32>, Dim<[usize; 1]>>.
How can I do this?
You can't. That's the whole point of MutexGuard: if you could take the data out of the MutexGuard, then you would be able to make a reference that can be accessed without locking the mutex, defeating the whole purpose of having a mutex in the first place.
Depending on what you really want to do, one of the following solutions might apply to you:
Most of the time, you don't need to take the data out of the mutex: MutexGuard<T> implements Deref<Target=T> and DerefMut<Target=T>, so you can use the MutexGuard everywhere you would use a &T or a &mut T. Note that if you change your code to call print_type_of(&*array) instead of print_type_of(&array), it will print the inner type.
If you really need to, you can take the data out of the Mutex itself (but not the MutexGuard) with into_inner, which consumes the mutex, ensuring that no one else can ever access it:
let array = Arc::try_unwrap(db).unwrap();
let array = array.into_inner().unwrap();
print_type_of(&array); // -> ndarray::ArrayBase<ndarray::data_repr::OwnedRepr<u32>, ndarray::dimension::dim::Dim<[usize; 1]>>
In my browser application, two closures access data stored in a Rc<RefCell<T>>. One closure mutably borrows the data, while the other immutably borrows it. The two closures are invoked independently of one another, and this will occasionally result in a BorrowError or BorrowMutError.
Here is my attempt at an MWE, though it uses a future to artificially inflate the likelihood of the error occurring:
use std::cell::RefCell;
use std::future::Future;
use std::pin::Pin;
use std::rc::Rc;
use std::task::{Context, Poll, Waker};
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsValue;
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(js_namespace = console)]
pub fn log(s: &str);
#[wasm_bindgen(js_name = setTimeout)]
fn set_timeout(closure: &Closure<dyn FnMut()>, millis: u32) -> i32;
#[wasm_bindgen(js_name = setInterval)]
fn set_interval(closure: &Closure<dyn FnMut()>, millis: u32) -> i32;
}
pub struct Counter(u32);
#[wasm_bindgen(start)]
pub async fn main() -> Result<(), JsValue> {
console_error_panic_hook::set_once();
let counter = Rc::new(RefCell::new(Counter(0)));
let counter_clone = counter.clone();
let log_closure = Closure::wrap(Box::new(move || {
let c = counter_clone.borrow();
log(&c.0.to_string());
}) as Box<dyn FnMut()>);
set_interval(&log_closure, 1000);
log_closure.forget();
let counter_clone = counter.clone();
let increment_closure = Closure::wrap(Box::new(move || {
let counter_clone = counter_clone.clone();
wasm_bindgen_futures::spawn_local(async move {
let mut c = counter_clone.borrow_mut();
// In reality this future would be replaced by some other
// time-consuming operation manipulating the borrowed data
SleepFuture::new(5000).await;
c.0 += 1;
});
}) as Box<dyn FnMut()>);
set_timeout(&increment_closure, 3000);
increment_closure.forget();
Ok(())
}
struct SleepSharedState {
waker: Option<Waker>,
completed: bool,
closure: Option<Closure<dyn FnMut()>>,
}
struct SleepFuture {
shared_state: Rc<RefCell<SleepSharedState>>,
}
impl Future for SleepFuture {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let mut shared_state = self.shared_state.borrow_mut();
if shared_state.completed {
Poll::Ready(())
} else {
shared_state.waker = Some(cx.waker().clone());
Poll::Pending
}
}
}
impl SleepFuture {
fn new(duration: u32) -> Self {
let shared_state = Rc::new(RefCell::new(SleepSharedState {
waker: None,
completed: false,
closure: None,
}));
let state_clone = shared_state.clone();
let closure = Closure::wrap(Box::new(move || {
let mut state = state_clone.borrow_mut();
state.completed = true;
if let Some(waker) = state.waker.take() {
waker.wake();
}
}) as Box<dyn FnMut()>);
set_timeout(&closure, duration);
shared_state.borrow_mut().closure = Some(closure);
SleepFuture { shared_state }
}
}
panicked at 'already mutably borrowed: BorrowError'
The error makes sense, but how should I go about resolving it?
My current solution is to have the closures use try_borrow or try_borrow_mut, and if unsuccessful, use setTimeout for an arbitrary amount of time before attempting to borrow again.
Think about this problem independently of Rust's borrow semantics. You have a long-running operation that's updating some shared state.
How would you do it if you were using threads? You would put the shared state behind a lock. RefCell is like a lock except that you can't block on unlocking it — but you can emulate blocking by using some kind of message-passing to wake up the reader.
How would you do it if you were using pure JavaScript? You don't automatically have anything like RefCell, so either:
The state can be safely read while the operation is still ongoing (in a concurrency-not-parallelism sense): in this case, emulate that by not holding a single RefMut (result of borrow_mut()) alive across an await boundary.
The state is not safe to be read: you'd either write something lock-like as described above, or perhaps arrange so that it's only written once when the operation is done, and until then, the long-running operation has its own private state not shared with the rest of the application (so there can be no BorrowError conflicts).
Think about what your application actually needs and pick a suitable solution. Implementing any of these solutions will most likely involve having additional interior-mutable objects used for communication.
I would like to be able to construct objects that contain immutable references to a mutable buffer object. The following code does not work but illustrates my use case, is there an idiomatic Rust method for handling this?
#[derive(Debug)]
struct Parser<'a> {
buffer: &'a String
}
fn main() {
let mut source = String::from("Peter");
let buffer = &source;
let parser = Parser { buffer };
// How can I legally change source?
source.push_str(" Pan");
println!("{:?}", parser);
}
The golden rule of the rust borrow checker is: Only one writer OR multiple readers can access a resource at a time. This ensures that algorithms are safe to run in multiple threads.
You breach this rule here:
#[derive(Debug)]
struct Parser<'a> {
buffer: &'a String
}
fn main() {
// mutable access begins here
let mut source = String::from("Peter");
// immutable access begins here
let buffer = &source;
let parser = Parser { buffer };
source.push_str(" Pan");
println!("{:?}", parser);
// Both immutable and mutable access end here
}
If you are sure that your program doesn't actively access resources at the same time mutably and immutably, you can move the check from compile time to run time by wrapping your resource in a RefCell:
use std::cell::RefCell;
use std::rc::Rc;
#[derive(Debug)]
struct Parser {
buffer: Rc<RefCell<String>>
}
fn main() {
let source = Rc::new(RefCell::new(String::from("Peter")));
let parser = Parser { buffer: source.clone() };
source.borrow_mut().push_str(" Pan");
println!("{:?}", parser);
}
If you plan on passing your resource around threads, you can use an RwLock to block the thread until the resource is available:
use std::sync::{RwLock, Arc};
#[derive(Debug)]
struct Parser {
buffer: Arc<RwLock<String>>
}
fn main() {
let source = Arc::new(RwLock::new(String::from("Peter")));
let parser = Parser { buffer: source.clone() };
source.write().unwrap().push_str(" Pan");
println!("{:?}", parser);
}
On another note, you should prefer &str over &String
It's hard to tell what exactly you want to achieve by mutating the source; I would assume you don't want it to happen while the parser is doing its work? You can always try (depending on your specific use case) to separate the immutable from the mutable with an extra scope:
fn main() {
let mut source = String::from("Peter");
{
let buffer = &source;
let parser = Parser { buffer };
println!("{:?}", parser);
}
source.push_str(" Pan");
}
If you don't want to use RefCell, unsafe (or to simply keep a mutable reference to source in Parser and use that), I'm afraid it doesn't get better than plain refactoring.
To elaborate on how this can be done unsafely, what you've described can be achieved by using a raw const pointer to avoid the borrowing rules, which of course is inherently unsafe, as the very concept of what you've described is pretty unsafe. There are ways to make it safer though, should you choose this path. But I would probably default to using an Arc<RwLock> or Arc<Mutex> should safety be important.
use std::fmt::{self, Display};
#[derive(Debug)]
struct Parser {
buffer: *const String
}
impl Display for Parser {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let buffer = unsafe { &*self.buffer };
write!(f, "{}", buffer)
}
}
fn main() {
let mut source = String::from("Peter");
let buffer = &source as *const String;
let parser = Parser { buffer };
source.push_str(" Pan");
println!("{}", parser);
}
Suppose I have a HashMap and I want to get a mutable reference to an entry, or if that entry does not exist I want a mutable reference to a new object, how can I do it? I've tried using unwrap_or(), something like this:
fn foo() {
let mut map: HashMap<&str, Vec<&str>> = HashMap::new();
let mut ref = map.get_mut("whatever").unwrap_or( &mut Vec::<&str>::new() );
// Modify ref.
}
But that doesn't work because the lifetime of the Vec isn't long enough. Is there any way to tell Rust that I want the returned Vec to have the same lifetime as foo()? I mean there is this obvious solution but I feel like there should be a better way:
fn foo() {
let mut map: HashMap<&str, Vec<&str>> = HashMap::new();
let mut dummy: Vec<&str> = Vec::new();
let mut ref = map.get_mut("whatever").unwrap_or( &dummy );
// Modify ref.
}
As mentioned by Shepmaster, here is an example of using the entry pattern. It seems verbose at first, but this avoids allocating an array you might not use unless you need it. I'm sure you could make a generic function around this to cut down on the chatter :)
use std::collections::HashMap;
use std::collections::hash_map::Entry::{Occupied, Vacant};
fn foo() {
let mut map = HashMap::<&str, Vec<&str>>::new();
let mut result = match map.entry("whatever") {
Vacant(entry) => entry.insert(Vec::new()),
Occupied(entry) => entry.into_mut(),
};
// Do the work
result.push("One thing");
result.push("Then another");
}
This can also be shortened to or_insert as I just discovered!
use std::collections::HashMap;
fn foo() {
let mut map = HashMap::<&str, Vec<&str>>::new();
let mut result = map.entry("whatever").or_insert(Vec::new());
// Do the work
result.push("One thing");
result.push("Then another");
}
If you want to add your dummy into the map, then this is a duplicate of How to properly use HashMap::entry? or Want to add to HashMap using pattern match, get borrow mutable more than once at a time (or any question about the entry API).
If you don't want to add it, then your code is fine, you just need to follow the compiler error messages to fix it. You are trying to use a keyword as an identifier (ref), and you need to get a mutable reference to dummy (& mut dummy):
use std::collections::HashMap;
fn foo() {
let mut map: HashMap<&str, Vec<&str>> = HashMap::new();
let mut dummy: Vec<&str> = Vec::new();
let f = map.get_mut("whatever").unwrap_or( &mut dummy );
}
fn main() {}
I'm trying to share a RwLock amongst several threads without using scoped threads but I can't figure out how to get the lifetimes correct. I assume that this is possible (what's the point of RwLocks otherwise?) but I can't find any examples of it.
Here is a toy example of what I'm trying to accomplish. Any advice would be appreciated.
rust playpen for this code
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(&mut stuff);
println!("done");
}
fn helper(stuff: &mut Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}
&mut references are not safe to send to a non-scoped thread, because the thread may still run after the referenced data has been deallocated. Furthermore, after helper returns, the main thread would still be able to mutate stuff, and the spawned thread would also be able to mutate stuff indirectly, which is not allowed in Rust (there can only be one mutable alias for a variable).
Instead, the RwLock should own the data, rather than borrow it. This means helper should receive a Stuff rather than a &mut Stuff.
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(stuff);
println!("done");
}
fn helper(stuff: Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}