Cannot share Arc variables when spawning threads - multithreading

I have the following struct to represent the server object:
pub struct Server {
client_managers: Arc<ClientManager>,
listener: Option<TcpListener>,
}
Here is the code that receives a client's connection and handles it in a new thread:
fn serve(&self) {
for stream in self.listener.as_ref().unwrap().incoming() {
match stream {
Ok(stream) => {
let client_manager = &mut self.client_managers.clone();
// let client_manager = Arc.new(self.client_managers);
thread::spawn(move || {
client_manager.do_something();
});
}
Err(e) => {
println!("connection error: {}", e);
}
}
}
}
However, I get the following error when compiling:
error[E0716]: temporary value dropped while borrowed
--> server/src/server.rs:37:47
|
37 | let client_manager = &mut self.client_managers.clone();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ creates a temporary which is freed while still in use
38 | // let client_manager = Arc.new(self.client_managers);
39 | / thread::spawn(move || {
40 | | client_manager.nothing();
41 | | });
| |______________________- argument requires that borrow lasts for `'static`
42 | }
| - temporary value is freed at the end of this statement
I understood why this error happened. My question is:
1) I use Arc by following some tutorials online. (Example) But why do their examples work but not mine?
2) How can I fix this error in my situation? (I still want to share the object client_manager).

thread::spawn takes a closure that is 'static, meaning that it cannot borrow data from outside the thread. However, this line will clone the Arc and borrow it, and passing the borrowed reference into the thread:
let client_manager = &mut self.client_managers.clone();
thread::spawn(move || {
client_manager.do_something();
// ^-- client_manager is a `&mut Arc<_>` borrowed from outside the thread
});
Instead, what you want is to just clone the Arc, not borrow it in any way before its passed into the thread:
let client_manager = self.client_managers.clone();
thread::spawn(move || {
client_manager.do_something();
// ^-- client_manager is a `Arc<_>` owned by the new thread
});
The Arc has shared ownership over the value, so it is only destroyed once all Arc pointers referring to it falls out of scope, even across threads.

Related

Problem regarding borrowing references in Rust

I am trying to write a program in which one thread writes to a queue and another thread
reads from the queue
But I am facing an issue regarding accessing the 'queue' in the thread that reads the queue
Below is the code which is not compiling
use ::std::collections::VecDeque;
use notify::{Config, RecommendedWatcher, RecursiveMode, Watcher};
use std::cell::RefCell;
use std::path::{Path, PathBuf};
use std::thread;
use std::time::Duration;
fn main() {
//let path = std::env::args()
// .nth(1)
// .expect("Argument 1 needs to be a path");
//println!("watching {}", path);
let path = "c:\\testfolder";
if let Err(e) = watch(path) {
println!("error: {:?}", e)
}
}
fn process_queue(queue: &VecDeque<String>) -> () {}
fn watch<P: AsRef<Path>>(path: P) -> notify::Result<()> {
let (tx, rx) = std::sync::mpsc::channel();
// Automatically select the best implementation for your platform.
// You can also access each implementation directly e.g. INotifyWatcher.
let mut watcher = RecommendedWatcher::new(tx, Config::default())?;
// Add a path to be watched. All files and directories at that path and
// below will be monitored for changes.
let mut queue: VecDeque<String> = VecDeque::new();
thread::spawn(|| {
// everything in here runs
process_queue(&queue)
});
watcher.watch(path.as_ref(), RecursiveMode::Recursive)?;
for res in rx {
match res {
Ok(event) => {
println!("changed: {:?}", event.paths);
let os_str: String = String::from(event.paths[0].to_str().unwrap());
//let my_str: String = os_str.unwrap().to_str().unwrap();
//let s =os_str.into_os_string();
queue.push_back(os_str);
}
Err(e) => println!("watch error: {:?}", e),
}
}
Ok(())
}
The output from the Rust compiler
error[E0373]: closure may outlive the current function, but it borrows `queue`, which is owned by the current function
--> src\main.rs:43:19
|
43 | thread::spawn(|| {
| ^^ may outlive borrowed value `queue`
...
47 | process_queue(&queue)
| ----- `queue` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src\main.rs:43:5
|
43 | / thread::spawn(|| {
44 | |
45 | | // everything in here runs
46 | |
47 | | process_queue(&queue)
48 | |
49 | | });
| |______^
help: to force the closure to take ownership of `queue` (and any other referenced variables), use the `move` keyword
|
43 | thread::spawn(move || {
| ++++
error[E0502]: cannot borrow `queue` as mutable because it is also borrowed as immutable
--> src\main.rs:63:17
|
43 | thread::spawn(|| {
| - -- immutable borrow occurs here
| _____|
| |
44 | |
45 | | // everything in here runs
46 | |
47 | | process_queue(&queue)
| | ----- first borrow occurs due to use of `queue` in closure
48 | |
49 | | });
| |______- argument requires that `queue` is borrowed for `'static`
...
63 | queue.push_back(os_str);
| ^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
From the errors I understand that the compiler does not allow both mutable and immutable references at the same time.
But I don't know how to implement what I am trying to do with these restrictions.
One way to solve this is by Box-ing the VecDeque so that you can pass a cloned reference to your process_queue function.
Using a Box allows you to allocate the VecDeque on the heap so that you can give your spawned thread a reference to the Vec and also still mutate the queue in the current thread.
This would look like:
let mut queue = Box::new(VecDeque::new());
let queue_clone = queue.clone();
thread::spawn(|| {
// queue_clone is now moved into the fn closure and is
// not accessible to the code below
process_queue(queue_clone)
});
and you can update process_queue to accept the correct type:
fn process_queue(queue: Box<VecDeque<String>>) -> () { }
Note that with this implementation, process_queue only runs once when the thread is spawned, and if you want to have process_queue do something every time the queue is changed, following the advice of others to use something like Channels makes the most sense.
Thanks for all your responses
From all the responses I understand that using channels and moving the receiver loop to the other thread as suggested bu user4815162342
will be the best solution
I successfully implemented what I was trying to do using channels based on your suggestions.
The final working code is pasted below
use std::thread;
use std::time::Duration;
use notify::{RecommendedWatcher, RecursiveMode, Watcher, Config};
use std::path::Path;
use std::path::PathBuf;
//
fn main() {
//let path = std::env::args()
// .nth(1)
// .expect("Argument 1 needs to be a path");
//println!("watching {}", path);
let path="c:\\testfolder";
if let Err(e) = watch(path) {
println!("error: {:?}", e)
}
}
fn watch<P: AsRef<Path>>(path: P) -> notify::Result<()> {
let (tx, rx) = std::sync::mpsc::channel();
// Automatically select the best implementation for your platform.
// You can also access each implementation directly e.g. INotifyWatcher.
let mut watcher = RecommendedWatcher::new(tx, Config::default())?;
// Add a path to be watched. All files and directories at that path and
// below will be monitored for changes.
let handle=thread::spawn(move || {
// everything in here runs
for res in rx {
match res {
Ok(event) =>{
// println!("changed: {:?}", event.paths);
let os_str:String = String::from(event.paths[0].to_str().unwrap());
println!("file name: {}", os_str);
},
Err(e) => println!("watch error: {:?}", e),
}
}
});
watcher.watch(path.as_ref(), RecursiveMode::Recursive)?;
handle.join();
Ok(())
}
In your situation, using Rust's MPSC (multi-producer single-consumer, ie essentially a queue) would probably be the best. You could also create a variable that is shared between multiple thread using Arc and Mutex structs, but that would be way overkilled and can have a performance impact (only one can access the variable at any time).
Here is an example of a multi-threaded MPSC, I will let you adapt it to your infrastructure :
use std::{sync::mpsc, thread};
fn main() {
let (sender, receiver) = mpsc::channel();
let handle_1 = thread::spawn(|| {
thread_1(sender);
});
let handle_2 = thread::spawn(|| {
thread_2(receiver);
});
handle_1.join().unwrap();
handle_2.join().unwrap();
}
// the enum must have the Send trait (automatically implemented)
enum Instruction {
Print(String),
Exit
}
fn thread_1(sender: mpsc::Sender<Instruction>) {
sender.send(Instruction::Print("I".to_owned())).unwrap();
sender.send(Instruction::Print("like".to_owned())).unwrap();
sender.send(Instruction::Print("Rust".to_owned())).unwrap();
sender.send(Instruction::Print(".".to_owned())).unwrap();
sender.send(Instruction::Exit).unwrap();
}
fn thread_2(receiver: mpsc::Receiver<Instruction>) {
'global_loop: loop {
for received in receiver.recv() {
match received {
Instruction::Print(string) => print!("{} ", string),
Instruction::Exit => {
println!("");
break 'global_loop;
}
}
}
}
}

Why is the compiler saying the variable is dropped while still borrowed if the borrower is already dropped?

I have a question about lifetimes. I have the following code:
async move {
let futures = FuturesUnordered::new();
let subscriptions = database.get_subscribers_for(&msg).await?;
for notifier in notifiers.iter() {
futures.push(notifier.notify(&msg, &subscriptions));
}
let results = futures.collect::<Vec<Result<()>>>().await;
results
.into_iter()
.filter(Result::is_err)
.for_each(|result| error!("Error sending notification: {}", result.unwrap_err()));
Ok(())
}
Signature from nofity:
async fn notify(&self, msg: &Notification, subs: &[Subscriptions]) -> Result<()>
I'm getting an error saying subscriptions might not live long enough and that subscriptions can be still be borrowed when dropped.
How can it be if the collect from the futures take the value from the futures and then I call into_iter which consumes all entries?
So basically the futures will be dropped before subscriptions. Am I understanding it wrong?
If I set the code like this, forcing the futures to be in a separate block, it compiles:
async move {
let subscriptions = database.get_subscribers_for(&msg).await?;
{
let futures = FuturesUnordered::new();
for notifier in notifiers.iter() {
futures.push(notifier.notify(&msg, &subscriptions));
}
let results = futures.collect::<Vec<Result<()>>>().await;
results
.into_iter()
.filter(Result::is_err)
.for_each(|result| error!("Error sending notification: {}", result.unwrap_err()));
}
Ok(())
}
Compiler message:
error[E0597]: `subscriptions` does not live long enough
--> src/notification/mod.rs:57:52
|
57 | futures.push(notifier.notify(&msg, &subscriptions));
| ^^^^^^^^^^^^^^ borrowed value does not live long enough
...
65 | })
| -
| |
| `subscriptions` dropped here while still borrowed
| borrow might be used here, when `futures` is dropped and runs the `Drop` code for type `FuturesUnordered`
|
= note: values in a scope are dropped in the opposite order they are defined

Why does i got lifetime error in rust closure

I'm testing some rust wasm features, and have some problem with closures.
I'm implemented this function, which setup callback on button click event.
pub fn setup_click(&mut self) {
let mut clicks = 0;
let ws_cloned = self.websocket.clone();
let num_clicks_cloned = self.num_clicks.clone();
let notifications = Rc::new(RefCell::new(Notificator::new(
NotificationConfig::new_default(),
)));
let cb = move |_: Event| {
clicks += 1;
num_clicks_cloned
.borrow_mut()
.set_inner_html(clicks.to_string());
let mut map: Map<String, Value> = serde_json::Map::new();
map.insert("key".to_string(), Value::String(clicks.to_string()));
if let Ok(ws) = ws_cloned.clone().try_borrow_mut() {
ws.send_rpc(
String::from("click"),
Params::Map(map),
Box::new(|payload: String| {
notifications.clone().borrow_mut().display(
payload,
"Click success".to_string(),
"success".to_string(),
)
}),
);
}
};
self.click_button.add_event_listener("click", cb);
}
where third param of the ws.send rpc is
pub type RPCHandler = Box<dyn Fn(String) + 'static>;
and add_event_listener has this sugnature
pub fn add_event_listener<T>(&mut self, event_name: &str, handler: T)
where
T: 'static + FnMut(web_sys::Event),
{
let cb = Closure::wrap(Box::new(handler) as Box<dyn FnMut(_)>);
if let Some(el) = self.el.take() {
let el_et: EventTarget = el.into();
el_et
.add_event_listener_with_callback(event_name, cb.as_ref().unchecked_ref())
.unwrap();
cb.forget();
if let Ok(el) = el_et.dyn_into::<web_sys::Element>() {
self.el = Some(el);
}
}
}
When i try to compile the code i got life time error
--> src/test_click_btn.rs:46:21
|
35 | let cb = move |_: Event| {
| --------------- lifetime `'1` represents this closure's body
...
46 | / Box::new(|payload: String| {
47 | | notifications.clone().borrow_mut().display(
48 | | payload,
49 | | "Click success".to_string(),
50 | | "success".to_string(),
51 | | )
52 | | }),
| |______________________^ cast requires that `'1` must outlive `'static`
|
= note: closure implements `FnMut`, so references to captured variables can't escape the closure```
I see that notifications not live long enough, but can't understand how to fix this error)
There's no guarantee in this code that the closure passed to send_rpc will last no longer than the event callback closure. Therefore, it needs to be made a move closure too, so that it can live independently rather than borrowing from the event handler closure.
Conveniently, you already have notifications wrapped in Rc, which is just what you need, but you've performed the clone in the wrong place. This line
notifications.clone().borrow_mut().display(
performs a clone and dereferences it immediately, so it's redundant. Instead, you need to clone it before creating the closure so that the closure (now move) can own it:
let notifications = notifications.clone(); // create a clone that will be moved into the closure
ws.send_rpc(
String::from("click"),
Params::Map(map),
Box::new(move |payload: String| { // now a move closure
notifications.borrow_mut().display( // no clone here
...

Lifetime issues in rust: borrowed value does not live long enough

I have reproduced my problem in the short code below.
Problem: The inner thread uses reference of variable v from the outer thread. The rust compiler throws an error because "technically" the outer thread could terminate before the inner thread and hence inner thread could loose access to variable v. However in the code below that clearly cannot happen.
Question: How shall I change this code so that it complies while maintaining the same functionality?
fn main() { //outer thread
let v = vec![0, 1];
let test = Test { v: &v }; //inner_thread
std::thread::spawn(move || test.print());
loop {
// this thread will never die because it will never leave this loop
}
}
pub struct Test<'a> {
v: &'a Vec<u32>,
}
impl<'a> Test<'a> {
fn print(&self) {
println!("{:?}", self.v);
}
}
error[E0597]: `v` does not live long enough
--> src/main.rs:3:26
|
3 | let test = Test { v: &v }; //inner_thread
| ^^ borrowed value does not live long enough
4 | std::thread::spawn(move || test.print());
| ---------------------------------------- argument requires that `v` is borrowed for `'static`
...
8 | }
| - `v` dropped here while still borrowed
The obvious solution would be to have Test own the vector instead of just have a reference.
But if you really need to borrow the value in the thread (probably because you want to use it after end of execution), then you may use crossbeam's scope:
let v = vec![0, 1];
let test = Test { v: &v }; //inner_thread
crossbeam::thread::scope(|scope| {
scope.spawn(|_| test.print());
}).unwrap();

Accessing a method of self inside a thread in Rust

I want to propagate the self struct object into a thread and then call its time_tick() method for increasing the HMS time.
pub fn start(&mut self) {
self.acti = true; // the time tick is activated now...
thread::spawn(move || {
let local_self: *mut Self = self; // this self live in the thread
loop {
thread::sleep(Duration::from_secs(1)); // wait for 1 sec
if (*local_self).acti == true { (*local_self).time_tick(); }
(*local_self).print_time(); // for debug
}
});
}
I get the error message:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/hmstimer/mod.rs:42:17
|
42 | thread::spawn(move || {
| _______________________^
43 | | let local_self: *mut Self = self; // this self live in the thread
44 | | loop {
45 | | thread::sleep(Duration::from_secs(1)); // wait for 1 sec
... |
48 | | }
49 | | });
| |_________^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 40:2...
--> src/hmstimer/mod.rs:40:2
|
40 | pub fn start(&mut self) {
| _____^
41 | | self.acti = true; // the time tick is activated now...
42 | | thread::spawn(move || {
43 | | let local_self: *mut Self = self; // this self live in the thread
... |
49 | | });
50 | | }
| |_____^
= note: ...so that the types are compatible:
expected &mut hmstimer::HMSTimer
found &mut hmstimer::HMSTimer
= note: but, the lifetime must be valid for the static lifetime...
note: ...so that the type `[closure#src/hmstimer/mod.rs:42:17: 49:7 self:&mut hmstimer::HMSTimer]` will meet its required lifetime bounds
But it seems that the about method is inappropriate. What is the best practice for doing the task?
You can't pass a closure that captures a mutable reference to thread::spawn. thread::spawn needs the function to be 'static, which means that either it captures no borrows, or that all borrows are 'static. That's because the thread can continue running after the referent has been dropped.
If you don't need to use self in the original thread after calling start, then you can just pass self by value.
pub fn start(self) {
self.acti = true;
thread::spawn(move || {
loop {
thread::sleep(Duration::from_secs(1));
if self.acti == true { self.time_tick(); }
self.print_time();
}
});
}
Otherwise, you'll need to use Arc to get the two threads to share ownership of Self, as well as Mutex or RwLock to synchronize reads and writes across threads.
// note: this is not a method anymore;
// invoke as `HMSTimer::start(arc.clone());`
pub fn start(this: Arc<Mutex<Self>>) {
this.lock().expect("mutex is poisoned").acti = true;
thread::spawn(move || {
loop {
thread::sleep(Duration::from_secs(1));
let lock = this.lock().expect("mutex is poisoned");
if lock.acti == true { lock.time_tick(); }
lock.print_time();
// `lock` is dropped here, unlocking the mutex
}
});
}

Resources