Can you specify a non-static lifetime for threads? [duplicate] - multithreading

This question already has answers here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Thread references require static lifetime?
(1 answer)
How do I use static lifetimes with threads?
(2 answers)
How can I send non-static data to a thread in Rust and is it needed in this example?
(1 answer)
Closed 3 years ago.
Here's a toy example of my problem:
use std::sync::{Arc, Mutex};
fn operate_in_chunks(vec: &mut Vec<f32>) {
let chunk_size = 10;
let mutex_vec: Arc<Mutex<&mut Vec<f32>>> = Arc::new(Mutex::new(vec));
let handles = Vec::new();
for chunk in 0..vec.len() / chunk_size {
handles.push(std::thread::spawn(move || {
operate(mutex_vec, chunk);
}));
}
for i in 0..handles.len() {
handles[i].join().unwrap();
}
}
fn operate(mutex_vec: Arc<Mutex<&mut Vec<f32>>>, chunk: usize) {}
I'd like to do some work on the passed-in struct, split among a number of threads, and then join them all back together before returning.
The error I get says:
error[E0621]: explicit lifetime required in the type of `vec`
--> src/lib.rs:10:22
|
3 | fn operate_in_chunks(vec: &mut Vec<f32>) {
| ------------- help: add explicit lifetime `'static` to the type of `vec`: `&'static mut std::vec::Vec<f32>`
...
10 | handles.push(std::thread::spawn(move || {
| ^^^^^^^^^^^^^^^^^^ lifetime `'static` required
I understand what it's complaining about: if the threads may have a 'static lifetime, and they reference vec, vec must have a 'static lifetime. However, my use-case should in theory be possible: I want to guarantee that the threads don't have a 'static lifetime, as they're all joined before the function returns, in which case I shouldn't need a 'static' lifetime onvec`.
Does Rust have a way of articulating this - unifying the lifetime of the threads with the lifetime of vec - or are all threads always 'static no matter what?

Rust actually used to allow scoped threads, which allow non-static lifetimes for any data passed. However, the API was found to be unsound and they were removed around two years ago. Fortunately, crossbeam, a crate, implements scoped threads with a different API allowing you to use this functionality safely. A sample from crossbeam's documentation is here:
let array = [1, 2, 3];
crossbeam::scope(|scope| {
for i in &array {
scope.spawn(move || {
println!("element: {}", i);
});
}
});

Related

Share function reference between threads in Rust [duplicate]

This question already has answers here:
Sending trait objects between threads in Rust
(1 answer)
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed 3 years ago.
I want to share a function reference between threads but the Rust compiler says `dyn for<'r> std::ops::Fn(&'r std::string::String) -> std::string::String` cannot be shared between threads safely. I'm well informed about Send, Sync, and Arc<T> when sharing "regular" values between threads but in this case I can't understand the problem. A function has a static address during the runtime of the program, therefore I can't see a problem here.
How can I make this work?
fn main() {
// pass a function..
do_sth_multithreaded(&append_a);
do_sth_multithreaded(&identity);
}
fn append_a(string: &String) -> String {
let mut string = String::from(string);
string.push('a');
string
}
fn identity(string: &String) -> String {
String::from(string)
}
fn do_sth_multithreaded(transform_fn: &dyn Fn(&String) -> String) {
for i in 0..4 {
let string = format!("{}", i);
thread::spawn(move || {
println!("Thread {}: {}", i, transform_fn(&string))
});
}
}
A function has a static address during the runtime of the program, therefore I can't see a problem here.
That's nice for functions, but you're passing a &dyn Fn, and that could just as well be a closure or (in unstable Rust) a custom object implementing that trait. And this object might not have a static address. So you can't guarantee that the object will outlive the threads you spawn.
But that's not even what the compiler is complaining about (yet!). It's actually complaining that it doesn't know whether you're allowed to access the Fn from another thread. Again, not relevant for function pointers, but relevant for closures.
Here's a signature that works for your example:
fn do_sth_multithreaded(transform_fn: &'static (dyn Fn(&String) -> String + Sync))
Note the 'static lifetime bound and the Sync bound.
But while the static lifetime works for this case, it probably means you can't ever send closures. To make that work, you need to use a scoped thread system (for example, from the crossbeam crate) to make sure do_sth_multithreaded waits for the threads to finish before returning. Then you can relax the static lifetime bound.

Send FnMut to other thread by wrapping it into a Weak<Mutex<FnMut>> complains about missing Send [duplicate]

This question already has answers here:
Sending trait objects between threads in Rust
(1 answer)
Sharing a struct with trait objects as properties across threads
(1 answer)
Closed 3 years ago.
I'm trying to write a simple Observer-pattern-ish thing with events and callbacks. The idea is that when the thing owning the mutating function dies (struct Obj {cb: Arc<Mutex<FnMut>>, .. }), it drops the Arc. After that, any Weak references will be (effectively) dead and can be detected on access. I was ALSO hoping that this would decouple the FnMut from Thread's 'static lifetime. I might need to wrap the whole thing in another Arc<Rwlock<T>> to prevent the Weak references from doing thread unsafe stuff, but that's a different problem.
The closest solution I've gotten was this: How can I send a function to another thread?
However, I seem to have the Arc<Mutex<T>> and adding lifetimes (I may have done that wrong, though) didn't seem to help. I'm kind of lost as to what is actually wrong.
I wrote a minimal example:
use std::{
collections::HashMap,
sync::{
mpsc::{self, Receiver},
Arc, Mutex, Weak,
},
thread::{self, JoinHandle},
};
struct Callback {
cb_w: Weak<Mutex<FnMut()>>,
}
type FnMapping = Arc<HashMap<String, Callback>>;
fn start_thread<'a>(rx: Receiver<String>, mapping: FnMapping) -> JoinHandle<()> {
thread::spawn(move || {
match rx.recv() {
Ok(v) => {
if let Some(o) = mapping.get(&v) {
match o.cb_w.upgrade() {
Some(cb_m) => match cb_m.lock() {
Ok(cb_lg) => (*cb_lg)(),
Err(e) => (),
},
None => { /* TODO owner is gone, mark for delete */ }
}
}
}
Err(e) => (),
}
})
}
fn main() {
let mapping: FnMapping = Arc::new(HashMap::new());
let (tx, rx) = mpsc::channel();
drop(tx);
start_thread(rx, mapping)
.join()
.expect("Could not join thread -- failed to terminate?");
println!("Leaving the test bay.");
}
This fails to compile with the following error:
error[E0277]: `(dyn std::ops::FnMut() + 'static)` cannot be sent between threads safely
--> src/main.rs:17:5
|
17 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `(dyn std::ops::FnMut() + 'static)` cannot be sent between threads safely
|
= help: the trait `std::marker::Send` is not implemented for `(dyn std::ops::FnMut() + 'static)`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Mutex<(dyn std::ops::FnMut() + 'static)>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Weak<std::sync::Mutex<(dyn std::ops::FnMut() + 'static)>>`
= note: required because it appears within the type `Callback`
= note: required because it appears within the type `(std::string::String, Callback)`
= note: required because it appears within the type `std::marker::PhantomData<(std::string::String, Callback)>`
= note: required because it appears within the type `std::collections::hash::table::RawTable<std::string::String, Callback>`
= note: required because it appears within the type `std::collections::HashMap<std::string::String, Callback>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::collections::HashMap<std::string::String, Callback>>`
= note: required because it appears within the type `[closure#src/main.rs:17:19: 32:6 rx:std::sync::mpsc::Receiver<std::string::String>, mapping:std::sync::Arc<std::collections::HashMap<std::string::String, Callback>>]`
= note: required by `std::thread::spawn`

Is it possible to call a member closure from another member closure? [duplicate]

This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Borrowing references to attributes in a struct
(2 answers)
Passing mutable self reference to method of owned object
(2 answers)
How to call method with &mut self from inside another method of the same type?
(2 answers)
Closed 4 years ago.
I'm working with callbacks on my struct, using boxed closures. However, I haven't been able to figure out how to call one closure from another:
struct MyStruct {
first: Box<Fn()>,
second: Box<Fn()>,
}
impl MyStruct {
fn method(&mut self) {
self.second = Box::new(|| {
// How can I make this work?
(self.first)();
});
}
}
fn main() {
let _ = MyStruct {
first: Box::new(|| {}),
second: Box::new(|| {}),
};
}
This gives:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:8:32
|
8 | self.second = Box::new(|| {
| ________________________________^
9 | | // How can I make this work?
10 | | (self.first)();
11 | | });
| |_________^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 7:5...
--> src/main.rs:7:5
|
7 | / fn method(&mut self) {
8 | | self.second = Box::new(|| {
9 | | // How can I make this work?
10 | | (self.first)();
11 | | });
12 | | }
| |_____^
= note: ...so that the types are compatible:
expected &&mut MyStruct
found &&mut MyStruct
= note: but, the lifetime must be valid for the static lifetime...
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() + 'static)>
found std::boxed::Box<dyn std::ops::Fn()>
I'm not quite sure what that means in this context.
I understand that the borrowing rules are the cause of the error, but is there any legal way in Rust to get the same effect of what I'm going for?
I'm not sure if it makes sense just from this why I would want to do this. It's a minimal reproduction. I can provide a bigger example, but it's much more involved.
Here is a partial solution to your problem: Do not take Box<Fn()>, but use Box<Fn(&MyStruct)> instead, i.e. pass the "self" explicitly to the closures.
However, as far as I can see, this only allows the closures to take &MyStruct (as opposed to &mut MyStruct), i.e. the closures cannot modify the given MyStruct (which may or may not be enough for your use case).
struct MyStruct {
first: Box<Fn(&MyStruct)>,
second: Box<Fn(&MyStruct)>,
}
impl MyStruct {
fn method(&mut self) {
self.second = Box::new(|self2: &MyStruct| {
(self2.first)(self2)
});
}
}
fn main() {
let _ = MyStruct {
first: Box::new(|_self_ignored| {}),
second: Box::new(|_self_ignored| {}),
};
}
If the closures should modify MyStruct, you run into problems with (self2.first)(self2), since this would then borrow self2 twice, once of them mutably. You could possibly get around this by swapping out first/second temporarily, but this way, you would need to be terribly careful whenever you call first/second, since it could at any time be impossible to call first/second.
There may be unsafe ways around this. That said, I suppose (or at least, I hope) there is a better way to implement what you are trying to achieve.

Rust Lifetime Issue with Threads [duplicate]

This question already has answers here:
Lifetime troubles sharing references between threads
(1 answer)
How do I use static lifetimes with threads?
(2 answers)
Thread references require static lifetime?
(1 answer)
Cannot call a function in a spawned thread because it "does not fulfill the required lifetime"
(2 answers)
"the type does not fulfill the required lifetime" when using a method in a thread
(1 answer)
Closed 4 years ago.
Here is the code:
use std::collections::HashMap;
use std::thread;
type MAP = HashMap<String, String>;
fn handle_n_times(count: i32, map: &mut MAP) {
for i in 0..count {
thread::spawn(move || {
map.insert(format!("key-{}", i), format!("value-{}", i));
});
}
}
fn main() {
let mut map: MAP = MAP::new();
handle_n_times(5, &mut map);
}
I cannot compile:
error[E0621]: explicit lifetime required in the type of `map`
--> src/main.rs:8:9
|
6 | fn handle_n_times(count: i32, map: &mut MAP) {
| -------- help: add explicit lifetime `'static` to the type of `map`: `&'static mut std::collections::HashMap<std::string::String, std::string::String>`
7 | for i in 0..count {
8 | thread::spawn(move || {
| ^^^^^^^^^^^^^ lifetime `'static` required
The hint it provides (add a &'static) is not helpful.
Question 1: How to make this code work?
Question 2: After 1 solved, I want to use std::sync::Arc to make map thread-safe, what's the Rust way to do it?
Question 1:
It is not thread safe to share variables into new threads. In Rust you can not compile this code. You can make the code work via using Arc<Mutex<Map>>
Question 2:
You need to change method signature as follows:
handle_n_times(count: i32, arc_map: Arc<Mutex<Map>>)
In case you are moving ownership in your thread::spawn, you need to clone Arc value just before sending it to other thread with
let clone_arc = arc_map.clone();
When you want to insert an item to the map you need to get reference from the Arc<Mutex> via lock and unwrap:
let map = clone_arc.lock().unwrap();
After that you can call map.insert with no explicit lifetime requirements.
Here is the full working code as an example

How to avoid mutex borrowing problems when using it's guard

I want my method of struct to perform in a synchronized way. I wanted to do this by using Mutex (Playground):
use std::sync::Mutex;
use std::collections::BTreeMap;
pub struct A {
map: BTreeMap<String, String>,
mutex: Mutex<()>,
}
impl A {
pub fn new() -> A {
A {
map: BTreeMap::new(),
mutex: Mutex::new(()),
}
}
}
impl A {
fn synchronized_call(&mut self) {
let mutex_guard_res = self.mutex.try_lock();
if mutex_guard_res.is_err() {
return
}
let mut _mutex_guard = mutex_guard_res.unwrap(); // safe because of check above
let mut lambda = |text: String| {
let _ = self.map.insert("hello".to_owned(),
"d".to_owned());
};
lambda("dd".to_owned());
}
}
Error message:
error[E0500]: closure requires unique access to `self` but `self.mutex` is already borrowed
--> <anon>:23:26
|
18 | let mutex_guard_res = self.mutex.try_lock();
| ---------- borrow occurs here
...
23 | let mut lambda = |text: String| {
| ^^^^^^^^^^^^^^ closure construction occurs here
24 | if let Some(m) = self.map.get(&text) {
| ---- borrow occurs due to use of `self` in closure
...
31 | }
| - borrow ends here
As I understand when we borrow anything from the struct we are unable to use other struct's fields till our borrow is finished. But how can I do method synchronization then?
The closure needs a mutable reference to the self.map in order to insert something into it. But closure capturing works with whole bindings only. This means, that if you say self.map, the closure attempts to capture self, not self.map. And self can't be mutably borrowed/captured, because parts of self are already immutably borrowed.
We can solve this closure-capturing problem by introducing a new binding for the map alone such that the closure is able to capture it (Playground):
let mm = &mut self.map;
let mut lambda = |text: String| {
let _ = mm.insert("hello".to_owned(), text);
};
lambda("dd".to_owned());
However, there is something you overlooked: since synchronized_call() accepts &mut self, you don't need the mutex! Why? Mutable references are also called exclusive references, because the compiler can assure at compile time that there is only one such mutable reference at any given time.
Therefore you statically know, that there is at most one instance of synchronized_call() running on one specific object at any given time, if the function is not recursive (calls itself).
If you have mutable access to a mutex, you know that the mutex is unlocked. See the Mutex::get_mut() method for more explanation. Isn't that amazing?
Rust mutexes do not work the way you are trying to use them. In Rust, a mutex protects specific data relying on the borrow-checking mechanism used elsewhere in the language. As a consequence, declaring a field Mutex<()> doesn't make sense, because it is protecting read-write access to the () unit object that has no values to mutate.
As Lukas explained, your call_synchronized as declared doesn't need to do synchronization because its signature already requests an exclusive (mutable) reference to self, which prevents it from being invoked from multiple threads on the same object. In other words, you need to change the signature of call_synchronized because the current one does not match the functionality it is intended to provide.
call_synchronized needs to accept a shared reference to self, which will signal to Rust that it can be called from multiple threads in the first place. Inside call_synchronized a call to Mutex::lock will simultaneously lock the mutex and provide a mutable reference to the underlying data, carefully scoped so that the lock is held for the duration of the reference:
use std::sync::Mutex;
use std::collections::BTreeMap;
pub struct A {
synced_map: Mutex<BTreeMap<String, String>>,
}
impl A {
pub fn new() -> A {
A {
synced_map: Mutex::new(BTreeMap::new()),
}
}
}
impl A {
fn synchronized_call(&self) {
let mut map = self.synced_map.lock().unwrap();
// omitting the lambda for brevity, but it would also work
// (as long as it refers to map rather than self.map)
map.insert("hello".to_owned(), "d".to_owned());
}
}

Resources