Rust Lifetime Issue with Threads [duplicate] - rust

This question already has answers here:
Lifetime troubles sharing references between threads
(1 answer)
How do I use static lifetimes with threads?
(2 answers)
Thread references require static lifetime?
(1 answer)
Cannot call a function in a spawned thread because it "does not fulfill the required lifetime"
(2 answers)
"the type does not fulfill the required lifetime" when using a method in a thread
(1 answer)
Closed 4 years ago.
Here is the code:
use std::collections::HashMap;
use std::thread;
type MAP = HashMap<String, String>;
fn handle_n_times(count: i32, map: &mut MAP) {
for i in 0..count {
thread::spawn(move || {
map.insert(format!("key-{}", i), format!("value-{}", i));
});
}
}
fn main() {
let mut map: MAP = MAP::new();
handle_n_times(5, &mut map);
}
I cannot compile:
error[E0621]: explicit lifetime required in the type of `map`
--> src/main.rs:8:9
|
6 | fn handle_n_times(count: i32, map: &mut MAP) {
| -------- help: add explicit lifetime `'static` to the type of `map`: `&'static mut std::collections::HashMap<std::string::String, std::string::String>`
7 | for i in 0..count {
8 | thread::spawn(move || {
| ^^^^^^^^^^^^^ lifetime `'static` required
The hint it provides (add a &'static) is not helpful.
Question 1: How to make this code work?
Question 2: After 1 solved, I want to use std::sync::Arc to make map thread-safe, what's the Rust way to do it?

Question 1:
It is not thread safe to share variables into new threads. In Rust you can not compile this code. You can make the code work via using Arc<Mutex<Map>>
Question 2:
You need to change method signature as follows:
handle_n_times(count: i32, arc_map: Arc<Mutex<Map>>)
In case you are moving ownership in your thread::spawn, you need to clone Arc value just before sending it to other thread with
let clone_arc = arc_map.clone();
When you want to insert an item to the map you need to get reference from the Arc<Mutex> via lock and unwrap:
let map = clone_arc.lock().unwrap();
After that you can call map.insert with no explicit lifetime requirements.
Here is the full working code as an example

Related

Share function reference between threads in Rust [duplicate]

This question already has answers here:
Sending trait objects between threads in Rust
(1 answer)
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed 3 years ago.
I want to share a function reference between threads but the Rust compiler says `dyn for<'r> std::ops::Fn(&'r std::string::String) -> std::string::String` cannot be shared between threads safely. I'm well informed about Send, Sync, and Arc<T> when sharing "regular" values between threads but in this case I can't understand the problem. A function has a static address during the runtime of the program, therefore I can't see a problem here.
How can I make this work?
fn main() {
// pass a function..
do_sth_multithreaded(&append_a);
do_sth_multithreaded(&identity);
}
fn append_a(string: &String) -> String {
let mut string = String::from(string);
string.push('a');
string
}
fn identity(string: &String) -> String {
String::from(string)
}
fn do_sth_multithreaded(transform_fn: &dyn Fn(&String) -> String) {
for i in 0..4 {
let string = format!("{}", i);
thread::spawn(move || {
println!("Thread {}: {}", i, transform_fn(&string))
});
}
}
A function has a static address during the runtime of the program, therefore I can't see a problem here.
That's nice for functions, but you're passing a &dyn Fn, and that could just as well be a closure or (in unstable Rust) a custom object implementing that trait. And this object might not have a static address. So you can't guarantee that the object will outlive the threads you spawn.
But that's not even what the compiler is complaining about (yet!). It's actually complaining that it doesn't know whether you're allowed to access the Fn from another thread. Again, not relevant for function pointers, but relevant for closures.
Here's a signature that works for your example:
fn do_sth_multithreaded(transform_fn: &'static (dyn Fn(&String) -> String + Sync))
Note the 'static lifetime bound and the Sync bound.
But while the static lifetime works for this case, it probably means you can't ever send closures. To make that work, you need to use a scoped thread system (for example, from the crossbeam crate) to make sure do_sth_multithreaded waits for the threads to finish before returning. Then you can relax the static lifetime bound.

Can you specify a non-static lifetime for threads? [duplicate]

This question already has answers here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Thread references require static lifetime?
(1 answer)
How do I use static lifetimes with threads?
(2 answers)
How can I send non-static data to a thread in Rust and is it needed in this example?
(1 answer)
Closed 3 years ago.
Here's a toy example of my problem:
use std::sync::{Arc, Mutex};
fn operate_in_chunks(vec: &mut Vec<f32>) {
let chunk_size = 10;
let mutex_vec: Arc<Mutex<&mut Vec<f32>>> = Arc::new(Mutex::new(vec));
let handles = Vec::new();
for chunk in 0..vec.len() / chunk_size {
handles.push(std::thread::spawn(move || {
operate(mutex_vec, chunk);
}));
}
for i in 0..handles.len() {
handles[i].join().unwrap();
}
}
fn operate(mutex_vec: Arc<Mutex<&mut Vec<f32>>>, chunk: usize) {}
I'd like to do some work on the passed-in struct, split among a number of threads, and then join them all back together before returning.
The error I get says:
error[E0621]: explicit lifetime required in the type of `vec`
--> src/lib.rs:10:22
|
3 | fn operate_in_chunks(vec: &mut Vec<f32>) {
| ------------- help: add explicit lifetime `'static` to the type of `vec`: `&'static mut std::vec::Vec<f32>`
...
10 | handles.push(std::thread::spawn(move || {
| ^^^^^^^^^^^^^^^^^^ lifetime `'static` required
I understand what it's complaining about: if the threads may have a 'static lifetime, and they reference vec, vec must have a 'static lifetime. However, my use-case should in theory be possible: I want to guarantee that the threads don't have a 'static lifetime, as they're all joined before the function returns, in which case I shouldn't need a 'static' lifetime onvec`.
Does Rust have a way of articulating this - unifying the lifetime of the threads with the lifetime of vec - or are all threads always 'static no matter what?
Rust actually used to allow scoped threads, which allow non-static lifetimes for any data passed. However, the API was found to be unsound and they were removed around two years ago. Fortunately, crossbeam, a crate, implements scoped threads with a different API allowing you to use this functionality safely. A sample from crossbeam's documentation is here:
let array = [1, 2, 3];
crossbeam::scope(|scope| {
for i in &array {
scope.spawn(move || {
println!("element: {}", i);
});
}
});

Is it possible to call a member closure from another member closure? [duplicate]

This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Borrowing references to attributes in a struct
(2 answers)
Passing mutable self reference to method of owned object
(2 answers)
How to call method with &mut self from inside another method of the same type?
(2 answers)
Closed 4 years ago.
I'm working with callbacks on my struct, using boxed closures. However, I haven't been able to figure out how to call one closure from another:
struct MyStruct {
first: Box<Fn()>,
second: Box<Fn()>,
}
impl MyStruct {
fn method(&mut self) {
self.second = Box::new(|| {
// How can I make this work?
(self.first)();
});
}
}
fn main() {
let _ = MyStruct {
first: Box::new(|| {}),
second: Box::new(|| {}),
};
}
This gives:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:8:32
|
8 | self.second = Box::new(|| {
| ________________________________^
9 | | // How can I make this work?
10 | | (self.first)();
11 | | });
| |_________^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 7:5...
--> src/main.rs:7:5
|
7 | / fn method(&mut self) {
8 | | self.second = Box::new(|| {
9 | | // How can I make this work?
10 | | (self.first)();
11 | | });
12 | | }
| |_____^
= note: ...so that the types are compatible:
expected &&mut MyStruct
found &&mut MyStruct
= note: but, the lifetime must be valid for the static lifetime...
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() + 'static)>
found std::boxed::Box<dyn std::ops::Fn()>
I'm not quite sure what that means in this context.
I understand that the borrowing rules are the cause of the error, but is there any legal way in Rust to get the same effect of what I'm going for?
I'm not sure if it makes sense just from this why I would want to do this. It's a minimal reproduction. I can provide a bigger example, but it's much more involved.
Here is a partial solution to your problem: Do not take Box<Fn()>, but use Box<Fn(&MyStruct)> instead, i.e. pass the "self" explicitly to the closures.
However, as far as I can see, this only allows the closures to take &MyStruct (as opposed to &mut MyStruct), i.e. the closures cannot modify the given MyStruct (which may or may not be enough for your use case).
struct MyStruct {
first: Box<Fn(&MyStruct)>,
second: Box<Fn(&MyStruct)>,
}
impl MyStruct {
fn method(&mut self) {
self.second = Box::new(|self2: &MyStruct| {
(self2.first)(self2)
});
}
}
fn main() {
let _ = MyStruct {
first: Box::new(|_self_ignored| {}),
second: Box::new(|_self_ignored| {}),
};
}
If the closures should modify MyStruct, you run into problems with (self2.first)(self2), since this would then borrow self2 twice, once of them mutably. You could possibly get around this by swapping out first/second temporarily, but this way, you would need to be terribly careful whenever you call first/second, since it could at any time be impossible to call first/second.
There may be unsafe ways around this. That said, I suppose (or at least, I hope) there is a better way to implement what you are trying to achieve.

How do I return a reference to an Option of a custom struct inside of a lazy-static Arc Mutex? [duplicate]

This question already has answers here:
How can I swap in a new value for a field in a mutable reference to a structure?
(2 answers)
How do I move out of a struct field that is an Option?
(1 answer)
Closed 4 years ago.
I'm experimenting with Rust's Arc<Mutex>, trying to use it with a struct that is not in the default examples, but rather a custom one that is lazily initialized:
use std::sync::{Arc, Mutex};
#[macro_use]
extern crate lazy_static; // lazy_static = "1.2.0"
pub struct FooBar {}
lazy_static! {
static ref FOO_BAR: Arc<Mutex<Option<FooBar>>> = Arc::new(Mutex::new(None));
}
pub fn unpack_foo_bar() {
let foo_bar_arc = Arc::clone(&FOO_BAR);
let foo_bar_mutex_result = foo_bar_arc.lock();
let foo_bar_mutex = foo_bar_mutex_result.unwrap();
let foo_bar = foo_bar_mutex.unwrap();
// do something
}
The static FOO_BAR variable is later initialized by replacing the content of the option.
The code above won't compile:
error[E0507]: cannot move out of borrowed content
--> src/lib.rs:15:19
|
15 | let foo_bar = foo_bar_mutex.unwrap();
| ^^^^^^^^^^^^^ cannot move out of borrowed content
It would if FooBar were replaced with e. g. u32. The code also doesn't compile with String, but that type has built-in methods for cloning, which my type may not necessarily have.
Short of using foo_bar_mutex.as_ref().unwrap(), what other options do I have, particularly if I'd like to abstract the extraction of the FooBar instance into a method, like this:
pub fn unpack_foo_bar() -> Option<FooBar> {
let foo_bar_arc = Arc::clone(&FOO_BAR);
let foo_bar_mutex_result = foo_bar_arc.lock();
let foo_bar_mutex = foo_bar_mutex_result.unwrap();
let foo_bar_option = *foo_bar_mutex;
foo_bar_option
}
In this case, the compiler throws almost the same error, which is
error[E0507]: cannot move out of borrowed content
--> src/main.rs:34:26
|
11 | let foo_bar_option = *foo_bar_mutex;
| ^^^^^^^^^^^^^^
| |
| cannot move out of borrowed content
| help: consider using a reference instead: `&*foo_bar_mutex`
I have a sense that this would be much simpler if FooBar were easily cloned, but in the real world scenario I'm basing this example on FooBar has a field that is an instance of a third party library object that does not derive the Clone trait.
Using Option::take would only make the contained variable usable once. I'm not trying to take ownership of the FooBar instance. I don't care if it's owned as long as I can call its methods. Returning a reference would be great, but when doing this, the compiler complains:
pub fn unpack_foo_bar() -> &Option<FooBar> {
let foo_bar_arc = Arc::clone(&FOO_BAR);
let foo_bar_mutex_result = foo_bar_arc.lock();
let foo_bar_mutex = foo_bar_mutex_result.unwrap();
let foo_bar_option = foo_bar_mutex.as_ref();
foo_bar_option
}
Compiler's response:
error[E0106]: missing lifetime specifier
--> src/main.rs:30:28
|
7 | pub fn unpack_foo_bar() -> &Option<FooBar> {
| ^ help: consider giving it a 'static lifetime: `&'static`
|
= help: this function's return type contains a borrowed value, but there is no value for it to be borrowed from
Experimenting with &*foo_bar_mutex and adding 'static to the return type each opens up subsequent cans of compiler errors.
I have also tried experimenting with to_owned() on the FooBar reference and stumbled upon the owning_ref crate, but alas, haven't figured out how to make returning a reference work.
Final Update
Given that it appears impossible to pass on the reference to the Option<FooBar> to an external function caller, I decided to avoid this problem altogether by allowing passing in the methods relying on the FooBar instance as a closure:
pub fn unpack_foo_bar(use_foo_bar: fn(&FooBar)) {
let foo_bar_arc = Arc::clone(&FOO_BAR);
let foo_bar_mutex_result = foo_bar_arc.lock();
let foo_bar_mutex = foo_bar_mutex_result.unwrap();
let foo_bar_reference = foo_bar_mutex.as_ref().unwrap();
// pass in the closure that needs the foo bar instance
use_foo_bar(foo_bar_reference);
}

How to avoid mutex borrowing problems when using it's guard

I want my method of struct to perform in a synchronized way. I wanted to do this by using Mutex (Playground):
use std::sync::Mutex;
use std::collections::BTreeMap;
pub struct A {
map: BTreeMap<String, String>,
mutex: Mutex<()>,
}
impl A {
pub fn new() -> A {
A {
map: BTreeMap::new(),
mutex: Mutex::new(()),
}
}
}
impl A {
fn synchronized_call(&mut self) {
let mutex_guard_res = self.mutex.try_lock();
if mutex_guard_res.is_err() {
return
}
let mut _mutex_guard = mutex_guard_res.unwrap(); // safe because of check above
let mut lambda = |text: String| {
let _ = self.map.insert("hello".to_owned(),
"d".to_owned());
};
lambda("dd".to_owned());
}
}
Error message:
error[E0500]: closure requires unique access to `self` but `self.mutex` is already borrowed
--> <anon>:23:26
|
18 | let mutex_guard_res = self.mutex.try_lock();
| ---------- borrow occurs here
...
23 | let mut lambda = |text: String| {
| ^^^^^^^^^^^^^^ closure construction occurs here
24 | if let Some(m) = self.map.get(&text) {
| ---- borrow occurs due to use of `self` in closure
...
31 | }
| - borrow ends here
As I understand when we borrow anything from the struct we are unable to use other struct's fields till our borrow is finished. But how can I do method synchronization then?
The closure needs a mutable reference to the self.map in order to insert something into it. But closure capturing works with whole bindings only. This means, that if you say self.map, the closure attempts to capture self, not self.map. And self can't be mutably borrowed/captured, because parts of self are already immutably borrowed.
We can solve this closure-capturing problem by introducing a new binding for the map alone such that the closure is able to capture it (Playground):
let mm = &mut self.map;
let mut lambda = |text: String| {
let _ = mm.insert("hello".to_owned(), text);
};
lambda("dd".to_owned());
However, there is something you overlooked: since synchronized_call() accepts &mut self, you don't need the mutex! Why? Mutable references are also called exclusive references, because the compiler can assure at compile time that there is only one such mutable reference at any given time.
Therefore you statically know, that there is at most one instance of synchronized_call() running on one specific object at any given time, if the function is not recursive (calls itself).
If you have mutable access to a mutex, you know that the mutex is unlocked. See the Mutex::get_mut() method for more explanation. Isn't that amazing?
Rust mutexes do not work the way you are trying to use them. In Rust, a mutex protects specific data relying on the borrow-checking mechanism used elsewhere in the language. As a consequence, declaring a field Mutex<()> doesn't make sense, because it is protecting read-write access to the () unit object that has no values to mutate.
As Lukas explained, your call_synchronized as declared doesn't need to do synchronization because its signature already requests an exclusive (mutable) reference to self, which prevents it from being invoked from multiple threads on the same object. In other words, you need to change the signature of call_synchronized because the current one does not match the functionality it is intended to provide.
call_synchronized needs to accept a shared reference to self, which will signal to Rust that it can be called from multiple threads in the first place. Inside call_synchronized a call to Mutex::lock will simultaneously lock the mutex and provide a mutable reference to the underlying data, carefully scoped so that the lock is held for the duration of the reference:
use std::sync::Mutex;
use std::collections::BTreeMap;
pub struct A {
synced_map: Mutex<BTreeMap<String, String>>,
}
impl A {
pub fn new() -> A {
A {
synced_map: Mutex::new(BTreeMap::new()),
}
}
}
impl A {
fn synchronized_call(&self) {
let mut map = self.synced_map.lock().unwrap();
// omitting the lambda for brevity, but it would also work
// (as long as it refers to map rather than self.map)
map.insert("hello".to_owned(), "d".to_owned());
}
}

Resources