I am trying to make parallel a prime sieve in Rust but Rust compiler not leave of give me a lifemetime error with the parameter true_block.
And data races are irrelevant because how primes are defined.
The error is:
error[E0621]: explicit lifetime required in the type of `true_block`
--> src/sieve.rs:65:22
|
50 | true_block: &mut Vec<bool>,
| -------------- help: add explicit lifetime `'static` to the type of `true_block`: `&'static mut Vec<bool>`
...
65 | handles.push(thread::spawn(move || {
| ^^^^^^^^^^^^^ lifetime `'static` required
The code is:
fn extend_helper(
primes: &Vec<usize>,
true_block: &mut Vec<bool>,
segment_min: usize,
segment_len: usize,
) {
let mut handles: Vec<thread::JoinHandle<()>> = vec![];
let arc_primes = Arc::new(true_block);
let segment_min = Arc::new(segment_min);
let segment_len = Arc::new(segment_len);
for prime in primes {
let prime = Arc::new(prime.clone());
let segment_min = Arc::clone(&segment_min);
let segment_len = Arc::clone(&segment_len);
let shared = Arc::clone(&arc_primes);
handles.push(thread::spawn(move || {
let tmp = smallest_multiple_of_n_geq_m(*prime, *segment_min) - *segment_min;
for j in (tmp..*segment_len).step_by(*prime) {
shared[j] = false;
}
}));
}
for handle in handles {
handle.join().unwrap();
}
}
The lifetime issue is with true_block which is a mutable reference passed into the function. Its lifetime is only as long as the call to the function, but because you're spawning a thread, the thread could live past the end of the function (the compiler can't be sure). Since it's not known how long the thread will live for, the lifetime of data passed to it must be 'static, meaning it will live until the end of the program.
To fix this, you can use .to_owned() to clone the array stored in the Arc, which will get around the lifetime issue. You would then need to copy the array date out of the Arc after joining the threads and write it back to the true_block reference. But it's also not possible to mutate an Arc, which only holds an immutable reference to the value. You need to use an Arc<Mutex<_>> for arc_primes and then you can mutate shared with shared.lock().unwrap()[j] = false; or something safer
Related
I am new to rust and I am attempting to create a Vec that will live on the main thread, and pass a reference to another thread, which then pushes members onto the vector, for the main thread to use.
use std::{thread};
fn main() {
let mut v: Vec<u8> = Vec::new();
let _ = thread::spawn(move || {
vec_push(&mut v, 0)
});
for i in v.iter_mut() {
println!("poo {}", i);
}
}
fn vec_push(v: &mut Vec<u8>, n: u8) {
v.push(n);
}
This is a simplified version of what I am trying to do. In my main code I am want it to be a Vec of TcpStreams.
I think this post would also apply to maintaining a struct (that doesn't implement Copy) between threads.
I get this error
error[E0382]: borrow of moved value: `v`
--> src/main.rs:8:11
|
4 | let mut v: Vec<u8> = Vec::new();
| ----- move occurs because `v` has type `Vec<u8>`, which does not implement the `Copy` trait
5 | let _ = thread::spawn(move || {
| ------- value moved into closure here
6 | vec_push(&mut v, 0)
| - variable moved due to use in closure
7 | });
8 | for i in v.iter_mut() {
| ^^^^^^^^^^^^ value borrowed here after move
Is there a better way to do this? Am I missing some basic concept?
Any help would be useful, I am used to C where I can just throw around references willy-nilly
What you are doing is wildly unsound. You are trying to have two mutable references to a object, which is strictly forbidden in rust. Rust forbids this to prevent you from having data races that would result in memory unsafety.
If you want to mutate an object from different threads you have to synchronize it somehow. The easiest way to do it is by using Mutex. This probably won't be very efficient in a high-congestion scenario (as locking a mutex can become your bottle neck), but it will be safe.
To share this Mutex between threads you can wrap it in an Arc (an atomic counted shared reference smart pointer). So your code can be transformed to something like this:
use std::thread;
use std::sync::{Arc, Mutex};
fn main() {
let v = Arc::new(Mutex::new(Vec::new()));
let v_clone = Arc::clone(&v);
let t = thread::spawn(move || {
vec_push(v_clone, 0)
});
t.join().unwrap();
for i in v.lock().unwrap().iter_mut() {
println!("poo {}", i);
}
}
fn vec_push(v: Arc<Mutex<Vec<u8>>>, n: u8) {
v.lock().unwrap().push(n);
}
You probably will also want to join your spawned thread, so you should name it.
I'm trying to map a 2d slice in parallel.
I'm doing this by assigning every child slice to one thread. So, if the slice is 10x10, 10 threads will be spawned.
I'm doing this by creating a Mutex for each child slice. The problem is that I'm getting an error for the parent slice:
error[E0759]: `collection` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
--> src\lib.rs:151:34
|
145 | fn gen_map_par<T: Send + Sync, F: Fn(f32, &mut T) + Send + Sync + 'static>(&self, collection: &[&[T]], predicate: F) {
| ------- this data with an anonymous lifetime `'_`...
...
151 | for (x, i) in collection.iter().enumerate() {
| ^^^^ ...is captured here...
...
155 | let handle = thread::spawn(move || {
| ------------- ...and is required to live as long as `'static` here
For more information about this error, try `rustc --explain E0759`.
The error shows that collection is being captured. Here is the associated code.
fn gen_map_par<T: Send + Sync, F: Fn(f32, &mut T) + Send + Sync + 'static>(
&self,
collection: &[&[T]],
predicate: F,
) {
let perm = Arc::new(self.perm);
let closure = Arc::new(predicate);
let mut handles = Vec::new();
for (x, i) in collection.iter().enumerate() {
// iterating through parent slice
let p = Arc::clone(&perm);
let c = Arc::clone(&closure);
let s = Arc::new(Mutex::new(*i)); // mutex created for one entire child slice
let handle = thread::spawn(move || {
// spawning the thread
let slice = s.lock().unwrap();
// collection gets moved into the thread for some reason.
for (y, e) in slice.iter().enumerate() {
let v = Simplex::convert_range(
max_2d,
min_2d,
1.0,
-1.0,
generate2d(x as f32, y as f32, &p),
);
(c)(v, &mut slice[y]);
}
});
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
}
I've tried wrapping the 'collection' in an arc/mutex, but neither of these works. I'm not sure how to solve this problem. The only solution I've found is to make collection static, but this would require the user to input a static variable, which I do not want to require.
After trying crossbeam::scope(), doing some research, and reading #Ceasars' suggestions, I have concluded that using a closure inside of a thread like this is either impossible or very difficult to do because the data has to be moved into the closure as a mutable reference, something the compiler will not allow as far as I can tell. Moving on to another project.
Thanks to everybody who responded, reading your suggestions helped me understand parallelism in rust better.
fn main() {
let mut a = String::from("dd");
let mut x = move || {
a.push_str("string: &str");
};
x();
x();
}
I have added move here to capture a but I am still able to call the x closure twice. Is a still borrowed as a mutable reference here? Why doesn't move force a move?
The variable a has indeed been moved into the closure:
fn main() {
let mut a = String::from("dd");
let mut x = move || {
a.push_str("string: &str");
};
x();
x();
a.len();
}
error[E0382]: borrow of moved value: `a`
--> src/main.rs:9:5
|
2 | let mut a = String::from("dd");
| ----- move occurs because `a` has type `std::string::String`, which does not implement the `Copy` trait
3 | let mut x = move || {
| ------- value moved into closure here
4 | a.push_str("string: &str");
| - variable moved due to use in closure
...
9 | a.len();
| ^ value borrowed here after move
It's unclear why you think that the closure x would become invalid after calling it, but it doesn't. No more than the same applied to a struct:
struct ClosureLike {
a: String,
}
impl ClosureLike {
fn call(&mut self) {
self.a.push_str("string: &str");
}
}
fn main() {
let a = String::from("dd");
let mut x = ClosureLike { a };
x.call();
x.call();
}
The question came from my wrong understanding of closures. The way it is documented in the Rust book also contributed to the confusion (I am not saying the book is bad). If anyone else had this same confusion, here is what I found.
Closures do not just store the scope and run it when its called. It captures the environment in the preferred way. The environment which contains a is stored in the closure. How the values are captured from the environment decides the trait.
The value of a persists until the closure exists, unless some operation moves it, such as if the closure returns a or a method consumes a. Here, nothing moves a out of the closure so the closure can be called as many times as I want.
A better understanding can be obtained from the FnOnce, FnMut, and Fn traits. These traits are decided by how the variables are captured by the closure, not by how the variables are moved into the closure. FnMut can be implemented on a closure where a value is moved .
In Rust, I have a BTreeSet that I'm using to keep my values in order. I have a loop that should retrieve and remove the first (lowest) member of the set. I'm using a cloned iterator to retrieve the first member. Here's the code:
use std::collections::BTreeSet;
fn main() {
let mut start_nodes = BTreeSet::new();
// add items to the set
while !start_nodes.is_empty() {
let mut start_iter = start_nodes.iter();
let mut start_iter_cloned = start_iter.cloned();
let n = start_iter_cloned.next().unwrap();
start_nodes.remove(&n);
}
}
This, however, gives me the following compile error:
error[E0502]: cannot borrow `start_nodes` as mutable because it is also borrowed as immutable
--> prog.rs:60:6
|
56 | let mut start_iter = start_nodes.iter();
| ----------- immutable borrow occurs here
...
60 | start_nodes.remove(&n);
| ^^^^^^^^^^^ mutable borrow occurs here
...
77 | }
| - immutable borrow ends here
Why is start_nodes.iter() considered an immutable borrow? What approach should I take instead to get the first member?
I'm using version 1.14.0 (not by choice).
Why is start_nodes.iter() considered an immutable borrow?
Whenever you ask a question like this one, you need to look at the prototype of the function, in this case the prototype of BTreeSet::iter():
fn iter(&self) -> Iter<T>
If we look up the Iter type that is returned, we find that it's defined as
pub struct Iter<'a, T> where T: 'a { /* fields omitted */ }
The lifetime 'a is not explicitly mentioned in the definition of iter(); however, the lifetime elision rules make the function definition equivalent to
fn iter<'a>(&'a self) -> Iter<'a, T>
From this expanded version, you can see that the return value has a lifetime that is bound to the lifetime of the reference to self that you pass in, which is just another way of stating that the function call creates a shared borrow that lives as long as the return value. If you store the return value in a variable, the borrow lives at least as long as the variable.
What approach should I take instead to get the first member?
As noted in the comments, your code works on recent versions of Rust due to non-lexical lifetimes – the compiler figures out by itself that start_iter and start_iter_cloned don't need to live longer than the call to next(). In older versions of Rust, you can artificially limit the lifetime by introducing a new scope:
while !start_nodes.is_empty() {
let n = {
let mut start_iter = start_nodes.iter();
let mut start_iter_cloned = start_iter.cloned();
start_iter_cloned.next().unwrap()
};
start_nodes.remove(&n);
}
However, note that this code is needlessly long-winded. The new iterator you create and its cloning version only live inside the new scope, and they aren't really used for any other purpose, so you could just as well write
while !start_nodes.is_empty() {
let n = start_nodes.iter().next().unwrap().clone();
start_nodes.remove(&n);
}
which does exactly the same, and avoids the issues with long-living borrows by avoiding to store the intermediate values in variables, to ensure their lifetime ends immediately after the expression.
Finally, while you don't give full details of your use case, I strongly suspect that you would be better off with a BinaryHeap instead of a BTreeSet:
use std::collections::BinaryHeap;
fn main() {
let mut start_nodes = BinaryHeap::new();
start_nodes.push(42);
while let Some(n) = start_nodes.pop() {
// Do something with `n`
}
}
This code is shorter, simpler, completely sidesteps the issue with the borrow checker, and will also be more efficient.
Not sure this is the best approach, but I fixed it by introducing a new scope to ensure that the immutable borrow ends before the mutable borrow occurs:
use std::collections::BTreeSet;
fn main() {
let mut start_nodes = BTreeSet::new();
// add items to the set
while !start_nodes.is_empty() {
let mut n = 0;
{
let mut start_iter = start_nodes.iter();
let mut start_iter_cloned = start_iter.cloned();
let x = &mut n;
*x = start_iter_cloned.next().unwrap();
}
start_nodes.remove(&n);
}
}
I am trying to fire up a new thread using some heap data in Rust and I am getting a bunch of errors that stem from the need of the data to have 'static lifetime. I've worked my way backwards up my program but hit a problem.
use std::sync::Arc;
use std::thread;
struct ThreadData {
vector_of_strings: Vec<String>,
terms: Vec<&'static str>,
quotient: usize,
}
fn perform_search(slice: &[String], terms: &[&str]) {
/* ... */
}
fn threaded_search(td_arc: &Arc<ThreadData>) {
let no_of_lines = td_arc.vector_of_strings.len();
let new_tda1 = td_arc.clone();
let strings_as_slice1 = new_tda1.vector_of_strings.as_slice();
thread::spawn(move || {
perform_search(&strings_as_slice1[0..td_arc.quotient], &new_tda1.terms);
});
}
fn main() {
let td = ThreadData {
vector_of_strings: Vec::new(),
terms: Vec::new(),
quotient: 0,
};
let td_arc = Arc::new(td);
threaded_search(&td_arc);
}
Error:
error[E0621]: explicit lifetime required in the type of `td_arc`
--> src/main.rs:20:5
|
14 | fn threaded_search(td_arc: &Arc<ThreadData>) {
| ---------------- help: add explicit lifetime `'static` to the type of `td_arc`: `&'static std::sync::Arc<ThreadData>`
...
20 | thread::spawn(move || {
| ^^^^^^^^^^^^^ lifetime `'static` required
The error about 'static is because the new thread created within thread::spawn may outlive the invocation of threaded_search during which the thread is initially created, which means the thread must not be permitted to use any local variables from threaded_search with a lifetime shorter than 'static.
In your code the new thread is referring to strings_as_slice1 and td_arc.
Generally with thread::spawn and Arc you will want to move ownership of one reference count into the thread and have the thread access whatever it needs through that reference counted pointer rather than from the enclosing short-lived scope directly.
fn threaded_search(td_arc: &Arc<ThreadData>) {
// Increment reference count that we can move into the new thread.
let td_arc = td_arc.clone();
thread::spawn(move || {
perform_search(&td_arc.vector_of_strings[0..td_arc.quotient], &td_arc.terms);
});
}