I try to process an array with multiple worker threads, just like Rayon's par_iter() function, but i want to have a mutable reference target, one for each thread.
I have a simple function like this
pub fn parallel_map_vector<A:Sync,T:Send,B:Send>(a:&[A], t:&mut [T], b:&mut [B], f:impl Fn(&A,&mut T)+Send+Sync){
assert_eq!(a.len(),b.len());
let atomic_counter = AtomicUsize::new(0);
crossbeam::scope(|scope| {
for target in t {
scope.spawn( |_| {
loop {
let idx = atomic_counter.fetch_add(1, Ordering::Relaxed);
if idx < a.len() {unsafe{std::slice::from_raw_parts_mut(b,b_len)};
f(&a[idx], target)
}else{
break
}
}
});
}
}).unwrap();
}
For some reason i keep getting the error
error[E0373]: closure may outlive the current function, but it borrows `target`, which is owned by the current function
--> htm/src/parallel.rs:11:26
|
9 | crossbeam::scope(|scope| {
| ----- has type `&crossbeam::thread::Scope<'1>`
10 | for target in t {
11 | scope.spawn( |_| {
| ^^^ may outlive borrowed value `target`
...
17 | f(&a[idx], target)
| ------ `target` is borrowed here
|
note: function requires argument type to outlive `'1`
--> htm/src/parallel.rs:11:13
|
11 | / scope.spawn( |_| {
12 | | loop {
13 | | let idx = atomic_counter.fetch_add(1, Ordering::Relaxed);
14 | | if idx < a.len() {
... |
21 | | }
22 | | });
| |______________^
help: to force the closure to take ownership of `target` (and any other referenced variables), use the `move` keyword
|
11 | scope.spawn( move |_| {
but when i try it in rust playground it compiles without problem
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0a10cbfd2f135975f4682c843664d539
What could be wrong here? The code looks safe to me.
The playground example to which you have linked is using the 2021 edition of Rust, which allows disjoint capture in closures (and hence the closure for each thread captures only target rather than the entire t slice). If you switch to the 2018 edition, it yields the same error as in your question.
Specify that you require the 2021 edition in your Cargo.toml (requires rustc >= 1.56.0).
Related
This question already has answers here:
Accessing a method of self inside a thread in Rust
(1 answer)
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed 6 months ago.
I have a program that has an object oriented struct which has functions in it that use threading (std::thread). In the threads I want to access "self". Unfortunately when I do that, I get this unhelpful error:
error[E0521]: borrowed data escapes outside of associated function
--> src\screens\map\map.rs:170:34
|
67 | &mut self,
| ---------
| |
| `self` is a reference that is only valid in the associated function body
| let's call the lifetime of this reference `'1`
...
170 | let handle = thread::spawn(move || {
| __________________________________^
171 | | let renderbytes = self.cloudslayer.render(
172 | | runtimechunksize as u32,
173 | | runtimechunksize as u32,
... |
179 | | renderbytes
180 | | });
| | ^
| | |
| |______________________`self` escapes the associated function body here
| argument requires that `'1` must outlive `'static`
error[E0382]: borrow of moved value: `self`
--> src\screens\map\map.rs:340:9
|
67 | &mut self,
| --------- move occurs because `self` has type `&mut screens::map::map::Map`, which does not implement the `Copy` trait
...
170 | let handle = thread::spawn(move || {
| ------- value moved into closure here, in previous iteration of loop
...
340 | / self.cachedrenderedchunks
341 | | .retain(|key, _| chunksonscreen.contains(key))
| |__________________________________________________________^ value borrowed here after move
Here is the thread:
let handle = thread::spawn(move || {
let renderbytes = self.cloudslayer.render(
runtimechunksize as u32,
runtimechunksize as u32,
x,
y,
lod,
);
renderbytes
});
I have only been programming in rust for about 3 weeks now, and so I am sure there is some dumb reason why this doesn't work, but any help would be greatly appreciated.
today I tried to write this MusicGuesser with GTK for fun and got this error:
error[E0507]: cannot move out of `guess`, a captured variable in an `Fn` closure
--> src/main.rs:63:32
|
15 | let mut guess = Arc::new(get_guess());
| --------- captured outer variable
16 |
17 | app.connect_activate(move |app| {
| ---------- captured by this `Fn` closure
...
63 | button.connect_clicked(move |_| {
| ^^^^^^^^ move out of `guess` occurs here
64 | let mut guess = Arc::clone(&guess);
| -----
| |
| variable moved due to use in closure
| move occurs because `guess` has type `Arc<Vec<std::string::String>>`, which does not implement the `Copy` trait
I found some same questions about this error, but I didn't understand them. There is source code: https://pastebin.com/1pNxEiB5
let guess = Arc::new(get_guess());
let cloned_guess = Arc::clone(&guess);
// ...
app.connect_activate(move |app| {
// ...
button.connect_clicked(move |_| {
let guess = cloned_guess;
// ...
If you use something in a move || closure, you move it into that closure. That means in your case you move the entire outer guess object in.
So you need to clone first, and then only move the cloned object in.
You will hit the next problem soon, though, because the content of Arc is always immutable. In Rust, you can never have multiple mutable references to the same thing. So in order to modify your guess, you will have to create interior mutability via Mutex or similar.
I am using the Cursive TUI crate and I am having trouble with a dialog button because I want to set an input to change with the dialog response. This is my code:
fn prompt_repeat(siv: &mut Cursive) -> bool {
let input = Cell::new(false);
siv.add_layer(views::Dialog::text("Do you want to retype the note again?")
.title("Redo?")
.button("Yes", |s| {
input.set(true);
s.quit();
})
.button("No", |s| {
s.quit();
}));
siv.run();
input.get()
}
This causes error E0373:
error[E0373]: closure may outlive the current function, but it borrows `input`, which is owned by the current function
--> src/main.rs:94:23
|
94 | .button("No", |s| {
| ^^^ may outlive borrowed value `input`
95 | input.set(false)
| ----- `input` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src/main.rs:89:19
|
89 | siv.add_layer(views::Dialog::text("Do you want to retype the note again?")
| ___________________^
90 | | .title("Redo?")
91 | | .button("Yes", |s| {
92 | | input.set(true)
... |
95 | | input.set(false)
96 | | })
| |______________^
help: to force the closure to take ownership of `input` (and any other referenced variables), use the `move` keyword
|
94 | .button("No", move |s| {
| ++++
I understand that this is caused because the compiler cannot know if input will still exist before the closure uses it, but I cannot use move because I need to reference the cell again to return it (I have tried this and the compiler tells me I have borrowed the value multiple times, which is correct).
How should I go about referencing the cell in multiple closures?
I agree with #Jmb, I'd probably go with a Rc reference counting smart pointer:
fn prompt_repeat(siv: &mut Cursive) -> bool {
let input = Rc::new(Cell::new(false));
let input_2 = input.clone();
siv.add_layer(
views::Dialog::text("Do you want to retype the note again?")
.title("Redo?")
.button("Yes", move |s| {
input_2.set(true);
s.quit();
})
.button("No", move |s| {
s.quit();
}),
);
siv.run();
input.get()
}
This is a super contrived example that shows what I want to do; I have a HashMap that I want to push data in to, but I want to do it in a set of threads - the real example is that I'm connecting to a remote service inside the thread::spawn and so I want to have all of it happen in the background as much as possible so it doesn't block the primary thread.
I'm not sure how to refactor my code to allow what I'm wanting to do - any suggestions would be helpful!
let mut item_array : HashMap<i32, i32> = HashMap::new();
let mut array: [i32; 3] = [0, 1, 2];
for item in &array {
thread::spawn(|| {
item_array.insert(*item, *item);
});
}
println!("{:?}", item_array);
The errors I receive are below
error[E0597]: `array` does not live long enough
--> src/main.rs:87:17
|
87 | for item in &array {
| ^^^^^^
| |
| borrowed value does not live long enough
| argument requires that `array` is borrowed for `'static`
...
153 | }
| - `array` dropped here while still borrowed
error[E0499]: cannot borrow `item_array` as mutable more than once at a time
--> src/main.rs:88:23
|
88 | thread::spawn(|| {
| - ^^ mutable borrow starts here in previous iteration of loop
| _________|
| |
89 | | item_array.insert(*item, *item);
| | ---------- borrows occur due to use of `item_array` in closure
90 | | });
| |__________- argument requires that `item_array` is borrowed for `'static`
error[E0373]: closure may outlive the current function, but it borrows `item`, which is owned by the current function
--> src/main.rs:88:23
|
88 | thread::spawn(|| {
| ^^ may outlive borrowed value `item`
89 | item_array.insert(*item, *item);
| ---- `item` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src/main.rs:88:9
|
88 | / thread::spawn(|| {
89 | | item_array.insert(*item, *item);
90 | | });
| |__________^
help: to force the closure to take ownership of `item` (and any other referenced variables), use the `move` keyword
|
88 | thread::spawn(move || {
| ^^^^^^^
error[E0373]: closure may outlive the current function, but it borrows `item_array`, which is owned by the current function
--> src/main.rs:88:23
|
88 | thread::spawn(|| {
| ^^ may outlive borrowed value `item_array`
89 | item_array.insert(*item, *item);
| ---------- `item_array` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src/main.rs:88:9
|
88 | / thread::spawn(|| {
89 | | item_array.insert(*item, *item);
90 | | });
| |__________^
help: to force the closure to take ownership of `item_array` (and any other referenced variables), use the `move` keyword
|
88 | thread::spawn(move || {
| ^^^^^^^
error[E0502]: cannot borrow `item_array` as immutable because it is also borrowed as mutable
--> src/main.rs:92:22
|
88 | thread::spawn(|| {
| - -- mutable borrow occurs here
| _________|
| |
89 | | item_array.insert(*item, *item);
| | ---------- first borrow occurs due to use of `item_array` in closure
90 | | });
| |__________- argument requires that `item_array` is borrowed for `'static`
91 | }
92 | println!("{:?}", item_array);
| ^^^^^^^^^^ immutable borrow occurs here
There are two problems here (which are typical problems with multithreaded Rust):
Your thread cannot borrow any data that may outlive it (which, when using std::thread::spawn, is any data1).
Multiple thread cannot borrow the same mutable data.
The first problem is typically solved by:
Copying data instead of referencing it. This is particularly useful for primitive types, such as integers.
Using scoped threads.
Using Arc for thread-safe shared pointer, to ensure that data outlives all threads using it.
Not all are possible in all cases, but I'd recommend the above solutions in that order.
The second problem is typically solved with locks such as Mutex or RwLock, which allows only a single thread to get a mutable reference at a time.
Combining these, I'd solve your example like this:
// mutable data is wrapped in a Mutex
let item_array: Mutex<HashMap<i32, i32>> = Mutex::new(HashMap::new());
// immutable data does not have to be wrapped with scoped threads
let array: [i32; 3] = [0, 1, 2];
// using crossbeam::scope from the crossbeam library, which lets
// us reference variables outside the scope (item_array and array)
crossbeam::scope(|s| {
for item in &array {
// copy item (since it is an integer)
let item = *item;
// explicitly reference item_array, since we'll later need to move this
let item_array_ref = &item_array;
// move item and item_array_ref into the closure (instead of referencing,
// which is by default)
s.spawn(move |_| {
// need to call .lock() to aquire a mutable reference to the HashMap
// will wait until the mutable reference is not used by any other threads
let mut item_array_lock = item_array_ref.lock().unwrap();
item_array_lock.insert(item, item);
});
}
// the scope will persist until all threads spawned inside it (with s.spawn) have completed, blocking the current thread,
// ensuring that any referenced data outlives all the threads
})
.unwrap();
// need to call .lock() here as well to get a reference to the HashMap
println!("{:?}", item_array.lock().unwrap());
Run in playground
1Except when the data has 'static lifetime, i.e. can exist for the entire lifetime of the program.
Here is very simple example of code based on source you have provieded:
use std::collections::HashMap;
use std::thread;
fn main() {
let item_array: HashMap<i32, i32> = HashMap::new();
let array: [i32; 3] = [0, 1, 2];
let mut handles = vec![];
let mutex = std::sync::Mutex::new(item_array);
let arc = std::sync::Arc::new(mutex);
for item in &array {
let item = item.to_owned();
let arc_cloned = std::sync::Arc::clone(&arc);
let th = thread::spawn(move || {
let mut guard = arc_cloned.lock().unwrap();
guard.insert(item, item);
});
handles.push(th);
}
for handle in handles {
handle.join().unwrap();
}
println!("{:?}", arc.lock().unwrap());
}
And you can play it here on rust playground
This question already has an answer here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed 3 years ago.
I am trying to write a binary tree that can be passed around between threads without having to be copied every time. I'm having a hard time understanding how to do this with Rust's restrictions around lifetimes.
use std::thread::spawn;
#[derive(Debug)]
struct Node<'a> {
left: &'a i32,
right: &'a i32,
}
fn main() {
let l = 3;
let r = 4;
let n = Node {
left: &l,
right: &r,
};
spawn(|| {
println!("{:?}", n);
});
}
error[E0597]: `l` does not live long enough
--> src/main.rs:13:15
|
13 | left: &l,
| ^^ borrowed value does not live long enough
...
17 | / spawn(|| {
18 | | println!("{:?}", n);
19 | | });
| |______- argument requires that `l` is borrowed for `'static`
20 | }
| - `l` dropped here while still borrowed
error[E0597]: `r` does not live long enough
--> src/main.rs:14:16
|
14 | right: &r,
| ^^ borrowed value does not live long enough
...
17 | / spawn(|| {
18 | | println!("{:?}", n);
19 | | });
| |______- argument requires that `r` is borrowed for `'static`
20 | }
| - `r` dropped here while still borrowed
error[E0373]: closure may outlive the current function, but it borrows `n`, which is owned by the current function
--> src/main.rs:17:11
|
17 | spawn(|| {
| ^^ may outlive borrowed value `n`
18 | println!("{:?}", n);
| - `n` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src/main.rs:17:5
|
17 | / spawn(|| {
18 | | println!("{:?}", n);
19 | | });
| |______^
help: to force the closure to take ownership of `n` (and any other referenced variables), use the `move` keyword
|
17 | spawn(move || {
| ^^^^^^^
I understand why it would think they don't live long enough, but how should I restructure this so that they do?
In your situation, you create a Node object with references to variables you defined in your main() function, and pass it to a new thread. The problem is that you have absolutely no guarantee that this thread will finish before the thread of your main(), and you risk your original variables going out of scope before the references, hence the error.
Given that you wish to build a tree, I think the easiest fix would be to have your Node type own its fields, like this:
#[derive(Debug)]
struct Node {
left: i32,
right: i32,
}
you will then have no more lifetime issues.
To share it between several threads without copying it, you can use the std::sync::Arc wrapper; it's for exactly this purpose. You can do something like:
use std::sync::Arc;
fn main() {
let n = Arc::new(Node { left: 3, right: 4 });
for _ in 0..10 {
let n_thread = n.clone();
spawn(move || {
println!("{:?}", n_thread);
});
}
}
If you need to give your other threads write access to the Node as well, you probably will want to wrap it in a Mutex or a RwLock, keeping whole thing inside an Arc.