I have the following minimal example of my code:
fn main()
{
let names : Vec<Vec<String>> = vec![
vec!["Foo1".to_string(), "Foo2".to_string()],
vec!["Bar1".to_string(), "Bar2".to_string()]
];
let ids : Vec<i64> = vec![10, 20];
names.iter().enumerate().flat_map(|(i,v)| {
let id : i64 = ids[i];
v.iter().map(|n|
(n.clone(), id)
)
});
}
Now, when I compile that with rustc I get the following error message:
error[E0597]: `id` does not live long enough
--> main.rs:12:16
|
11 | v.iter().map(|n|
| --- capture occurs here
12 | (n.clone(), id)
| ^^ borrowed value does not live long enough
13 | )
14 | });
| -- borrowed value needs to live until here
| |
| borrowed value only lives until here
But in my understanding, id is of type i64 and should therefore be able to be copied into the capture, with would be exactly what I need?
I've also tried to inline the id variable but to no avail:
error[E0597]: `i` does not live long enough
--> main.rs:11:21
|
10 | v.iter().map(|n|
| --- capture occurs here
11 | (n.clone(), ids[i])
| ^ borrowed value does not live long enough
12 | )
13 | });
| -- borrowed value needs to live until here
| |
| borrowed value only lives until here
So how can I copy my integer into the closure instead of borrowing it?
I tried using move, but rustc doesn't like that either:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> main.rs:10:17
|
7 | let ids : Vec<i64> = vec![10, 20];
| --- captured outer variable
...
10 | v.iter().map(move |n|
| ^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
So I'd somehow need to get rustc to only move/copy some but not the other variable?
When you create a closure in Rust, it captures the variables either by value or by reference. A mix of both is impossible. By default, it captures by reference, but with the move keyword, it captures by value (i.e. it moves the captured variables inside the closure).
So, in your first code, you need to move id inside the closure:
fn main() {
let names: Vec<Vec<String>> = vec![
vec!["Foo1".to_string(), "Foo2".to_string()],
vec!["Bar1".to_string(), "Bar2".to_string()],
];
let ids: Vec<i64> = vec![10, 20];
names.iter().enumerate().flat_map(|(i, v)| {
let id: i64 = ids[i];
v.iter().map(move |n| (n.clone(), id))
});
}
Then you ask if you can "inline" ids:
fn main() {
let names: Vec<Vec<String>> = vec![
vec!["Foo1".to_string(), "Foo2".to_string()],
vec!["Bar1".to_string(), "Bar2".to_string()],
];
let ids: Vec<i64> = vec![10, 20];
names.iter().enumerate().flat_map(|(i, v)| {
v.iter().map(|n| (n.clone(), ids[i]))
});
}
You cannot put ids at all in your inner closure, because you are already inside a FnMut closure (that requires exclusive access). Thus, you cannot borrow or move ids because it is already borrowed by the FnMut closure. Minimal reproduction:
fn main() {
let mut i = 0;
let mut closure = || {
i = 2;
|| {
println!("i = {}", i);
}
};
closure()();
}
You can move the variable into closure with move keyword. Here you need to change the closure like:
v.iter().map(move |n| // move is the keyword for moving variables into closure scope.
(n.clone(), id)
)
Playground
Related
Please help me, I'm completely confused.
How do I make this code work?
I need to change the structure members in the thread...
#[derive(Debug)]
struct S {
str: String,
b: bool,
i: u128,
}
fn main() {
let mut vec_s = vec![];
vec_s.push(S {
str: "a".to_string(),
b: false,
i: 0,
});
let mut threads = vec![];
for s in vec_s {
{
let mut _s = &s;
threads.push(std::thread::spawn(move || {
_s.b = true;
_s.str = "b".to_string();
_s.i = 1;
}));
}
}
for thread in threads {
let _ = thread.join();
}
dbg!(&vec_s);
}
The compiler outputs a lot of errors:
error[E0594]: cannot assign to `_s.b`, which is behind a `&` reference
--> src/main.rs:23:17
|
23 | _s.b = true;
| ^^^^^^^^^^^ cannot assign
error[E0594]: cannot assign to `_s.str`, which is behind a `&` reference
--> src/main.rs:24:17
|
24 | _s.str = "b".to_string();
| ^^^^^^ cannot assign
error[E0594]: cannot assign to `_s.i`, which is behind a `&` reference
--> src/main.rs:25:17
|
25 | _s.i = 1;
| ^^^^^^^^ cannot assign
error[E0597]: `s` does not live long enough
--> src/main.rs:21:26
|
21 | let mut _s = &s;
| ^^ borrowed value does not live long enough
22 | threads.push(std::thread::spawn(move || {
| __________________________-
23 | | _s.b = true;
24 | | _s.str = "b".to_string();
25 | | _s.i = 1;
26 | | }));
| |______________- argument requires that `s` is borrowed for `'static`
27 | }
28 | }
| - `s` dropped here while still borrowed
error[E0382]: borrow of moved value: `vec_s`
--> src/main.rs:34:10
|
9 | let mut vec_s = vec![];
| --------- move occurs because `vec_s` has type `Vec<S>`, which does not implement the `Copy` trait
...
19 | for s in vec_s {
| ----- `vec_s` moved due to this implicit call to `.into_iter()`
...
34 | dbg!(&vec_s);
| ^^^^^^ value borrowed here after move
|
note: this function takes ownership of the receiver `self`, which moves `vec_s`
--> /home/martin/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/collect.rs:261:18
|
261 | fn into_iter(self) -> Self::IntoIter;
| ^^^^
help: consider iterating over a slice of the `Vec<S>`'s content to avoid moving into the `for` loop
|
19 | for s in &vec_s {
| +
You are attempting to do multithreading. Everything accessed by multiple threads has to be thread-safe. Further, with Rust being very strict (zero undefined behaviour tolerance), the compiler has to understand that your code is thread-safe.
The biggest problem here is that Rust's borrow checker doesn't understand that your threads get joined at some point. Therefore it doesn't allow you to create references to your S objects, because it cannot prove how long those references exist. Rust must be able to prove that they are destroyed before your S object is dropped; that's part of Rust's borrow checker safety guarantees.
For your specific usecase, Rust introduced thread scopes. With them, the compiler can finally understand thread lifetimes.
With some minor ownership issues fixed (use &mut vec_s for your loop, otherwise the vec_s object gets consumed by the loop), this now works:
#[derive(Debug)]
struct S {
s: String,
b: bool,
i: u128,
}
fn main() {
let mut vec_s = vec![];
vec_s.push(S {
s: "a".to_string(),
b: false,
i: 0,
});
std::thread::scope(|scope| {
let mut threads = vec![];
for s in &mut vec_s {
threads.push(scope.spawn(move || {
s.b = true;
s.s = "b".to_string();
s.i = 1;
}));
}
for thread in threads {
let _ = thread.join();
}
});
dbg!(&vec_s);
}
[src/main.rs:31] &vec_s = [
S {
s: "b",
b: true,
i: 1,
},
]
Another optimization:
If you don't actually use the JoinHandles for error propagation or similar, you don't need them at all in this example. The scope already automatically joins all threads it spawned at the end of the scope:
#[derive(Debug)]
struct S {
s: String,
b: bool,
i: u128,
}
fn main() {
let mut vec_s = vec![];
vec_s.push(S {
s: "a".to_string(),
b: false,
i: 0,
});
std::thread::scope(|scope| {
for s in &mut vec_s {
scope.spawn(move || {
s.b = true;
s.s = "b".to_string();
s.i = 1;
});
}
});
dbg!(&vec_s);
}
[src/main.rs:27] &vec_s = [
S {
s: "b",
b: true,
i: 1,
},
]
Context
I have a function that will use rayon to paralellize some work. Each thread should write its output to a separate consumer. In practice, these consumers will usually write to (separate) files on disc. For testing, I want them to append to vectors, which I can then make assertions on.
Problem
In fact I hit an issue even with a serial version of this code.
The function I want to test is func_under_test. I want get_sink to create and hand out a new 'sink' (a Vec<i32>) each time it is called, which will be inside func_under_test. But I also want to keep a reference to these vectors in my main function, so I can make assertions on them. I've tried writing this:
fn func_under_test<GetSink, F>(
mut get_sink: GetSink
)
where
GetSink: FnMut(usize) -> F,
F: FnMut(i32) -> (),
{
let idxs : Vec<usize> = (0..2).collect();
idxs.iter().for_each(|&i| {
let mut sink = get_sink(i);
sink(0i32);
sink(1i32);
sink(2i32);
});
}
fn main() {
let mut sinks : Vec<&Vec<i32>> = vec![];
let get_sink = |i: usize| {
let mut sink : Vec<i32> = vec![];
sinks.push(&sink);
move |x : i32| sink.push(x)
};
func_under_test(get_sink);
assert_eq!(**sinks.get(0).unwrap(), vec![0i32, 1i32, 2i32]);
}
But I get this compilation error:
error[E0597]: `sink` does not live long enough
--> src/main.rs:23:20
|
20 | let mut sinks : Vec<&Vec<i32>> = vec![];
| --------- lifetime `'1` appears in the type of `sinks`
...
23 | sinks.push(&sink);
| -----------^^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `sink` is borrowed for `'1`
24 | move |x : i32| sink.push(x)
25 | };
| - `sink` dropped here while still borrowed
error[E0505]: cannot move out of `sink` because it is borrowed
--> src/main.rs:24:9
|
20 | let mut sinks : Vec<&Vec<i32>> = vec![];
| --------- lifetime `'1` appears in the type of `sinks`
...
23 | sinks.push(&sink);
| -----------------
| | |
| | borrow of `sink` occurs here
| argument requires that `sink` is borrowed for `'1`
24 | move |x : i32| sink.push(x)
| ^^^^^^^^^^^^^^ ---- move occurs due to use in closure
| |
| move out of `sink` occurs here
Am I correct in thinking that I need to use something like an Rc in this context, since I need to have two references to each Vec<i32>?
Since only one of those references needs to be mutable, do I need to use Rc<Cell<Vec<i32>>> in one case, and just Rc<Vec<i32>> in the other?
You could use an Arc<Mutex<i32>> to share mutable access to the vectors. (Rc is not thread-safe.)
A tidier solution would be typed_arena::Arena, which would allow you to write essentially the code you tried, but with the roles swapped: the Vec<i32>s are always owned by the arena (thus it outlives func_under_tests). Then, after all of the references to the arena are gone, you can convert it into a vector of its elements.
use typed_arena::Arena;
fn main() {
let sinks: Arena<Vec<i32>> = Arena::new();
let get_sink = |_i: usize| {
let sink_ref = sinks.alloc(Vec::new());
move |x| sink_ref.push(x)
};
func_under_test(get_sink);
assert_eq!(sinks.into_vec()[0], vec![0i32, 1i32, 2i32]);
}
I think I've figured out the answer to this. The thread doing the testing needs to have access to the inner vectors, and so does the thread that is populating this. So they need to be wrapped in an Arc and a Mutex. But the outer vector does not, since it is only borrowed by the get_sink closure.
So sinks can have type Vec<Arc<Mutex<Vec<i32>>>.
Anyway, I can write this and it does compile and behave and expected:
use rayon::prelude::*;
use std::sync::{Arc, Mutex};
fn func_under_test<G, S>(
get_sink: G)
where
G: Fn(&usize) -> S + Sync,
S: FnMut(i32) -> (),
{
let idxs : Vec<usize> = vec![0, 1, 2];
idxs.par_iter().for_each(|i| {
let mut sink : S = get_sink(i);
for j in 0..3 {
sink(j + 3*(*i) as i32);
}
});
}
fn main() {
let sinks : Vec<Arc<Mutex<Vec<i32>>>> = vec![
Arc::new(Mutex::new(vec![])),
Arc::new(Mutex::new(vec![])),
Arc::new(Mutex::new(vec![])),
];
let get_sink = |i: &usize| {
let m : Arc<Mutex<Vec<i32>>> = sinks.get(*i).unwrap().clone();
move |x: i32| {
m.lock().unwrap().push(x);
}
};
func_under_test(get_sink);
assert_eq!(*sinks.get(0).unwrap().lock().unwrap(), vec![0, 1, 2]);
assert_eq!(*sinks.get(1).unwrap().lock().unwrap(), vec![3, 4, 5]);
assert_eq!(*sinks.get(2).unwrap().lock().unwrap(), vec![6, 7, 8]);
}
It also seems like G implements the correct traits here: it is Sync, since it needs to be shared between threads. But is is not Send, since it does not need to send information between threads.
I'm trying to build some sort of linked list, each node has a keyword and vectors of the same node. In that for loop, i will traverse the linked list to get the vector, either already exist if keyword is the same, or create a new vector. I find it impossible to *not * violate the double mut reference error since I'll need to do iter_mut(), push() on the same path. Anyone has general idea on how to approach this?
pub struct Knowledge {
pub title: String,
}
pub struct Entry {
pub keyword: String,
pub next: Vec<Box<KnowledgeEntry>>
}
pub enum KnowledgeEntry {
Normal(Knowledge),
Entry(Entry)
}
impl KnowledgeEntry {
pub fn get_entry_mut (&mut self) -> Option<&mut Vec<Box<KnowledgeEntry>>> {
match self {
KnowledgeEntry::Entry(e) => {
Some(&mut e.next)
}
KnowledgeEntry::Normal(k) => {None}
}
}
}
fn main () {
let mut entry = Entry{keyword: "".to_owned(), next: vec![]};
let mut current_container = &mut entry.next;
let new_knowledge = Knowledge{title: "title1".to_owned()};
// new knowledge to be insert in these hierarchy.
let remaining_comp: Vec<_> = vec!["hier1", "hier2", "hier3"];
// each loop will set current_container, either created from new, or found in the vector.
for comp in remaining_comp.iter() {
// true will be replace with body checking with keyword.
let mut _result = current_container.iter_mut().find(|en| {true});
if let Some(mut _res) = _result {
// found the container. Set the current container
if let KnowledgeEntry::Entry(ref mut entry) = &mut **_res {
current_container = &mut entry.next;
} else {
break;
}
} else {
// can't find the resulting container. Create a new vector, and move the container.
let new_container: Vec<Box<KnowledgeEntry>> = vec![];
let mut _temp = KnowledgeEntry::Entry(Entry {
keyword: (*comp).to_string(),
next: new_container
});
current_container.push(Box::new(_temp));
let last_var = current_container.last_mut().unwrap();
// get_entry_mut() to retrieve `_temp` that we have just moved.
current_container = (**last_var).get_entry_mut().unwrap();
}
}
// found the container correspond to "hier1" -> "hier2" -> "hier3", add the entry.
current_container.push(Box::new(KnowledgeEntry::Normal(new_knowledge)));
}
Playground
Produces the following errors:
error[E0499]: cannot borrow `*current_container` as mutable more than once at a time
--> src/main.rs:50:13
|
35 | let mut _result = current_container.iter_mut().find(|en| {true});
| ----------------- first mutable borrow occurs here
...
50 | current_container.push(Box::new(_temp));
| ^^^^^^^^^^^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
error[E0499]: cannot borrow `*current_container` as mutable more than once at a time
--> src/main.rs:51:28
|
35 | let mut _result = current_container.iter_mut().find(|en| {true});
| ----------------- first mutable borrow occurs here
...
51 | let last_var = current_container.last_mut().unwrap();
| ^^^^^^^^^^^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
error[E0499]: cannot borrow `*current_container` as mutable more than once at a time
--> src/main.rs:57:1
|
35 | let mut _result = current_container.iter_mut().find(|en| {true});
| ----------------- first mutable borrow occurs here
...
57 | current_container.push(Box::new(KnowledgeEntry::Normal(new_knowledge)));
| ^^^^^^^^^^^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
I have a borrow in the empty variable and I want to extend its life. In the commented code-block, I attempt to address it, but the reference is no longer available. I have to loop through the loop again to find the match in order to act on it.
How can I loop through a query looking for a best-match and then act on it once I know it's the best match, without having to loop through to find it again?
use bevy::prelude::*;
struct Person;
struct Name(String);
fn main() {
App::build()
.add_default_plugins()
.add_startup_system(startup.system())
.add_system(boot.system())
.run();
}
fn boot(mut query: Query<(&Person, &mut Name)>) {
let mut temp_str = String::new();
let mut empty: Option<&mut Name> = None;
for (_p, mut n_val) in &mut query.iter() {
if n_val.0.to_lowercase() > temp_str.to_lowercase() {
temp_str = n_val.0.clone();
empty = Some(&mut n_val);
}
}
println!("{}", temp_str);
if let Some(n) = empty {
// ...
}
// for (_p, mut n_val) in &mut query.iter() {
// if n_val.0 == temp_str {
// n_val.0 = "a".to_string();
// }
// }
}
fn startup(mut commands: Commands) {
commands
.spawn((Person, Name("Gene".to_string())))
.spawn((Person, Name("Candace".to_string())))
.spawn((Person, Name("Zany".to_string())))
.spawn((Person, Name("Sarah".to_string())))
.spawn((Person, Name("Carl".to_string())))
.spawn((Person, Name("Robert".to_string())));
}
Cargo.toml:
[package]
name = "sample"
version = "0.1.0"
authors = [""]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bevy = "0.1.3"
Specific error:
error[E0716]: temporary value dropped while borrowed
--> src/main.rs:17:33
|
17 | for (_p, mut n_val) in &mut query.iter() {
| ^^^^^^^^^^^-
| | |
| | temporary value is freed at the end of this statement
| creates a temporary which is freed while still in use
...
24 | if let Some(n) = empty {
| ----- borrow later used here
|
= note: consider using a `let` binding to create a longer lived value
error[E0597]: `n_val` does not live long enough
--> src/main.rs:20:26
|
20 | empty = Some(&mut n_val);
| ^^^^^^^^^^ borrowed value does not live long enough
21 | }
22 | }
| - `n_val` dropped here while still borrowed
23 | println!("{}", temp_str);
24 | if let Some(n) = empty {
| ----- borrow later used here
You cannot extend the lifetime of a reference, but that is not your issue here, the error says temporary value dropped while borrowed, so you must extend the lifetime of your temporary.
If you wonder what temporary, the compiler also points to that (literally) in the error message: query.iter(). This is a function call, and the returned value is not bound to anything, so the compiler creates a temporary value for that. Then you iterate using a reference to that temporary. When the for loop ends, the temporary is dropped and any reference lifetime to it expires.
The solution is to bind the temporary to a local variable. This way you extend the lifetime of the object to the scope of the variable:
let mut iter = query.iter();
for (_p, n_val) in &mut iter {
if n_val.0.to_lowercase() > temp_str.to_lowercase() {
temp_str = n_val.0.clone();
empty = Some(n_val);
}
}
PS: I find quite bizarre the pattern of iterating over &mut iter. I would expect the return of iter() to implement Iterator or IntoIterator directly, but it looks like this is not the case.
I have the following minimal example of my code:
fn main()
{
let names : Vec<Vec<String>> = vec![
vec!["Foo1".to_string(), "Foo2".to_string()],
vec!["Bar1".to_string(), "Bar2".to_string()]
];
let ids : Vec<i64> = vec![10, 20];
names.iter().enumerate().flat_map(|(i,v)| {
let id : i64 = ids[i];
v.iter().map(|n|
(n.clone(), id)
)
});
}
Now, when I compile that with rustc I get the following error message:
error[E0597]: `id` does not live long enough
--> main.rs:12:16
|
11 | v.iter().map(|n|
| --- capture occurs here
12 | (n.clone(), id)
| ^^ borrowed value does not live long enough
13 | )
14 | });
| -- borrowed value needs to live until here
| |
| borrowed value only lives until here
But in my understanding, id is of type i64 and should therefore be able to be copied into the capture, with would be exactly what I need?
I've also tried to inline the id variable but to no avail:
error[E0597]: `i` does not live long enough
--> main.rs:11:21
|
10 | v.iter().map(|n|
| --- capture occurs here
11 | (n.clone(), ids[i])
| ^ borrowed value does not live long enough
12 | )
13 | });
| -- borrowed value needs to live until here
| |
| borrowed value only lives until here
So how can I copy my integer into the closure instead of borrowing it?
I tried using move, but rustc doesn't like that either:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> main.rs:10:17
|
7 | let ids : Vec<i64> = vec![10, 20];
| --- captured outer variable
...
10 | v.iter().map(move |n|
| ^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
So I'd somehow need to get rustc to only move/copy some but not the other variable?
When you create a closure in Rust, it captures the variables either by value or by reference. A mix of both is impossible. By default, it captures by reference, but with the move keyword, it captures by value (i.e. it moves the captured variables inside the closure).
So, in your first code, you need to move id inside the closure:
fn main() {
let names: Vec<Vec<String>> = vec![
vec!["Foo1".to_string(), "Foo2".to_string()],
vec!["Bar1".to_string(), "Bar2".to_string()],
];
let ids: Vec<i64> = vec![10, 20];
names.iter().enumerate().flat_map(|(i, v)| {
let id: i64 = ids[i];
v.iter().map(move |n| (n.clone(), id))
});
}
Then you ask if you can "inline" ids:
fn main() {
let names: Vec<Vec<String>> = vec![
vec!["Foo1".to_string(), "Foo2".to_string()],
vec!["Bar1".to_string(), "Bar2".to_string()],
];
let ids: Vec<i64> = vec![10, 20];
names.iter().enumerate().flat_map(|(i, v)| {
v.iter().map(|n| (n.clone(), ids[i]))
});
}
You cannot put ids at all in your inner closure, because you are already inside a FnMut closure (that requires exclusive access). Thus, you cannot borrow or move ids because it is already borrowed by the FnMut closure. Minimal reproduction:
fn main() {
let mut i = 0;
let mut closure = || {
i = 2;
|| {
println!("i = {}", i);
}
};
closure()();
}
You can move the variable into closure with move keyword. Here you need to change the closure like:
v.iter().map(move |n| // move is the keyword for moving variables into closure scope.
(n.clone(), id)
)
Playground