I'm doing some exercises with using threads in Rust. I want to print chunked string, but I can't get pass lifetimes issue I've stumbled upon. Here's my code:
let input = "some sample string".to_string();
let mut threads = vec![];
for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
threads.push(thread::spawn({
move || -> () {
println!("{:?}", chunk);
}
}))
}
When I'm trying to run it, I get a following error:
error[E0716]: temporary value dropped while borrowed
--> src\main.rs:7:18
|
7 | for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^----------
| | |
| | temporary value is freed at the end of this statement
| creates a temporary which is freed while still in use
| argument requires that borrow lasts for `'static`
I understand that the compiler doesn't know if the chunk within the thread will not outlive the input variable. But what I can do to fix this code?
I've tried to clone() the vector, but it didn't help.
The chunks are of the type &[char], with the same lifetime as the string.
Simply cloning here won't work, as at compile time it doesn't know the size of the char array. To get a owned copy of a unknown sized array, you must convert it to a Vec, which is done easily with the to_vec() function.
let input = "some sample string".to_string();
let mut threads = vec![];
for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
let owned_chunk = chunk.to_vec();
threads.push(thread::spawn({
move || -> () {
println!("{:?}", owned_chunk);
}
}))
}
Alternatively, since we know the size of the array, we can create an array on the stack to copy into. This removes the need to allocate any heap memory.
let input = "some sample string".to_string();
let mut threads = vec![];
for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
let mut owned_array = ['\0'; 2];
owned_array.copy_from_slice(chunk);
threads.push(thread::spawn({
move || -> () {
println!("{:?}", owned_array);
}
}))
}
Related
This question already has an answer here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed last month.
How do I move a vector reference into threads? The closest I get is the (minimized) code below. (I realize that the costly calculation still isn't parallel, as it is locked by the mutex, but one problem at a time.)
Base problem: I'm calculating values based on information saved in a vector. Then I'm storing the results as nodes per vector element. So vector in vector (but only one vector in the example code below). The calculation takes time so I would like to divide it into threads. The structure is big, so I don't want to copy it.
use std::sync::{Arc, Mutex};
use std::thread;
fn main() {
let n = Nodes::init();
n.calc();
println!("Result: nodes {:?}", n);
}
#[derive(Debug)]
struct Nodes {
nodes: Vec<Info>,
}
impl Nodes {
fn init() -> Self {
let mut n = Nodes { nodes: Vec::new() };
n.nodes.push(Info::init(1));
n.nodes.push(Info::init(2));
n
}
fn calc(&self) {
Nodes::calc_associative(&self.nodes);
}
fn calc_associative(nodes: &Vec<Info>) {
let mut handles = vec![];
let arc_nodes = Arc::new(nodes);
let counter = Arc::new(Mutex::new(0));
for _ in 0..2 {
let arc_nodes = Arc::clone(&arc_nodes);
let counter = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut idx = counter.lock().unwrap();
// costly calculation
arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
*idx += 1;
});
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
}
}
#[derive(Debug)]
struct Info {
length: u32,
}
impl Info {
fn init(length: u32) -> Self {
Info { length }
}
fn get_length(&self) -> u32 {
self.length
}
fn set_length(&mut self, x: u32) {
self.length = x;
}
}
The compiler complains that life time of the reference isn't fulfilled, but isn't that what Arc::clone() should do? Then Arc require a deref, but maybe there are better solutions before starting to dig into that...?
Compiling threads v0.1.0 (/home/freefox/proj/threads)
error[E0596]: cannot borrow data in an `Arc` as mutable
--> src/main.rs:37:17
|
37 | arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
| ^^^^^^^^^ cannot borrow as mutable
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Arc<&Vec<Info>>`
error[E0521]: borrowed data escapes outside of associated function
--> src/main.rs:34:26
|
25 | fn calc_associative(nodes: &Vec<Info>) {
| ----- - let's call the lifetime of this reference `'1`
| |
| `nodes` is a reference that is only valid in the associated function body
...
34 | let handle = thread::spawn(move || {
| __________________________^
35 | | let mut idx = counter.lock().unwrap();
36 | | // costly calculation
37 | | arc_nodes[*idx].set_length(arc_nodes[*idx].get_length() * 2);
38 | | *idx += 1;
39 | | });
| | ^
| | |
| |______________`nodes` escapes the associated function body here
| argument requires that `'1` must outlive `'static`
Some errors have detailed explanations: E0521, E0596.
For more information about an error, try `rustc --explain E0521`.
error: could not compile `threads` due to 2 previous errors
You wrap a reference with Arc. Now the type is Arc<&Vec<Info>>. There is still a reference here, so the variable could still be destroyed before the thread return and we have a dangling reference.
Instead, you should take a &Arc<Vec<Info>>, and on the construction of the Vec wrap it in Arc, or take &[Info] and clone it (let arc_nodes = Arc::new(nodes.to_vec());). You also need a mutex along the way (either Arc<Mutex<Vec<Info>>> or Arc<Vec<Mutex<Info>>>), since you want to change the items.
Or better, since you immediately join() the threads, use scoped threads:
fn calc_associative(nodes: &[Mutex<Info>]) {
let counter = std::sync::atomic::AtomicUsize::new(0); // Changed to atomic, prefer it to mutex wherever possible
std::thread::scope(|s| {
for _ in 0..2 {
s.spawn(|| {
let idx = counter.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let node = &mut *nodes[idx].lock().unwrap();
// costly calculation
node.set_length(node.get_length() * 2);
});
}
});
}
I am writing a small function that reads /proc/bus/input/devices line by line and looks for a pattern in each block.
let mut handlers = Vec::<&str>::new();
let entry = RefCell::new(String::new());
let re = Regex::new(r"(?m)(event\d+)").unwrap();
for line in lines {
let l = line.unwrap();
if !l.is_empty() {
entry.borrow_mut().push_str(&l);
continue
}
if entry.borrow().contains("EV=120013") {
if let Some(captures) = re.captures(entry.borrow().as_str().clone()) {
if let Some(m) = captures.get(0) {
&handlers.push(m.as_str());
}
}
}
entry.borrow_mut().clear();
}
However, the build fails with the following error:
error[E0716]: temporary value dropped while borrowed
--> src/linux/keylogger.rs:50:53
|
50 | if let Some(captures) = re.captures(entry.borrow().as_str().clone()) {
| ^^^^^^^^^^^^^^ creates a temporary which is freed while still in use
51 | if let Some(m) = captures.get(0) {
52 | &handlers.push(m.as_str());
| ------------------------- borrow later used here
...
55 | }
| - temporary value is freed at the end of this statement
|
= note: consider using a `let` binding to create a longer lived value
I tried to create such binding, but I couldn't get it to work ...
entry.borrow() returns a so called guard. You have to keep this guard alive for the entire time you access the content.
As you are only using it inline, it goes out of scope right away and the borrowed value gets returned.
You need to store it in a local variable that stays alive for as long as you need the borrowed content.
That's only the first problem, though. The second one is that you can't you can't store &str that aren't &'static str in a vector. &str do not keep the content of the string alive, they are only references. To store them somewhere, you need the owning version, String.
use std::cell::RefCell;
use regex::Regex;
fn main() {
let lines = [Some("aaa"), Some("bbb")];
let mut handlers = Vec::<String>::new();
let entry = RefCell::new(String::new());
let re = Regex::new(r"(?m)(event\d+)").unwrap();
for line in lines {
let l = line.unwrap();
if !l.is_empty() {
entry.borrow_mut().push_str(&l);
continue;
}
if entry.borrow().contains("EV=120013") {
let entry_guard = entry.borrow();
if let Some(captures) = re.captures(&entry_guard) {
if let Some(m) = captures.get(0) {
handlers.push(m.as_str().to_string());
}
}
}
entry.borrow_mut().clear();
}
}
I'm new to Rust and still reading the Rust book. Below is my program.
use clap::{App, Arg};
type GenericError = Box<dyn std::error::Error + Send + Sync + 'static>;
type GenericResult<T> = Result<T, GenericError>;
fn main() -> GenericResult<()> {
let matches = App::new("test")
.arg(Arg::new("latency")
.takes_value(true))
.get_matches();
let latency_present = matches.is_present("latency");
let latency = matches.value_of("latency").unwrap_or("1000:10,300:30");
let latency_pairs: Vec<&str> = latency.split(",").collect();
let checker = std::thread::spawn(move || -> GenericResult<()>{
loop {
if latency_present {
for (i, latency_pair) in latency_pairs.iter().enumerate() {
// let latency_pair: Vec<&str> = latency_pair.split(":").collect();
// let latency = latency_pair[0].parse::<f64>().unwrap();
}
}
}
});
checker.join().unwrap()?;
Ok(())
}
When I run it, it tells me this:
error[E0597]: `matches` does not live long enough
--> src\main.rs:14:19
|
14 | let latency = matches.value_of("latency").unwrap_or("1000:10,300:30");
| ^^^^^^^--------------------
| |
| borrowed value does not live long enough
| argument requires that `matches` is borrowed for `'static`
...
30 | }
| - `matches` dropped here while still borrowed
I don't quite understand the error messages here. But I guess it's because I use latency_pairs in the checker thread and latency_pairs could get dropped while checker is still executing. Is my understanding correct? How to fix the error? I tried for (i, latency_pair) in latency_pairs.clone().iter().enumerate() { in order to pass a cloned value for the thread, but it doesn't help.
latency_pairs holds references into latency which in turn references matches. Thus cloning latency_pairs just clones the references into latency and matches.
Your code would require that latency's type is &'static str but it's actually &'a str where 'a is bound to matches' lifetime.
You can call to_owned() on latency to get an owned value and split the string inside the closure or you can call to_owned() on each of the splits collected in latency_pairs and move that Vec<String> into the closure:
let latency_pairs: Vec<String> = latency.split(",").map(ToOwned::to_owned).collect();
let checker = std::thread::spawn(move || -> GenericResult<()>{
loop {
if latency_present {
for (i, latency_pair) in latency_pairs.iter().enumerate() {
// let latency_pair: Vec<&str> = latency_pair.split(":").collect();
// let latency = latency_pair[0].parse::<f64>().unwrap();
}
}
}
});
If you need to use latency_pairs outside of the closure, you can clone it before moving it into the closure:
let latency_pairs: Vec<String> = latency.split(",").map(ToOwned::to_owned).collect();
let latency_pairs_ = latency_pairs.clone();
let checker = std::thread::spawn(move || -> GenericResult<()>{
loop {
if latency_present {
for (i, latency_pair) in latency_pairs_.iter().enumerate() {
// let latency_pair: Vec<&str> = latency_pair.split(":").collect();
// let latency = latency_pair[0].parse::<f64>().unwrap();
}
}
}
});
println!("{:?}", latency_pairs);
Here is the code:
fn test(){
let mut numbers = vec![2];
let f = || {
for _ in numbers.iter(){
}
false
};
while false {
let res = f();
if res {
numbers.push(10);
}
}
}
The error is:
|
15 | let f = || {
| -- immutable borrow occurs here
16 | for _ in numbers.iter(){
| ------- first borrow occurs due to use of `numbers` in closure
...
22 | let res = f();
| - immutable borrow later used here
23 | if res {
24 | numbers.push(10);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
But if I change the while keyword to if, it can be compiled. How to fix this? I want to call the anonymous function in a loop.
We can simplify your example even more by replacing the closure by a simple immutable reference
let mut numbers = vec![2];
let r = &numbers;
while false {
println!("{:?}", r);
numbers.push(10);
}
Here we get this error:
error[E0502]: cannot borrow `numbers` as mutable because it is also borrowed as immutable
--> src/lib.rs:7:5
|
3 | let r = &numbers;
| -------- immutable borrow occurs here
...
6 | println!("{:?}", r); // use reference
| - immutable borrow later used here
7 | numbers.push(10);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
And like in your example, replacing the while with if makes the error go away. Why?
You probably know about the important Rust rule: Aliasing nand mutability. It states that, at any given time, a value can either be borrowed immutably arbitrarily many times or mutably exactly once.
The statement numbers.push(10) borrows numbers mutably temporarily (just for the statement). But we also have r which is an immutable reference. In order for numbers.push(10) to work, the compiler has to make sure that no other borrow exists at that time. But there exists the reference r! This reference cannot exist at the same time as numbers.push(10) exists.
Let's see for the if case first:
let mut numbers = vec![2];
let r = &numbers; // <------+ (creation of r)
// |
if false { // |
println!("{:?}", r); // <------+ (last use of r)
numbers.push(10);
}
While the lexical scope means the variable r is only dropped at the end of the function, due to non-lexical lifetimes, the compiler can see that the last use of r is in the println line. Then the compiler can mark r as "dead" after this line. And this in turn means, that there is no other borrow in the line numbers.push(10) and everything works out fine.
And for the loop case? Let's imagine the loop iterating three times:
let mut numbers = vec![2];
let r = &numbers; // <------+ (creation of r)
// |
// First iteration // |
println!("{:?}", r); // |
numbers.push(10); // | <== oh oh!
// |
// Second iteration // |
println!("{:?}", r); // |
numbers.push(10); // |
// |
// Third iteration // |
println!("{:?}", r); // <------+ (last use of r)
numbers.push(10);
As can be seen here, the time in which r is active overlaps numbers.push(10) (except the last one). And as a result, the compiler will produce an error because this code violates the central Rust rule.
And the explanation is the same for your closure case: the closure borrows numbers immutably and f(); uses that closure. In the loop case, the compiler is not able to shrink the "alive time" of the closure enough to make sure it doesn't overlap the mutable borrow for push.
How to fix?
Well, you could pass numbers into the closure each time:
let mut numbers = vec![2];
let f = |numbers: &[i32]| {
for _ in numbers.iter(){
}
false
};
while false {
let res = f(&numbers);
if res {
numbers.push(10);
}
}
This works because now, numbers is borrowed immutably also just temporarily for the f(&numbers); statement.
You can also use a RefCell as the other answer suggested, but that should be a last resort.
It's not exactly sure what you're trying to accomplish, but one way to solve this, without changing your code too drastically, would be to use std::cell::RefCell (described in the std and in the book):
use std::cell::RefCell;
fn test(){
let numbers = RefCell::new(vec![2]);
let f = || {
for _ in numbers.borrow().iter(){
}
false
};
while false {
let res = f();
if res {
numbers.borrow_mut().push(10);
}
}
}
Here's a little bit tweaked demo, which actually does something:
use std::cell::RefCell;
fn main() {
test();
}
fn test() {
let numbers = RefCell::new(vec![0]);
let f = || {
for n in numbers.borrow().iter() {
println!("In closure: {}", n);
}
println!();
true
};
let mut i = 1;
while i <= 3 {
let res = f();
if res {
numbers.borrow_mut().push(i);
}
i += 1;
}
println!("End of test(): {:?}", numbers.borrow());
}
Output:
In closure: 0
In closure: 0
In closure: 1
In closure: 0
In closure: 1
In closure: 2
End of test(): [0, 1, 2, 3]
Rust Playground demo
Consider the following code:
use std::io::{self, BufRead, Read};
fn main() {
let mut stdin = io::stdin();
let mut content_length = 0;
for line_wrapped in stdin.lock().lines() {
let line = line_wrapped.unwrap();
if line == "" {
let mut buf = vec![0u8; content_length];
stdin.read_exact(&mut buf).unwrap();
print!("{:?}", buf);
}
if line.starts_with("Content-Length: ") {
content_length = line
.split("Content-Length: ")
.nth(1)
.unwrap()
.parse()
.unwrap();
}
}
}
And the compiler output:
error[E0502]: cannot borrow `stdin` as mutable because it is also borrowed as immutable
--> src/main.rs:11:13
|
7 | for line_wrapped in stdin.lock().lines() {
| ----- immutable borrow occurs here
...
11 | stdin.read_exact(&mut buf).unwrap();
| ^^^^^ mutable borrow occurs here
...
22 | }
| - immutable borrow ends here
Is there a way I could fix the error while keeping a similar structure of the program (read within a .lines())?
Alternating between buffered and non-buffered reads of the same stream can be quite tricky. If you didn't have to lock standard input in order to call lines(), the internal buffer used to implement StdinLock could consume beyond the \n of the empty line, and the subsequent read_exact call would not start at the right place.
So you have to lock it only once, and you have to call read_exact on the same buffered reader that gave you the Lines, to be sure no bytes are lost. At first glance this looks impossible: lines() takes self by value, so once you've called it, you can't call read_exact on the same object. But there's a bit of a trick you can use.
The documentation for BufRead contains this blanket impl:
impl<'a, B: BufRead + ?Sized> BufRead for &'a mut B
&mut references to things that implement BufRead also implement BufRead. So you can take a temporary &mut reference of your StdinLock, call lines() on that, discard the Lines in time to read_exact the payload into buf, and then start over again with another &mut reference and another Lines.
This approach necessitates adding another loop, with a flag has_header to indicate whether to break the outer loop. It's not very pretty, but maybe you can work with it.
let stdin = io::stdin();
let mut stdin_buf = stdin.lock();
'record: loop {
let mut content_length = 0;
let mut has_header = false;
'header: for line_wrapped in (&mut stdin_buf).lines() {
let line = line_wrapped.unwrap();
if line.starts_with("Content-Length: ") {
content_length = line["Content-Length: ".len()..].parse().unwrap();
}
if line.is_empty() {
has_header = true;
break 'header;
}
}
if has_header {
let mut buf = vec![0u8; content_length];
stdin_buf.read_exact(&mut buf).unwrap();
println!("{:?}", buf);
} else {
break 'record;
}
}
A final note: It's unclear what should happen when the Content-Length header is absent. If your original code worked, it would reuse the previously defined value (whatever the last content length was, or 0 for the first record). My version just uses 0 all the time. It's idiomatic in Rust to use Option to represent possibly-uninitialized values such as content_length, so initializing it to None could be a better option if the absence of a Content-Length header is an error that needs to be detected.