What's the proper way to use variables defined in the main thread in a child thread in Rust? - multithreading

I'm new to Rust and still reading the Rust book. Below is my program.
use clap::{App, Arg};
type GenericError = Box<dyn std::error::Error + Send + Sync + 'static>;
type GenericResult<T> = Result<T, GenericError>;
fn main() -> GenericResult<()> {
let matches = App::new("test")
.arg(Arg::new("latency")
.takes_value(true))
.get_matches();
let latency_present = matches.is_present("latency");
let latency = matches.value_of("latency").unwrap_or("1000:10,300:30");
let latency_pairs: Vec<&str> = latency.split(",").collect();
let checker = std::thread::spawn(move || -> GenericResult<()>{
loop {
if latency_present {
for (i, latency_pair) in latency_pairs.iter().enumerate() {
// let latency_pair: Vec<&str> = latency_pair.split(":").collect();
// let latency = latency_pair[0].parse::<f64>().unwrap();
}
}
}
});
checker.join().unwrap()?;
Ok(())
}
When I run it, it tells me this:
error[E0597]: `matches` does not live long enough
--> src\main.rs:14:19
|
14 | let latency = matches.value_of("latency").unwrap_or("1000:10,300:30");
| ^^^^^^^--------------------
| |
| borrowed value does not live long enough
| argument requires that `matches` is borrowed for `'static`
...
30 | }
| - `matches` dropped here while still borrowed
I don't quite understand the error messages here. But I guess it's because I use latency_pairs in the checker thread and latency_pairs could get dropped while checker is still executing. Is my understanding correct? How to fix the error? I tried for (i, latency_pair) in latency_pairs.clone().iter().enumerate() { in order to pass a cloned value for the thread, but it doesn't help.

latency_pairs holds references into latency which in turn references matches. Thus cloning latency_pairs just clones the references into latency and matches.
Your code would require that latency's type is &'static str but it's actually &'a str where 'a is bound to matches' lifetime.
You can call to_owned() on latency to get an owned value and split the string inside the closure or you can call to_owned() on each of the splits collected in latency_pairs and move that Vec<String> into the closure:
let latency_pairs: Vec<String> = latency.split(",").map(ToOwned::to_owned).collect();
let checker = std::thread::spawn(move || -> GenericResult<()>{
loop {
if latency_present {
for (i, latency_pair) in latency_pairs.iter().enumerate() {
// let latency_pair: Vec<&str> = latency_pair.split(":").collect();
// let latency = latency_pair[0].parse::<f64>().unwrap();
}
}
}
});
If you need to use latency_pairs outside of the closure, you can clone it before moving it into the closure:
let latency_pairs: Vec<String> = latency.split(",").map(ToOwned::to_owned).collect();
let latency_pairs_ = latency_pairs.clone();
let checker = std::thread::spawn(move || -> GenericResult<()>{
loop {
if latency_present {
for (i, latency_pair) in latency_pairs_.iter().enumerate() {
// let latency_pair: Vec<&str> = latency_pair.split(":").collect();
// let latency = latency_pair[0].parse::<f64>().unwrap();
}
}
}
});
println!("{:?}", latency_pairs);

Related

Cannot borrow value from a hashmap as mutable because it is also borrowed as immutable

I want to get two values from a hashmap at the same time, but I can't escape the following error, I have simplified the code as follows, can anyone help me to fix this error.
#[warn(unused_variables)]
use hashbrown::HashMap;
fn do_cal(a: &[usize], b: &[usize]) -> usize {
a.iter().sum::<usize>() + b.iter().sum::<usize>()
}
fn do_check(i: usize, j:usize) -> bool {
i/2 < j - 10
}
fn do_expensive_cal(i: usize) -> Vec<usize> {
vec![i,i,i]
}
fn main() {
let size = 1000000;
let mut hash: HashMap<usize, Vec<usize>> = HashMap::new();
for i in 0..size{
if i > 0 {
hash.remove(&(i - 1));
}
if !hash.contains_key(&i){
hash.insert(i, do_expensive_cal(i));
}
let data1 = hash.get(&i).unwrap();
for j in i + 1..size {
if do_check(i, j) {
break
}
if !hash.contains_key(&j){
hash.insert(j, do_expensive_cal(j));
}
let data2 = hash.get(&j).unwrap();
let res = do_cal(data1, data2);
println!("res:{}", res);
}
}
}
Playground
error[E0502]: cannot borrow hash as mutable because it is also borrowed as immutable
--> src/main.rs:26:8
|
19 | let data1 = hash.get(&i).unwrap();
| ------------ immutable borrow occurs here
...
26 | hash.insert(j, vec![1,2,3]);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
...
29 | let res = do_cal(data1, data2);
| ----- immutable borrow later used here
For more information about this error, try rustc --explain E0502.
error: could not compile playground due to previous error
Consider this: the borrow checker doesn't know that hash.insert(j, …) will leave the data you inserted with hash.insert(i, …) alone. For the borrow checker, hash.insert(…) may do anything to any element in hash, including rewriting or removing it. So you can't be allowed to hold the reference data1 over hash.insert(j, …).
How to get over that? The easiest is probably to move let data1 = hash.get(…) so it doesn't have to live for so long:
let data1 = hash.get(&i).unwrap();
let data2 = hash.get(&j).unwrap();
let res = do_cal(data1, data2);
This will of course look up data1 every loop iteration (and it must, since hash.insert(j, …) may have resized and thus realocated the content of the hashmap, giving data1 a new storage location in the hashmap). For completeness's sake, there are ways to get around that, but I don't recommend you do any of this:
Clone: let data1 = hash.get(&i).unwrap().clone() (if your vecs are short, this may actually be reasonable…)
As a way of making the cloning cheap, you could use a HashMap<usize, Rc<Vec<usize>>> instead (where you only need to clone the Rc, no the entire Vec)
If you ever need mutable references to both arguments of do_call, you could combine the Rc with a RefCell: Rc<RefCell<Vec<…>>>
If you need to overengineer it even more, you could replace the Rcs with references obtained from allocating in a bump allocator, e.g. bumpalo.
Since the keys to the hash table are the integers 0..100 you can use a Vec to perform these steps, temporarily splitting the Vec into 2 slices to allow the mutation on one side. If you need a HashMap for later computations, you can then create a HashMap from the Vec.
The following code compiles but panics because the j - 10 calculation underflows:
fn do_cal(a: &[usize], b: &[usize]) -> usize {
a.iter().sum::<usize>() + b.iter().sum::<usize>()
}
fn do_check(i: usize, j:usize) -> bool {
i/2 < j - 10
}
fn main() {
let size = 100;
let mut v: Vec<Option<Vec<usize>>> = vec![None; size];
for i in 0..size {
let (v1, v2) = v.split_at_mut(i + 1);
if v1[i].is_none() {
v1[i] = Some(vec![1,2,3]);
}
let data1 = v1[i].as_ref().unwrap();
for (j, item) in (i + 1..).zip(v2.iter_mut()) {
if do_check(i, j) {
break
}
if item.is_none() {
*item = Some(vec![1,2,3]);
}
let data2 = item.as_ref().unwrap();
let res = do_cal(data1, data2);
println!("res:{}", res);
}
}
}

Creating threads from a list of tasks

I try to set up my first-ever thread machine and ran into the following issue.
We have this state:
use std::{thread, sync::mpsc};
fn main() {
let tasks = vec![String::from("task1"), String::from("task2")];
let (tx, rx) = mpsc::channel();
let tx1 = tx.clone();
let handle = thread::spawn(move || {
for i in 0..(tasks.len()/2 - 1){
let send = format!("{}, {}", tasks[i], String::from("done"));
tx.send(send).unwrap();
}
});
for i in tasks.len()/2..tasks.len() - 1 {
let send = format!("{}, {}", tasks[i], String::from("done"));
tx1.send(send).unwrap();
}
handle.join().unwrap();
for string in rx {
println!("{}", string);
}
}
I try to set up parallel threads, which do something with the strings in the vector. They should work and send until the whole vector is covered. The vector has reliable 100 entries.
The corresponding error message:
error[E0382]: borrow of moved value: `tasks`
--> src/main.rs:19:14
|
6 | let tasks = vec![String::from("task1"), String::from("task2")];
| ----- move occurs because `tasks` has type `Vec<String>`, which does not implement the `Copy` trait
...
11 | let handle = thread::spawn(move || {
| ------- value moved into closure here
12 | for i in 0..(tasks.len()/2 - 1){
| ----- variable moved due to use in closure
...
19 | for i in tasks.len()/2..tasks.len() - 1 {
| ^^^^^ value borrowed here after move
error: aborting due to previous error
For more information about this error, try `rustc --explain E0382`.
It seems like I really do not understand how to set up parallel threads and how to treat channels together with threads and a list of tasks.
As the compiler indicates, the problem is that you moved the vector of tasks when spawning a thread, and then you try to borrow the vector from the main thread while it was already moved.
The problem you need to solve here is how to share the vectors' ownership between the main thread and the spawned thread. One way to do it is to use Arc.
Based on your code, I have wrote an example,
use std::{thread, sync::mpsc};
use std::sync::Arc;
fn main() {
let tasks = Arc::new(vec![String::from("task1"), String::from("task2")]);
let tasks_arc_clone = Arc::clone(&tasks);
let (tx, rx) = mpsc::channel();
let tx1 = tx.clone();
let mid_idx = tasks.len() / 2 - 1;
let handle = thread::spawn(move || {
for i in 0..mid_idx {
let send = format!("{}, {}", tasks_arc_clone[i], String::from("done"));
tx.send(send).unwrap();
}
});
for i in mid_idx..tasks.len() {
let send = format!("{}, {}", tasks[i], String::from("done"));
tx1.send(send).unwrap();
}
drop(tx1);
for string in rx {
println!("{}", string);
}
handle.join().unwrap()
}
Rust playground link, in case you want to play with it.

How to fix use of moved value in Rust?

I am trying to convert a yaml file to xml using Rust and I am not able to figure out how to fix this error regarding the use of moved value. I think I understand why this error is coming, but haven't got a clue about what to do next.
Here's the code:
struct Element {
element_name: String,
indentation_count: i16,
}
struct Attribute<'a> {
attribute_name: &'a str,
attribute_value: &'a str,
}
fn convert_yaml_to_xml(content: String, indentation_count: i16) -> String {
let mut xml_elements: Vec<Element> = vec![];
let mut attributes: Vec<Attribute> = vec![];
xml_elements.push(Element {element_name: "xmlRoot".to_string(), indentation_count: -1});
let mut target: Vec<u8> = Vec::new();
let mut xml_data_writer = EmitterConfig::new().perform_indent(true).create_writer(&mut target);
let mut attribute_written_flag = false;
let mut xml_event;
xml_event = XmlEvent::start_element("xmlRoot");
for line in content.lines() {
let current_line = line.trim();
let caps = indentation_count_regex.captures(current_line).unwrap();
let current_indentation_count = caps.get(1).unwrap().as_str().to_string().len() as i16;
if ELEMENT_REGEX.is_match(current_line) {
loop {
let current_attribute_option = attributes.pop();
match current_attribute_option {
Some(current_attribute_option) => {
xml_event.attr(current_attribute_option.attribute_name, current_attribute_option.attribute_value)
},
None => {
break;
},
};
}
xml_data_writer.write(xml_event);
// Checking if the line is an element
let caps = ELEMENT_REGEX.captures(current_line).unwrap();
let element_name = caps.get(2);
let xml_element_struct = Element {
indentation_count: current_indentation_count,
element_name: element_name.unwrap().as_str().to_string(),
};
xml_elements.push(xml_element_struct);
xml_event = XmlEvent::start_element(element_name.unwrap().as_str());
attribute_written_flag = false;
} else if ATTR_REGEX.is_match(current_line) {
// Checking if the line is an attribute
let caps = ATTR_REGEX.captures(current_line).unwrap();
let attr_name = caps.get(2);
let attr_value = caps.get(3);
// Saving attributes to a stack
attributes.push(Attribute{ attribute_name: attr_name.unwrap().as_str(), attribute_value: attr_value.unwrap().as_str() });
// xml_event.attr(attr_name.unwrap().as_str(), attr_value.unwrap().as_str());
}/* else if NEW_ATTR_SET_REGEX.is_match(current_line) {
let caps = NEW_ATTR_SET_REGEX.captures(current_line).unwrap();
let new_attr_set_name = caps.get(2);
let new_attr_set_value = caps.get(3);
current_xml_hash.insert("name".to_string(), new_attr_set_name.unwrap().as_str().to_string());
current_xml_hash.insert("value".to_string(), new_attr_set_value.unwrap().as_str().to_string());
} */
}
if attribute_written_flag {
xml_data_writer.write(xml_event);
}
for item in xml_elements.iter() {
let event = XmlEvent::end_element();
let event_name = item.element_name.to_string();
xml_data_writer.write(event.name(event_name.as_str()));
}
println!("OUTPUT");
println!("{:?}", target);
return "".to_string();
}
And here's the error:
error[E0382]: use of moved value: `xml_event`
--> src/main.rs:77:25
|
65 | let mut xml_event;
| ------------- move occurs because `xml_event` has type `StartElementBuilder<'_>`, which does not implement the `Copy` trait
...
77 | xml_event.attr(current_attribute_option.attribute_name, current_attribute_option.attribute_value)
| ^^^^^^^^^ --------------------------------------------------------------------------------------- `xml_event` moved due to this method call, in previous iteration of loop
|
note: this function takes ownership of the receiver `self`, which moves `xml_event`
--> /Users/defiant/.cargo/registry/src/github.com-1ecc6299db9ec823/xml-rs-0.8.4/src/writer/events.rs:193:24
|
193 | pub fn attr<N>(mut self, name: N, value: &'a str) -> StartElementBuilder<'a>
| ^^^^
From XmlEvent::start_element() documentation we see that it produces a StartElementBuilder<'a>.
From StartElementBuilder<'a>::attr() documentation we see that it consumes the StartElementBuilder<'a> (the first parameter is self, not &mut self) and produces a new StartElementBuilder<'a> (which is probably similar to self but considers the expected effect of .attr()).
This approach is known as the consuming builder pattern, which is used in Rust (for example std::thread::Builder).
The typical usage of such an approach consists in chaining the function calls: something.a().b().c().d() such as something is consumed by a(), its result is consumed by b(), the same about c() and finally d() does something useful with the last result.
The alternative would be to use mutable borrows in order to modify in place something but dealing with mutable borrows is known as difficult in some situations.
In your case, you can just reassign the result of .attr() to xml_event because otherwise the .attr() function would have no effect (its result is discarded) and xml_event would become unusable because it is consumed; reassigning it makes it usable again afterwards (at least i guess, i didn't try).

Iterate through a whole file one character at a time

I'm new to Rust and I'm struggle with the concept of lifetimes. I want to make a struct that iterates through a file a character at a time, but I'm running into issues where I need lifetimes. I've tried to add them where I thought they should be but the compiler isn't happy. Here's my code:
struct Advancer<'a> {
line_iter: Lines<BufReader<File>>,
char_iter: Chars<'a>,
current: Option<char>,
peek: Option<char>,
}
impl<'a> Advancer<'a> {
pub fn new(file: BufReader<File>) -> Result<Self, Error> {
let mut line_iter = file.lines();
if let Some(Ok(line)) = line_iter.next() {
let char_iter = line.chars();
let mut advancer = Advancer {
line_iter,
char_iter,
current: None,
peek: None,
};
// Prime the pump. Populate peek so the next call to advance returns the first char
let _ = advancer.next();
Ok(advancer)
} else {
Err(anyhow!("Failed reading an empty file."))
}
}
pub fn next(&mut self) -> Option<char> {
self.current = self.peek;
if let Some(char) = self.char_iter.next() {
self.peek = Some(char);
} else {
if let Some(Ok(line)) = self.line_iter.next() {
self.char_iter = line.chars();
self.peek = Some('\n');
} else {
self.peek = None;
}
}
self.current
}
pub fn current(&self) -> Option<char> {
self.current
}
pub fn peek(&self) -> Option<char> {
self.peek
}
}
fn main() -> Result<(), Error> {
let file = File::open("input_file.txt")?;
let file_buf = BufReader::new(file);
let mut advancer = Advancer::new(file_buf)?;
while let Some(char) = advancer.next() {
print!("{}", char);
}
Ok(())
}
And here's what the compiler is telling me:
error[E0515]: cannot return value referencing local variable `line`
--> src/main.rs:37:13
|
25 | let char_iter = line.chars();
| ---- `line` is borrowed here
...
37 | Ok(advancer)
| ^^^^^^^^^^^^ returns a value referencing data owned by the current function
error[E0597]: `line` does not live long enough
--> src/main.rs:49:34
|
21 | impl<'a> Advancer<'a> {
| -- lifetime `'a` defined here
...
49 | self.char_iter = line.chars();
| -----------------^^^^--------
| | |
| | borrowed value does not live long enough
| assignment requires that `line` is borrowed for `'a`
50 | self.peek = Some('\n');
51 | } else {
| - `line` dropped here while still borrowed
error: aborting due to 2 previous errors
Some errors have detailed explanations: E0515, E0597.
For more information about an error, try `rustc --explain E0515`.
error: could not compile `advancer`.
Some notes:
The Chars iterator borrows from the String it was created from. So you can't drop the String while the iterator is alive. But that's what happens in your new() method, the line variable owning the String disappears while the iterator referencing it is stored in the struct.
You could also try storing the current line in the struct, then it would live long enough, but that's not an option – a struct cannot hold a reference to itself.
Can you make a char iterator on a String that doesn't store a reference into the String? Yes, probably, for instance by storing the current position in the string as an integer – it shouldn't be the index of the char, because chars can be more than one byte long, so you'd need to deal with the underlying bytes yourself (using e.g. is_char_boundary() to take the next bunch of bytes starting from your current index that form a char).
Is there an easier way? Yes, if performance is not of highest importance, one solution is to make use of Vec's IntoIterator instance (which uses unsafe magic to create an object that hands out parts of itself) :
let char_iter = file_buf.lines().flat_map(|line_res| {
let line = line_res.unwrap_or(String::new());
line.chars().collect::<Vec<_>>()
});
Note that just returning line.chars() would have the same problem as the first point.
You might think that String should have a similar IntoIterator instance, and I wouldn't disagree.

How does ownership and borrowing work when making slices?

Here are two code snippets, but they show a different behavior and I couldn't understand what's going on in there. Their main function is identical. If the borrowing and ownership concept applies to one code (i.e to code 2) why not to another code i.e (code 1)?
code 1:
This code compiles with no errors and prompt the result.
fn main() {
let mut s = String::from("Hello world");
let result = first_word(&s);
s.clear();
println!("result:{:#?}", result);
}
fn first_word(s: &String) -> usize {
let s = s.as_bytes();
//println!("{:?}",s);
for (i, &item) in s.iter().enumerate() {
if item == 32 {
return i;
}
}
s.len()
}
Code 1 Output :
Finished dev [unoptimized + debuginfo] target(s) in 0.28s
Running `target/debug/rust_Slices`
result:5
Code 2:
This code won't compile and gives an error.
fn main() {
let mut s = String::from("Hello world");
let result = first_word(&s);
s.clear();
println!("{:#?}", result);
}
fn first_word(s: &String) -> &str {
let bytes = s.as_bytes();
for (i, &item) in bytes.iter().enumerate() {
if item == b' ' {
return &s[0..i];
}
}
&s[..]
}
Code 2 Output:
cannot borrow `s` as mutable because it is also borrowed as immutable
--> src/main.rs:4:4
|
3 | let result = first_word(&s);
| -- immutable borrow occurs here
4 | s.clear();
| ^^^^^^^^^ mutable borrow occurs here
5 | println!("{:#?}",result);
| ------ immutable borrow later used here
Let's decompose:
// Let's build a string, which basically is a growable array of chars
let mut s = String::from("Hello world");
// now make result a slice over that string, that is a reference
// to a part of the underlying chars
let result = first_word(&s);
// now let's remove the content of the string (which of course removes
// what the slice was referring to)
s.clear();
// and let's... print it ?
println!("{:#?}", result);
Hopefully the borrow checker prevents you from doing this with this exact error:
cannot borrow s as mutable because it is also borrowed as immutable
And if you've understood this, the solution should be obvious: don't make result a window over another string but a string by itself, having its own content: change the second line to
let result = first_word(&s).to_string();
Now you can clear the source string and keep the first word. Of course to_string() isn't a costless operation so you might want to try keep the source string around in real applications.
The key thing here is lifetimes. By default lifetimes argument for functions with one input reference and output reference are the same (liftime elision). So compiler implicitly changes the code following way:
fn first_word<'a>(s: &'a String) -> &'a str { // note 'a here
let bytes = s.as_bytes();
for (i, &item) in bytes.iter().enumerate() {
if item == b' ' {
return &s[0..i];
}
}
&s[..]
}
That means that the result borrows the input argument. You can explicitly make lifetimes different and eliminate error in the main but in this case first_word will not compile:
fn first_word1<'a, 'b>(s: &'a String) -> &'b str {
let bytes = s.as_bytes();
for (i, &item) in bytes.iter().enumerate() {
if item == b' ' {
return &s[0..i];
}
}
&s[..]
}
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> src/main.rs:7:21
|
7 | return &s[0..i];
| ^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime 'a as defined on the function body

Resources