from rust beginner.
I have some problems about ownership I think. What i wanna do is changing "ret" which is boolean
type variable inside the pool block. But when i ran the code and checked the ret, it changed well inside the pool block but outside the block, ret alway behave as true,,,
plz fix my headache...
let mut pool = Pool::new(max_worker);
let mut ret = true;
pool.scoped(|scoped| {
for i in 0..somevalue{
scoped.execute( move || {
let ret_ref = &mut ret;
// Do Something
if success {
*ret_ref = false
}
});
}
});
if ret == true { /* Do Something */ }
scoped_threadpool::Pool::scoped(&mut self, <closure>) returns a closure that impls FnOnce which means you can only call it once. You had it inside a for loop which is why the compiler kept giving you errors with confusing suggestions. Once you refactor the code to move the for outside the call to scoped then it compiles and works as expected:
use scoped_threadpool::Pool;
fn main() {
let max_workers = 1;
let somevalue = 1;
let mut pool = Pool::new(max_workers);
let mut ret = true;
let ret_ref = &mut ret;
for i in 0..somevalue {
pool.scoped(|scoped| {
scoped.execute(|| {
// do something
let success = true;
if success {
*ret_ref = false
}
});
});
}
if ret == true {
println!("ret stayed true");
} else {
// prints this
println!("ret was changed to false in the scoped thread");
}
}
playground
Related
I'm trying to do some parallel processing on a list of values:
fn process_list(list: Vec<f32>) -> Vec<f32> { // Block F
let chunk_size = 100;
let output_list = vec![0.0f32;list.len()];
thread::scope(|s| { // Block S
(0..list.len()).collect::<Vec<_>>().chunks(chunk_size).for_each(|chunk| { // Block T
s.spawn(|| {
chunk.into_iter().for_each(|&idx| {
let value = calc_value(list[idx]);
unsafe {
let out = (output_list.as_ptr() as *mut f32).offset(idx as isize);
*out = value;
}
});
});
});
});
output_list
}
The API says that thread::scope() only returns once each thread spawned by the scope it creates has finished. However, the compiler is telling me that the temporary range object (0..list.len()) is deconstructed while the threads that use it might still be alive.
I'm curious about what's actually happening under the hood. My intuition tells me that each thread spawned and variable created within Block S would both have Block S's lifetime. But clearly the threads have a lifetime longer than Block S.
Why aren't these lifetimes be the same?
Is the best practice here to create a variable in Block F that serves the purpose of the temporary like so:
fn process_list(list: Vec<f32>) -> Vec<f32> { // Block F
let chunk_size = 100;
let output_list = vec![0.0f32;list.len()];
let range = (0..list.len()).collect::<Vec<_>>();
thread::scope(|s| { // Block S
range.chunks(chunk_size).for_each(|chunk| { // Block T
s.spawn(|| {
chunk.into_iter().for_each(|&idx| {
let value = calc_value(list[idx]);
unsafe {
let out = (output_list.as_ptr() as *mut f32).offset(idx as isize);
*out = value;
}
});
});
});
});
output_list
}
You don't have to ask SO to find out what happens under the hood of std functions, their source code is readily available, but since you asked I'll try to explain a little more in the comments.
pub fn scope<'env, F, T>(f: F) -> T
where
F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T,
{
// `Scope` creation not very relevant to the issue at hand
let scope = ...;
// here your 'Block S' gets run and returns.
let result = catch_unwind(AssertUnwindSafe(|| f(&scope)));
// the above is just a fancy way of calling `f` while catching any panics.
// but we're waiting for threads to finish running until here
while scope.data.num_running_threads.load(Ordering::Acquire) != 0 {
park();
}
// further not so relevant cleanup code
//...
}
So as you can see your assumption that 'Block S' will stick around as long as any of the threads is wrong.
And yes the solution is to capture the owner of the chunks before you call thread::scope.
There is also no reason to dive into unsafe for your example, you can use zip instead:
fn process_list(list: Vec<f32>) -> Vec<f32> { // Block F
let chunk_size = 100;
let mut output_list = vec![0.0f32; list.len()];
let mut zipped = list
.into_iter()
.zip(output_list.iter_mut())
.collect::<Vec<_>>();
thread::scope(|s| { // Block S
zipped.chunks_mut(chunk_size).for_each(|chunk| { // Block T
s.spawn(|| {
chunk.into_iter().for_each(|(v, out)| {
let value = calc_value(*v);
**out = value;
});
});
});
});
output_list
}
The test code below considers a situation in which there are three different threads.
Each thread has to do certain asynchronous tasks, that may take a certain time to finish.
This is "simulated" in the code below with a sleep.
On top of that, two of the threads collect information that they have to send to the third one for further processing. This is done using mpsc channels.
Due to the fact that there are out of our control information obtained from outside of the Rust application, the threads may get interrupted. This is emulated by generating a random number, and the loop on each thread breaks when that happens.
What I'm trying to achieve is a system in which whenever one of the threads has an error (simulated with the random number = 9), every other thread is cancelled too.
`
use std::sync::mpsc::channel;
use std::sync::mpsc::{Sender, Receiver, TryRecvError};
use std::thread::sleep;
use tokio::time::Duration;
use rand::distributions::{Uniform, Distribution};
#[tokio::main]
async fn main() {
execution_cycle().await;
}
async fn execution_cycle() {
let (tx_first, rx_first) = channel::<Message>();
let (tx_second, rx_second) = channel::<Message>();
let handle_sender_first = tokio::spawn(sender_thread(tx_first));
let handle_sender_second = tokio::spawn(sender_thread(tx_second));
let handle_receiver = tokio::spawn(receiver_thread(rx_first, rx_second));
let mut thread_rng = rand::thread_rng();
let rng_generator = Uniform::from(1..10);
let mut cancel_from_cycle = rng_generator.sample(&mut thread_rng);
while !&handle_sender_first.is_finished() && !&handle_sender_second.is_finished() && !&handle_receiver.is_finished() {
cancel_from_cycle = rng_generator.sample(&mut thread_rng);
if (cancel_from_cycle == 9) {
println!("Aborting from the execution cycle.");
handle_receiver.abort();
handle_sender_first.abort();
handle_sender_second.abort();
}
}
if handle_sender_first.is_finished() {
println!("handle_sender_first finished.");
} else {
println!("handle_sender_first ongoing.");
}
if handle_sender_second.is_finished() {
println!("handle_sender_second finished.");
} else {
println!("handle_sender_second ongoing.");
}
if handle_receiver.is_finished() {
println!("handle_receiver finished.");
} else {
println!("handle_receiver ongoing.");
}
}
async fn sender_thread(tx: Sender<Message>) {
let mut thread_rng = rand::thread_rng();
let rng_generator = Uniform::from(1..20);
let mut random_id = rng_generator.sample(&mut thread_rng);
while random_id != 9 {
let msg = Message {
id: random_id,
text: "hello".to_owned()
};
println!("Sending message {}.", msg.id);
random_id = rng_generator.sample(&mut thread_rng);
println!("Generated id {}.", random_id);
let result = tx.send(msg);
match result {
Ok(res) => {},
Err(error) => {
println!("Sending error {:?}", error);
random_id = 9;
}
}
sleep(Duration::from_millis(2000));
}
}
async fn receiver_thread(rx_first: Receiver<Message>, rx_second: Receiver<Message>) {
let mut channel_open_first = true;
let mut channel_open_second = true;
let mut thread_rng = rand::thread_rng();
let rng_generator = Uniform::from(1..15);
let mut random_event = rng_generator.sample(&mut thread_rng);
while channel_open_first && channel_open_second && random_event != 9 {
channel_open_first = receiver_inner(&rx_first);
channel_open_second = receiver_inner(&rx_second);
random_event = rng_generator.sample(&mut thread_rng);
println!("Generated event {}.", random_event);
sleep(Duration::from_millis(800));
}
}
fn receiver_inner(rx: &Receiver<Message>) -> bool {
let value = rx.try_recv();
match value {
Ok(msg) => {
println!("Message {} received: {}", msg.id, msg.text);
},
Err(error) => {
if error != TryRecvError::Empty {
println!("{}", error);
return false;
} else { /* Channel is empty.*/ }
}
}
return true;
}
struct Message {
id: usize,
text: String,
}
`
In the working example here, it does exactly that, however, it does it only from inside the threads, and I would like to add a "kill switch" in the execution_cycle() method, allowing to cancel all the three threads when a certain event takes place (the random number cancel_from_cycle == 9), and do that in the most simple way possible... I tried drop(handler_sender), and also panic!() from the execution_cycle() but the spawn threads keep running, preventing the application to finish. I also tried handle_receiver().abort() without success.
How can I achieve the wished result?
i'm new to rust.
I'm trying to write a file_sensor that will start a counter after a file is created. The plan is that after an amount of time, if a second file is not received the sensor will exit with a zero exit code.
I could write the code to continue that work but i feel the code below illustrates the problem (i have also missed for example the post function referred to)
I have been struggling with this problem for several hours, i've tried Arc and mutex's and even global variables.
The Timer implementation is Ticktock-rs
I need to be able to either get heartbeat in the match body for EventKind::Create(CreateKind::Folder) or file_count in the loop
The code i've attached here runs but file_count is always zero in the loop.
use std::env;
use std::path::Path;
use std::{thread, time};
use std::process::ExitCode;
use ticktock::Timer;
use notify::{
Watcher,
RecommendedWatcher,
RecursiveMode,
Result,
event::{EventKind, CreateKind, ModifyKind, Event}
};
fn main() -> Result<()> {
let now = time::Instant::now();
let mut heartbeat = Timer::apply(
|_, count| {
*count += 1;
*count
},
0,
)
.every(time::Duration::from_millis(500))
.start(now);
let mut file_count = 0;
let args = Args::parse();
let REQUEST_SENSOR_PATH = env::var("REQUEST_SENSOR_PATH").expect("$REQUEST_SENSOR_PATH} is not set");
let mut watcher = notify::recommended_watcher(move|res: Result<Event>| {
match res {
Ok(event) => {
match event.kind {
EventKind::Create(CreateKind::File) => {
file_count += 1;
// do something with file
}
_ => { /* something else changed */ }
}
println!("{:?}", event);
},
Err(e) => {
println!("watch error: {:?}", e);
ExitCode::from(101);
},
}
})?;
watcher.watch(Path::new(&REQUEST_SENSOR_PATH), RecursiveMode::Recursive)?;
loop {
let now = time::Instant::now();
if let Some(n) = heartbeat.update(now){
println!("Heartbeat: {}, fileCount: {}", n, file_count);
if n > 10 {
heartbeat.set_value(0);
// This function will reset timer when a file arrives
}
}
}
Ok(())
}
Your compiler warnings show you the problem:
warning: unused variable: `file_count`
--> src/main.rs:31:25
|
31 | file_count += 1;
| ^^^^^^^^^^
|
= note: `#[warn(unused_variables)]` on by default
= help: did you mean to capture by reference instead?
The problem here is that you use file_count inside of a move || closure. file_count is an i32, which is Copy. Using it in a move || closure actually creates a copy of it, which does no longer update the original variable if you assign to it.
Either way, it's impossible to modify a variable in main() from an event handler. Event handlers require 'static lifetime if they reference things, because Rust cannot guarantee that the event handler lives shorter than main.
One solution for this problem is to use reference counters and interior mutability. In this case, I will use Arc for reference counters and AtomicI32 for interior mutability. Note that notify::recommended_watcher requires thread safety, otherwise instead of an Arc<AtomicI32> we could have used an Rc<Cell<i32>>, which is the same thing but only for single-threaded environments, with a little less overhead.
use notify::{
event::{CreateKind, Event, EventKind},
RecursiveMode, Result, Watcher,
};
use std::time;
use std::{env, sync::atomic::Ordering};
use std::{path::Path, sync::Arc};
use std::{process::ExitCode, sync::atomic::AtomicI32};
use ticktock::Timer;
fn main() -> Result<()> {
let now = time::Instant::now();
let mut heartbeat = Timer::apply(
|_, count| {
*count += 1;
*count
},
0,
)
.every(time::Duration::from_millis(500))
.start(now);
let file_count = Arc::new(AtomicI32::new(0));
let REQUEST_SENSOR_PATH =
env::var("REQUEST_SENSOR_PATH").expect("$REQUEST_SENSOR_PATH} is not set");
let mut watcher = notify::recommended_watcher({
let file_count = Arc::clone(&file_count);
move |res: Result<Event>| {
match res {
Ok(event) => {
match event.kind {
EventKind::Create(CreateKind::File) => {
file_count.fetch_add(1, Ordering::AcqRel);
// do something with file
}
_ => { /* something else changed */ }
}
println!("{:?}", event);
}
Err(e) => {
println!("watch error: {:?}", e);
ExitCode::from(101);
}
}
}
})?;
watcher.watch(Path::new(&REQUEST_SENSOR_PATH), RecursiveMode::Recursive)?;
loop {
let now = time::Instant::now();
if let Some(n) = heartbeat.update(now) {
println!(
"Heartbeat: {}, fileCount: {}",
n,
file_count.load(Ordering::Acquire)
);
if n > 10 {
heartbeat.set_value(0);
// This function will reset timer when a file arrives
}
}
}
}
Also, note that the ExitCode::from(101); gives you a warning. It does not actually exit the program, it only creates an exit code variable and then discards it again. You probably intended to write std::process::exit(101);. Although I would discourage it, because it does not properly clean up (does not call any Drop implementations). I'd use panic here, instead. This is the exact usecase panic is meant for.
How can I print the frequencies HashSet after the loop terminates? Problem seems to be there is no guarantee the loop terminates.
use std::collections::HashSet;
use std::fs;
fn main() {
let f = fs::read_to_string("input.txt").expect("Unable to open file");
let mut total = 0;
let mut frequencies = HashSet::new();
frequencies.insert(0);
loop {
for line in f.lines() {
let line_trimmed = line.trim();
let something = line_trimmed.parse::<i32>().unwrap();
total += something;
// println!("{:?}",total);
if frequencies.contains(&total) {
println!("duplicated found {:?}", total);
return;
}
frequencies.insert(total);
}
}
println!("duplicated found {:?}", frequencies);
}
println!("duplicated found {:?}", frequencies); is unreachable because, when you are calling return, you are returning out of the main function.
Change loop to a while, and set a variable to exit the while loop.
let mut duplicated_found = false;
while !duplicated_found {
for line in f.lines() {
let line_trimmed = line.trim();
let something = line_trimmed.parse::<i32>().unwrap();
total += something;
// println!("{:?}",total);
if frequencies.contains(&total) {
println!("duplicated found {:?}", total);
duplicated_found = true;
break; // This breaks out of the for loop
}
frequencies.insert(total);
}
}
println!("duplicated found {:?}", frequencies);
OR you can use labels to specify which loop you want to break out of:
'outer: loop {
for line in f.lines() {
let line_trimmed = line.trim();
let something = line_trimmed.parse::<i32>().unwrap();
total += something;
// println!("{:?}",total);
if frequencies.contains(&total) {
println!("duplicated found {:?}", total);
break 'outer; // This breaks out of the outer loop
}
frequencies.insert(total);
}
}
println!("duplicated found {:?}", frequencies);
I'm trying to solve the thread-ring problem. In each thread I read the token value
if it is not mine, check if it's the end of the program
if it is then finish the thread
otherwise, read again and repeat
if it is mine (i.e. has my id) then acquire the write lock, increase the value of the token, check if it's the end then tell main thread that I finished it and finish the current thread loop
If it not over, then release the write lock, and start to read again
There is no unlock. Is there any unlock like the one I need in here?
It seems that I should release the read lock as well, because the write lock won't happen if there is someone reading the data. Is it necessary?
fn main() {
use std::sync::{Arc, RwLock};
use std::thread;
use std::sync::mpsc::channel;
const N: usize = 5; //503;
const STOP_POINT: usize = 100;
let n = Arc::new(RwLock::new(1));
let (sender, reciever) = channel();
for i in 1..N {
let (n_c, channel) = (n.clone(), sender.clone());
// println!("Thread n.{} beeing created!", i);
let a = thread::Builder::new()
.name(i.to_string())
.spawn(move || -> () {
loop {
let mut read_only = n_c.read().unwrap();
let say_my_name = (*thread::current().name().unwrap()).to_string();
// println!("Thread {} says: gonna try!", say_my_name);
while (*read_only % N) != i {
if *read_only == 0 {
break;
}
// println!("Thread {} says: aint mine!", say_my_name);
read_only = n_c.read().unwrap();
} // WAIT
println!("Thread {} says: my turn!", say_my_name);
let mut ref_to_num = n_c.write().unwrap();
*ref_to_num += 1;
if *ref_to_num == STOP_POINT {
channel.send(say_my_name).unwrap();
break;
}
}
()
});
assert_eq!(a.is_ok(), true);
// thread::spawn();
// println!("Thread n.{} created!", i);
}
println!("{}", reciever.recv().unwrap());
}
To release a lock, you let it fall out of scope or explicitly invoke its destructor by calling drop.
Here's how your program could be written using drop in two places:
fn main() {
use std::sync::{Arc, RwLock};
use std::sync::mpsc::channel;
use std::thread;
use std::time::Duration;
const N: usize = 503;
const STOP_POINT: usize = 100;
let n = Arc::new(RwLock::new(1));
let (sender, receiver) = channel();
for i in 1..N {
let (n_c, channel) = (n.clone(), sender.clone());
// println!("Thread n.{} beeing created!", i);
thread::Builder::new()
.name(i.to_string())
.spawn(move || {
loop {
let mut read_only = n_c.read().unwrap();
let say_my_name = (*thread::current().name().unwrap()).to_string();
// println!("Thread {} says: gonna try!", say_my_name);
while (*read_only % N) != i {
if *read_only == 0 {
break;
}
drop(read_only); // release the lock before sleeping
// println!("Thread {} says: aint mine!", say_my_name);
thread::sleep(Duration::from_millis(1));
read_only = n_c.read().unwrap();
}
println!("Thread {} says: my turn!", say_my_name);
drop(read_only); // release the read lock before taking a write lock
let mut ref_to_num = n_c.write().unwrap();
*ref_to_num += 1;
if *ref_to_num == STOP_POINT {
channel.send(say_my_name).unwrap();
break;
}
}
})
.expect("failed to spawn a thread");
// println!("Thread n.{} created!", i);
}
println!("{}", receiver.recv().unwrap());
}
Note that if we don't reassign read_lock in the while loop, the compiler will give an error because read_lock doesn't hold a valid value after we call drop(read_lock). Rust is fine with local variables that are temporarily uninitialized, but of course we need to reinitialize them before we can use them again.
Here's how the thread's main loop could be written to use a scope to replace one of the drops:
loop {
let say_my_name = (*thread::current().name().unwrap()).to_string();
{
let mut read_only = n_c.read().unwrap();
// println!("Thread {} says: gonna try!", say_my_name);
while (*read_only % N) != i {
if *read_only == 0 {
break;
}
drop(read_only);
thread::sleep(Duration::from_millis(1));
// println!("Thread {} says: aint mine!", say_my_name);
read_only = n_c.read().unwrap();
}
println!("Thread {} says: my turn!", say_my_name);
} // read_only is dropped here
let mut ref_to_num = n_c.write().unwrap();
*ref_to_num += 1;
if *ref_to_num == STOP_POINT {
channel.send(say_my_name).unwrap();
break;
}
}