Consider the following code, I append each of my threads to a Vector in order to join them up to the main thread after I have spawned each thread, however I am not able to call iter() on my vector of JoinHandlers.
How can I go about doing this?
fn main() {
let requests = Arc::new(Mutex::new(Vec::new()));
let threads = Arc::new(Mutex::new(Vec::new()));
for _x in 0..100 {
println!("Spawning thread: {}", _x);
let mut client = Client::new();
let thread_items = requests.clone();
let handle = thread::spawn(move || {
for _y in 0..100 {
println!("Firing requests: {}", _y);
let start = time::precise_time_s();
let _res = client.get("http://jacob.uk.com")
.header(Connection::close())
.send().unwrap();
let end = time::precise_time_s();
thread_items.lock().unwrap().push((Request::new(end-start)));
}
});
threads.lock().unwrap().push((handle));
}
// src/main.rs:53:22: 53:30 error: type `alloc::arc::Arc<std::sync::mutex::Mutex<collections::vec::Vec<std::thread::JoinHandle<()>>>>` does not implement any method in scope named `unwrap`
for t in threads.iter(){
println!("Hello World");
}
}
First, you don't need threads to be contained in Arc in Mutex. You can keep it just Vec:
let mut threads = Vec::new();
...
threads.push(handle);
This is so because you don't share threads between, well, threads. You only access it from the main thread.
Second, if for some reason you do need to keep it in Arc (e.g. if your example does not reflect the actual structure of your program which is more complex), then you need to lock the mutex to obtain a reference to the contained vector, just as you do when pushing:
for t in threads.lock().unwrap().iter() {
...
}
Related
I am writing a program that pings a set of targets 100 times, and stores each RTT value returned from the ping into a vector, thus giving me a set of RTT values for each target. Say I have n targets, I would like all of the pinging to be done concurrently. The rust code looks like this:
let mut sample_rtts_map = HashMap::new();
for addr in targets.to_vec() {
let mut sampleRTTvalues: Vec<f32> = vec![];
//sample_rtts_map.insert(addr, sampleRTTvalues);
thread::spawn(move || {
while sampleRTTvalues.len() < 100 {
let sampleRTT = ping(addr);
sampleRTTvalues.push(sampleRTT);
// thread::sleep(Duration::from_millis(5000));
}
});
}
The hashmap is used to tell which vector of values belongs to which target. The problem is, how do I retrieve the updated sampleRTTvalues from each thread after the thread is done executing? I would like something like:
let (name, sampleRTTvalues) = thread::spawn(...)
The name, being the name of the thread, and sampleRTTvalues being the vector. However, since I'm creating threads in a for loop, each thread is being instantiated the same way, so how I differentiate them?
Is there some better way to do this? I've looked into schedulers, future, etc., but it seems my case can just be done with simple threads.
I go the desired behavior with the following code:
use std::thread;
use std::sync::mpsc;
use std::collections::HashMap;
use rand::Rng;
use std::net::{Ipv4Addr,Ipv6Addr,IpAddr};
const RTT_ONE: IpAddr = IpAddr::V4(Ipv4Addr::new(127,0,0,1));
const RTT_TWO: IpAddr = IpAddr::V6(Ipv6Addr::new(0,0,0,0,0,0,0,1));
const RTT_THREE: IpAddr = IpAddr::V4(Ipv4Addr::new(127,0,1,1));//idk how ip adresses work, forgive if this in invalid but you get the idea
fn ping(address: IpAddr) -> f32 {
rand::thread_rng().gen_range(5.0..107.0)
}
fn main() {
let targets = [RTT_ONE,RTT_TWO,RTT_THREE];
let mut sample_rtts_map: HashMap<IpAddr,Vec<f32>> = HashMap::new();
for addr in targets.into_iter() {
let (sample_values,moved_values) = mpsc::channel();
let mut sampleRTTvalues: Vec<f32> = vec![];
thread::spawn(move || {
while sampleRTTvalues.len() < 100 {
let sampleRTT = ping(addr);
sampleRTTvalues.push(sampleRTT);
//thread::sleep(Duration::from_millis(5000));
}
});
sample_rtts_map.insert(addr,moved_values.recv().unwrap());
}
}
note that the use rand::Rng can be removed when implementing, as it is only so the example works. what this does is pass data from the spawned thread to the main thread, and in the method used it waits until the data is ready before adding it to the hash map. If this is problematic (takes a long time, etc.) then you can use try_recv instead of recv which will add an error / option type that will return a recoverable error if the value is ready when unwrapped, or return the value if it's ready
You can use a std::sync::mpsc channel to collect your data:
use std::collections::HashMap;
use std::sync::mpsc::channel;
use std::thread;
fn ping(_: &str) -> f32 { 0.0 }
fn main() {
let targets = ["a", "b"]; // just for example
let mut sample_rtts_map = HashMap::new();
let (tx, rx) = channel();
for addr in targets {
let tx = tx.clone();
thread::spawn(move || {
for _ in 0..100 {
let sampleRTT = ping(addr);
tx.send((addr, sampleRTT));
}
});
}
drop(tx);
// exit loop when all thread's tx have dropped
while let Ok((addr, sampleRTT)) = rx.recv() {
sample_rtts_map.entry(addr).or_insert(vec![]).push(sampleRTT);
}
println!("sample_rtts_map: {:?}", sample_rtts_map);
}
This will run all pinging threads simultaneously, and collect data in main thread synchronously, so that we can avoid using locks. Do not forget to drop sender in main thread after cloning to all pinging threads, or the main thread will hang forever.
I'm new to Rust. I'm supposed to use a Mutex and an Arc to create a critical section within the print_lots function to stop the race condition from happening. Any Ideas?
fn main() {
let num_of_threads = 4;
let mut array_of_threads = vec![];
for id in 0..num_of_threads {
array_of_threads.push(std::thread::spawn(move || print_lots(id)));
}
for t in array_of_threads {
t.join().expect("Thread join failure");
}
}
fn print_lots(id: u32) {
println!("Begin [{}]", id);
for _i in 0..100 {
print!("{} ", id);
}
println!("\nEnd [{}]", id);
}
Mutex in Rust perhaps works differently to how locks work in some other languages you might be used to. Instead of tracking the lock independently from the value, a Rust Mutex owns the data and prevents accessing it without first obtaining a lock, which is enforced at compile time.
The warning you are getting is because you have locked the Mutex, but then done nothing with the value. The warning is there because this is almost certainly a mistake.
fn main() {
let foo = Mutex::new(0);
// It's often best to just unwrap and panic if the lock is poisoned
if let Ok(mut lock) = foo.lock() {
*lock = 2;
// The mutex is unlocked automatically when lock goes out of scope here
}
println!("{:?}", foo); // Mutex { data: 2 }
}
I am guessing that your real problem is that you want to synchronise the print statements so that output from different threads is not intermingled.
One way to do that is to obtain a lock on StdOut which actually uses a lock internally and provides a similar API to Mutex:
fn print_lots(id: u32) {
let stdout = io::stdout();
println!("Begin [{}]", id);
let mut handle = stdout.lock();
for _i in 0..100 {
write!(&mut handle, "{} ", id).unwrap();
}
println!("\nEnd [{}]", id);
// handle is dropped here, unlocking stdout
}
In your simplified example, creating a long-lived lock in each thread is counterproductive since each thread will block the others and the result is sequential rather than concurrent. This might still make sense though if your real-world code has more going on.
use std::sync::{Arc, Mutex};
fn main() {
let num_of_threads = 4;
let mut array_of_threads = vec![];
let counter = Arc::new(Mutex::new(0));
for id in 0..num_of_threads {
let counter_clone = counter.clone();
array_of_threads.push(std::thread::spawn(move || print_lots(id, counter_clone)));
}
for t in array_of_threads {
t.join().expect("Thread join failure");
}
}
fn print_lots(id: u32, c: Arc<Mutex<u32>>) {
println!("Begin [{}]", id);
let _guard = c.lock().unwrap();
for _i in 0..100 {
print!("{} ", id);
}
println!("\nEnd [{}]", id);
}
Given several threads that complete with an Output value, how do I get the first Output that's produced? Ideally while still being able to get the remaining Outputs later in the order they're produced, and bearing in mind that some threads may or may not terminate.
Example:
struct Output(i32);
fn main() {
let mut spawned_threads = Vec::new();
for i in 0..10 {
let join_handle: ::std::thread::JoinHandle<Output> = ::std::thread::spawn(move || {
// pretend to do some work that takes some amount of time
::std::thread::sleep(::std::time::Duration::from_millis(
(1000 - (100 * i)) as u64,
));
Output(i) // then pretend to return the `Output` of that work
});
spawned_threads.push(join_handle);
}
// I can do this to wait for each thread to finish and collect all `Output`s
let outputs_in_order_of_thread_spawning = spawned_threads
.into_iter()
.map(::std::thread::JoinHandle::join)
.collect::<Vec<::std::thread::Result<Output>>>();
// but how would I get the `Output`s in order of completed threads?
}
I could solve the problem myself using a shared queue/channels/similar, but are there built-in APIs or existing libraries which could solve this use case for me more elegantly?
I'm looking for an API like:
fn race_threads<A: Send>(
threads: Vec<::std::thread::JoinHandle<A>>
) -> (::std::thread::Result<A>, Vec<::std::thread::JoinHandle<A>>) {
unimplemented!("so far this doesn't seem to exist")
}
(Rayon's join is the closest I could find, but a) it only races 2 closures rather than an arbitrary number of closures, and b) the thread pool w/ work stealing approach doesn't make sense for my use case of having some closures that might run forever.)
It is possible to solve this use case using pointers from How to check if a thread has finished in Rust? just like it's possible to solve this use case using an MPSC channel, however here I'm after a clean API to race n threads (or failing that, n closures on n threads).
These problems can be solved by using a condition variable:
use std::sync::{Arc, Condvar, Mutex};
#[derive(Debug)]
struct Output(i32);
enum State {
Starting,
Joinable,
Joined,
}
fn main() {
let pair = Arc::new((Mutex::new(Vec::new()), Condvar::new()));
let mut spawned_threads = Vec::new();
let &(ref lock, ref cvar) = &*pair;
for i in 0..10 {
let my_pair = pair.clone();
let join_handle: ::std::thread::JoinHandle<Output> = ::std::thread::spawn(move || {
// pretend to do some work that takes some amount of time
::std::thread::sleep(::std::time::Duration::from_millis(
(1000 - (100 * i)) as u64,
));
let &(ref lock, ref cvar) = &*my_pair;
let mut joinable = lock.lock().unwrap();
joinable[i] = State::Joinable;
cvar.notify_one();
Output(i as i32) // then pretend to return the `Output` of that work
});
lock.lock().unwrap().push(State::Starting);
spawned_threads.push(Some(join_handle));
}
let mut should_stop = false;
while !should_stop {
let locked = lock.lock().unwrap();
let mut locked = cvar.wait(locked).unwrap();
should_stop = true;
for (i, state) in locked.iter_mut().enumerate() {
match *state {
State::Starting => {
should_stop = false;
}
State::Joinable => {
*state = State::Joined;
println!("{:?}", spawned_threads[i].take().unwrap().join());
}
State::Joined => (),
}
}
}
}
(playground link)
I'm not claiming this is the simplest way to do it. The condition variable will awake the main thread every time a child thread is done. The list can show the state of each thread, if one is (about to) finish, it can be joined.
No, there is no such API.
You've already been presented with multiple options to solve your problem:
Use channels
Use a CondVar
Use futures
Sometimes when programming, you have to go beyond sticking pre-made blocks together. This is supposed to be a fun part of programming. I encourage you to embrace it. Go create your ideal API using the components available and publish it to crates.io.
I really don't see what's so terrible about the channels version:
use std::{sync::mpsc, thread, time::Duration};
#[derive(Debug)]
struct Output(i32);
fn main() {
let (tx, rx) = mpsc::channel();
for i in 0..10 {
let tx = tx.clone();
thread::spawn(move || {
thread::sleep(Duration::from_millis((1000 - (100 * i)) as u64));
tx.send(Output(i)).unwrap();
});
}
// Don't hold on to the sender ourselves
// Otherwise the loop would never terminate
drop(tx);
for r in rx {
println!("{:?}", r);
}
}
One thread calculates some data that takes about 1GB of RAM and another thread only reads this data. What is the best way to implement this?
use std::thread;
use std::time::Duration;
fn main() {
let mut shared: i32 = 0; // will be changed to big structure
thread::spawn(move || {
loop {
shared += 1;
println!("write shared {}", shared);
thread::sleep(Duration::from_secs(2));
}
});
thread::spawn(move || {
loop {
thread::sleep(Duration::from_secs(1));
println!("read shared = ???"); // <---------------- ????
}
});
thread::sleep(Duration::from_secs(4));
println!("main");
}
You can run this code online (play.rust-lang.org)
The code and your statements don't really make sense together. For example, there's nothing that prevents the second thread from finishing before the first thread ever has a chance to start. Yes, I see the sleeps, but sleeping is not a viable concurrency solution.
For the question as asked, I'd use a channel. This allows one thread to produce a value and then transfer ownership of that value to another thread:
use std::thread;
use std::sync::mpsc;
fn main() {
let (tx, rx) = mpsc::channel();
let a = thread::spawn(move || {
let large_value = 1;
println!("write large_value {}", large_value);
tx.send(large_value).expect("Unable to send");
});
let b = thread::spawn(move || {
let large_value = rx.recv().expect("Unable to receive");
println!("read shared = {}", large_value);
});
a.join().expect("Unable to join a");
b.join().expect("Unable to join b");
println!("main");
}
For the code as presented, there's really no other options besides a Mutex or a RwLock. This allows one thread to mutate the shared value for a while, then the other thread may read it for a while (subject to the vagaries of the OS scheduler):
use std::thread;
use std::time::Duration;
use std::sync::{Arc, Mutex};
fn main() {
let shared = Arc::new(Mutex::new(0));
let shared_1 = shared.clone();
thread::spawn(move || {
loop {
let mut shared = shared_1.lock().expect("Unable to lock");
*shared += 1;
println!("write large_value {}", *shared);
}
});
thread::spawn(move || {
loop {
let shared = shared.lock().expect("Unable to lock");
println!("read shared = {}", *shared);
}
});
thread::sleep(Duration::from_secs(1));
println!("main");
}
None of this is particularly unique to Rust; channels are quite popular in Go and Clojure and mutexes have existed for A Very Long Time. I'd suggest checking out any of the numerous beginner's guides on the Internet for multithreading and the perils therein.
I'm trying to share a RwLock amongst several threads without using scoped threads but I can't figure out how to get the lifetimes correct. I assume that this is possible (what's the point of RwLocks otherwise?) but I can't find any examples of it.
Here is a toy example of what I'm trying to accomplish. Any advice would be appreciated.
rust playpen for this code
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(&mut stuff);
println!("done");
}
fn helper(stuff: &mut Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}
&mut references are not safe to send to a non-scoped thread, because the thread may still run after the referenced data has been deallocated. Furthermore, after helper returns, the main thread would still be able to mutate stuff, and the spawned thread would also be able to mutate stuff indirectly, which is not allowed in Rust (there can only be one mutable alias for a variable).
Instead, the RwLock should own the data, rather than borrow it. This means helper should receive a Stuff rather than a &mut Stuff.
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(stuff);
println!("done");
}
fn helper(stuff: Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}