This question already has answers here:
How can I swap in a new value for a field in a mutable reference to a structure?
(2 answers)
Closed 3 years ago.
I have a bucket of objects that need to accumulate values. It's protected by an RwLock, and as such I also keep around its write lock. I want to keep a single write lock for the duration of the process.
For example:
use std::sync::RwLock;
fn main() {
let locked = RwLock::new(Vec::<u32>::new());
// this is the entry point for real-world code
let mut writer = locked.write().unwrap();
// copy into 'locked' until it is full (has 4 items)
for v in 0..100 {
if writer.len() > 4 {
// discard 'writer' and 'locked', create anew
locked = RwLock::new(Vec::<u32>::new());
writer = locked.write().unwrap();
}
writer.push(v);
}
}
While my example operates on fixed data, and so appears to not need the RwLock at all, the real code would enter at "real code" and not necessarily exit on the boundary of locked becoming "full".
How do I create a new locked and writer object when needed without the borrow-checker disagreeing?
I agree with David Grayson, there's no obvious need to recreate the RwLock. Assuming you need the vector after filling it up, use mem::replace to switch out the Vec:
use std::sync::RwLock;
use std::mem;
fn main() {
let locked = RwLock::new(Vec::<u32>::new());
let mut writer = locked.write().unwrap();
for v in 0..100 {
if writer.len() > 4 {
let old_vec = mem::replace(&mut *writer, Vec::new());
}
writer.push(v);
}
}
If you don't need the Vec, then just call Vec::clear.
Related
This question already has answers here:
What does 'let x = x' do in Rust?
(2 answers)
Closed 2 years ago.
I've created a simple list that has mutable push behaviour but no need to have that same mutability for peek fn
fn peek(){
let mut list = List::new();//take a mutable ref here to perform push
list.push(1);
let list = list; //shadow the variable with the same name and take a immut ref
assert_eq!(list.peek().unwrap(),&1);
}
Your way is fine. It's typical to include a comment like this:
let list = list; // discard mut
Another way to do it (it's arguable whether it's better or not, complexity vs. readability vs. separation of concerns) is to separate the initialization.
fn peek() {
let list = {
let mut list = List::new();
list.push(1);
list
};
assert_eq!(list.peek().unwrap(),&1);
}
Your solution is correct enough. Maybe it would be a bit more readable to do it this way:
fn peek(){
let list = {
let mut li = List::new();//take a mutable ref here to perform push
li.push(1);
li
};
assert_eq!(list.peek().unwrap(),&1);
}
But in your case there is another option:
fn peek(){
let list = std::iter::once(1).collect::<List<_>>();
assert_eq!(list.peek().unwrap(),&1);
}
This question already has answers here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Concurrent access to vector from multiple threads using a mutex lock
(1 answer)
How do I pass disjoint slices from a vector to different threads?
(1 answer)
Lifetime of variables passed to a new thread
(1 answer)
Processing vec in parallel: how to do safely, or without using unstable features?
(2 answers)
Closed 2 years ago.
I have a struct which contains a Vec of instances of another base class of struct. I am trying to iterate over the Vec and spawn threads which each run a single impl fn from the base struct. Nothing needs mutable access at any time after the iteration of thread spawning begins; just some basic math returning an f64 (based on values in a HashMap using keys stored in a fixed Vec in each base struct).
I am running into lifetime issues which I don't fully understand and which the compiler error messages (for once) don't help with.
Here is a stripped down version of what I want to implement (with some annotation of the errors encountered):
struct BaseStruct {
non_copy_field: Vec<&'static str>, // BaseStruct has vector members (thus can't implement Copy).
}
impl BaseStruct {
fn target_function(&self, value_dict: &HashMap<&'static str, f64>) -> f64 {
// Does some calculations, returns the result.
// Uses self.non_copy_field to get values from value_dict.
0.0
}
}
struct StructSet {
values: HashMap<&'static str, f64>, // This will be set prior to passing to base_struct.target_function().
all_structs: Vec<BaseStruct>, // Vector to be iterated over.
}
impl StructSet {
fn parallel_calculation(&self) -> f64 {
let mut result = 0.0;
let handles: Vec<_> = self.all_structs.iter().map(|base_struct| {
// Complains about lifetime here ^^^^^ or ^ here if I switch to &base_struct
thread::spawn(move || {
base_struct.target_function(&self.values)
})
}).collect();
for process in handles.iter() {
result += process.join().unwrap();
};
// Shouldn't all base_structs from self.all_structs.iter() be processed by this point?
result
} // Why does it say "...so that reference does not outlive borrowed content" here?
}
I have been trying various combinations of RwLock/Arc/Mutex wrapping the contents of the fields of StructSet to attempt to gain thread-safe, read-only access to each of the elements iterated/passed, but nothing seems to work. I'm looking to keep the codebase light, but I guess I'd consider rayon or similar as I'll need to follow this same process multiple places in the full module.
Can anyone point me in the correct direction?
Rust's lifetime system needs to know that if you borrow something (i.e. have a reference to it), that underlying value will exist for the whole time that you borrow it. For regular function calls, this is easy, but threads cause problems here - a thread you start may outlive the function you start it from. So you can't borrow values into a thread, you have to move values into a thread (which is why you have to write thread::spawn(move || { not just thread::spawn(|| {.
When you call .iter() on a Vec, the values the iterator produces are references to the values in the Vec - they're being borrowed. So you can't use them as-is from another thread. You need to move some owned value into the thread.
There are a few ways you can go about this:
If you don't need the Vec after your processing, you could switch to use .into_iter() rather than .iter(). This will iterate over the owned values in the Vec, rather than borrowing them, which means you can move them into the threads. But because the Vec is giving up ownership of the items, your Vec stops being usable after that.
If you do need your Vec after, and your values are clonable (i.e. they implement the Clone trait), you could call .iter().cloned() instead of .iter() - this will make copies of each of the values, which you can then move into the thread.
If you need the Vec afterwards, and either your values aren't clonable, or you don't want to clone them (maybe because it's expensive, or because it matters to you that both threads are using the exact same object), you can have your Vec store Arc<BaseStruct> instead of just BaseStruct - but that's not quite enough - you'll also need to explicitly clone the values before moving them into the thread (perhaps by using .iter().cloned() like above).
A downside of using Arcs is that no threads will be able to modify the values in the future. If you want to use Arcs, but you ever want o be able to modify the values in the future, you'll need instead of storing an Arc<BaseStruct>, to store an Arc<Mutex<BaseStruct>> (or an Arc<RwLock<BaseStruct>>. The Mutex ensures that only one thread can be modifying (or indeed reading) the value at a time, and the Arc allows for the cloning (so you can move a copy into the other thread).
I found the issue. Specifically it was the use of self.all_structs.iter().map(). Apparently iter() was requiring static lifetimes.
I switched to the following and it's working now:
fn parallel_calculation(&self) -> f64 {
let mut handles: Vec<_> = Vec::new();
for base_struct in &self.all_structs {
let for_solve = self.for_solve.clone();
let base_struct = base_struct.clone();
handles.push(thread::spawn(move || {
let for_solve = for_solve.read().unwrap();
base_struct.target_function(&for_solve)
}));
};
let mut result = 0.0;
for process in handles { result += process.join().unwrap(); };
return result
}
With Arcs and an RwLock in the main struct as follows:
pub struct StructSet {
all_structs: Vec<Arc<BaseStruct>>,
for_solve: Arc<RwLock<HashMap<&'static str, f64>>>,
}
I have the following code which uses both Rc and Box; what is the difference between those? Which one is better?
use std::rc::Rc;
fn main() {
let a = Box::new(1);
let a1 = &a;
let a2 = &a;
let b = Rc::new(1);
let b1 = b.clone();
let b2 = b.clone();
println!("{} {}", a1, a2); //=> 1 1
println!("{} {}", b1, b2); //=> 1 1
}
playground link
Rc provides shared ownership so by default its contents can't be mutated, while Box provides exclusive ownership and thus mutation is allowed:
use std::rc::Rc;
fn main() {
let mut a = Box::new(1);
let mut b = Rc::new(1);
*a = 2; // works
*b = 2; // doesn't
}
In addition Rc cannot be sent between threads, because it doesn't implement Send.
The bottom line is they are meant for different things: if you don't need shared access, use Box; otherwise, use Rc (or Arc for multi-threaded shared usage) and keep in mind you will be needing Cell or RefCell for internal mutability.
Looking at the example given in the description, I think the real question here is "when to use Rc versus &Box" (note the ampersand).
Both Rc and &Box store the underlying data on the heap, neither can be sent across threads, and both allow immutable sharing (demonstrated by the aforementioned example). However, the biggest difference is that Rc gives you a shared (immutable) owned value while with &Box you get a shared (immutable) reference.
In the Rc case, the underlying data will be dropped (freed/deallocated) whenever the last owner (whether the original one or any cloned one) gets dropped – that's the idea of reference counting. In the &Box case, however, there is only one owner: any shared references to it will become invalid immediately after the owner gets out of scope.
Said differently, contrary to a Rc::clone(), binding a variable to a new &Box (let a2 = &a; in the example) will not make it live any longer than it would otherwise.
As a concrete example, the following is valid:
use std::rc::Rc;
fn main() {
let rc_clone;
{
let rc = Rc::new(1);
rc_clone = rc.clone();
// rc gets out of scope here but as a "shared owner", rc_clone
// keeps the underlying data alive.
}
println!("{}", rc_clone); // Ok.
}
But this isn't:
fn main() {
let b_ref;
{
let b = Box::new(1);
b_ref = &b;
// b gets out of scope here and since it is the only owner,
// the underlying data gets dropped.
}
println!("{}", b_ref); // Compilation error: `b` does not live long enough.
}
This question already has answers here:
Temporarily move out of borrowed content
(3 answers)
How do you replace the value of a mutable variable by taking ownership of it?
(2 answers)
Closed 5 years ago.
I have a mutable reference to a container, and I want to modify one of its interior values:
struct BigContainer {
contained: BigUint,
}
fn double(container: &mut BigContainer) {
container.contained = container.contained * 2_usize;
}
BigUint doesn't implement MulAssign, so we can't just use the *= operator. Consequently, the code as written fails, because Mul takes ownership of the value; we need to temporarily move out of borrowed content. (playground)
We can get around that by initializing a temporary value, and using std::mem::replace: (playground)
fn double(container: &mut BigContainer) {
let prev_value = mem::replace(&mut container.contained, 0_usize.into());
container.contained = prev_value * 2_usize;
}
This works, but it's a bit ugly, and we need to create that pointless filler value to temporarily hold the position open for us. What I want is some unsafe method which lets me just override the borrow checker. Unfortunately, the following does not work: (playground)
fn double(container: &mut BigContainer) {
unsafe {
container.contained = container.contained * 2_usize;
}
}
The unsafe block does nothing for us in this case. Is there any way to accomplish this, safe or not, which eliminates the temporary value allocation? If not, is the compiler at least smart enough to skip it?
I'm trying to get my head around Rust. I've got an alpha version of 1.
Here's the problem I'm trying to program: I have a vector of floats. I want to set up some threads asynchronously. Each thread should wait for the number of seconds specified by each element of the vector, and return the value of the element, plus 10. The results need to be in input order.
It's an artificial example, to be sure, but I wanted to see if I could implement something simple before moving onto more complex code. Here is my code so far:
use std::thread;
use std::old_io::timer;
use std::time::duration::Duration;
fn main() {
let mut vin = vec![1.4f64, 1.2f64, 1.5f64];
let mut guards: Vec<thread::scoped> = Vec::with_capacity(3);
let mut answers: Vec<f64> = Vec::with_capacity(3);
for i in 0..3 {
guards[i] = thread::scoped( move || {
let ms = (1000.0f64 * vin[i]) as i64;
let d = Duration::milliseconds(ms);
timer::sleep(d);
println!("Waited {}", vin[i]);
answers[i] = 10.0f64 + (vin[i] as f64);
})};
for i in 0..3 {guards[i].join(); };
for i in 0..3 {println!("{}", vin[i]); }
}
So the input vector is [1.4, 1.2, 1.5], and I'm expecting the output vector to be [11.4, 11.2, 11.5].
There appear to be a number of problems with my code, but the first one is that I get a compilation error:
threads.rs:7:25: 7:39 error: use of undeclared type name `thread::scoped`
threads.rs:7 let mut guards: Vec<thread::scoped> = Vec::with_capacity(3);
^~~~~~~~~~~~~~
error: aborting due to previous error
There also seem to be a number of other problems, including using vin within a closure. Also, I have no idea what move does, other than the fact that every example I've seen seems to use it.
Your error is due to the fact that thread::scoped is a function, not a type. What you want is a Vec<T> where T is the result type of the function. Rust has a neat feature that helps you here: It automatically detects the correct type of your variables in many situations.
If you use
let mut guards = Vec::with_capacity(3);
the type of guards will be chosen when you use .push() the first time.
There also seem to be a number of other problems.
you are accessing guards[i] in the first for loop, but the length of the guards vector is 0. Its capacity is 3, which means that you won't have any unnecessary allocations as long as the vector never contains more than 3 elements. use guards.push(x) instead of guards[i] = x.
thread::scoped expects a Fn() -> T, so your closure can return an object. You get that object when you call .join(), so you don't need an answer-vector.
vin is moved to the closure. Therefore in the second iteration of the loop that creates your guards, vin isn't available anymore to be moved to the "second" closure. Every loop iteration creates a new closure.
i is moved to the closure. I have no idea what's going on there. But the solution is to let inval = vin[i]; outside the closure, and then use inval inside the closure. This also solves Point 3.
vin is mutable. Yet you never mutate it. Don't bind variables mutably if you don't need to.
vin is an array of f64. Therefore (vin[i] as f64) does nothing. Therefore you can simply use vin[i] directly.
join moves out of the guard. Since you cannot move out of an array, your cannot index into an array of guards and join the element at the specified index. What you can do is loop over the elements of the array and join each guard.
Basically this means: don't iterate over indices (for i in 1..3), but iterate over elements (for element in vector) whenever possible.
All of the above implemented:
use std::thread;
use std::old_io::timer;
use std::time::duration::Duration;
fn main() {
let vin = vec![1.4f64, 1.2f64, 1.5f64];
let mut guards = Vec::with_capacity(3);
for inval in vin {
guards.push(thread::scoped( move || {
let ms = (1000.0f64 * inval) as i64;
let d = Duration::milliseconds(ms);
timer::sleep(d);
println!("Waited {}", inval);
10.0f64 + inval
}));
}
for guard in guards {
let answer = guard.join();
println!("{}", answer);
};
}
In supplement of Ker's answer: if you really need to mutate arrays within a thread, I suppose the most closest valid solution for your task will be something like this:
use std::thread::spawn;
use std::old_io::timer;
use std::sync::{Arc, Mutex};
use std::time::duration::Duration;
fn main() {
let vin = Arc::new(vec![1.4f64, 1.2f64, 1.5f64]);
let answers = Arc::new(Mutex::new(vec![0f64, 0f64, 0f64]));
let mut workers = Vec::new();
for i in 0..3 {
let worker_vin = vin.clone();
let worker_answers = answers.clone();
let worker = spawn( move || {
let ms = (1000.0f64 * worker_vin[i]) as i64;
let d = Duration::milliseconds(ms);
timer::sleep(d);
println!("Waited {}", worker_vin[i]);
let mut answers = worker_answers.lock().unwrap();
answers[i] = 10.0f64 + (worker_vin[i] as f64);
});
workers.push(worker);
}
for worker in workers { worker.join().unwrap(); }
for answer in answers.lock().unwrap().iter() {
println!("{}", answer);
}
}
In order to share vectors between several threads, I have to prove, that these vectors outlive all of my threads. I cannot use just Vec, because it will be destroyed at the end of main block, and another thread could live longer, possibly accessing freed memory. So I took Arc reference counter, which guarantees, that my vectors will be destroyed only when the counter downs to zero.
Arc allows me to share read-only data. In order to mutate answers array, I should use some synchronize tools, like Mutex. That is how Rust prevents me to make data races.