How can I share references across threads? - multithreading

I am unable to share a reference between threads.
trait Facade { /*some functions*/ }
struct Client<'a> {
facade: &'a mut Facade,
join_grd: thread::JoinGuard<'a()>,
}
impl<'a> Client<'a> {
pub fn new(my_facade: &'a mut Facade) -> Client<'a> {
Client {
facade: my_facade,
join_grd: thread::scoped(|| Client::start(my_facade)),
}
}
fn start(my_facade: &'a mut Facade) { unimplemented!() }
}
Given my newbie status in Rust, I'm getting confused with concepts and errors. How do I achieve the above ?

I'm pretty sure you can't do this due to the mutable aliasing guarantees in Rust. In Rust you can't have two mutable references to the same thing at the same time, but this is exactly what happens in your code: you store my_facade to the field of Client and at the same time you are trying to pass it to start() method in another thread. This would require having two mutable references to the same Facade which is disallowed.
The actual errors which compiler emits on your code are caused by that you're using a non-moving closure. If you change thread::scoped() instantiation to this:
join_grd: thread::scoped(move || Client::start(my_facade))
the error would be more sensible:
test.rs:16:60: 16:69 error: cannot move `my_facade` into closure because it is borrowed
test.rs:16 join_grd: thread::scoped(move || Client::start(my_facade))
^~~~~~~~~
test.rs:15:21: 15:30 note: borrow of `*my_facade` occurs here
test.rs:15 facade: my_facade,
^~~~~~~~~
This essentially means that since &mut references are unique and are moved instead of copied, you can't duplicate them. Similar code with the regular & reference instead of &mut (and an additional Sync parent trait on Facade) works fine.
You have to rethink your architecture to fix this error. It is difficult to understand what you want from this piece of code alone, so I can't give any exact advices, but you may consider using Arc and Mutex if you want to share mutable state between threads.
Naive usage of Arc/Mutex like this:
fn start(my_facade: Arc<Mutex<Facade>>)
won't work because Facade is a trait, not a regular type. When you use traits as types, you're in fact opting into dynamic dispatch in form of trait objects. In short, trait objects can't be used directly; they should always be behind a pointer. Your original program also used trait objects (&'a mut Facade is a trait object). Ideally we should be able to form trait objects with any kind of smart pointer, and ideally Arc<Mutex<Facade>> should work, but unfortunately for now trait objects can only be created with &, &mut or Box:
fn start(my_facade: Arc<Mutex<Box<Facade>>>)
This is the reason of the error about Sized that you observe.
However, you should also consider not using trait objects at all and just use generics:
trait Facade: Send { fn f(&self); }
struct Client<'a, F: Facade> { // '
facade: Arc<Mutex<F>>,
join_grd: thread::JoinGuard<'a, ()>, // '
}
impl<'a, F: Facade+'a> Client<'a, F> { // '
pub fn new(my_facade: Arc<Mutex<F>>) -> Client<'a, F> { // '
let my_facade_2 = my_facade.clone(); // clone the Arc pointer
Client {
facade: my_facade,
join_grd: thread::scoped(move || Client::start(my_facade_2)),
}
}
fn start(my_facade: Arc<Mutex<F>>) { unimplemented!() }
}
You also need to add Send bound either on the trait itself (as in the example above) or on F type variable (as in F: Facade+Send+'a) because only Send data may be transferred between threads safely, so you need to specify that F is Send, either directly or as a supertrait constraint on Facade.

Related

Can I implement Index/IndexMut for a type that has locked data?

I've got a struct that contains some locked data. The real world is complex, but here's a minimal example (or as minimal as I can make it):
use std::fmt::Display;
use std::ops::{Index, IndexMut};
use std::sync::Mutex;
struct LockedVector<T> {
stuff: Mutex<Vec<T>>,
}
impl<T> LockedVector<T> {
pub fn new(v: Vec<T>) -> Self {
LockedVector {
stuff: Mutex::new(v),
}
}
}
impl<T> Index<usize> for LockedVector<T> {
type Output = T;
fn index(&self, index: usize) -> &Self::Output {
todo!()
}
}
impl<T> IndexMut<usize> for LockedVector<T> {
fn index_mut(&mut self, index: usize) -> &mut Self::Output {
let thing = self.stuff.get_mut().unwrap();
&mut thing[index]
}
}
impl<T: Display> Display for LockedVector<T> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let strings: Vec<String> = self
.stuff
.lock()
.unwrap()
.iter()
.map(|s| format!("{}", s))
.collect();
write!(f, "{}", strings.join(", "))
}
}
fn main() {
let mut my_stuff = LockedVector::new(vec![0, 1, 2, 3, 4]);
println!("initially: {}", my_stuff);
my_stuff[2] = 5;
println!("then: {}", my_stuff);
let a_mut_var: &mut usize = &mut my_stuff[3];
*a_mut_var = 54;
println!("Still working: {}", my_stuff);
}
What I'm trying to do here is implement the Index and IndexMut traits on a struct, where the data being indexed is behind a Mutex lock. My very fuzzy reasoning for why this should be possible is that the result of locking a mutex is sort-of like a reference, and it seems like you could map a reference onto another reference, or somehow make a sort of reference that wraps the entire lock but only de-references the specific index.
My much less fuzzy reasoning is that the code above compiles and runs (note the todo!) - I'm able to get back mutable references, and I assume I haven't somehow snuck past the mutex in an unthread-safe way. (I made an attempt to test the threaded behavior, but ran into other issues trying to get a mutable reference into another thread at all.)
The weird issue is, I can't do the same for Index - there is no get_immut() I can use, and I haven't found another approach. I can get a mutable reference out of Mutex, but not an immutable one (and of course, I can't get the mutable one if I only have an immutable reference to begin with)
My expectation is that indexing would acquire a lock, and the returned reference (in both mutable and immutable cases) would maintain the lock for their lifetimes. As a bonus, it would be nice if RwLock-ed things could only grab/hold the read lock for the immutable cases, and the write lock for mutable ones.
For context as to why I'd do this: I have a Grid trait that is used by a bunch of different code, but backed by different implementations, some of which are thread-safe. I was hoping to put the Index and IndexMut traits on it for the nice syntax. Threads don't generally have mutable references to the thread-safe Grids at all, so the IndexMut trait would see little use there, but I could see it being valuable during setup or for the non-thread-safe cases. The immutable Index behavior seems like it would be useful everywhere.
Bonus question: I absolutely hate that Display code, how can I make it less hideous?
If you look at the documentation of get_mut you'll see it's only possible precisely because a mutable reference ensures that there is no other reference or a lock to it, unfortunately for you that means that a get_ref for Mutex would only be possible by taking a mutable reference, that's just an artificially limited get_mut though.
Unfortunately for you since Index only gives you a shared reference you can't safely get a shared reference to it's contents, so you can't implement an Index so that it indexes into something behind a Mutex.

Can I mark an immutable borrow as exclusive in Rust?

I have a data structure, that is somewhat like a RwLock. It is not re-entrant, but the actual locking mechanism is a const function. Is there any way that I can mark this function as "exclusive_borrow" without switching it to be be a mutable function. That way multiple calls to 'read' will be caught at compile time instead of panicking.
struct MyRwLock<T> {
t: T,
}
impl MyRwLock {
// Works fine, but doesn't enforce on compile time that there is
// only 1 Guard.
pub fn read(&self) -> ReadGuard<'_, T> { ... }
// Enforces only 1 ReadGuard at compile time, but unnecessarily
// requires MyMutex to be mutable to read.
pub fn mut_read(&mut self) -> ReadGuard<'_, T> { ... }
}
&mut is a bit of a misnomer. It actually means exclusive reference, not mutable. If that's what you want, then it's correct to use &mut.
In fact, there was a proposal to rename &mut back in 2014. It never came through, but you might occasionally hear whispers of the "mutpocalypse" today.

Destructuring a struct containing a borrow in a function argument

I am trying to implement a system that would use borrow checking/lifetimes in order to provide safe custom indices on a collection. Consider the following code:
struct Graph(i32);
struct Edge<'a>(&'a Graph, i32);
impl Graph {
pub fn get_edge(&self) -> Edge {
Edge(&self, 0)
}
pub fn split(&mut self, Edge(_, edge_id): Edge) {
self.0 = self.0 + edge_id;
}
pub fn join(&mut self, Edge(_, edge0_id): Edge, Edge(_, edge1_id): Edge) {
self.0 = self.0 + edge0_id + edge1_id;
}
}
fn main() {
let mut graph = Graph(0);
let edge = graph.get_edge();
graph.split(edge)
}
References to the graph borrowed by the Edge struct should be dropped when methods such as split or join are called. This would fulfill the API invariant that all edge indices must be destroyed when the graph is mutated. However, the compiler doesn't get it. It fails with messages like
error[E0502]: cannot borrow `graph` as mutable because it is also borrowed as immutable
--> src/main.rs:23:5
|
22 | let edge = graph.get_edge();
| ----- immutable borrow occurs here
23 | graph.split(edge)
| ^^^^^ mutable borrow occurs here
24 | }
| - immutable borrow ends here
If I understand this correctly, the compiler fails to realise that the borrowing of the graph that happened in the edge struct is actually being released when the function is called. Is there a way to teach the compiler what I am trying to do here?
Bonus question: is there a way to do exactly the same but without actually borrowing the graph in the Edge struct? The edge struct is only used as a temporary for the purpose of traversal and will never be part of an external object state (I have 'weak' versions of the edge for that).
Addendum: After some digging around, it seems to be really far from trivial. First of all, Edge(_, edge_id) does not actually destructure the Edge, because _ does not get bound at all (yes, i32 is Copy which makes things even more complicated, but this is easily remedied by wrapping it into a non-Copy struct). Second, even if I completely destructure Edge (i.e. by doing it in a separate scope), the reference to the graph is still there, even though it should have been moved (this must be a bug). It only works if I perform the destructuring in a separate function. Now, I have an idea how to circumvent it (by having a separate object that describes a state change and destructures the indices as they are supplied), but this becomes very awkward very quickly.
You have a second problem that you didn’t mention: how does split know that the user didn’t pass an Edge from a different Graph? Fortunately, it’s possible to solve both problems with higher-rank trait bounds!
First, let’s have Edge carry a PhantomData marker instead of a real reference to the graph:
pub struct Edge<'a>(PhantomData<&'a mut &'a ()>, i32);
Second, let’s move all the Graph operations into a new GraphView object that gets consumed by operations that should invalidate the identifiers:
pub struct GraphView<'a> {
graph: &'a mut Graph,
marker: PhantomData<&'a mut &'a ()>,
}
impl<'a> GraphView<'a> {
pub fn get_edge(&self) -> Edge<'a> {
Edge(PhantomData, 0)
}
pub fn split(self, Edge(_, edge_id): Edge) {
self.graph.0 = self.graph.0 + edge_id;
}
pub fn join(self, Edge(_, edge0_id): Edge, Edge(_, edge1_id): Edge) {
self.graph.0 = self.graph.0 + edge0_id + edge1_id;
}
}
Now all we have to do is guard the construction of GraphView objects such that there’s never more than one with a given lifetime parameter 'a.
We can do this by (1) forcing GraphView<'a> to be invariant over 'a with a PhantomData member as above, and (2) only ever providing a constructed GraphView to a closure with a higher-rank trait bound that creates a fresh lifetime each time:
impl Graph {
pub fn with_view<Ret>(&mut self, f: impl for<'a> FnOnce(GraphView<'a>) -> Ret) -> Ret {
f(GraphView {
graph: self,
marker: PhantomData,
})
}
}
fn main() {
let mut graph = Graph(0);
graph.with_view(|view| {
let edge = view.get_edge();
view.split(edge);
});
}
Full demo on Rust Playground.
This isn’t totally ideal, since the caller may have to go through contortions to put all its operations inside the closure. But I think it’s the best we can do in the current Rust language, and it does allow us to enforce a huge class of compile-time guarantees that almost no other language can express at all. I’d love to see more ergonomic support for this pattern added to the language somehow—perhaps a way to create a fresh lifetime via a return value rather than a closure parameter (pub fn view(&mut self) -> exists<'a> GraphView<'a>)?

Borrowing reference in structure

I'm trying to put in one structure information about inotify events and hashmap with inotify watch id as key and name of the file as value.
extern crate inotify;
use inotify::INotify;
use std::sync::{Arc, Mutex};
use std::collections::HashMap;
struct Notificator {
inotify: INotify,
watch_service: Arc<Mutex<HashMap<inotify::wrapper::Watch, Arc<String>>>>,
}
impl Notificator {
pub fn new() -> Notificator {
Notificator {
inotify: INotify::init().unwrap(),
watch_service: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn resolving_events(&mut self) {
{
let mut events = self.check_for_events();
self.events_execution(events);
}
}
fn check_for_events(&mut self) -> &[inotify::wrapper::Event] {
self.inotify.available_events().unwrap()
}
fn events_execution(&self, events: &[inotify::wrapper::Event]) {
for event in events.iter() {
}
}
}
During compilation I am receiving an error
src/main.rs:204:13: 204:17 error: cannot borrow `*self` as immutable because it is also borrowed as mutable [E0502]
src/main.rs:204 self.events_execution(events);
I thought the best solution would be to separate somehow inotify variable in Notificator structure with watch_service, but I can't dereference self.check_for_events(); because I receive
src/main.rs:203:17: 203:27 error: the trait bound `[inotify::wrapper::Event]: std::marker::Sized` is not satisfied [E0277]
src/main.rs:203 let mut events = *self.check_for_events();
I understand the core of the problem: I'm trying to borrow reference by check_for_events and then using it as parameter in events_execution which also requires self as parameter, but I have no idea how to resolve it.
One thing you could do, although it's not very elegant, is to have your mutable method consume its mutable borrow and return an immutable one that you can then use:
pub fn resolving_events(&mut self) {
let (slf, events) = self.check_for_events();
slf.events_execution(events);
}
fn check_for_events(&mut self) -> (&Self, &[inotify::wrapper::Event]) {
let events = self.inotify.available_events().unwrap();
(&self, events)
}
I've made a small proof-of-concept on the playground (using vecs of u64 as the mutable state, but the principle is similar). It might be cleaner to refactor your code so that some external client can (mutably) borrow the Notifier, produce the events, release the borrow, and borrow it (immutably) to process them...
This is a known issue in the borrow checker (more discussion on internals). You cannot have a function taking a &mut T to an object and returning a &T without losing the ability to access the object again until the &T goes out of scope. You can't work around it due to the way inotify is implemented.
But you can ask the inotify authors to create a get_available_notifications method that doesn't fetch new data. This way you can call available_notifications once and drop the returned value. Then call get_available_notifications (which doesn't take &mut INotify, but just &INotify) and work from there.

Can I write an Iterator that mutates itself and then yields a reference into itself?

I ran into a problem that simplifies into the following:
struct MyIter {
vec: Vec<i8>,
}
fn fill_with_useful_data(v: &mut Vec<i8>) {
/* ... */
}
impl<'a> Iterator for MyIter {
type Item = &'a [i8];
fn next(&mut self) -> Option<&'a [i8]> {
fill_with_useful_data(&mut self.vec);
Some(&self.vec)
}
}
fn main() {
for slice in (MyIter { vec: Vec::new() }) {
println!("{}", slice);
}
}
This generates the error:
error[E0207]: the lifetime parameter `'a` is not constrained by the impl trait, self type, or predicates
--> src/main.rs:9:6
|
9 | impl<'a> Iterator for MyIter {
| ^^ unconstrained lifetime parameter
The idea is that the iterator does a bunch of work that reflects in its fields and at each step, it yields a reference into itself to the calling code. In this case I could model it as yielding a copy of the state instead of the reference, but let's pretend that's not possible or just inconveniently expensive.
Intuitively this shouldn't be a problem because the borrow checker can ensure that .next() isn't called again while the yielded reference can still be used to inspect the iterator's state, but the Iterator trait doesn't seem to provide for that sort of thing directly. Even with some permutations like only holding on to a reference to the vector in the iterator itself or making the iterator a reference or something to get the lifetimes baked into the type earlier on, I can't get anything past the borrow checker.
I read the "Iterators yielding mutable references" blogpost but I'm not sure if/how it applies to my problem that doesn't involve mutable references.
This is not possible. If it were allowed one could call next again and thus modify data that is also visible via & or even invalidate the reference entirely. This is because there is no connection between the self object itself and the returned reference: there is no explicit lifetime linking them.
For the compiler to reason about this and allow returning a reference into self next needs a signature like
fn next(&'a mut self) -> Option<&'a [i8]>
However, this differs from the signature of the trait which is not allowed as generic code that just takes an T: Iterator<...> cannot tell that there are different requirements on the use of the return value for some T; all have to be handled identically.
The Iterator trait is designed for return values that are independent of the iterator object, which is necessary for iterator adaptors like .collect to be correct and safe. This is more restrictive than necessary for many uses (e.g. a transient use inside a for loop) but it is just how it is at the moment. I don't think we have the tools for generalising this trait/the for loop properly now (specifically, I think we need associated types with higher rank lifetimes), but maybe in the future.

Resources