Reference got from mutex'a value doesn't live enought - multithreading

I'm learning Rust and trying to create a Admin panel like with egui and multi-threading;
One thread which update the Student struct in an Hashmap && another which run egui and display Student information;
here the problem, I have:
students: Arc<Hashmap<PathBuf, Mutex<Student>>>
actual_student : Option<Mutex<Student>>
// The student in the popup
// Here when I would like to get the &Student after clicking on a button 'See more'
self.actual_student = Some(self.students.get(path.deref()).unwrap());
// But got the error: lifetime may not live enough assignment requires that `'1` must outlive `'a`
Here the complete code of the implementation:
pub struct Maestro<'a> {
pub students: Arc<HashMap<PathBuf, Mutex<Student>>>,
pub logger: Logger,
pub actual_student: Option<&'a Mutex<Student>>,
// Would like to make like this: pub student: Option<&Student> but cannot get reference to
// Student from mutex's value
}
impl<'a> Maestro<'a> {
pub fn new() -> Self {
let config = load_config();
let watcher = config.watcher;
let students = Arc::new(watcher.init());
let mut students_for_thread = Arc::clone(&students);
std::thread::spawn(move || {
watcher.run(&mut students_for_thread);
});
Maestro {
students,
logger: config.logger,
actual_student: None,
}
}
}
impl<'a> eframe::App for Maestro<'a> {
fn update(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
egui::CentralPanel::default().show(ctx, |ui| {
for (path, student) in self.students.iter() {
if let Ok(mutex) = student.try_lock() {
ui.horizontal(|ui| {
ui.label(&mutex.name);
match &mutex.bashrc_editable {
true => ui.colored_label(egui::Color32::GREEN, "true"),
false => ui.colored_label(egui::Color32::RED, "false"),
};
if ui.button("See more").clicked() {
if self.students.contains_key(path) {
self.actual_student = Some(self.students.get(path.deref()).unwrap());
// FIXME 10/27/22 ectaclick: lifetime may not live long enough assignment requires that `'1` must outlive `'a`
}
}
});
}
}
});
// Would like to create a 'popup' on the right of the list
if self.actual_student.is_some() {
let student = self.actual_student.unwrap().lock().unwrap();
egui::SidePanel::right("student").show(ctx, |ui| {
ui.label(student.name.as_str());
match student.bashrc_editable {
true => ui.colored_label(Color32::GREEN, "true"),
false => ui.colored_label(Color32::RED, "false"),
}
});
}
std::thread::sleep(Duration::from_millis(100));
}
I tried to many type of actual_student:
Option<Student>
Box<Student>
Option<&Student>
Tried to clone the value but the actual_student is not the same of the Student in the Hashmap which is the Student struct updated by the second thread.
But the problem is still the same.

You're getting the lifetime error because if the value is removed the hashmap, the the reference you have in actual_student becomes invalid.
A direct solution to this is to use
Arc<Mutex<Student>> instead of Mutex<Student>
And the use simply use clone of the Arc object, this is relatively lightweight as it is not copying the whole object.
But in general, you're trying to think in OOP. a better approach is to use an id, for example:
actual_student_id : Option<u64>
which means you'll have to use
students.iter().find(|it|it.id == actual_student_id)
everytime you want to get a reference to actual student, but that avoids using Arc, and will probably get rid of Mutex too which is even better as mutexes are generally a source of performance hit, sometimes hard to debug.

Related

Lifetime of return value not directly related to parameter lifetime

I am new to rust, and having trouble figuring out the correct design pattern for what I am trying to do. In my program, there are Position structs, which reference a single coordinate system that they "belong" to. I have understood that rust must therefore be able to guarantee that coordinate_system reference outlives the Position. So far so good.
The issue is in a function such as transform_to_parent, where I want to create a new Position which is dependent on the lifetime of the exact coordinate_system that it later references, and not on the lifetime of the self parameter through which it accesses the coordinate system. This seems to be the only thing that lifetime specifiers would allow.
If I add a lifetime specifier to the following code, it compiles, but understandably complains when I let the old Position that I transformed from goes out of scope.
pub struct Position<'a> {
// Position data omitted
coordinate_system: &'a CoordinateSystem<'a>,
}
impl<'a> Position<'a> {
fn transform_to_parent<'b>(self: &'b Self) -> Option<Position<'b>> {
Some(Position {
coordinate_system: self.coordinate_system.origin?.coordinate_system
})
}
}
pub struct CoordinateSystem<'a> {
origin: Option<&'a Position<'a>>,
}
// Test case
fn main() {
// child_system is the child coordinate system of root_system, and has its origin at child_origin
let root_system = CoordinateSystem { origin: None };
let child_origin = Position { coordinate_system: &root_system };
let child_system = CoordinateSystem { origin: Some(&child_origin) };
let mut p2 = Position { coordinate_system: &child_system };
{
let p1 = Position { coordinate_system: &child_system };
if let Some(x) = p1.transform_to_parent() { // Error: "borrowed value does not live long enough"
p2 = x;
}
}
// No-op, just pretend to use p2 again, after p1 has gone out of scope
p2;
}
Is it possible for Rust to bind the lifetime of the function result to the lifetime of self.coordinate_system.get_origin()?.coordinate_system (i.e. the parent coordinate system), instead of the self? Is there a correct design pattern for something like this?
I assume that a ref-counting system would work, but I think that would be bad design, because there is clear ownership of the coordinate systems, and because the lifetime information should be deducible somehow.
Just use the 'a lifetime from the field:
impl<'a> Position<'a> {
fn transform_to_parent(&self) -> Option<Position<'a>> {
Some(Position {
coordinate_system: self.coordinate_system.origin?.coordinate_system
})
}
}

Rust: joining thread fails with: cannot move out of dereference of `std::sync::MutexGuard<'_, models::worker::Worker>`

I am having a hard time figuring out how to sort out this issue.
So I have a class ArcWorker holding a shared reference to Worker (as you can remark below).
I wrote a function in ArcWorker called join() in which the line self.internal.lock().unwrap().join(); fails with the following error:
cannot move out of dereference of std::sync::MutexGuard<'_, models::worker::Worker>
What I attempt through that line is to lock the mutex, unwrap and call the join() function from the Worker class.
As far as I understand, once that the lock function is called and it borrows a reference to self (&self), then I need some way to get to pass self by value to join (std::thread's join function requires passing self by value).
What can I do to make this work? Tried to find an answer to my question for hours but to no avail.
pub struct Worker {
accounts: Vec<Arc<Mutex<Account>>>,
thread_join_handle: Option<thread::JoinHandle<()>>
}
pub struct ArcWorker {
internal: Arc<Mutex<Worker>>
}
impl ArcWorker {
pub fn new(accounts: Vec<Arc<Mutex<Account>>>) -> ArcWorker {
return ArcWorker {
internal: Arc::new(Mutex::new(Worker {
accounts: accounts,
thread_join_handle: None
}))
}
}
pub fn spawn(&self) {
let local_self_1 = self.internal.clone();
self.internal.lock().unwrap().thread_join_handle = Some(thread::spawn(move || {
println!("Spawn worker");
local_self_1.lock().unwrap().perform_random_transactions();
}));
}
pub fn join(&self) {
self.internal.lock().unwrap().join();
}
}
impl Worker {
fn join(self) {
if let Some(thread_join_handle) = self.thread_join_handle {
thread_join_handle.join().expect("Couldn't join the associated threads.")
}
}
fn perform_random_transactions(&self) {
}
}
Since you already hold JoinHandle in an option, you can make Worker::join() take &mut self instead of self and change the if let condition to:
// note added `.take()`
if let Some(thread_join_handle) = self.thread_join_handle.take() {
Option::take() will move the handle out of the option and give you ownership over it, while leaving None in self.thread_join_handle. With this change ArcWorker::join() should compile as-is.

How to get State in FromRequest implementation with Rocket?

I initialized a Mutex<Pool<Postgres>> instance and passed it to manage method on Rocket instance in order to access the database on my controllers. What I usually do is:
// this is purely for example
#[get("/foo/bar")]
pub async fn controller<'a>(
// ... others ...
db_pool: &State<Mutex<PgPool>>,
// ... others ...
) {
// ... do other things ...
let query_r = sqlx::query("SELECT * FROM users WHERE id=$1")
.bind(&id)
.fetch_optional(&mut db_pool.inner().lock().await.acquire().await.unwrap()) // i reach `inner` of state, then `lock` the mutex, lastly `acquire` a pool connection
.await; // type is Result<Option<PgRow>> i suppose, rust-analyzer is not very good at inferring it
// ... do other things ...
}
This is cool and all but here's my problem: I wrote a struct as below...
pub struct User {
id: usize,
username: String,
email: String,
}
...and I actually want to use it as request guard so that the client cannot hit the controllers if they haven't provided the correct credentials.
Official guide of Rocket tells to implement FromRequest trait so that it can be resolved as guard.
However, in order to initialize a User instance, I need to get a PoolConnection inside this FromRequest trait. I have dived into Rocket's API documentation and seems like request parameter on FromRequest has a method named rocket with returns a Rocket<Orbit> instance, and, with this, I can access the state.
However, the problem is that state method returns &Mutex<Pool<Postgres>>, which is an immutable reference, so I cannot do any operations on database with this.
I have come this far and would like to ask if anyone knows a way to access Mutex<Pool<Postgres>> without a reference.
Thanks in advance.
How Far I Have Come
Below is how I have implemented FromRequest.
pub struct User {
id: usize,
username: String,
email: String,
}
#[derive(Debug)]
pub enum UserError {
TokenError(TokenError),
MissingToken,
DatabasePoolError,
}
#[rocket::async_trait]
impl<'r> FromRequest<'r> for User {
type Error = UserError;
async fn from_request(request: &'r rocket::Request<'_>) -> request::Outcome<Self, Self::Error> {
let encrypted_token_o = request.headers().get_one("Authorization");
match encrypted_token_o {
Some(encrypted_token) => {
let claims_r = Claims::try_from(encrypted_token.to_owned());
match claims_r {
Ok(claims) => {
let db_pool_o = request.rocket().state::<Mutex<PgPool>>();
match db_pool_o {
Some(db_pool) => {
let id = 0; // a test id
let query_r = sqlx::query(
"SELECT id, username, email FROM users WHERE id=$1",
)
.bind(&id)
.fetch_optional(
// ERROR: no method named `inner` found for reference `&rocket::futures::lock::Mutex<Pool<Postgres>>` in the current scope
&mut db_pool.inner().lock().await.acquire().await.unwrap(),
)
.await;
todo!()
}
None => request::Outcome::Failure((
http::Status::InternalServerError,
UserError::DatabasePoolError,
)),
}
}
Err(e) => request::Outcome::Failure((
http::Status::InternalServerError,
UserError::TokenError(e),
)),
}
}
None => request::Outcome::Failure((http::Status::BadRequest, UserError::MissingToken)),
}
}
}
Environment
rustc 1.55.0
rocket 0.5.0-rc.1 with features default and json
sqlx 0.5.7 with features runtime-tokio-native-tls, all-types, postgres and migrate

Variable in loop does not live long enough

I've been playing with Rust for the past few days,
and I'm still struggling with the memory management (figures).
I wrote a simple project that creates a hierarchy of structs ("keywords") from lexing/parsing a text file.
A keyword is defined like this:
pub struct Keyword<'a> {
name: String,
code: u32,
parent: Option<&'a Keyword<'a>>,
}
which is my equivalent for this C struct:
typedef struct Keyword Keyword;
struct Keyword {
char* name;
unsigned int code;
Keyword* parent;
};
A hierarchy is just a container for keywords, defined like this:
pub struct KeywordHierarchy<'a> {
keywords: Vec<Box<Keyword<'a>>>,
}
impl<'a> KeywordHierarchy<'a> {
fn add_keyword(&mut self, keyword: Box<Keyword<'a>>) {
self.keywords.push(keyword);
}
}
In the parse function (which is a stub of the complete parser), I recreated the condition that spawns the lifetime error in my code.
fn parse<'a>() -> Result<KeywordHierarchy<'a>, String> {
let mut hierarchy = KeywordHierarchy { keywords: Vec::new() };
let mut first_if = true;
let mut second_if = false;
while true {
if first_if {
// All good here.
let root_keyword = Keyword {
name: String::from("ROOT"),
code: 0,
parent: None,
};
hierarchy.add_keyword(Box::new(root_keyword));
first_if = false;
second_if = true;
}
if second_if {
// Hierarchy might have expired here?
// Find parent
match hierarchy.keywords.iter().find(|p| p.code == 0) {
None => return Err(String::from("No such parent")),
Some(parent) => {
// If parent is found, create a child
let child = Keyword {
name: String::from("CHILD"),
code: 1,
parent: Some(&parent),
};
hierarchy.add_keyword(Box::new(child));
}
}
second_if = false;
}
if !first_if && !second_if {
break;
}
}
Ok(hierarchy)
}
There's a while loop that goes through the lexer tokens.
In the first if, I add the ROOT keyword to the hierarchy, which is the only one that doesn't have a parent, and everything goes smoothly as expected.
In the second if, I parse the children keywords and I get a lifetime error when invoking KeywordHierarchy.add_keyword().
hierarchy.keywords` does not live long enough
Could you guys recommend an idiomatic way to fix this?
Cheers.
P.S. Click here for the playground
The immediate problem I can see with your design is that your loop is going to modify the hierarchy.keywords vector (in the first_if block), but it also borrows elements from the hierarchy.keywords vector (in the second_if block).
This is problematic, because modifying a vector may cause its backing buffer to be reallocated, which, if it were allowed, would invalidate all existing borrows to the vector. (And thus it is not allowed.)
Have you considered using an arena to hold your keywords instead of a Vec? Arenas are designed so that you can allocate new things within them while still having outstanding borrows to elements previously allocated within the arena.
Update: Here is a revised version of your code that illustrates using an arena (in this case a rustc_arena::TypedArena, but that's just so I can get this running on the play.rust-lang.org service) alongside a Vec<&Keyword> to handle the lookups.
https://play.rust-lang.org/?gist=fc6d81cb7efa7e4f32c481ab6482e587&version=nightly&backtrace=0
The crucial bits of code are this:
First: the KeywordHierarchy now holds a arena alongside a vec:
pub struct KeywordHierarchy<'a> {
keywords: Vec<&'a Keyword<'a>>,
kw_arena: &'a TypedArena<Keyword<'a>>,
}
Second: Adding a keyword now allocates the spot in the arena, and stashes a reference to that spot in the vec:
fn add_keyword(&mut self, keyword: Keyword<'a>) {
let kw = self.kw_arena.alloc(keyword);
self.keywords.push(kw);
}
Third: the fn parse function now takes an arena (reference) as input, because we need the arena to outlive the stack frame of fn parse:
fn parse<'a>(arena: &'a TypedArena<Keyword<'a>>) -> Result<KeywordHierarchy<'a>, String> {
...
Fourth: To avoid borrow-checker issues with borrowing hierarchy as mutable while also iterating over it, I moved the hierarchy modification outside of your Find parent match:
// Find parent
let parent = match hierarchy.keywords.iter().find(|p| p.code == 0) {
None => return Err(String::from("No such parent")),
Some(parent) => *parent, // (this deref is important)
};
// If parent is found, create a child
let child = Keyword {
name: String::from("CHILD"),
code: 1,
parent: Some(parent),
};
hierarchy.add_keyword(child);
second_if = false;

In memory database design

I am trying to create an in-memory database using HashMap. I have a struct Person:
struct Person {
id: i64,
name: String,
}
impl Person {
pub fn new(id: i64, name: &str) -> Person {
Person {
id: id,
name: name.to_string(),
}
}
pub fn set_name(&mut self, name: &str) {
self.name = name.to_string();
}
}
And I have struct Database:
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::Mutex;
struct Database {
db: Arc<Mutex<HashMap<i64, Person>>>,
}
impl Database {
pub fn new() -> Database {
Database {
db: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn add_person(&mut self, id: i64, person: Person) {
self.db.lock().unwrap().insert(id, person);
}
pub fn get_person(&self, id: i64) -> Option<&mut Person> {
self.db.lock().unwrap().get_mut(&id)
}
}
And code to use this database:
let mut db = Database::new();
db.add_person(1, Person::new(1, "Bob"));
I want to change person's name:
let mut person = db.get_person(1).unwrap();
person.set_name("Bill");
The complete code in the Rust playground.
When compiling, I get a problem with Rust lifetimes:
error[E0597]: borrowed value does not live long enough
--> src/main.rs:39:9
|
39 | self.db.lock().unwrap().get_mut(&id)
| ^^^^^^^^^^^^^^^^^^^^^^^ temporary value does not live long enough
40 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the method body at 38:5...
--> src/main.rs:38:5
|
38 | / pub fn get_person(&self, id: i64) -> Option<&mut Person> {
39 | | self.db.lock().unwrap().get_mut(&id)
40 | | }
| |_____^
How to implement this approach?
The compiler rejects your code because it violates the correctness model enforced by Rust and could cause crashes. For one, if get_person() were allowed to compile, one might call it from two threads and modify the underlying object without the protection of the mutex, causing data races on the String object inside. Worse, one could wreak havoc even in a single-threaded scenario by doing something like:
let mut ref1 = db.get_person(1).unwrap();
let mut ref2 = db.get_person(1).unwrap();
// ERROR - two mutable references to the same object!
let vec: Vec<Person> = vec![];
vec.push(*ref1); // move referenced object to the vector
println!(*ref2); // CRASH - object already moved
To correct the code, you need to adjust your design to satisfy the following constraints:
No reference can be allowed to outlive the referred-to object;
During the lifetime of a mutable reference, no other reference (mutable or immutable) to the object may exist..
The add_person method already complies with both rules because it eats the object you pass it, moving it to the database.
What if we modified get_person() to return an immutable reference?
pub fn get_person(&self, id: i64) -> Option<&Person> {
self.db.lock().unwrap().get(&id)
}
Even this seemingly innocent version still doesn't compile! That is because it violates the first rule. Rust cannot statically prove that the reference will not outlive the database itself, since the database is allocated on the heap and reference-counted, so it can be dropped at any time. But even if it were possible to somehow explicitly declare the lifetime of the reference to one that provably couldn't outlive the database, retaining the reference after unlocking the mutex would allow data races. There is simply no way to implement get_person() and still retain thread safety.
A thread-safe implementation of a read can opt to return a copy of the data. Person can implement the clone() method and get_person() can invoke it like this:
#[derive(Clone)]
struct Person {
id: i64,
name: String
}
// ...
pub fn get_person(&self, id: i64) -> Option<Person> {
self.db.lock().unwrap().get(&id).cloned()
}
This kind of change won't work for the other use case of get_person(), where the method is used for the express purpose of obtaining a mutable reference to change the person in the database. Obtaining a mutable reference to a shared resource violates the second rule and could lead to crashes as shown above. There are several ways to make it safe. One is by providing a proxy in the database for setting each Person field:
pub fn set_person_name(&self, id: i64, new_name: String) -> bool {
match self.db.lock().unwrap().get_mut(&id) {
Some(mut person) => {
person.name = new_name;
true
}
None => false
}
}
As the number of fields on Person grows, this would quickly get tedious. It could also get slow, as a separate mutex lock would have to be acquired for each access.
There is fortunately a better way to implement modification of the entry. Remember that using a mutable reference violates the rules unless Rust can prove that the reference won't "escape" the block where it is being used. This can be ensured by inverting the control - instead of a get_person() that returns the mutable reference, we can introduce a modify_person() that passes the mutable reference to a callable, which can do whatever it likes with it. For example:
pub fn modify_person<F>(&self, id: i64, f: F) where F: FnOnce(Option<&mut Person>) {
f(self.db.lock().unwrap().get_mut(&id))
}
The usage would look like this:
fn main() {
let mut db = Database::new();
db.add_person(1, Person::new(1, "Bob"));
assert!(db.get_person(1).unwrap().name == "Bob");
db.modify_person(1, |person| {
person.unwrap().set_name("Bill");
});
}
Finally, if you're worried about the performance of get_person() cloning Person for the sole reason of inspecting it, it is trivial to create an immutable version of modify_person that serves as a non-copying alternative to get_person():
pub fn read_person<F, R>(&self, id: i64, f: F) -> R
where F: FnOnce(Option<&Person>) -> R {
f(self.db.lock().unwrap().get(&id))
}
Besides taking a shared reference to Person, read_person is also allowing the closure to return a value if it chooses, typically something it picks up from the object it receives. Its usage would be similar to the usage of modify_person, with the added possibility of returning a value:
// if Person had an "age" field, we could obtain it like this:
let person_age = db.read_person(1, |person| person.unwrap().age);
// equivalent to the copying definition of db.get_person():
let person_copy = db.read_person(1, |person| person.cloned());
This post use the pattern cited as "inversion of control" in the well explained answer and just add only sugar for demonstrating another api for an in-memory db.
With a macro rule it is possible to expose a db client api like that:
fn main() {
let db = Database::new();
let person_id = 1234;
// probably not the best design choice to duplicate the person_id,
// for the purpose here is not important
db.add_person(person_id, Person::new(person_id, "Bob"));
db_update!(db #person_id => set_name("Gambadilegno"));
println!("your new name is {}", db.get_person(person_id).unwrap().name);
}
My opinionated macro has the format:
<database_instance> #<object_key> => <method_name>(<args>)
Below the macro implementation and the full demo code:
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::Mutex;
macro_rules! db_update {
($db:ident # $id:expr => $meth:tt($($args:tt)*)) => {
$db.modify_person($id, |person| {
person.unwrap().$meth($($args)*);
});
};
}
#[derive(Clone)]
struct Person {
id: u64,
name: String,
}
impl Person {
pub fn new(id: u64, name: &str) -> Person {
Person {
id: id,
name: name.to_string(),
}
}
fn set_name(&mut self, value: &str) {
self.name = value.to_string();
}
}
struct Database {
db: Arc<Mutex<HashMap<u64, Person>>>, // access from different threads
}
impl Database {
pub fn new() -> Database {
Database {
db: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn add_person(&self, id: u64, person: Person) {
self.db.lock().unwrap().insert(id, person);
}
pub fn modify_person<F>(&self, id: u64, f: F)
where
F: FnOnce(Option<&mut Person>),
{
f(self.db.lock().unwrap().get_mut(&id));
}
pub fn get_person(&self, id: u64) -> Option<Person> {
self.db.lock().unwrap().get(&id).cloned()
}
}

Resources