Callbacks that mutate local state - rust

I have two structs:
Client, which stores a callback and calls it in response to receiving new data. As an example, you can think of this as a websocket client, and we want to provide a hook for incoming messages.
BusinessLogic, which wants to hold a Client initialized with a callback that will update its local value in response to changes that the Client sees.
After following compiler hints, I arrived at the following minimal example:
Rust playground link
use rand::Rng;
struct Client<'cb> {
callback: Box<dyn FnMut(i64) + 'cb>,
}
impl<'cb> Client<'cb> {
fn do_thing(&mut self) {
// does stuff
let value = self._get_new_value();
// does more stuff
(self.callback)(value);
// does even more stuff
}
fn _get_new_value(&self) -> i64 {
let mut rng = rand::thread_rng();
rng.gen()
}
}
struct BusinessLogic<'cb> {
value: Option<i64>,
client: Option<Client<'cb>>,
}
impl<'cb> BusinessLogic<'cb> {
fn new() -> Self {
Self {
value: None,
client: None,
}
}
fn subscribe(&'cb mut self) {
self.client = Some(Client {
callback: Box::new(|value| {
self.value = Some(value);
})
})
}
}
fn main() {
let mut bl = BusinessLogic::new();
bl.subscribe();
println!("Hello, world!");
}
Problem is, I am still getting the following compiler error:
Compiling playground v0.0.1 (/playground)
error[E0597]: `bl` does not live long enough
--> src/main.rs:51:5
|
51 | bl.subscribe();
| ^^^^^^^^^^^^^^ borrowed value does not live long enough
...
54 | }
| -
| |
| `bl` dropped here while still borrowed
| borrow might be used here, when `bl` is dropped and runs the destructor for type `BusinessLogic<'_>`
For more information about this error, try `rustc --explain E0597`.
error: could not compile `playground` due to previous error
I understand why I'm seeing this error: the call to subscribe uses a borrow of bl with a lifetime of 'cb, which is not necessarily contained within the scope of main(). However, I don't see how to resolve this issue. Won't I always need to provide a lifetime for the callback stored in Client, which will end up bleeding through my code in the form of 'cb lifetime annotations?
More generally, I'm interested in understanding what is the canonical way of solving this callback/hook problem in Rust. I'm open to designs different from the one I have proposed, and if there are relevant performance concerns for various options, that would be useful to know also.

What you've created is a self-referential structure, which is problematic and not really expressible with references and lifetime annotations. See: Why can't I store a value and a reference to that value in the same struct? for the potential problems and workarounds. Its an issue here because you want to be able to mutate the BusinessLogic in the callback, but since it holds the Client, you can mutate the callback while its running, which is no good.
I would instead suggest that the callback has full ownership of the BusinessLogic which does not directly reference the Client:
use rand::Rng;
struct Client {
callback: Box<dyn FnMut(i64)>,
}
impl Client {
fn do_thing(&mut self) {
let value = rand::thread_rng().gen();
(self.callback)(value);
}
}
struct BusinessLogic {
value: Option<i64>,
}
fn main() {
let mut bl = BusinessLogic {
value: None
};
let mut client = Client {
callback: Box::new(move |value| {
bl.value = Some(value);
})
};
client.do_thing();
println!("Hello, world!");
}
if you need the subscriber to have backwards communication to the Client, you can pass an additional parameter that the callback can mutate, or simply do it via return value
if you need more complicated communication from the Client to the callback, either send a Message enum as the argument, or make the callback a custom trait instead of just FnMut with additional methods
if you need a single BusinessLogic to operate from multiple Clients use Arc+Mutex to allow shared ownership

Related

How to borrow ctx in a module chain

I have this entrypoint where ctx is passed by a parent:
pub mod instructions;
#[program]
pub mod solana_anchor_coinflip {
use super::*;
pub fn flip(ctx: Context<Play>, data: FlipArgs) -> Result<()> {
instructions::play::play(ctx, data)
}
}
Then, instructions/play.rs has this:
pub fn play(ctx: Context<Play>, data: FlipArgs) -> Result<()> {
ctx.accounts.game.flip(ctx, data) // <== cannot move out of `ctx` because it is borrowed
}
#[derive(Accounts)]
pub struct Play<'info> {
#[account(mut)]
pub game: Account<'info, Game>,
pub player: Signer<'info>,
}
and ctx is finally passed to game.rs:
impl Game {
pub fn flip(&mut self, ctx: Context<Play>, data: FlipArgs) -> Result<()> {
self.charge_fee(ctx);
match data.heads_or_tails {
true => self.play_heads(ctx), // <== use of moved value: `ctx`
false => self.play_tails(ctx),
}
}
fn charge_fee(&self, ctx: Context<Play>) -> Result<()> {
let player = &ctx.accounts.player;
// ...
Ok(())
}
}
How to correctly borrow ctx from lib.rs > play.rs > game.rs?
As said in the comments, you are not borrowing ctx, you're just moving it. See the relevant documentation to understand the differences, alongside with examples to illustrate that.
If you read through that, you will understand why you just need to change the signature of your functions:
// in instructions/play.rs
pub fn play(ctx: &Context<Play>, data: FlipArgs) -> Result<()> { // <-- takes a `&Context<Play>`
ctx.accounts.game.flip(ctx, data) // and passes the borrow
}
// game.rs
impl Game {
pub fn flip(&mut self, ctx: &Context<Play>, data: FlipArgs) -> Result<()> { // <-- takes a &Context<Play>
self.charge_fee(ctx); // <-- here you pass the borrow
// also, unused `Result`, which is _bad_
if data.heads_or_tails { // an `if` statement is a `match` over a `bool`
self.play_heads(ctx) // <-- here you pass the borrow too, which is fine, because `&T: Copy`
} else {
self.play_tails(ctx) // <-- same as above
}
}
fn charge_fee(&self, ctx: &Context<Play>) -> Result<()> { // <-- takes a `&Context<Play>`
let player = &ctx.accounts.player;
// ...
Ok(())
}
}
However, despite this patch, there is a chance that you need to refactor your code anyways after understanding borrow, for multiple motives:
Given the context of your question, it's impossible to tell if play should take borrow to a Context<Play>, or an owned value, because both could work (in the second case, you would have to pass a borrow to the method call, ie ctx.accounts.game.flip(&ctx, data).
You seem to pass data as an owned value all along, just like you do with ctx, but since it was a mistake with ctx, it might be for data too (and for many other parts of your code).
I'm not sure whether the current version will compile anyways do the flip requiring &mut self, that is, it needs a mutable borrow to ctx.accounts.game in the play function. However, that would also require a mutable borrow of ctx (unless it doesn't, but that's a bit too advanced: learn about borrows before learning about interior mutability) for the duration of the call, which would invalidate any borrow to be passed as an argument.
As an advice, I would suggest you to wrap your head around Rust's core concepts (which can be efficiently done by reading the Rust Book) before designing the architecture of a complex application, because Rust has some very peculiar patterns, even if you are used other programming languages. Otherwise, you will keep fighting against the compiler trying to adapt the code you had in mind when you still did not fully understand the borrowing and ownership in Rust to something that works.

Can't find my way arround lifetime and borrowing

This problem is very simple, yet I can't figure out how to implement this in Rust:
I have a TCP server that awaits for new clients connection. Each Client stores it's TCPStream and a UUID and is stored in a ClientHolder struct. This struct implements an update function that will call Client::update for each Client.
The Client::update function looks like this:
pub fn update(&mut self) -> bool {
let mut msg = String::new();
match self.bufreader.read_line(&mut msg) {
Ok(size) => {
if size > 1 {
self.parse(&msg);
}
true
}
Err(_) => {
println!(
"Terminating connection with {}",
self.stream.peer_addr().unwrap()
);
self.stream.shutdown(Shutdown::Both).unwrap();
false
}
}
}
I need the parse function to be able to execute code that'll modify a Client (for example to transfer a command from a client to another). I can't find a way to get a reference to a Client from inside the parse function without triggering borrowing or lifetime errors that I can't fix.
I would really appreciate if someone could point me in the right direction. I've been struggling for two days on this issue and tried many things.
EDIT:
This is an example of what could be parse:
fn parse(&mut self, msg: &String, holder: &ClientHolder) {
let v: Value = serde_json::from_str(&msg).unwrap();
if v["type"] == "init" {
self.set_uuid(v["uuid"].to_string());
}
else {
let uuid = v["uuid"];
let client = holder.get_client(&uuid);
command_service.try_execute(v["type"], client);
}
}
If &ClientHolder is forwarded using update, this is what happens:
Inside ClientHolder, the update function calls Client::update and pass an immutable reference of self. If the client is disconnected, it removes it from the Vec.
pub fn update(&mut self) {
for i in (0..self.clients.len()).rev() {
let client = &mut self.clients[i];
if !client.update(&*self) {
self.clients.swap_remove(i);
}
}
}
The compiler gives me this error:
error[E0502]: cannot borrow `*self` as immutable because it is also borrowed as mutable
--> src\server\client_holder.rs:25:31
|
24 | let client = &mut self.clients[i];
| ------------ mutable borrow occurs here
25 | if !client.update(&*self) {
| ------ ^^^^^^ immutable borrow occurs here
| |
| mutable borrow later used by call
I understand why it does, but couldn't find a solution that works for me.

Cannot borrow in a Rc as mutable

First of all I'm new with Rust :-)
The problem:
I want to create a module called RestServer that contain the methods ( actix-web ) to add routes and start the server.
struct Route
{
url: String,
request: String,
handler: Box<dyn Fn(HttpRequest) -> HttpResponse>
}
impl PartialEq for Route {
fn eq(&self, other: &Self) -> bool {
self.url == other.url
}
}
impl Eq for Route {}
impl Hash for Route {
fn hash<H: Hasher>(&self, hasher: &mut H) {
self.url.hash(hasher);
}
}
this is the route structure, this structure containe the the route url, the request type ( GET, POST etc ) and hanlder is the function that have to catch the request and return a HTTPResponse
pub struct RestServer
{
scopes: HashMap<String, Rc<HashSet<Route>>>,
routes: HashSet<Route>,
host: String,
}
impl RestServer {
pub fn add_route(self, req: &str, funct: impl Fn(HttpRequest) -> HttpResponse + 'static,
route: &str, scope: Option<&str>) -> RestServer
{
let mut routes_end = self.routes;
let mut scopes_end = self.scopes;
let url = self.host;
let route = Route {
url: String::from(route),
request: String::from(req),
handler: Box::new(funct)
};
if let Some(x) = scope {
if let Some(y) = scopes_end.get(x) {
let mut cloned_y = Rc::clone(y);
cloned_y.insert(route);
scopes_end.insert(String::from(x), cloned_y);
}else {
let mut hash_scopes = HashSet::new();
hash_scopes.insert(route);
scopes_end.insert(String::from(x), Rc::new(hash_scopes));
}
} else {
routes_end.insert(route);
}
RestServer {
scopes: scopes_end,
routes: routes_end,
host: String::from(url)
}
}
the latest code is the implementation of RestServer.
The most important part is the add_route function, this function receive as paramente the route that is a string, the function handler, the request string and the scope.
First i create the route object.
I check if the scope exist into the HashMap, if yes i have to take the actual scope and update the HashSet.
When i build the code i get the following error
error[E0596]: cannot borrow data in an `Rc` as mutable
--> interface/src/rest/mod.rs:60:17
|
60 | cloned_y.insert(route);
| ^^^^^^^^ cannot borrow as mutable
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not
implemented for `std::rc::Rc<std::collections::HashSet<rest::Route>>`
I know that the compiler give me some help but honestly i have no idea how to do that or if i can do with some easy solution.
After a large search in google i found a solution in RefCell, but is not so much clear
Thanks in advance for your help
You cannot borrow a reference-counting pointer as mutable; this is because one of the guarantees it provides is only possible if the structure is read-only.
You can, however, get around it, but it will require some signature changes.
Enter interior mutability
Interior mutability is a concept you may know from other programming languages in the form of mutexes, atomics and synchronization primitives. In practice, those structures allow you to temporarily guarantee that you are the only accessor of a given variable.
In Rust, this is particularly good, as it allows us to extract a mutable reference to an interior member from a structure that only requires immutable references to itself to function. Perfect to fit in Rc.
Depending on what you need for your needs, you will find the Cell and RefCell structures to be exactly what you need for this. These are not thread-safe, but then again, neither is Rc so it's not exactly a deal-breaker.
In practice, it works very simply:
let data = Rc::new(RefCell::new(true));
{
let mut reference = data.borrow_mut();
*reference = false;
}
println!("{:?}", data);
playground
(If you ever want the threaded versions, Arc replaces Rc and Mutex or RwLock replaces Cell/RefCell)

Spawning tasks with non-static lifetimes with tokio 0.1.x

I have a tokio core whose main task is running a websocket (client). When I receive some messages from the server, I want to execute a new task that will update some data. Below is a minimal failing example:
use tokio_core::reactor::{Core, Handle};
use futures::future::Future;
use futures::future;
struct Client {
handle: Handle,
data: usize,
}
impl Client {
fn update_data(&mut self) {
// spawn a new task that updates the data
self.handle.spawn(future::ok(()).and_then(|x| {
self.data += 1; // error here
future::ok(())
}));
}
}
fn main() {
let mut runtime = Core::new().unwrap();
let mut client = Client {
handle: runtime.handle(),
data: 0,
};
let task = future::ok::<(), ()>(()).and_then(|_| {
// under some conditions (omitted), we update the data
client.update_data();
future::ok::<(), ()>(())
});
runtime.run(task).unwrap();
}
Which produces this error:
error[E0477]: the type `futures::future::and_then::AndThen<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:13:51: 16:10 self:&mut &mut Client]>` does not fulfill the required lifetime
--> src/main.rs:13:21
|
13 | self.handle.spawn(future::ok(()).and_then(|x| {
| ^^^^^
|
= note: type must satisfy the static lifetime
The problem is that new tasks that are spawned through a handle need to be static. The same issue is described here. Sadly it is unclear to me how I can fix the issue. Even some attempts with and Arc and a Mutex (which really shouldn't be needed for a single-threaded application), I was unsuccessful.
Since developments occur rather quickly in the tokio landscape, I am wondering what the current best solution is. Do you have any suggestions?
edit
The solution by Peter Hall works for the example above. Sadly when I built the failing example I changed tokio reactor, thinking they would be similar. Using tokio::runtime::current_thread
use futures::future;
use futures::future::Future;
use futures::stream::Stream;
use std::cell::Cell;
use std::rc::Rc;
use tokio::runtime::current_thread::{Builder, Handle};
struct Client {
handle: Handle,
data: Rc<Cell<usize>>,
}
impl Client {
fn update_data(&mut self) {
// spawn a new task that updates the data
let mut data = Rc::clone(&self.data);
self.handle.spawn(future::ok(()).and_then(move |_x| {
data.set(data.get() + 1);
future::ok(())
}));
}
}
fn main() {
// let mut runtime = Core::new().unwrap();
let mut runtime = Builder::new().build().unwrap();
let mut client = Client {
handle: runtime.handle(),
data: Rc::new(Cell::new(1)),
};
let task = future::ok::<(), ()>(()).and_then(|_| {
// under some conditions (omitted), we update the data
client.update_data();
future::ok::<(), ()>(())
});
runtime.block_on(task).unwrap();
}
I obtain:
error[E0277]: `std::rc::Rc<std::cell::Cell<usize>>` cannot be sent between threads safely
--> src/main.rs:17:21
|
17 | self.handle.spawn(future::ok(()).and_then(move |_x| {
| ^^^^^ `std::rc::Rc<std::cell::Cell<usize>>` cannot be sent between threads safely
|
= help: within `futures::future::and_then::AndThen<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]>`, the trait `std::marker::Send` is not implemented for `std::rc::Rc<std::cell::Cell<usize>>`
= note: required because it appears within the type `[closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]`
= note: required because it appears within the type `futures::future::chain::Chain<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]>`
= note: required because it appears within the type `futures::future::and_then::AndThen<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]>`
So it does seem like in this case I need an Arc and a Mutex even though the entire code is single-threaded?
In a single-threaded program, you don't need to use Arc; Rc is sufficient:
use std::{rc::Rc, cell::Cell};
struct Client {
handle: Handle,
data: Rc<Cell<usize>>,
}
impl Client {
fn update_data(&mut self) {
let data = Rc::clone(&self.data);
self.handle.spawn(future::ok(()).and_then(move |_x| {
data.set(data.get() + 1);
future::ok(())
}));
}
}
The point is that you no longer have to worry about the lifetime because each clone of the Rc acts as if it owns the data, rather than accessing it via a reference to self. The inner Cell (or RefCell for non-Copy types) is needed because the Rc can't be dereferenced mutably, since it has been cloned.
The spawn method of tokio::runtime::current_thread::Handle requires that the future is Send, which is what is causing the problem in the update to your question. There is an explanation (of sorts) for why this is the case in this Tokio Github issue.
You can use tokio::runtime::current_thread::spawn instead of the method of Handle, which will always run the future in the current thread, and does not require that the future is Send. You can replace self.handle.spawn in the code above and it will work just fine.
If you need to use the method on Handle then you will also need to resort to Arc and Mutex (or RwLock) in order to satisfy the Send requirement:
use std::sync::{Mutex, Arc};
struct Client {
handle: Handle,
data: Arc<Mutex<usize>>,
}
impl Client {
fn update_data(&mut self) {
let data = Arc::clone(&self.data);
self.handle.spawn(future::ok(()).and_then(move |_x| {
*data.lock().unwrap() += 1;
future::ok(())
}));
}
}
If your data is really a usize, you could also use AtomicUsize instead of Mutex<usize>, but I personally find it just as unwieldy to work with.

Storing a boxed closure which references an object in that object

I'm trying to implement a console system for the game I'm writing and have found a fairly simple system: I define a Console object that stores commands as boxed closures (specifically Box<FnMut + 'a> for some 'a). This works for any component of the engine so long as the Console is created before anything else.
Unfortunately, this prevents me from adding commands that modify the Console itself, which means I can't create commands that simply print text or define other variables or commands. I've written a small example that replicates the error:
use std::cell::Cell;
struct Console<'a> {
cmds: Vec<Box<FnMut() + 'a>>,
}
impl<'a> Console<'a> {
pub fn println<S>(&self, msg: S)
where S: AsRef<str>
{
println!("{}", msg.as_ref());
}
pub fn add_cmd(&mut self, cmd: Box<FnMut() + 'a>) {
self.cmds.push(cmd);
}
}
struct Example {
val: Cell<i32>,
}
fn main() {
let ex = Example {
val: Cell::new(0),
};
let mut con = Console {
cmds: Vec::new(),
};
// this works
con.add_cmd(Box::new(|| ex.val.set(5)));
(con.cmds[0])();
// this doesn't
let cmd = Box::new(|| con.println("Hello, world!"));
con.add_cmd(cmd);
(con.cmds[1])();
}
And the error:
error: `con` does not live long enough
--> console.rs:34:31
|
34 | let cmd = Box::new(|| con.println("Hello, world!"));
| -- ^^^ does not live long enough
| |
| capture occurs here
35 | con.add_cmd(cmd);
36 | }
| - borrowed value dropped before borrower
|
= note: values in a scope are dropped in the opposite order they are created
error: aborting due to previous error
Is there a workaround for this, or a better system I should look into? This is on rustc 1.18.0-nightly (53f4bc311 2017-04-07).
This is one of those fairly tricky resource borrowing conundrums that the compiler could not allow. Basically, we have a Console that owns multiple closures, which in turn capture an immutable reference to the same console. This means two constraints:
Since Console owns the closures, they will live for as long as the console itself, and the inner vector will drop them right after Console is dropped.
At the same time, each closure must not outlive Console, because otherwise we would end up with dangling references to the console.
It may seem harmless from the fact that the console and respective closures go out of scope at once. However, the drop method follows a strict order here: first the console, then the closures.
Not to mention of course, that if you wish for closures to freely apply modifications to the console without interior mutability, you would have to mutably borrow it, which cannot be done over multiple closures.
An approach to solving the problem is to separate the two: let the console not own the closures, instead having them in a separate registry, and let the closures only borrow the console when calling the closure.
This can be done by passing the console as an argument to the closures and moving the closure vector to another object (Playground):
use std::cell::Cell;
struct CommandRegistry<'a> {
cmds: Vec<Box<Fn(&mut Console) + 'a>>,
}
impl<'a> CommandRegistry<'a> {
pub fn add_cmd(&mut self, cmd: Box<Fn(&mut Console) + 'a>) {
self.cmds.push(cmd);
}
}
struct Console {
}
impl Console {
pub fn println<S>(&mut self, msg: S)
where S: AsRef<str>
{
println!("{}", msg.as_ref());
}
}
struct Example {
val: Cell<i32>,
}
fn main() {
let ex = Example {
val: Cell::new(0),
};
let mut reg = CommandRegistry{ cmds: Vec::new() };
let mut con = Console {};
// this works
reg.add_cmd(Box::new(|_: &mut Console| ex.val.set(5)));
(reg.cmds[0])(&mut con);
// and so does this now!
let cmd = Box::new(|c: &mut Console| c.println("Hello, world!"));
reg.add_cmd(cmd);
(reg.cmds[1])(&mut con);
}
I also took the liberty of making closures accept a mutable reference. No conflicts emerge here because we are no longer borrowing the console that was already borrowed when fetching the borrowing closure. This way, the closures can also outlive the console.

Resources