This problem is very simple, yet I can't figure out how to implement this in Rust:
I have a TCP server that awaits for new clients connection. Each Client stores it's TCPStream and a UUID and is stored in a ClientHolder struct. This struct implements an update function that will call Client::update for each Client.
The Client::update function looks like this:
pub fn update(&mut self) -> bool {
let mut msg = String::new();
match self.bufreader.read_line(&mut msg) {
Ok(size) => {
if size > 1 {
self.parse(&msg);
}
true
}
Err(_) => {
println!(
"Terminating connection with {}",
self.stream.peer_addr().unwrap()
);
self.stream.shutdown(Shutdown::Both).unwrap();
false
}
}
}
I need the parse function to be able to execute code that'll modify a Client (for example to transfer a command from a client to another). I can't find a way to get a reference to a Client from inside the parse function without triggering borrowing or lifetime errors that I can't fix.
I would really appreciate if someone could point me in the right direction. I've been struggling for two days on this issue and tried many things.
EDIT:
This is an example of what could be parse:
fn parse(&mut self, msg: &String, holder: &ClientHolder) {
let v: Value = serde_json::from_str(&msg).unwrap();
if v["type"] == "init" {
self.set_uuid(v["uuid"].to_string());
}
else {
let uuid = v["uuid"];
let client = holder.get_client(&uuid);
command_service.try_execute(v["type"], client);
}
}
If &ClientHolder is forwarded using update, this is what happens:
Inside ClientHolder, the update function calls Client::update and pass an immutable reference of self. If the client is disconnected, it removes it from the Vec.
pub fn update(&mut self) {
for i in (0..self.clients.len()).rev() {
let client = &mut self.clients[i];
if !client.update(&*self) {
self.clients.swap_remove(i);
}
}
}
The compiler gives me this error:
error[E0502]: cannot borrow `*self` as immutable because it is also borrowed as mutable
--> src\server\client_holder.rs:25:31
|
24 | let client = &mut self.clients[i];
| ------------ mutable borrow occurs here
25 | if !client.update(&*self) {
| ------ ^^^^^^ immutable borrow occurs here
| |
| mutable borrow later used by call
I understand why it does, but couldn't find a solution that works for me.
Related
I have two structs:
Client, which stores a callback and calls it in response to receiving new data. As an example, you can think of this as a websocket client, and we want to provide a hook for incoming messages.
BusinessLogic, which wants to hold a Client initialized with a callback that will update its local value in response to changes that the Client sees.
After following compiler hints, I arrived at the following minimal example:
Rust playground link
use rand::Rng;
struct Client<'cb> {
callback: Box<dyn FnMut(i64) + 'cb>,
}
impl<'cb> Client<'cb> {
fn do_thing(&mut self) {
// does stuff
let value = self._get_new_value();
// does more stuff
(self.callback)(value);
// does even more stuff
}
fn _get_new_value(&self) -> i64 {
let mut rng = rand::thread_rng();
rng.gen()
}
}
struct BusinessLogic<'cb> {
value: Option<i64>,
client: Option<Client<'cb>>,
}
impl<'cb> BusinessLogic<'cb> {
fn new() -> Self {
Self {
value: None,
client: None,
}
}
fn subscribe(&'cb mut self) {
self.client = Some(Client {
callback: Box::new(|value| {
self.value = Some(value);
})
})
}
}
fn main() {
let mut bl = BusinessLogic::new();
bl.subscribe();
println!("Hello, world!");
}
Problem is, I am still getting the following compiler error:
Compiling playground v0.0.1 (/playground)
error[E0597]: `bl` does not live long enough
--> src/main.rs:51:5
|
51 | bl.subscribe();
| ^^^^^^^^^^^^^^ borrowed value does not live long enough
...
54 | }
| -
| |
| `bl` dropped here while still borrowed
| borrow might be used here, when `bl` is dropped and runs the destructor for type `BusinessLogic<'_>`
For more information about this error, try `rustc --explain E0597`.
error: could not compile `playground` due to previous error
I understand why I'm seeing this error: the call to subscribe uses a borrow of bl with a lifetime of 'cb, which is not necessarily contained within the scope of main(). However, I don't see how to resolve this issue. Won't I always need to provide a lifetime for the callback stored in Client, which will end up bleeding through my code in the form of 'cb lifetime annotations?
More generally, I'm interested in understanding what is the canonical way of solving this callback/hook problem in Rust. I'm open to designs different from the one I have proposed, and if there are relevant performance concerns for various options, that would be useful to know also.
What you've created is a self-referential structure, which is problematic and not really expressible with references and lifetime annotations. See: Why can't I store a value and a reference to that value in the same struct? for the potential problems and workarounds. Its an issue here because you want to be able to mutate the BusinessLogic in the callback, but since it holds the Client, you can mutate the callback while its running, which is no good.
I would instead suggest that the callback has full ownership of the BusinessLogic which does not directly reference the Client:
use rand::Rng;
struct Client {
callback: Box<dyn FnMut(i64)>,
}
impl Client {
fn do_thing(&mut self) {
let value = rand::thread_rng().gen();
(self.callback)(value);
}
}
struct BusinessLogic {
value: Option<i64>,
}
fn main() {
let mut bl = BusinessLogic {
value: None
};
let mut client = Client {
callback: Box::new(move |value| {
bl.value = Some(value);
})
};
client.do_thing();
println!("Hello, world!");
}
if you need the subscriber to have backwards communication to the Client, you can pass an additional parameter that the callback can mutate, or simply do it via return value
if you need more complicated communication from the Client to the callback, either send a Message enum as the argument, or make the callback a custom trait instead of just FnMut with additional methods
if you need a single BusinessLogic to operate from multiple Clients use Arc+Mutex to allow shared ownership
I would like to return binary data in chunks of specific size. Here is a minimal example.
I made a wrapper struct for hyper::Response to hold my data like status, status text, headers and the resource to return:
pub struct Response<'a> {
pub resource: Option<&'a Resource>
}
This struct has a build method that creates the hyper::Response:
impl<'a> Response<'a> {
pub fn build(&mut self) -> Result<hyper::Response<hyper::Body>, hyper::http::Error> {
let mut response = hyper::Response::builder();
match self.resource {
Some(r) => {
let chunks = r.data
.chunks(100)
.map(Result::<_, std::convert::Infallible>::Ok);
response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
},
None => response.body(hyper::Body::from("")),
}
}
}
There is also another struct holding the database content:
pub struct Resource {
pub data: Vec<u8>
}
Everything works until I try to create a chunked response. The Rust compiler gives me the following error:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:14:15
|
14 | match self.resource {
| ^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime `'a` as defined on the impl at 11:6...
--> src/main.rs:11:6
|
11 | impl<'a> Response<'a> {
| ^^
note: ...so that the types are compatible
--> src/main.rs:14:15
|
14 | match self.resource {
| ^^^^^^^^^^^^^
= note: expected `Option<&Resource>`
found `Option<&'a Resource>`
= note: but, the lifetime must be valid for the static lifetime...
note: ...so that the types are compatible
--> src/main.rs:19:31
|
19 | response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
| ^^^^^^^^^^^^^^^^^^^^^^^^
= note: expected `From<&[u8]>`
found `From<&'static [u8]>`
I don't know how to fulfill these lifetime requirements. How can I do this correctly?
The problem is not in the 'a itself, but in the fact that the std::slice::chunks() function returns an iterator that borrows the original slice. You are trying to create a stream future from this Chunks<'_, u8> value, but the stream requires it to be 'static. Even if your Resource did not have the 'a lifetime, you would still have the r.data borrowed, and it would still fail.
Remember that here 'static does not mean that the value lives forever, but that it can be made to live as long as necessary. That is, the future must not hold any (non-'static) borrows.
You could clone all the data, but if it is very big, it can be costly. If so, you could try using Bytes, that is just like Vec<u8> but reference counted.
It looks like there is no Bytes::chunks() function that returns an iterator of Bytes. Fortunately it is easy to do it by hand.
Lastly, remember that iterators in Rust are lazy, so they keep the original data borrowed, even if it is a Bytes. So we need to collect them into a Vec to actually own the data (playground):
pub struct Resource {
pub data: Bytes,
}
impl<'a> Response<'a> {
pub fn build(&mut self) -> Result<hyper::Response<hyper::Body>, hyper::http::Error> {
let mut response = hyper::Response::builder();
match self.resource {
Some(r) => {
let len = r.data.len();
let chunks = (0..len)
.step_by(100)
.map(|x| {
let range = x..len.min(x + 100);
Ok(r.data.slice(range))
})
.collect::<Vec<Result<Bytes, std::convert::Infallible>>>();
response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
}
None => response.body(hyper::Body::from("")),
}
}
}
UPDATE: We can avoid the call to collect() if we notice that stream::iter() takes ownership of an IntoIterator that can be evaluated lazily, as long as we make it 'static. It can be done if we do a (cheap) clone of r.data and move it into the lambda (playground):
let data = r.data.clone();
let len = data.len();
let chunks = (0..len).step_by(100)
.map(move |x| {
let range = x .. len.min(x + 100);
Result::<_, std::convert::Infallible>::Ok(data.slice(range))
});
response.body(hyper::Body::wrap_stream(stream::iter(chunks)))
This question already has answers here:
Joining a thread in a method that takes `&mut self` (like drop) results in "cannot move out of borrowed content"
(4 answers)
Closed 1 year ago.
I have some multi-threaded code that's giving me trouble. This is as simple as I can reproduce it:
use std::thread;
use std::time;
use std::sync::{Arc, atomic::{Ordering, AtomicBool}};
use std::ops::Drop;
struct Container {
// Join Handle for a thread
th: Option<thread::JoinHandle<()>>,
// Gets set to true when we want the thread to exit
stop_thread: Arc<AtomicBool>,
}
impl Container {
fn new() -> Self {
// Create new instance
let mut inst = Self {
th: None,
stop_thread: Arc::new(AtomicBool::new(false)),
};
let stop_thread = inst.stop_thread.clone();
// Start a new thread that does some work
let t = thread::spawn(move || {
// Keep doing work until stop_thread gets set to true
while !stop_thread.load(Ordering::SeqCst) {
println!("Doing stuff...");
thread::sleep(time::Duration::from_secs(1));
}
println!("Thread exited");
});
inst.th = Some(t);
inst
}
}
impl Drop for Container {
fn drop(&mut self) {
self.stop_thread.store(true, Ordering::SeqCst);
if let Some(t) = self.th {
t.join().unwrap();
}
}
}
fn main() {
let c = Container::new();
thread::sleep(time::Duration::from_secs(3));
drop(c);
}
The idea is that when a new instance of the Container struct is created, a background thread is started that does something. It keeps running until the instance is destroyed, at which point, I need the thread to be notified that it needs to exit. I also need to actually wait for the thread to exit before proceeding.
Everything works great, except for the code in the drop function. Rust is unhappy with if let Some(t) = self.th. It says:
error[E0507]: cannot move out of `self.th.0` which is behind a mutable reference
--> src/main.rs:45:26
|
45 | if let Some(t) = self.th {
| - ^^^^^^^ help: consider borrowing here: `&self.th`
| |
| data moved here
| move occurs because `t` has type `JoinHandle<()>`, which does not implement the `Copy` trait
Why can't I do this? What is self.th.0?
When I try to take Rust's suggestion, and do if let Some(t) = &self.th instead, it still doesn't compile:
error[E0507]: cannot move out of `*t` which is behind a shared reference
--> src/main.rs:46:13
|
46 | t.join().unwrap();
| ^ move occurs because `*t` has type `JoinHandle<()>`, which does not implement the `Copy` trait
What am I doing wrong?
As specified in this answer (linked by Rabbid76), this can be worked around by using the .take() function:
impl Drop for Container {
fn drop(&mut self) {
self.stop_thread.store(true, Ordering::SeqCst);
if let Some(t) = self.th.take() {
t.join().unwrap();
}
}
}
Though you might wanna consider if waiting on another thread inside the drop implementation is a good idea as explained here
I'm trying to implement a console system for the game I'm writing and have found a fairly simple system: I define a Console object that stores commands as boxed closures (specifically Box<FnMut + 'a> for some 'a). This works for any component of the engine so long as the Console is created before anything else.
Unfortunately, this prevents me from adding commands that modify the Console itself, which means I can't create commands that simply print text or define other variables or commands. I've written a small example that replicates the error:
use std::cell::Cell;
struct Console<'a> {
cmds: Vec<Box<FnMut() + 'a>>,
}
impl<'a> Console<'a> {
pub fn println<S>(&self, msg: S)
where S: AsRef<str>
{
println!("{}", msg.as_ref());
}
pub fn add_cmd(&mut self, cmd: Box<FnMut() + 'a>) {
self.cmds.push(cmd);
}
}
struct Example {
val: Cell<i32>,
}
fn main() {
let ex = Example {
val: Cell::new(0),
};
let mut con = Console {
cmds: Vec::new(),
};
// this works
con.add_cmd(Box::new(|| ex.val.set(5)));
(con.cmds[0])();
// this doesn't
let cmd = Box::new(|| con.println("Hello, world!"));
con.add_cmd(cmd);
(con.cmds[1])();
}
And the error:
error: `con` does not live long enough
--> console.rs:34:31
|
34 | let cmd = Box::new(|| con.println("Hello, world!"));
| -- ^^^ does not live long enough
| |
| capture occurs here
35 | con.add_cmd(cmd);
36 | }
| - borrowed value dropped before borrower
|
= note: values in a scope are dropped in the opposite order they are created
error: aborting due to previous error
Is there a workaround for this, or a better system I should look into? This is on rustc 1.18.0-nightly (53f4bc311 2017-04-07).
This is one of those fairly tricky resource borrowing conundrums that the compiler could not allow. Basically, we have a Console that owns multiple closures, which in turn capture an immutable reference to the same console. This means two constraints:
Since Console owns the closures, they will live for as long as the console itself, and the inner vector will drop them right after Console is dropped.
At the same time, each closure must not outlive Console, because otherwise we would end up with dangling references to the console.
It may seem harmless from the fact that the console and respective closures go out of scope at once. However, the drop method follows a strict order here: first the console, then the closures.
Not to mention of course, that if you wish for closures to freely apply modifications to the console without interior mutability, you would have to mutably borrow it, which cannot be done over multiple closures.
An approach to solving the problem is to separate the two: let the console not own the closures, instead having them in a separate registry, and let the closures only borrow the console when calling the closure.
This can be done by passing the console as an argument to the closures and moving the closure vector to another object (Playground):
use std::cell::Cell;
struct CommandRegistry<'a> {
cmds: Vec<Box<Fn(&mut Console) + 'a>>,
}
impl<'a> CommandRegistry<'a> {
pub fn add_cmd(&mut self, cmd: Box<Fn(&mut Console) + 'a>) {
self.cmds.push(cmd);
}
}
struct Console {
}
impl Console {
pub fn println<S>(&mut self, msg: S)
where S: AsRef<str>
{
println!("{}", msg.as_ref());
}
}
struct Example {
val: Cell<i32>,
}
fn main() {
let ex = Example {
val: Cell::new(0),
};
let mut reg = CommandRegistry{ cmds: Vec::new() };
let mut con = Console {};
// this works
reg.add_cmd(Box::new(|_: &mut Console| ex.val.set(5)));
(reg.cmds[0])(&mut con);
// and so does this now!
let cmd = Box::new(|c: &mut Console| c.println("Hello, world!"));
reg.add_cmd(cmd);
(reg.cmds[1])(&mut con);
}
I also took the liberty of making closures accept a mutable reference. No conflicts emerge here because we are no longer borrowing the console that was already borrowed when fetching the borrowing closure. This way, the closures can also outlive the console.
I'm learning Rust and I'm trying to cargo-cult this code into compiling:
use std::vec::Vec;
use std::collections::BTreeMap;
struct Occ {
docnum: u64,
weight: f32,
}
struct PostWriter<'a> {
bytes: Vec<u8>,
occurrences: BTreeMap<&'a [u8], Vec<Occ>>,
}
impl<'a> PostWriter<'a> {
fn new() -> PostWriter<'a> {
PostWriter {
bytes: Vec::new(),
occurrences: BTreeMap::new(),
}
}
fn add_occurrence(&'a mut self, term: &[u8], occ: Occ) {
let occurrences = &mut self.occurrences;
match occurrences.get_mut(term) {
Some(x) => x.push(occ),
None => {
// Add the term bytes to the big vector of all terms
let termstart = self.bytes.len();
self.bytes.extend(term);
// Create a new occurrences vector
let occs = vec![occ];
// Take the appended term as a slice to use as a key
// ERROR: cannot borrow `*occurrences` as mutable more than once at a time
occurrences.insert(&self.bytes[termstart..], occs);
}
}
}
}
fn main() {}
I get an error:
error[E0499]: cannot borrow `*occurrences` as mutable more than once at a time
--> src/main.rs:34:17
|
24 | match occurrences.get_mut(term) {
| ----------- first mutable borrow occurs here
...
34 | occurrences.insert(&self.bytes[termstart..], occs);
| ^^^^^^^^^^^ second mutable borrow occurs here
35 | }
36 | }
| - first borrow ends here
I don't understand... I'm just calling a method on a mutable reference, why would that line involve borrowing?
I'm just calling a method on a mutable reference, why would that line involve borrowing?
When you call a method on an object that's going to mutate the object, you can't have any other references to that object outstanding. If you did, your mutation could invalidate those references and leave your program in an inconsistent state. For example, say that you had gotten a value out of your hashmap and then added a new value. Adding the new value hits a magic limit and forces memory to be reallocated, your value now points off to nowhere! When you use that value... bang goes the program!
In this case, it looks like you want to do the relatively common "append or insert if missing" operation. You will want to use entry for that:
use std::collections::BTreeMap;
fn main() {
let mut map = BTreeMap::new();
{
let nicknames = map.entry("joe").or_insert(Vec::new());
nicknames.push("shmoe");
// Using scoping to indicate that we are done with borrowing `nicknames`
// If we didn't, then we couldn't borrow map as
// immutable because we could still change it via `nicknames`
}
println!("{:?}", map)
}
Because you're calling a method that borrows as mutable
I had a similar question yesterday about Hash, until I noticed something in the docs. The docs for BTreeMap show a method signature for insert starting with fn insert(&mut self..
So when you call .insert, you're implicitly asking that function to borrow the BTreeMap as mutable.