I'm writing a game engine. In the engine, I've got a game state which contains the list of entities in the game.
I want to provide a function on my gamestate update which will in turn tell each entity to update. Each entity needs to be able to refer to the gamestate in order to correctly update itself.
Here's a simplified version of what I have so far.
pub struct GameState {
pub entities: Vec<Entity>,
}
impl GameState {
pub fn update(&mut self) {
for mut t in self.entities.iter_mut() {
t.update(self);
}
}
}
pub struct Entity {
pub value: i64,
}
impl Entity {
pub fn update(&mut self, container: &GameState) {
self.value += container.entities.len() as i64;
}
}
fn main() {
let mut c = GameState { entities: vec![] };
c.entities.push(Entity { value: 1 });
c.entities.push(Entity { value: 2 });
c.entities.push(Entity { value: 3 });
c.update();
}
The problem is the borrow checker doesn't like me passing the gamestate to the entity:
error[E0502]: cannot borrow `*self` as immutable because `self.entities` is also borrowed as mutable
--> example.rs:8:22
|
7 | for mut t in self.entities.iter_mut() {
| ------------- mutable borrow occurs here
8 | t.update(self);
| ^^^^ immutable borrow occurs here
9 | }
| - mutable borrow ends here
error: aborting due to previous error
Can anyone give me some suggestions on better ways to design this that fits with Rust better?
Thanks!
First, let's answer the question you didn't ask: Why is this not allowed?
The answer lies around the guarantees that Rust makes about & and &mut pointers. A & pointer is guaranteed to point to an immutable object, i.e. it's impossible for the objects behind the pointer to mutate while you can use that pointer. A &mut pointer is guaranteed to be the only active pointer to an object, i.e. you can be sure that nobody is going to observe or mutate the object while you're mutating it.
Now, let's look at the signature of Entity::update:
impl Entity {
pub fn update(&mut self, container: &GameState) {
// ...
}
}
This method takes two parameters: a &mut Entity and a &GameState. But hold on, we can get another reference to self through the &GameState! For example, suppose that self is the first entity. If we do this:
impl Entity {
pub fn update(&mut self, container: &GameState) {
let self_again = &container.entities[0];
// ...
}
}
then self and self_again alias each other (i.e. they refer to the same thing), which is not allowed as per the rules I mentioned above because one of the pointers is a mutable pointer.
What can you do about this?
One option is to remove an entity from the entities vector before calling update on it, then inserting it back after the call. This solves the aliasing problem because we can't get another alias to the entity from the game state. However, removing the entity from the vector and reinserting it are operations with linear complexity (the vector needs to shift all the following items), and if you do it for each entity, then the main update loop runs in quadratic complexity. You can work around that by using a different data structure; this can be as simple as a Vec<Option<Entity>>, where you simply take the Entity from each Option, though you might want to wrap this into a type that hides all None values to external code. A nice consequence is that when an entity has to interact with other entities, it will automatically skip itself when iterating on the entities vector, since it's no longer there!
A variation on the above is to simply take ownership of the whole vector of entities and temporarily replace the game state's vector of entities with an empty one.
impl GameState {
pub fn update(&mut self) {
let mut entities = std::mem::replace(&mut self.entities, vec![]);
for mut t in entities.iter_mut() {
t.update(self);
}
self.entities = entities;
}
}
This has one major downside: Entity::update will not be able to interact with the other entities.
Another option is to wrap each entity in a RefCell.
use std::cell::RefCell;
pub struct GameState {
pub entities: Vec<RefCell<Entity>>,
}
impl GameState {
pub fn update(&mut self) {
for t in self.entities.iter() {
t.borrow_mut().update(self);
}
}
}
By using RefCell, we can avoid retaining a mutable borrow on self. Here, we can use iter instead of iter_mut to iterate on entities. In return, we now need to call borrow_mut to obtain a mutable pointer to the value wrapped in the RefCell.
RefCell essentially performs borrow checking at runtime. This means that you can end up writing code that compiles fine but panics at runtime. For example, if we write Entity::update like this:
impl Entity {
pub fn update(&mut self, container: &GameState) {
for entity in container.entities.iter() {
self.value += entity.borrow().value;
}
}
}
the program will panic:
thread 'main' panicked at 'already mutably borrowed: BorrowError', ../src/libcore/result.rs:788
That's because we end up calling borrow on the entity that we're currently updating, which is still borrowed by the borrow_mut call done in GameState::update. Entity::update doesn't have enough information to know which entity is self, so you would have to use try_borrow or borrow_state (which are both unstable as of Rust 1.12.1) or pass additional data to Entity::update to avoid panics with this approach.
Related
Running into an ownership issue when attempting to reference multiple values from a HashMap in a struct as parameters in a function call. Here is a PoC of the issue.
use std::collections::HashMap;
struct Resource {
map: HashMap<String, String>,
}
impl Resource {
pub fn new() -> Self {
Resource {
map: HashMap::new(),
}
}
pub fn load(&mut self, key: String) -> &mut String {
self.map.get_mut(&key).unwrap()
}
}
fn main() {
// Initialize struct containing a HashMap.
let mut res = Resource {
map: HashMap::new(),
};
res.map.insert("Item1".to_string(), "Value1".to_string());
res.map.insert("Item2".to_string(), "Value2".to_string());
// This compiles and runs.
let mut value1 = res.load("Item1".to_string());
single_parameter(value1);
let mut value2 = res.load("Item2".to_string());
single_parameter(value2);
// This has ownership issues.
// multi_parameter(value1, value2);
}
fn single_parameter(value: &String) {
println!("{}", *value);
}
fn multi_parameter(value1: &mut String, value2: &mut String) {
println!("{}", *value1);
println!("{}", *value2);
}
Uncommenting multi_parameter results in the following error:
28 | let mut value1 = res.load("Item1".to_string());
| --- first mutable borrow occurs here
29 | single_parameter(value1);
30 | let mut value2 = res.load("Item2".to_string());
| ^^^ second mutable borrow occurs here
...
34 | multi_parameter(value1, value2);
| ------ first borrow later used here
It would technically be possible for me to break up the function calls (using the single_parameter function approach), but it would be more convenient to pass the
variables to a single function call.
For additional context, the actual program where I'm encountering this issue is an SDL2 game where I'm attempting to pass multiple textures into a single function call to be drawn, where the texture data may be modified within the function.
This is currently not possible, without resorting to unsafe code or interior mutability at least. There is no way for the compiler to know if two calls to load will yield mutable references to different data as it cannot always infer the value of the key. In theory, mutably borrowing both res.map["Item1"] and res.map["Item2"] would be fine as they would refer to different values in the map, but there is no way for the compiler to know this at compile time.
The easiest way to do this, as already mentioned, is to use a structure that allows interior mutability, like RefCell, which typically enforces the memory safety rules at run-time before returning a borrow of the wrapped value. You can also work around the borrow checker in this case by dealing with mut pointers in unsafe code:
pub fn load_many<'a, const N: usize>(&'a mut self, keys: [&str; N]) -> [&'a mut String; N] {
// TODO: Assert that keys are distinct, so that we don't return
// multiple references to the same value
keys.map(|key| self.load(key) as *mut _)
.map(|ptr| unsafe { &mut *ptr })
}
Rust Playground
The TODO is important, as this assertion is the only way to ensure that the safety invariant of only having one mutable reference to any value at any time is upheld.
It is, however, almost always better (and easier) to use a known safe interior mutation abstraction like RefCell rather than writing your own unsafe code.
I have to iterate on keys, find the value in HashMap by key, possibly do some heavy computation in the found struct as a value (lazy => mutate the struct) and cached return it in Rust.
I'm getting the following error message:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:25:26
|
23 | fn it(&mut self) -> Option<&Box<Calculation>> {
| - let's call the lifetime of this reference `'1`
24 | for key in vec!["1","2","3"] {
25 | let result = self.find(&key.to_owned());
| ^^^^ `*self` was mutably borrowed here in the previous iteration of the loop
...
28 | return result
| ------ returning this value requires that `*self` is borrowed for `'1`
Here is the code in playground.
use std::collections::HashMap;
struct Calculation {
value: Option<i32>
}
struct Struct {
items: HashMap<String, Box<Calculation>> // cache
}
impl Struct {
fn find(&mut self, key: &String) -> Option<&Box<Calculation>> {
None // find, create, and/or calculate items
}
fn it(&mut self) -> Option<&Box<Calculation>> {
for key in vec!["1","2","3"] {
let result = self.find(&key.to_owned());
if result.is_some() {
return result
}
}
None
}
}
I can't avoid the loop as I have to check multiple keys
I have to make it mutable (self and the structure) as the possible calculation changes it
Any suggestion on how to change the design (as Rust forces to think in a bit different way that makes sense) or work around it?
PS. There are some other issues with the code, but let's split the problems and solve this one first.
You can't do caching with exclusive access. You can't treat Rust references like general-purpose pointers (BTW: &String and &Box<T> are double indirection, and very unidiomatic in Rust. Use &str or &T for temporary borrows).
&mut self means not just mutable, but exclusive and mutable, so your cache supports returning only one item, because the reference it returns has to keep self "locked" for as long as it exists.
You need to convince the borrow checker that the thing that find returns won't suddenly disappear next time you call it. Currently there's no such guarantee, because the interface doesn't stop you from calling e.g. items.clear() (borrow checker checks what function's interface allows, not what function actually does).
You can do that either by using Rc, or using a crate that implements a memory pool/arena.
struct Struct {
items: HashMap<String, Rc<Calculation>>,
}
fn find(&mut self, key: &str) -> Rc<Calculation>
This way if you clone the Rc, it will live for as long as it needs, independently of the cache.
You can also make it nicer with interior mutability.
struct Struct {
items: RefCell<HashMap<…
}
This will allow your memoizing find method to use a shared borrow instead of an exclusive one:
fn find(&self, key: &str) -> …
which is much easier to work with for the callers of the method.
Probably not the cleanest way to do that, but it compiles. The idea is not to store the value found in a temporary result, to avoid aliasing: if you store the result, self is kept borrowed.
impl Struct {
fn find(&mut self, key: &String) -> Option<&Box<Calculation>> {
None
}
fn it(&mut self) -> Option<&Box<Calculation>> {
for key in vec!["1","2","3"] {
if self.find(&key.to_owned()).is_some() {
return self.find(&key.to_owned());
}
}
None
}
}
I had similar issues, and I found a workaround by turning the for loop into a fold, which convinced the compiler that self was not mutably borrowed twice.
It works without using interior mutability or duplicated function call; the only downside is that if the result was found early, it will not short-circuit but continue iterating until the end.
Before:
for key in vec!["1","2","3"] {
let result = self.find(&key.to_owned());
if result.is_some() {
return result
}
}
After:
vec!["1", "2,", "3"]
.iter()
.fold(None, |result, key| match result {
Some(result) => Some(result),
None => self.find(&key.to_string())
})
Working playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=92bc73e4bac556ce163e0790c7d3f154
I have a struct and I want to call one of the struct's methods every time a mutable borrow to it has ended. To do so, I would need to know when the mutable borrow to it has been dropped. How can this be done?
Disclaimer: The answer that follows describes a possible solution, but it's not a very good one, as described by this comment from Sebastien Redl:
[T]his is a bad way of trying to maintain invariants. Mostly because dropping the reference can be suppressed with mem::forget. This is fine for RefCell, where if you don't drop the ref, you will simply eventually panic because you didn't release the dynamic borrow, but it is bad if violating the "fraction is in shortest form" invariant leads to weird results or subtle performance issues down the line, and it is catastrophic if you need to maintain the "thread doesn't outlive variables in the current scope" invariant.
Nevertheless, it's possible to use a temporary struct as a "staging area" that updates the referent when it's dropped, and thus maintain the invariant correctly; however, that version basically amounts to making a proper wrapper type and a kind of weird way to use it. The best way to solve this problem is through an opaque wrapper struct that doesn't expose its internals except through methods that definitely maintain the invariant.
Without further ado, the original answer:
Not exactly... but pretty close. We can use RefCell<T> as a model for how this can be done. It's a bit of an abstract question, but I'll use a concrete example to demonstrate. (This won't be a complete example, but something to show the general principles.)
Let's say you want to make a Fraction struct that is always in simplest form (fully reduced, e.g. 3/5 instead of 6/10). You write a struct RawFraction that will contain the bare data. RawFraction instances are not always in simplest form, but they have a method fn reduce(&mut self) that reduces them.
Now you need a smart pointer type that you will always use to mutate the RawFraction, which calls .reduce() on the pointed-to struct when it's dropped. Let's call it RefMut, because that's the naming scheme RefCell uses. You implement Deref<Target = RawFraction>, DerefMut, and Drop on it, something like this:
pub struct RefMut<'a>(&'a mut RawFraction);
impl<'a> Deref for RefMut<'a> {
type Target = RawFraction;
fn deref(&self) -> &RawFraction {
self.0
}
}
impl<'a> DerefMut for RefMut<'a> {
fn deref_mut(&mut self) -> &mut RawFraction {
self.0
}
}
impl<'a> Drop for RefMut<'a> {
fn drop(&mut self) {
self.0.reduce();
}
}
Now, whenever you have a RefMut to a RawFraction and drop it, you know the RawFraction will be in simplest form afterwards. All you need to do at this point is ensure that RefMut is the only way to get &mut access to the RawFraction part of a Fraction.
pub struct Fraction(RawFraction);
impl Fraction {
pub fn new(numerator: i32, denominator: i32) -> Self {
// create a RawFraction, reduce it and wrap it up
}
pub fn borrow_mut(&mut self) -> RefMut {
RefMut(&mut self.0)
}
}
Pay attention to the pub markings (and lack thereof): I'm using those to ensure the soundness of the exposed interface. All three types should be placed in a module by themselves. It would be incorrect to mark the RawFraction field pub inside Fraction, since then it would be possible (for code outside the module) to create an unreduced Fraction without using new or get a &mut RawFraction without going through RefMut.
Supposing all this code is placed in a module named frac, you can use it something like this (assuming Fraction implements Display):
let f = frac::Fraction::new(3, 10);
println!("{}", f); // prints 3/10
f.borrow_mut().numerator += 3;
println!("{}", f); // prints 3/5
The types encode the invariant: Wherever you have Fraction, you can know that it's fully reduced. When you have a RawFraction, &RawFraction, etc., you can't be sure. If you want, you may also make RawFraction's fields non-pub, so that you can't get an unreduced fraction at all except by calling borrow_mut on a Fraction.
Basically the same thing is done in RefCell. There you want to reduce the runtime borrow-count when a borrow ends. Here you want to perform an arbitrary action.
So let's re-use the concept of writing a function that returns a wrapped reference:
struct Data {
content: i32,
}
impl Data {
fn borrow_mut(&mut self) -> DataRef {
println!("borrowing");
DataRef { data: self }
}
fn check_after_borrow(&self) {
if self.content > 50 {
println!("Hey, content should be <= {:?}!", 50);
}
}
}
struct DataRef<'a> {
data: &'a mut Data
}
impl<'a> Drop for DataRef<'a> {
fn drop(&mut self) {
println!("borrow ends");
self.data.check_after_borrow()
}
}
fn main() {
let mut d = Data { content: 42 };
println!("content is {}", d.content);
{
let b = d.borrow_mut();
//let c = &d; // Compiler won't let you have another borrow at the same time
b.data.content = 123;
println!("content set to {}", b.data.content);
} // borrow ends here
println!("content is now {}", d.content);
}
This results in the following output:
content is 42
borrowing
content set to 123
borrow ends
Hey, content should be <= 50!
content is now 123
Be aware that you can still obtain an unchecked mutable borrow with e.g. let c = &mut d;. This will be silently dropped without calling check_after_borrow.
I am trying to implement a system that would use borrow checking/lifetimes in order to provide safe custom indices on a collection. Consider the following code:
struct Graph(i32);
struct Edge<'a>(&'a Graph, i32);
impl Graph {
pub fn get_edge(&self) -> Edge {
Edge(&self, 0)
}
pub fn split(&mut self, Edge(_, edge_id): Edge) {
self.0 = self.0 + edge_id;
}
pub fn join(&mut self, Edge(_, edge0_id): Edge, Edge(_, edge1_id): Edge) {
self.0 = self.0 + edge0_id + edge1_id;
}
}
fn main() {
let mut graph = Graph(0);
let edge = graph.get_edge();
graph.split(edge)
}
References to the graph borrowed by the Edge struct should be dropped when methods such as split or join are called. This would fulfill the API invariant that all edge indices must be destroyed when the graph is mutated. However, the compiler doesn't get it. It fails with messages like
error[E0502]: cannot borrow `graph` as mutable because it is also borrowed as immutable
--> src/main.rs:23:5
|
22 | let edge = graph.get_edge();
| ----- immutable borrow occurs here
23 | graph.split(edge)
| ^^^^^ mutable borrow occurs here
24 | }
| - immutable borrow ends here
If I understand this correctly, the compiler fails to realise that the borrowing of the graph that happened in the edge struct is actually being released when the function is called. Is there a way to teach the compiler what I am trying to do here?
Bonus question: is there a way to do exactly the same but without actually borrowing the graph in the Edge struct? The edge struct is only used as a temporary for the purpose of traversal and will never be part of an external object state (I have 'weak' versions of the edge for that).
Addendum: After some digging around, it seems to be really far from trivial. First of all, Edge(_, edge_id) does not actually destructure the Edge, because _ does not get bound at all (yes, i32 is Copy which makes things even more complicated, but this is easily remedied by wrapping it into a non-Copy struct). Second, even if I completely destructure Edge (i.e. by doing it in a separate scope), the reference to the graph is still there, even though it should have been moved (this must be a bug). It only works if I perform the destructuring in a separate function. Now, I have an idea how to circumvent it (by having a separate object that describes a state change and destructures the indices as they are supplied), but this becomes very awkward very quickly.
You have a second problem that you didn’t mention: how does split know that the user didn’t pass an Edge from a different Graph? Fortunately, it’s possible to solve both problems with higher-rank trait bounds!
First, let’s have Edge carry a PhantomData marker instead of a real reference to the graph:
pub struct Edge<'a>(PhantomData<&'a mut &'a ()>, i32);
Second, let’s move all the Graph operations into a new GraphView object that gets consumed by operations that should invalidate the identifiers:
pub struct GraphView<'a> {
graph: &'a mut Graph,
marker: PhantomData<&'a mut &'a ()>,
}
impl<'a> GraphView<'a> {
pub fn get_edge(&self) -> Edge<'a> {
Edge(PhantomData, 0)
}
pub fn split(self, Edge(_, edge_id): Edge) {
self.graph.0 = self.graph.0 + edge_id;
}
pub fn join(self, Edge(_, edge0_id): Edge, Edge(_, edge1_id): Edge) {
self.graph.0 = self.graph.0 + edge0_id + edge1_id;
}
}
Now all we have to do is guard the construction of GraphView objects such that there’s never more than one with a given lifetime parameter 'a.
We can do this by (1) forcing GraphView<'a> to be invariant over 'a with a PhantomData member as above, and (2) only ever providing a constructed GraphView to a closure with a higher-rank trait bound that creates a fresh lifetime each time:
impl Graph {
pub fn with_view<Ret>(&mut self, f: impl for<'a> FnOnce(GraphView<'a>) -> Ret) -> Ret {
f(GraphView {
graph: self,
marker: PhantomData,
})
}
}
fn main() {
let mut graph = Graph(0);
graph.with_view(|view| {
let edge = view.get_edge();
view.split(edge);
});
}
Full demo on Rust Playground.
This isn’t totally ideal, since the caller may have to go through contortions to put all its operations inside the closure. But I think it’s the best we can do in the current Rust language, and it does allow us to enforce a huge class of compile-time guarantees that almost no other language can express at all. I’d love to see more ergonomic support for this pattern added to the language somehow—perhaps a way to create a fresh lifetime via a return value rather than a closure parameter (pub fn view(&mut self) -> exists<'a> GraphView<'a>)?
I'm trying to put in one structure information about inotify events and hashmap with inotify watch id as key and name of the file as value.
extern crate inotify;
use inotify::INotify;
use std::sync::{Arc, Mutex};
use std::collections::HashMap;
struct Notificator {
inotify: INotify,
watch_service: Arc<Mutex<HashMap<inotify::wrapper::Watch, Arc<String>>>>,
}
impl Notificator {
pub fn new() -> Notificator {
Notificator {
inotify: INotify::init().unwrap(),
watch_service: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn resolving_events(&mut self) {
{
let mut events = self.check_for_events();
self.events_execution(events);
}
}
fn check_for_events(&mut self) -> &[inotify::wrapper::Event] {
self.inotify.available_events().unwrap()
}
fn events_execution(&self, events: &[inotify::wrapper::Event]) {
for event in events.iter() {
}
}
}
During compilation I am receiving an error
src/main.rs:204:13: 204:17 error: cannot borrow `*self` as immutable because it is also borrowed as mutable [E0502]
src/main.rs:204 self.events_execution(events);
I thought the best solution would be to separate somehow inotify variable in Notificator structure with watch_service, but I can't dereference self.check_for_events(); because I receive
src/main.rs:203:17: 203:27 error: the trait bound `[inotify::wrapper::Event]: std::marker::Sized` is not satisfied [E0277]
src/main.rs:203 let mut events = *self.check_for_events();
I understand the core of the problem: I'm trying to borrow reference by check_for_events and then using it as parameter in events_execution which also requires self as parameter, but I have no idea how to resolve it.
One thing you could do, although it's not very elegant, is to have your mutable method consume its mutable borrow and return an immutable one that you can then use:
pub fn resolving_events(&mut self) {
let (slf, events) = self.check_for_events();
slf.events_execution(events);
}
fn check_for_events(&mut self) -> (&Self, &[inotify::wrapper::Event]) {
let events = self.inotify.available_events().unwrap();
(&self, events)
}
I've made a small proof-of-concept on the playground (using vecs of u64 as the mutable state, but the principle is similar). It might be cleaner to refactor your code so that some external client can (mutably) borrow the Notifier, produce the events, release the borrow, and borrow it (immutably) to process them...
This is a known issue in the borrow checker (more discussion on internals). You cannot have a function taking a &mut T to an object and returning a &T without losing the ability to access the object again until the &T goes out of scope. You can't work around it due to the way inotify is implemented.
But you can ask the inotify authors to create a get_available_notifications method that doesn't fetch new data. This way you can call available_notifications once and drop the returned value. Then call get_available_notifications (which doesn't take &mut INotify, but just &INotify) and work from there.