This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Closed 3 months ago.
I'm trying to write a renderer for my abstract node system.
Each node is on a circuit, which looks like:
struct Circuit<'a> {
nodes: Vec<Node>,
connections: Vec<Connection<'a>>,
}
Where each nodes contains some arbitrary data, and a connection connects those two nodes.
struct Connection<'a> {
from: &'a Node,
to: &'a Node,
}
In one instance, I'm trying to mutably iterate over Circuit.nodes and change some data, while still keeping the refences to Node inside Connections. I want the Connection references to still hold the reference to the same Node structs, but point to the updated data after the mutable borrow.
But with this setup, I get an cannot borrow 'circuit.nodes' as mutable because it is also borrowed as immutable error.
I have tried to use RefCell and a Mutex, but to no success. No resource I could find explained this problem well.
Maybe you want to wrap your Nodes in a Rc (or Arc if in multithreaded contexts),
that way connections and the circuit can share ownership over the nodes
use std::rc::Rc;
use std::cell::RefCell;
fn main() {
let a = Rc::new(RefCell::new(Node { data: "a".into() }));
let b = Rc::new(RefCell::new(Node { data: "b".into() }));
let conn = Connection {
from: a.clone(),
to: b.clone(),
};
let cir = Circuit {
nodes: vec![a.clone(), b.clone()],
connections: vec![conn],
};
}
#[derive(Debug)]
struct Node {
data: String,
}
#[derive(Debug)]
struct Circuit {
nodes: Vec<Rc<RefCell<Node>>>,
connections: Vec<Connection>,
}
#[derive(Debug)]
struct Connection {
from: Rc<RefCell<Node>>,
to: Rc<RefCell<Node>>,
}
Related
This question already has answers here:
How do I express mutually recursive data structures in safe Rust?
(4 answers)
Closed 1 year ago.
I have two structs that refer to each other. Once initialized, they are never mutated for the rest of the application's lifetime.
It's possible to wrap them in a Mutex or RwLock or something, but it'd be nice not to deal with those throughout the codebase, just to get things initialized.
Here's the example code (does not compile):
use std::sync::Arc;
struct First {
second: Option<Second>,
}
struct Second {
first: Option<Arc<First>>,
}
fn main() {
let first = Arc::new(First { second: None });
let mut second = Second { first: None };
second.first = Some(first.clone());
first.second = Some(second);
}
The problem:
error[E0594]: cannot assign to data in an `Arc`
--> src/main.rs:14:5
|
14 | first.second = Some(second);
| ^^^^^^^^^^^^ cannot assign
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Arc<First>`
It's clear what's happening here; it's not possible to assign to first, because it's inside an Arc, which does not allow interior mutability.
Changing the order of operations doesn't get around the problem.
Using Arc::get_mut() won't work, because the Arc is cloned for storage in second.first.
So, is it possible to create objects with this pattern without needing runtime locking?
You have two options, depending on if you need Sync or not.
With Sync you need some synchronization structure, a RwLock should do:
use std::borrow::BorrowMut;
use std::sync::RwLock;
use std::sync::Arc;
struct First {
second: Option<Second>,
}
struct Second {
first: Option<Arc<RwLock<First>>>,
}
fn main() {
let first = Arc::new(RwLock::new(First { second: None }));
let mut second = Second { first: None };
second.first = Some(first.clone());
let mut inner = first.write().unwrap();
inner.second = Some(second);
}
Playground
If you do not need Sync, use Rc instead of Arc and RefCell:
use std::rc::Rc;
use std::cell::RefCell;
struct First {
second: Option<Second>,
}
struct Second {
first: Option<Rc<RefCell<First>>>,
}
fn main() {
let first = Rc::new(RefCell::new(First { second: None }));
let mut second = Second { first: None };
second.first = Some(first.clone());
first.borrow_mut().second = Some(second);
}
Playground
This question already has answers here:
How can I borrow from a HashMap to read and write at the same time?
(2 answers)
Borrow two mutable values from the same HashMap
(2 answers)
How to get mutable references to two array elements at the same time?
(8 answers)
Closed 2 years ago.
The question says "a collection in a struct", but I'm uncertain whether the "in a struct" part is actually relevant. I may not understand the problem well enough to properly ask the question. In any case, I'm trying to do the following:
Create a graph. Each node in the graph has a collection of string identifiers indicating which other nodes it is connected to. Of note: This collection also includes nodes which it is implicitly connected to via other nodes, which it distinguishes by indicating their distance (implicit connections will have distance > 1). Here's the code:
use core::mem::MaybeUninit;
use core::ptr;
use std::collections::HashMap;
use std::sync::Once;
// In my actual code, this map is global for WebAssembly reasons.
// I'm leaving it global to avoid introducing new borrow-checker issues on top of my existing ones.
// Please roll with it. I realize there may be better implementations.
static mut NODE_MAP_UNSAFE: MaybeUninit<HashMap<String, Node>> = MaybeUninit::uninit();
static NODE_MAP_INIT: Once = Once::new();
// For readability
macro_rules! node_map { () => { unsafe { &(*NODE_MAP_UNSAFE.as_ptr()) } }; }
macro_rules! node_map_mut { () => { unsafe { &mut (*NODE_MAP_UNSAFE.as_mut_ptr()) } }; }
// Contains the node's identifier and its connections
struct Node {
name: String,
connections: HashMap<String, Connect>,
}
// Contains the identifier of the node we're connected to, as well as its distance from us
struct Connect {
name: String,
dist: i32,
}
fn main() {
// Again, please roll with it
NODE_MAP_INIT.call_once(|| unsafe { ptr::write(NODE_MAP_UNSAFE.as_mut_ptr(), HashMap::new()); });
// Create nodes 1-3 and add them to the map
let node1 = Node::new("1");
let node2 = Node::new("2");
let node3 = Node::new("3");
node_map_mut!().insert(String::from("1"), node1);
node_map_mut!().insert(String::from("2"), node2);
node_map_mut!().insert(String::from("3"), node3);
// Note: I will be skipping all Option::None checks for brevity's sake.
// The actual code would of course perform the checks properly.
// Connect node 1 to node 2, and vice versa
node_map_mut!().get_mut(&String::from("1")).unwrap().add_connection(String::from("2"));
node_map_mut!().get_mut(&String::from("2")).unwrap().add_connection(String::from("1"));
// Connect node 2 to node 3, and vice versa
node_map_mut!().get_mut(&String::from("2")).unwrap().add_connection(String::from("3"));
node_map_mut!().get_mut(&String::from("3")).unwrap().add_connection(String::from("2"));
// Implicitly connect node 1 to node 3, via node 2
node_map_mut!().get_mut(&String::from("1")).unwrap().update_connections(String::from("2"));
}
impl Node {
fn new(name: &str) -> Node {
return Node {
name: String::from(name),
connections: HashMap::new(),
}
}
// Add a connection
fn add_connection(&mut self, comp_to_add: String) {
// Create the new connection
let new_connection = Connect { name: comp_to_add.clone(), dist: 1 };
// Implicitly connect to distant nodes via our new connection
// No problem here, since the connection doesn't belong to self.connections yet
self.infer(&node_map!().get(&new_connection.name).unwrap().connections);
// Add the connection to self.connections
self.connections.insert(comp_to_add.clone(), new_connection);
}
// Check for new implicit connections and create them, if they exist.
fn update_connections(&mut self, comp_to_update_via: String) {
// Retrieve the existing connection from self.connections (thus borrowing it).
let existing_connection = &self.connections.get(&comp_to_update_via).unwrap().name;
// Implicitly connect to distant nodes via the existing connection
self.infer(&node_map!().get(existing_connection).unwrap().connections); // problem here
}
fn infer(&mut self, map: &HashMap<String, Connect>) {
for element in map {
// Ignore connections to us
if element.0 == &self.name { continue; }
// Add the implicit connection to self.connections (thus borrowing it).
// Actual code would instead update the connection if it already existed.
self.connections.insert(element.0.clone(), Connect { name: element.0.clone(), dist: element.1.dist + 1 });
}
}
}
This gives the following warning:
warning: cannot borrow `*self` as mutable because it is also borrowed as immutable
--> src/main.rs:76:9
|
73 | let existing_connection = self.connections.get(&comp_to_update_via).unwrap();
| ---------------- immutable borrow occurs here
...
76 | self.infer(&node_map!().get(&existing_connection.name).unwrap().connections); // problem here
| ^^^^ ------------------------- immutable borrow later used here
| |
| mutable borrow occurs here
|
= note: `#[warn(mutable_borrow_reservation_conflict)]` on by default
= warning: this borrowing pattern was not meant to be accepted, and may become a hard error in the future
= note: for more information, see issue #59159 <https://github.com/rust-lang/rust/issues/59159>
In my actual code, that's an error, but in this abbreviated version it's a warning for reasons I don't understand (maybe the error is being optimized out?). Either way, the problem is the same: self.connections is being borrowed to retrieve the name, and then self.connections is modified before the borrow ends. I could fix this in either of the following ways:
By cloning the String:
let existing_connection = self.connections.get(&comp_to_update_via).unwrap().name.clone();
self.infer(&node_map!().get(&existing_connection).unwrap().connections);
By doing the entire borrow inline:
self.infer(&node_map!().get(&self.connections.get(&comp_to_update_via).unwrap().name).unwrap().connections);
I dislike option 1 because throwing .clone()s around seems like a bad habit to be in. If the type in question were something more complex than a String, the performance cost could potentially be quite high.
I dislike option 2 because 1: it's ugly and hard to read, and 2: it wouldn't be possible (I don't think) if I were properly checking for Option::None. Also, I don't even understand why it works in the first place.
This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
How to return a reference to a sub-value of a value that is under a mutex?
(5 answers)
Returning a RWLockReadGuard independently from a method
(2 answers)
How can I store a Chars iterator in the same struct as the String it is iterating on?
(2 answers)
Closed 3 years ago.
I would like to wrap a low-level third-party API with my own struct and functions to make it more friendly. Unfortunately, the third-party API needs a reference to a socket in its constructor, which means I'd like my Struct to "own" the socket (someone has to own it so that it can be borrowed, right? and I'd like that hidden as an implementation detail in my API).
The third-party API looks something like this:
struct LowLevelApi<'a> {
stream: &'a mut TcpStream,
// ...
}
impl<'a> LowLevelApi<'a> {
pub fn new(socket: &'a mut TcpStream, ... ) -> LowLevelApi<'a> {
// ...
}
}
I would like to make the interface to my function look like:
pub fn new(host: String, port: u16, ...) -> HighLevelApi {
// ...
}
I tried this:
pub struct HighLevelApi<'a> {
stream: TcpStream,
low: LowLevelApi<'a>
}
impl <'a> HighLevelApi<'a> {
pub fn new(host: String, port: u16) -> HighLevelApi<'a> {
// Ignore lack of error checking for now
let mut stream = TcpStream::connect(format!("{}:{}", host, port)).unwrap();
HighLevelApi {
stream,
low: LowLevelApi::new(&mut stream)
}
}
}
Rust is (rightly) angry: It has no way of knowing that I'm not going to do something bad with the low field later. And even worse, I would need to somehow guarantee that when my structure gets dropped, low gets dropped first, and stream second (since by that point, any relationship between the two is lost - or rather, there never is/was a relationship between the two).
(actually, it's worse than that, because the stream local variable gets moved into the new struct, so the local can't possibly be borrowed by LowLevelApi, but I can't think of a way to initialize HighLevelApi with the stream from the struct, since there's no way to get a handle to that from within the struct's initialization, is there? But based on my guess about what would happen in the paragraph above, it doesn't really matter since it still wouldn't do what I wanted)
What are examples of the various techniques that can be used to store a wrap a third-party (not under my control) struct that needs a reference to something?
The Rental crate seems to do what is needed here, albeit with documentation and examples that leave a lot to the imagination (i.e. trial and error).
Here's roughly what solves this
rental! {
pub mod rentals {
#[rental_mut]
pub struct Wrapper {
stream: Box<TcpStream>,
low: LowLevelApi<'stream>
}
}
}
pub struct HighLevelApi {
wrapper: rentals::Wrapper
}
impl HighLevelApi {
pub fn new(host: String, port: u16) -> HighLevelApi {
Api {
// Ignore lack of error checking for now
wrapper: rentals::Wrapper::new(
Box::new(TcpStream::connect(format!("{}:{}", host, port)).unwrap()),
|s| LowLevelApi::new(s)
)
}
}
pub fn do_something(&mut self) {
self.wrapper.rent_mut(|ll| ll.do_something()) // ll is the LowLevelApi
}
}
I noticed two important things that made this work:
The lifetime name on low in the Wrapper struct must match the name of the "owning" field (in this case "'stream")
You never get direct access to the reference - you get it through a callback/closure:
In the auto-generated constructor (new()) the second parameter isn't a LowLevelApi, it's a closure that gets the &mut TcpStream, and that closure is then expected to return a LowLevelApi
When you want to actually use the LowLevelApi you can "rent" it, hence the wrapper.rent_mut(f) where f is the closure that gets passed a LowLevelApi (ll) and can do what it needs
With these facts, the rest of the Rental documentation makes a lot more sense.
This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Implement graph-like data structure in Rust
(3 answers)
What is the right smart pointer to have multiple strong references and allow mutability?
(1 answer)
Mutating one field while iterating over another immutable field
(3 answers)
How can I borrow from a HashMap to read and write at the same time?
(2 answers)
Closed 3 years ago.
I am trying to implement the JVM in Rust as a fun project but I am struggling with an issue involving references. When classes are loaded from class files, their references to other classes are represented as Strings. After the loading phase, a JVM should link classes with actual references to each other. I cannot figure out how to do this with Rust's references. Here is a stripped down version of what I am trying to achieve. (an MCV example)
use ClassRef::{Symbolic, Static};
use std::collections::HashMap;
fn main() {
let mut loader = ClassLoader { classes: HashMap::new() };
loader.load_class("java/lang/String", "java/lang/Object");
// java.lang.Object is the only class without a direct superclass
loader.load_class("java/lang/Object", "");
loader.link_classes();
}
struct ClassLoader<'a> {
classes: HashMap<String, Class<'a>>
}
impl<'a> ClassLoader<'a> {
pub fn load_class(&mut self, name: &str, super_name: &str) {
self.classes.insert(name.to_owned(), Class::new(name, super_name));
}
pub fn link_classes(&mut self) {
// Issue 1: Editing the stored class requires a mutable borrow
for (name, class) in self.classes.iter_mut() {
// Issue 2: I am not allowed to move this value to itself
class.super_class = match class.super_class {
// Issue 3: Storing a reference to another value in the map
Symbolic(ref super_name) => Static(self.classes.get(super_name)),
a => a
}
}
}
}
struct Class<'a> {
super_class: ClassRef<'a>,
name: String
}
impl<'a> Class<'a> {
pub fn new(name: &str, super_name: &str) -> Self {
Class {
name: name.to_owned(),
super_class: Symbolic(super_name.to_owned())
}
}
}
enum ClassRef<'a> {
Symbolic(String),
Static(Option<&'a Class<'a>>)
}
Currently, there are three issues with the process that I am unable to solve.
First, when I iterate through the list of loaded classes, I need to be able to change the field Class#super_class, so I need a mutable iterator. This means that I have a mutable borrow on self.classes. I also need an immutable borrow on self.classes during the iteration to look up a reference to the superclass. The way I feel like this should be solved is having a way of proving to the compiler that I am not mutating the map itself, but only a key. I feel like it has to do with std::cell::Cell as shown in Choosing your Guarantees, but the examples seem so simple I don't know how to correlate it to what I am trying to do here.
Second, I need the previous value of Class#super_class in order to calculate the new one. Essentially, I am moving the previous value out, changing it, and then moving it back in. I feel like this could be solved with some sort of higher order Map function to work on the value in place, but once again I'm not sure.
Third, when I give one class a reference to another class, there is no way of proving to the compiler that the other class won't be removed from the hashmap, causing the reference to point to deallocated memory. When classes are loaded, one class name will always point to the same class and classes are never unloaded, so I know that the value will always be there. The way I think this should be solved is with a Hashmap implementation which doesn't allow values to be overwritten or removed. I tried to poke around for such a thing but I couldn't find it.
EDIT: Solution
The duplicates linked were certainly helpful, but I ended up not using reference counting pointers (Rc<>) because the relationships between classes can sometimes be cyclic, so trying to drop the ClassLoader would end up leaking memory. The solution I settled on is using an immutable reference to a
Typed-Arena allocator. This allocator is an insertion only allocator which ensures the references it distributes are valid for the entire life of the reference to the allocator, as long as said reference is immutable. I then used the hashmap to store the references. That way everything is dropped when I drop the ClassLoader. Hopefully anyone that trying to implement a relationship similar to this finds the following code helpful.
extern crate typed_arena;
use ClassRef::{Symbolic, Static};
use std::collections::HashMap;
use std::cell::RefCell;
use core::borrow::BorrowMut;
use typed_arena::Arena;
fn main() {
let mut loader = ClassLoader { class_map: HashMap::new(), classes: &Arena::new() };
loader.load_class("java/lang/String", "java/lang/Object");
// java.lang.Object is the only class without a direct superclass
loader.load_class("java/lang/Object", "");
loader.link_classes();
}
struct ClassLoader<'a> {
classes: &'a Arena<RefCell<Class<'a>>>,
class_map: HashMap<String, &'a RefCell<Class<'a>>>
}
impl<'a> ClassLoader<'a> {
pub fn load_class(&mut self, name: &str, super_name: &str) {
let super_opt = if super_name.len() > 0 {
Some(super_name)
} else {
None
};
let class = Class::new(name, super_opt);
let class_ref = self.classes.alloc(RefCell::new(class));
self.class_map.insert(name.to_owned(), class_ref);
}
pub fn link_classes(&mut self) {
for (_name, class_ref) in &self.class_map {
let mut class = (*class_ref).borrow_mut();
if let Some(Symbolic(super_name)) = &class.super_class {
let super_class = self.class_map.get(super_name);
let super_class = super_class.map(|c| Static(c.clone()));
*class.super_class.borrow_mut() = super_class;
}
}
}
}
struct Class<'a> {
super_class: Option<ClassRef<'a>>,
name: String
}
impl<'a> Class<'a> {
pub fn new(name: &str, super_name: Option<&str>) -> Self {
Class {
name: name.to_owned(),
super_class: super_name.map(|name| Symbolic(name.to_owned()))
}
}
}
enum ClassRef<'a> {
Symbolic(String),
Static(&'a RefCell<Class<'a>>)
}
I'm trying to spawn a given set of threads and have each perform a long running operation. I would be passing a structure to each worker thread as the internal state of the given thread. The collection of said structs is kept in a vector, part of a Master struct.
The compiler rejects me passing the internal member of a struct to Arc::new():
use std::thread;
use std::sync::Arc;
struct Worker {
name: String,
}
struct Master {
workers: Vec<Worker>,
}
impl Worker {
fn start(&self) {
println!("My name is {} and I'm working!", self.name);
thread::sleep_ms(100_000);
}
}
impl Master {
pub fn run_test(&mut self) {
for i in 0..10 {
self.workers.push(Worker {
name: String::new() + "Worker" + &i.to_string()
});
}
let mut data = Arc::new(self.workers);
for i in 0..10 {
let local_data = data.clone();
thread::spawn(move || {
local_data[i].start();
});
}
thread::sleep_ms(100_000);
}
}
fn main() {
let mut master = Master { workers: vec![] };
}
The error message:
error[E0507]: cannot move out of borrowed content
--> <anon>:26:33
|
26 | let mut data = Arc::new(self.workers);
| ^^^^ cannot move out of borrowed content
What am I doing wrong? Is this idiomatic Rust?
Welcome to Ownership.
In Rust, any single piece of data has one and exactly one owner. Don't be fooled by Rc and Arc: they are a shared interface on top of a single (invisible) owner.
The simplest way of expressing ownership is by value:
struct Master {
workers: Vec<Worker>
}
Here, Master owns a Vec<Worker> which itself owns multiple Worker.
Similarly, functions that take their argument by value (fn new(t: T) -> Arc<T> for example) receive ownership of their argument.
And that is where the issue lies:
Arc::new(self.workers)
means that you are, at the same time:
claiming that Master is the owner of workers
claiming that Arc is the owner of workers
Given the rule of one and exactly one owner, this is clearly intractable.
So, how do you cheat and have multiple co-owners for a single piece of data?
Well... use Rc or Arc!
struct Master {
workers: Arc<Vec<Worker>>
}
And now creating data is as simple as:
let data = self.workers.clone();
which creates a new Arc (which just bumps the reference count).
That's not quite all, though. The core tenet of the Borrowing system is: Aliasing XOR Mutability.
Since Arc is about aliasing, it prevents mutability. You cannot insert workers into self.workers any longer!
There are multiple solutions, such as deferring the initialization of self.workers until the vector is built, however the most common is to use cells or mutexes, that is Rc<RefCell<T>> or Arc<Mutex<T>> (or Arc<RwLock<T>>).
RefCell and Mutex are wrappers that move borrow checking from compile-time to run-time. This gives a bit more flexibility, but may result in run-time panics instead of compile-time errors, so is best used as a last resort.