This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
How to return a reference to a sub-value of a value that is under a mutex?
(5 answers)
Returning a RWLockReadGuard independently from a method
(2 answers)
How can I store a Chars iterator in the same struct as the String it is iterating on?
(2 answers)
Closed 3 years ago.
I would like to wrap a low-level third-party API with my own struct and functions to make it more friendly. Unfortunately, the third-party API needs a reference to a socket in its constructor, which means I'd like my Struct to "own" the socket (someone has to own it so that it can be borrowed, right? and I'd like that hidden as an implementation detail in my API).
The third-party API looks something like this:
struct LowLevelApi<'a> {
stream: &'a mut TcpStream,
// ...
}
impl<'a> LowLevelApi<'a> {
pub fn new(socket: &'a mut TcpStream, ... ) -> LowLevelApi<'a> {
// ...
}
}
I would like to make the interface to my function look like:
pub fn new(host: String, port: u16, ...) -> HighLevelApi {
// ...
}
I tried this:
pub struct HighLevelApi<'a> {
stream: TcpStream,
low: LowLevelApi<'a>
}
impl <'a> HighLevelApi<'a> {
pub fn new(host: String, port: u16) -> HighLevelApi<'a> {
// Ignore lack of error checking for now
let mut stream = TcpStream::connect(format!("{}:{}", host, port)).unwrap();
HighLevelApi {
stream,
low: LowLevelApi::new(&mut stream)
}
}
}
Rust is (rightly) angry: It has no way of knowing that I'm not going to do something bad with the low field later. And even worse, I would need to somehow guarantee that when my structure gets dropped, low gets dropped first, and stream second (since by that point, any relationship between the two is lost - or rather, there never is/was a relationship between the two).
(actually, it's worse than that, because the stream local variable gets moved into the new struct, so the local can't possibly be borrowed by LowLevelApi, but I can't think of a way to initialize HighLevelApi with the stream from the struct, since there's no way to get a handle to that from within the struct's initialization, is there? But based on my guess about what would happen in the paragraph above, it doesn't really matter since it still wouldn't do what I wanted)
What are examples of the various techniques that can be used to store a wrap a third-party (not under my control) struct that needs a reference to something?
The Rental crate seems to do what is needed here, albeit with documentation and examples that leave a lot to the imagination (i.e. trial and error).
Here's roughly what solves this
rental! {
pub mod rentals {
#[rental_mut]
pub struct Wrapper {
stream: Box<TcpStream>,
low: LowLevelApi<'stream>
}
}
}
pub struct HighLevelApi {
wrapper: rentals::Wrapper
}
impl HighLevelApi {
pub fn new(host: String, port: u16) -> HighLevelApi {
Api {
// Ignore lack of error checking for now
wrapper: rentals::Wrapper::new(
Box::new(TcpStream::connect(format!("{}:{}", host, port)).unwrap()),
|s| LowLevelApi::new(s)
)
}
}
pub fn do_something(&mut self) {
self.wrapper.rent_mut(|ll| ll.do_something()) // ll is the LowLevelApi
}
}
I noticed two important things that made this work:
The lifetime name on low in the Wrapper struct must match the name of the "owning" field (in this case "'stream")
You never get direct access to the reference - you get it through a callback/closure:
In the auto-generated constructor (new()) the second parameter isn't a LowLevelApi, it's a closure that gets the &mut TcpStream, and that closure is then expected to return a LowLevelApi
When you want to actually use the LowLevelApi you can "rent" it, hence the wrapper.rent_mut(f) where f is the closure that gets passed a LowLevelApi (ll) and can do what it needs
With these facts, the rest of the Rental documentation makes a lot more sense.
Related
I want to offer a safe API like below FooManager. It should be able to store arbitrary user-defined values that implement a trait Foo. It should also be able to hand them back later - not as trait object (Box<dyn Foo>) but as the original type (Box<T> where T: Foo). At least conceptually it should be possible to offer this as a safe API, by using generic handles (Handle<T>), see below.
Additional criteria:
The solution should work in stable Rust (internal usage of unsafe blocks is perfectly okay though).
I don't want to modify the trait Foo, as e.g. suggested in How to get a reference to a concrete type from a trait object?. It should work without adding a method as_any(). Reasoning: Foo shouldn't have any knowledge about the fact that it might be stored in containers and be restored to the actual type.
trait Foo {}
struct Handle<T> {
// ...
}
struct FooManager {
// ...
}
impl FooManager {
// A real-world API would complain if the value is already stored.
pub fn keep_foo<T: Foo>(&mut self, foo: Box<T>) -> Handle<T> {
// ...
}
// In a real-world API this would return an `Option`.
pub fn return_foo<T: Foo>(&mut self, handle: Handle<T>) -> Box<T> {
// ...
}
}
I came up with this (Rust Playground) but not sure if there's a better way or if it's safe even. What do you think of that approach?
This question already has answers here:
Why are borrows of struct members allowed in &mut self, but not of self to immutable methods?
(2 answers)
cannot borrow `self.x` as immutable because `*self` is also borrowed as mutable
(1 answer)
How can I call a mutating method while holding a reference to self?
(1 answer)
Is it possible to borrow parts of a struct as mutable and other parts as immutable?
(2 answers)
Passing mutable self reference to method of owned object
(2 answers)
Closed 3 years ago.
This question is not about why the compiler gives these errors — I think I understand that. This question is about how to manage the situation on the mutable path while hoping to be able to call other functions.
This is a simplification of what I am trying to do. I have a lot of Foos inside World and I need to update them based on other data from World. I have several private functions inside World that I would like to be able to call.
pub struct Foo {
id: i32,
//...
}
pub struct Bar {
//...
}
pub struct World {
foos: Vec<Foo>,
bars: Vec<Bar>,
//...
}
impl World {
pub fn create() -> World {
World {
foos: Vec::new(),
bars: Vec::new(),
}
}
pub fn update(&mut self) {
for foo in self.get_foos_mut() {
//error[E0XXX]: cannot borrow `*self` as mutable/imutable ...
let _bar = self.get_bar_given_foo(foo);
//alter foo based on bar and other world things calling other world functions...
//...
}
}
//some useful utility functions that should be called several times
//from different World places...
fn get_foos_mut(&mut self) -> &mut [Foo] {
//select slice interval based on foos or any other calculation using self data...
&mut self.foos[..]
}
fn get_bar_given_foo(&self, foo: &Foo) -> &Bar {
//select bar based on foo or any other calculation using self data...
&self.bars[foo.id as usize]
}
//other utility functions that abstract e sequence of operations that I would
//like to use on the update path...
}
fn main() {
let mut world = World::create();
world.update();
}
You can also run the code.
It looks a very limiting situation because when on a mutable path like the update function, I am unable to call any of the private methods from self.
Since I don't want to clone all the data, I could think of three solutions. I tried 1 and 2 and both work.
Inline everything. Since this solution works for the example above, it looks like the compilation error is caused by the limitation of the Rust compiler that as I understand: it just looks the function signature and not the implementation of the functions.
Create functions that receive references to what they need. There is no need borrow self again. The function World::get_foos_mut(&mut self) -> &mut [Foo] could be removed and created a new one outside World: fn world_get_foos_mut(foos: &mut Vec<Foo>) -> &mut [Foo] The code on the playground shows this solution.
Create macros instead of regular functions? Since they are macros I suspect that there won't be any borrow involved. Since I am not familiar with macros I am unable to tell if this works or how to do this yet.
I really need other options because Solution 1 is not viable in my opinion and solution 2 looks very cumbersome and fragile (it looks that I might have the same problems but with the references inside World).
I am really frustrated about not being able to find the correct model to handle this.
Will the Rust compiler try to analyze these situations in the future?
This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Implement graph-like data structure in Rust
(3 answers)
What is the right smart pointer to have multiple strong references and allow mutability?
(1 answer)
Mutating one field while iterating over another immutable field
(3 answers)
How can I borrow from a HashMap to read and write at the same time?
(2 answers)
Closed 3 years ago.
I am trying to implement the JVM in Rust as a fun project but I am struggling with an issue involving references. When classes are loaded from class files, their references to other classes are represented as Strings. After the loading phase, a JVM should link classes with actual references to each other. I cannot figure out how to do this with Rust's references. Here is a stripped down version of what I am trying to achieve. (an MCV example)
use ClassRef::{Symbolic, Static};
use std::collections::HashMap;
fn main() {
let mut loader = ClassLoader { classes: HashMap::new() };
loader.load_class("java/lang/String", "java/lang/Object");
// java.lang.Object is the only class without a direct superclass
loader.load_class("java/lang/Object", "");
loader.link_classes();
}
struct ClassLoader<'a> {
classes: HashMap<String, Class<'a>>
}
impl<'a> ClassLoader<'a> {
pub fn load_class(&mut self, name: &str, super_name: &str) {
self.classes.insert(name.to_owned(), Class::new(name, super_name));
}
pub fn link_classes(&mut self) {
// Issue 1: Editing the stored class requires a mutable borrow
for (name, class) in self.classes.iter_mut() {
// Issue 2: I am not allowed to move this value to itself
class.super_class = match class.super_class {
// Issue 3: Storing a reference to another value in the map
Symbolic(ref super_name) => Static(self.classes.get(super_name)),
a => a
}
}
}
}
struct Class<'a> {
super_class: ClassRef<'a>,
name: String
}
impl<'a> Class<'a> {
pub fn new(name: &str, super_name: &str) -> Self {
Class {
name: name.to_owned(),
super_class: Symbolic(super_name.to_owned())
}
}
}
enum ClassRef<'a> {
Symbolic(String),
Static(Option<&'a Class<'a>>)
}
Currently, there are three issues with the process that I am unable to solve.
First, when I iterate through the list of loaded classes, I need to be able to change the field Class#super_class, so I need a mutable iterator. This means that I have a mutable borrow on self.classes. I also need an immutable borrow on self.classes during the iteration to look up a reference to the superclass. The way I feel like this should be solved is having a way of proving to the compiler that I am not mutating the map itself, but only a key. I feel like it has to do with std::cell::Cell as shown in Choosing your Guarantees, but the examples seem so simple I don't know how to correlate it to what I am trying to do here.
Second, I need the previous value of Class#super_class in order to calculate the new one. Essentially, I am moving the previous value out, changing it, and then moving it back in. I feel like this could be solved with some sort of higher order Map function to work on the value in place, but once again I'm not sure.
Third, when I give one class a reference to another class, there is no way of proving to the compiler that the other class won't be removed from the hashmap, causing the reference to point to deallocated memory. When classes are loaded, one class name will always point to the same class and classes are never unloaded, so I know that the value will always be there. The way I think this should be solved is with a Hashmap implementation which doesn't allow values to be overwritten or removed. I tried to poke around for such a thing but I couldn't find it.
EDIT: Solution
The duplicates linked were certainly helpful, but I ended up not using reference counting pointers (Rc<>) because the relationships between classes can sometimes be cyclic, so trying to drop the ClassLoader would end up leaking memory. The solution I settled on is using an immutable reference to a
Typed-Arena allocator. This allocator is an insertion only allocator which ensures the references it distributes are valid for the entire life of the reference to the allocator, as long as said reference is immutable. I then used the hashmap to store the references. That way everything is dropped when I drop the ClassLoader. Hopefully anyone that trying to implement a relationship similar to this finds the following code helpful.
extern crate typed_arena;
use ClassRef::{Symbolic, Static};
use std::collections::HashMap;
use std::cell::RefCell;
use core::borrow::BorrowMut;
use typed_arena::Arena;
fn main() {
let mut loader = ClassLoader { class_map: HashMap::new(), classes: &Arena::new() };
loader.load_class("java/lang/String", "java/lang/Object");
// java.lang.Object is the only class without a direct superclass
loader.load_class("java/lang/Object", "");
loader.link_classes();
}
struct ClassLoader<'a> {
classes: &'a Arena<RefCell<Class<'a>>>,
class_map: HashMap<String, &'a RefCell<Class<'a>>>
}
impl<'a> ClassLoader<'a> {
pub fn load_class(&mut self, name: &str, super_name: &str) {
let super_opt = if super_name.len() > 0 {
Some(super_name)
} else {
None
};
let class = Class::new(name, super_opt);
let class_ref = self.classes.alloc(RefCell::new(class));
self.class_map.insert(name.to_owned(), class_ref);
}
pub fn link_classes(&mut self) {
for (_name, class_ref) in &self.class_map {
let mut class = (*class_ref).borrow_mut();
if let Some(Symbolic(super_name)) = &class.super_class {
let super_class = self.class_map.get(super_name);
let super_class = super_class.map(|c| Static(c.clone()));
*class.super_class.borrow_mut() = super_class;
}
}
}
}
struct Class<'a> {
super_class: Option<ClassRef<'a>>,
name: String
}
impl<'a> Class<'a> {
pub fn new(name: &str, super_name: Option<&str>) -> Self {
Class {
name: name.to_owned(),
super_class: super_name.map(|name| Symbolic(name.to_owned()))
}
}
}
enum ClassRef<'a> {
Symbolic(String),
Static(&'a RefCell<Class<'a>>)
}
This question already has answers here:
How can I swap in a new value for a field in a mutable reference to a structure?
(2 answers)
Closed 4 years ago.
(Following "cannot move out of borrowed content" when replacing a struct field)
I have a third-party API where one struct have a method that consumes the instance and returns a new instance; I want to wrap this API in my own wrapper and abstract away the detail that the struct is consumed in the operation.
This example explains what I'm trying to achieve:
// Third-party API
struct Item {
x: u32,
}
impl Item {
pub fn increment(self, amount: u32) -> Self {
Item { x: self.x + amount }
}
}
// My API
struct Container {
item: Item,
}
impl Container {
pub fn increment_item(&mut self, amount: u32) {
// This line causes "cannot move out of borrowed content" but this is exactly what I want to do
self.item = self.item.increment(amount);
}
}
While now I understand the error I'm wondering how can I implement this without taking ownership of self inside Container::increment_item.
Proposed solutions:
Change Item::increment to take &mut self instead of self: I can't, Item::increment comes from a crate.
Use mem::replace (from here): Unfortunately constructing an Item instance it's not that easy, and to be honest I don't completely understand how the mem::replace solutions works.
Change Container::increment_item to take self instead of &mut self: I don't want to consume the Container instance in the process, I'm trying to design my wrapper to be as ergonomic as possible and abstract away completely the fact that Item must be consumed when changing it.
Some ideas on how to do it? Or am I trying an impossible design in Rust?
The simplest way is to use an Option:
use take to take ownership of the item,
then assign back Some(...).
If a panic occurs while the Option is empty, this is perfectly safe. No double-destruction occurs, and the container can even remain usable if you so desire.
Or in code:
// Third-party API
struct Item {
x: u32,
}
impl Item {
pub fn increment(self, amount: u32) -> Self {
Item { x: self.x + amount }
}
}
// My API
struct Container {
item: Option<Item>,
}
impl Container {
pub fn increment_item(&mut self, amount: u32) {
let item = self.item.take().unwrap();
self.item = Some(self.item.increment(amount));
}
}
I'm trying to port some C++ code to Rust. It composes a virtual (.mp4) file from a few kinds of slices (string reference, lazy-evaluated string reference, part of a physical file) and serves HTTP requests based on the result. (If you're curious, see Mp4File which takes advantage of the FileSlice interface and its concrete implementations in http.h.)
Here's the problem: I want require as few heap allocations as possible. Let's say I have a few implementations of resource::Slice that I can hopefully figure out on my own. Then I want to make the one that composes them all:
pub trait Slice : Send + Sync {
/// Returns the length of the slice in bytes.
fn len(&self) -> u64;
/// Writes bytes indicated by `range` to `out.`
fn write_to(&self, range: &ByteRange,
out: &mut io::Write) -> io::Result<()>;
}
// (used below)
struct SliceInfo<'a> {
range: ByteRange,
slice: &'a Slice,
}
/// A `Slice` composed of other `Slice`s.
pub struct Slices<'a> {
len: u64,
slices: Vec<SliceInfo<'a>>,
}
impl<'a> Slices<'a> {
pub fn new() -> Slices<'a> { ... }
pub fn append(&mut self, slice: &'a resource::Slice) { ... }
}
impl<'a> Slice for Slices<'a> { ... }
and use them to append lots and lots of slices with as few heap allocations as possible. Simplified, something like this:
struct ThingUsedWithinMp4Resource {
slice_a: resource::LazySlice,
slice_b: resource::LazySlice,
slice_c: resource::LazySlice,
slice_d: resource::FileSlice,
}
struct Mp4Resource {
slice_a: resource::StringSlice,
slice_b: resource::LazySlice,
slice_c: resource::StringSlice,
slice_d: resource::LazySlice,
things: Vec<ThingUsedWithinMp4Resource>,
slices: resource::Slices
}
impl Mp4Resource {
fn new() {
let mut f = Mp4Resource{slice_a: ...,
slice_b: ...,
slice_c: ...,
slices: resource::Slices::new()};
// ...fill `things` with hundreds of things...
slices.append(&f.slice_a);
for thing in f.things { slices.append(&thing.slice_a); }
slices.append(&f.slice_b);
for thing in f.things { slices.append(&thing.slice_b); }
slices.append(&f.slice_c);
for thing in f.things { slices.append(&thing.slice_c); }
slices.append(&f.slice_d);
for thing in f.things { slices.append(&thing.slice_d); }
f;
}
}
but this isn't working. The append lines cause errors "f.slice_* does not live long enough", "reference must be valid for the lifetime 'a as defined on the block at ...", "...but borrowed value is only valid for the block suffix following statement". I think this is similar to this question about the self-referencing struct. That's basically what this is, with more indirection. And apparently it's impossible.
So what can I do instead?
I think I'd be happy to give ownership to the resource::Slices in append, but I can't put a resource::Slice in the SliceInfo used in Vec<SliceInfo> because resource::Slice is a trait, and traits are unsized. I could do a Box<resource::Slice> instead but that means a separate heap allocation for each slice. I'd like to avoid that. (There can be thousands of slices per Mp4Resource.)
I'm thinking of doing an enum, something like:
enum BasicSlice {
String(StringSlice),
Lazy(LazySlice),
File(FileSlice)
};
and using that in the SliceInfo. I think I can make this work. But it definitely limits the utility of my resource::Slices class. I want to allow it to be used easily in situations I didn't anticipate, preferably without having to define a new enum each time.
Any other options?
You can add a User variant to your BasicSlice enum, which takes a Box<SliceInfo>. This way only the specialized case of users will take the extra allocation, while the normal path is optimized.