The following code compiles and runs but emits a mutable_borrow_reservation_conflict warning.
My goal is to have a field all_ops owning a set of Op's implementations (readonly) where each op can be referenced in another container in the same struct (and when the main all_ops container is cleared, used_ops access becomes illegal as expected)
Of course, one could use Rc but it causes performance issues.
Do you have an idea to do that properly ? (i.e. a way which will not become an hard error in the (near?) future).
trait Op {
fn f(&self);
}
struct OpA;
impl Op for OpA {
fn f(&self) {
println!("OpA");
}
}
struct OpB;
impl Op for OpB {
fn f(&self) {
println!("OpB");
}
}
struct Container<'a> {
all_ops: Vec<Box<dyn Op>>,
used_ops: Vec<&'a Box<dyn Op>>, // data pointing to data in all_ops field
}
fn main() {
let v: Vec<Box<dyn Op>> = vec![Box::new(OpA), Box::new(OpB)];
let mut c = Container { all_ops: v, used_ops: Vec::new() };
c.used_ops.push(&c.all_ops.get(0).unwrap());
c.used_ops.push(&c.all_ops.get(1).unwrap());
c.used_ops.push(&c.all_ops.get(0).unwrap());
for op in c.used_ops {
op.f();
}
c.all_ops.clear();
// c.used.first().unwrap().f(); // cannot borrow `c.all` as mutable because it is also borrowed as immutable
}
Rust playground
If I replace used_ops: Vec<&'a Box<dyn Op>>
by used_ops: Vec<&'a dyn Op>, it seems sufficient to fix the warning.
Unfortunately, Container isn't movable even if all references in used_ops are on objects allocated in the heap (and I understand why since there are references to (inner parts of) the object).
trait Op {
fn f(&self);
}
struct OpA;
impl Op for OpA {
fn f(&self) {
println!("OpA");
}
}
struct OpB;
impl Op for OpB {
fn f(&self) {
println!("OpB");
}
}
struct Container<'a> {
all_ops: Vec<Box<dyn Op>>,
used_ops: Vec<&'a dyn Op>, // data pointing to data in all_ops field
}
fn main() {
let v: Vec<Box<dyn Op>> = vec![Box::new(OpA), Box::new(OpB)];
let mut c = Container { all_ops: v, used_ops: Vec::new() };
c.used_ops.push(c.all_ops.get(0).unwrap().as_ref());
c.used_ops.push(c.all_ops.get(1).unwrap().as_ref());
c.used_ops.push(c.all_ops.get(0).unwrap().as_ref());
for op in c.used_ops.iter() {
op.f();
}
// let c2 = c; // cannot move out of `c` because it is borrowed
}
Playground
Related
Is my understanding correct that in Rust it is not possible to protect reference members of a struct from modification while having the reference target values mutable? (Without runtime borrow checking that is.) For example:
struct MyData<'a> {
pub some_ref: &'a mut i32,
}
fn doit<'a>(data: &mut MyData<'a>, other_ref: &'a mut i32) {
// I want to be able to do the following here:
*data.some_ref = 22;
// but make it impossible to do the following:
data.some_ref = other_ref;
}
Not being able to change the reference value may be useful in certain FFI situations. FFI and the performance requirements reasons prevent the use of runtime borrow checking here.
In C++ it can be expressed like this:
struct MyData {
int* const some_ref;
};
void doit(const MyData &data, int* other_ref) {
// this is allowed:
*data.some_ref = 22;
// this is not:
data.some_ref = other_ref; // compile error
}
You can create a wrapper type around the reference. If the constructor is private, and so is the wrapped reference field, you cannot replace the reference itself. You can then implement DerefMut to allow changing the referent.
pub struct ImmRef<'a> {
inner: &'a mut i32,
}
impl<'a> ImmRef<'a> {
fn new(inner: &'a mut i32) -> Self { Self { inner } }
}
impl std::ops::Deref for ImmRef<'_> {
type Target = i32;
fn deref(&self) -> &Self::Target { &*self.inner }
}
impl std::ops::DerefMut for ImmRef<'_> {
fn deref_mut(&mut self) -> &mut Self::Target { &mut *self.inner }
}
struct MyData<'a> {
pub some_ref: ImmRef<'a>,
}
fn doit<'a>(data: &mut MyData<'a>, other_ref: &'a mut i32) {
// I want to be able to do the following here:
*data.some_ref = 22;
// but make it impossible to do the following:
// data.some_ref = other_ref;
}
You can mark the newtype #[repr(transparent)] for FFI purposes.
But do note that if the code has some ImmRef<'a> available it can use tools such as std::mem::replace() to replace the reference.
Rust does not allow you to specify the mutability of individual fields like you can via const in C++. Instead, you should simply encapsulate the data by making it private and only allow modification through methods that you dictate:
struct MyData<'a> {
some_ref: &'a mut i32,
}
impl MyData<'_> {
pub fn set_ref(&mut self, other: i32) {
*self.some_ref = other;
}
}
That way, the field some_ref cannot be modified directly (outside of the module) and must use the available method.
I've been struggling with trying to understand how to properly annotate lifetimes in my application. I've simplified the actual code I had to this:
struct Item<'a> {
rf: &'a i32,
}
impl<'a> Item<'a> {
fn new(main: &'a App) -> Item<'a> {
Item{
rf: &main.i
}
}
}
struct App<'a> {
i: i32,
vec: Vec<Item<'a>>
}
impl<'a> App<'a> {
fn new() -> App<'a> {
App {
i: 32,
vec: vec![]
}
}
fn init(&mut self) {
self.vec.push(Item::new(self))
}
fn update(&self) {
for item in self.vec.iter() {
println!("{}", item.rf)
}
}
}
fn main() {
let app = App::new();
app.init();
app.update();
}
So there's a vector of items that hold a reference to something in App, which I know would exist as long as the app is alive, but the borrow-checker doesn't accept this. Could someone explain what's wrong with this code and how I could fix it?
You can find this code in rust playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=ee8980c9e1b2a0622548525dbcf9f50f
I think the problem lays with how rust handles references and infers them. Here's a possible solution.
In short, we tell the rust compiler that in the App struct the i member lives for 'a length. This member we can then share with another struct that lives for at least 'a. we achieve this by telling the Item struct when creating it, that it at least has to live for at least the lifetime of the App struct. Because the compiler can be a bit picky about inferring and anonymous lifetimes we have to be explicit. We do this by adding a 'b to the new method of the Item struct. When we then call this method with the lifetime 'a of the app struct, the compiler knows that Item lives for as long as App. Sorry for the short explanation cause more than I probably know goes on here.
struct Item<'a> {
rf: &'a i32,
}
impl<'a> Item<'a> {
fn new<'b>(main: &'b App<'a>) -> Item<'a> {
Item{
rf: main.i
}
}
}
struct App<'a> {
i: &'a i32,
vec: Vec<Item<'a>>
}
impl<'a> App<'a> {
fn new() -> App<'a> {
App {
i: &32,
vec: vec![]
}
}
fn init(&mut self) {
let item = Item::new(self);
self.vec.push(item)
}
fn update(&self) {
for item in self.vec.iter() {
println!("{}", item.rf)
}
}
}
fn main() {
let mut app = App::new();
app.init();
app.update();
}
I'm trying to understand how to work with interior mutability. This question is strongly related to my previous question.
I have a generic struct Port<T> that owns a Vec<T>. We can "chain" port B to port A so, when reading the content of port A, we are able to read the content of port B. However, this chaining is hidden to port A's reader. That is why I implemented the iter(&self) method:
use std::rc::Rc;
pub struct Port<T> {
values: Vec<T>,
ports: Vec<Rc<Port<T>>>,
}
impl <T> Port<T> {
pub fn new() -> Self {
Self { values: vec![], ports: vec![] }
}
pub fn add_value(&mut self, value: T) {
self.values.push(value);
}
pub fn is_empty(&self) -> bool {
self.values.is_empty() && self.ports.is_empty()
}
pub fn chain_port(&mut self, port: Rc<Port<T>>) {
if !port.is_empty() {
self.ports.push(port)
}
}
pub fn iter(&self) -> impl Iterator<Item = &T> {
self.values.iter().chain(
self.ports.iter()
.flat_map(|p| Box::new(p.iter()) as Box<dyn Iterator<Item = &T>>)
)
}
pub fn clear(&mut self) {
self.values.clear();
self.ports.clear();
}
}
The application has the following pseudo-code behavior:
create ports
loop:
fill ports with values
chain ports
iterate over ports' values
clear ports
The main function should look like this:
fn main() {
let mut port_a = Rc::new(Port::new());
let mut port_b = Rc::new(Port::new());
loop {
port_a.add_value(1);
port_b.add_value(2);
port_a.chain_port(port_b.clone());
for val in port_a.iter() {
// read data
};
port_a.clear();
port_b.clear();
}
}
However, the compiler complains:
error[E0596]: cannot borrow data in an `Rc` as mutable
--> src/modeling/port.rs:46:9
|
46 | port_a.add_value(1);
| ^^^^^^ cannot borrow as mutable
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Rc<Port<i32>>`
I've been reading several posts etc., and it seems that I need to work with Rc<RefCell<Port<T>>> to be able to mutate the ports. I changed the implementation of Port<T>:
use std::cell::RefCell;
use std::rc::Rc;
pub struct Port<T> {
values: Vec<T>,
ports: Vec<Rc<RefCell<Port<T>>>>,
}
impl<T> Port<T> {
// snip
pub fn chain_port(&mut self, port: Rc<RefCell<Port<T>>>) {
if !port.borrow().is_empty() {
self.ports.push(port)
}
}
pub fn iter(&self) -> impl Iterator<Item = &T> {
self.values.iter().chain(
self.ports
.iter()
.flat_map(|p| Box::new(p.borrow().iter()) as Box<dyn Iterator<Item = &T>>),
)
}
// snip
}
This does not compile either:
error[E0515]: cannot return value referencing temporary value
--> src/modeling/port.rs:35:31
|
35 | .flat_map(|p| Box::new(p.borrow().iter()) as Box<dyn Iterator<Item = &T>>),
| ^^^^^^^^^----------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| | |
| | temporary value created here
| returns a value referencing data owned by the current function
I think I know what the problem is: p.borrow() returns a reference to the port being chained. We use that reference to create the iterator, but as soon as the function is done, the reference goes out of scope and the iterator is no longer valid.
I have no clue on how to deal with this. I managed to implement the following unsafe method:
pub fn iter(&self) -> impl Iterator<Item = &T> {
self.values.iter().chain(self.ports.iter().flat_map(|p| {
Box::new(unsafe { (&*p.as_ref().as_ptr()).iter() }) as Box<dyn Iterator<Item = &T>>
}))
}
While this works, it uses unsafe code, and there must be a safe workaround.
I set a playground for more details of my application. The application compiles and outputs the expected result (but uses unsafe code).
You can't modify anything behind an Rc, that's correct. While this might be solved with a RefCell, you don't want to go down that road. You might come into a situation where you'd need to enforce a specific clean() order or similar horrors.
More important: your main is fundamentally flawed, ownership-wise. Take these lines:
let mut port_a = Port::new();
let mut port_b = Port::new();
loop {
// Creates an iummutable borrow of port_b with same lifetime as port_a!
port_a.chain_port(port_b);
// ...
// A mutable borrow of port_b.
// But the immutable borrow from above persists across iterations.
port_b.clear();
// Or, even if you do fancy shenanigans at least until this line.
port_a.clear();
}
To overcome this, just constrain the ports lifetime to one iteration. You currently manually clean them up anyway, so that's already what you're doing conceptually.
Also, I got rid of that recursive iteration, just to simplify things a little more.
#[derive(Clone)]
pub struct Port<'a, T> {
values: Vec<T>,
ports: Vec<&'a Port<'a, T>>,
}
impl<'a, T> Port<'a, T> {
pub fn new() -> Self {
Self {
values: vec![],
ports: vec![],
}
}
pub fn add_value(&mut self, value: T) {
self.values.push(value);
}
pub fn is_empty(&self) -> bool {
self.values.is_empty() && self.ports.is_empty()
}
pub fn chain_port(&mut self, port: &'a Port<T>) {
if !port.is_empty() {
self.ports.push(&port)
}
}
pub fn iter(&self) -> impl Iterator<Item = &T> {
let mut port_stack: Vec<&Port<T>> = vec![self];
// Sensible estimate I guess.
let mut values: Vec<&T> = Vec::with_capacity(self.values.len() * (self.ports.len() + 1));
while let Some(port) = port_stack.pop() {
values.append(&mut port.values.iter().collect());
port_stack.extend(port.ports.iter());
}
values.into_iter()
}
}
fn main() {
loop {
let mut port_a = Port::new();
let mut port_b = Port::new();
port_a.add_value(1);
port_b.add_value(2);
port_a.chain_port(&port_b);
print!("values in port_a: [ ");
for val in port_a.iter() {
print!("{} ", val);
}
println!("]");
}
}
use std::marker;
use std::ops;
pub struct Shared<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> Shared<'r, T> {
pub fn new(value: T) -> Shared<'r, T> {
let boxed = Box::new(value);
Shared {
data: Box::into_raw(boxed),
_pd: marker::PhantomData,
}
}
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
}
impl<'r, T> ops::Deref for Shared<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
pub struct SharedRef<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> ops::Deref for SharedRef<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
impl<'r, T> Drop for Shared<'r, T> {
fn drop(&mut self) {
unsafe {
Box::from_raw(self.data);
}
}
}
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
What I wanted to express was a mix between a Box and an Arc. A uniquely owned pointer that is also capable of giving out references.
The problem is that I want to be able to move Shared around even if there are currently immutable borrows to it. It should be legal in this scenario because it is heap allocated.
The problem is that I have no idea how to express this.
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
Here I move s into a scope with "less" lifetime than it had before. But now after I have moved s into s1, s_ref should not be accessible anymore. So what I want to say is that it is okay to move a Shared if the lifetime does not get smaller.
Can this be expressed in Rust?
The reason Rust allows you to move out of the Shared is that you haven't tied the lifetime of the returned SharedRef to it:
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
Annotating the &self fixes that:
pub fn as_ref(&'r self) -> SharedRef<'r, T> { .. }
My current understanding is that the key difference here is that this says that the lifetime of the SharedRef now matches the lifetime of the borrow of self, keeping the borrow alive. Indeed it doesn't have to be the same lifetime ('r) as in the Shared; it works with a new lifetime just for the borrow/return:
pub fn as_ref<'b>(&'b self) -> SharedRef<'b, T> { .. }
This also disallows the move.
As for the bonus part of the question, where you want to allow moving as long as it's to something with a long enough lifetime, I think the answer is no. The only way I know to stop something being moved at all is to borrow it, and that stops any move.
I am trying to make some kind of ffi to a library written in C, but got stuck. Here is a test case:
extern crate libc;
use libc::{c_void, size_t};
// this is C library api call
unsafe fn some_external_proc(_handler: *mut c_void, value: *const c_void,
value_len: size_t) {
println!("received: {:?}" , std::slice::from_raw_buf(
&(value as *const u8), value_len as usize));
}
// this is Rust wrapper for C library api
pub trait MemoryArea {
fn get_memory_area(&self) -> (*const u8, usize);
}
impl MemoryArea for u64 {
fn get_memory_area(&self) -> (*const u8, usize) {
(unsafe { std::mem::transmute(self) }, std::mem::size_of_val(self))
}
}
impl <'a> MemoryArea for &'a str {
fn get_memory_area(&self) -> (*const u8, usize) {
let bytes = self.as_bytes();
(bytes.as_ptr(), bytes.len())
}
}
#[allow(missing_copy_implementations)]
pub struct Handler<T> {
obj: *mut c_void,
}
impl <T> Handler<T> {
pub fn new() -> Handler<T> { Handler{obj: std::ptr::null_mut(),} }
pub fn invoke_external_proc(&mut self, value: T) where T: MemoryArea {
let (area, area_len) = value.get_memory_area();
unsafe {
some_external_proc(self.obj, area as *const c_void,
area_len as size_t)
};
}
}
// this is Rust wrapper user code
fn main() {
let mut handler_u64 = Handler::new();
let mut handler_str = Handler::new();
handler_u64.invoke_external_proc(1u64); // OK
handler_str.invoke_external_proc("Hello"); // also OK
loop {
match std::io::stdin().read_line() {
Ok(line) => {
let key =
line.trim_right_matches(|&: c: char| c.is_whitespace());
//// error: `line` does not live long enough
// handler_str.invoke_external_proc(key)
}
Err(std::io::IoError { kind: std::io::EndOfFile, .. }) => break ,
Err(error) => panic!("io error: {}" , error),
}
}
}
Rust playpen
I get "line does not live long enough" error if I uncomment line inside the loop. In fact, I realize that Rust is afraid that I could store short-living reference to a slice somewhere inside Handler object, but I quite sure that I wouldn't, and I also know, that it is safe to pass pointers to the external proc (actually, memory is immidiately copied at the C library side).
Is there any way for me to bypass this check?
The problem is that you are incorrectly parameterizing your struct, when you really want to do it for the function. When you create your current Handler, the struct will be specialized with a type that includes a lifetime. However, the lifetime of line is only for the block, so there can be no lifetime for Handler that lasts multiple loop iterations.
What you want is for the lifetime to be tied to the function call, not the life of the struct. As you noted, if you put the lifetime on the struct, then the struct is able to store references of that length. You don't need that, so put the generic type on the function instead:
impl Handler {
pub fn new() -> Handler { Handler{obj: std::ptr::null_mut(),} }
pub fn invoke_external_proc<T>(&mut self, value: T) where T: MemoryArea {
let (area, area_len) = value.get_memory_area();
unsafe {
some_external_proc(self.obj, area as *const c_void,
area_len as size_t)
};
}
}
Amended answer
Since you want to specialize the struct on a type, but don't care too much about the lifetime of the type, let's try this:
#[allow(missing_copy_implementations)]
pub struct Handler<T: ?Sized> {
obj: *mut c_void,
}
impl<T: ?Sized> Handler<T> {
pub fn new() -> Handler<T> { Handler{ obj: std::ptr::null_mut() } }
pub fn invoke_external_proc(&mut self, value: &T) where T: MemoryArea {
let (area, area_len) = value.get_memory_area();
unsafe {
some_external_proc(self.obj, area as *const c_void,
area_len as size_t)
};
}
}
Here, we allow the type to be unsized. Since you can't pass an unsized value as a parameter, we now have to take a reference instead. We also have to change the impl:
impl MemoryArea for str {
fn get_memory_area(&self) -> (*const u8, usize) {
let bytes = self.as_bytes();
(bytes.as_ptr(), bytes.len())
}
}