I've been working on creating a Table struct that can resolve to a Val trait object along with other data types (Number implemented as an example) and can be seen in the Rust Playground here. I haven't been able to get past this borrow/lifetime issue but the idea is that Number and Table are both Val objects. However, any attempt to write the object to a String buffer causes a failure in compilation.
What am I doing wrong here? And what is the rationale for the borrow checker to operate in this way?
use std::fmt;
use std::fmt::Write;
struct Number(f64);
struct Table<'a> {
rows: Vec<Row<'a>>
// Will have column vector as a member
}
impl <'a>Table<'a> {
fn new(rows: Vec<Vec<Box<dyn Val>>>) -> Table {
let mut table = Table {
rows: rows.into_iter().map(|r| {
Row {
parent: std::ptr::null_mut(),
cells: r
}
}).collect()
};
let parent = &mut table as *mut Table;
table.rows = table.rows.into_iter().map(|mut r| {
r.parent = parent;
r
}).collect();
table
}
}
struct Row<'a> {
parent: *mut Table<'a>,
cells: Vec<Box<dyn Val<'a>>>
}
impl <'a>Row<'a> {
fn to_str(&'a self, buf: &mut String) -> fmt::Result {
let mut cell_iter = self.cells.iter().enumerate().peekable();
let _parent = unsafe { self.parent.as_ref() }; // Real implementation will need parent ref
while let Some((idx,_c)) = cell_iter.next() { // Real implementation will cycle through columns in parent struct
match self.cells.get(idx) {
Some(v) => v.to_str(buf),
None => Ok(())
}?;
if let Some(_) = cell_iter.peek() {
write!(buf, ",")?;
}
}
Ok(())
}
}
pub trait Val<'a> {
fn to_str(&'a self, buf: &mut String) -> fmt::Result;
}
pub trait ObjWriter<'a> {
fn to_str(&'a self, buf: &'a mut String) -> fmt::Result;
}
impl <'a>ObjWriter<'a> for dyn Val<'a> {
fn to_str(&'a self, buf: &mut String) -> fmt::Result { self.to_str(buf) }
}
impl <'a>Val<'a> for Table<'a> {
fn to_str(&'a self, buf: &mut String) -> fmt::Result {
write!(buf, "(START TABLE:")?;
let mut row_iter = self.rows.iter().peekable();
while let Some(r) = row_iter.next() {
r.to_str(buf)?;
write!(buf, "\n")?;
}
write!(buf, "END TABLE)")
}
}
impl Number {
fn to_str(&self, buf: &mut String) -> fmt::Result {
write!(buf,"{}",self.0)
}
}
impl <'a>Val<'a> for Number {
fn to_str(&self, buf: &mut String) -> fmt::Result {
self.to_str(buf)
}
}
fn main() {
let table = Table::new(vec![
vec![Box::new(Number(0.5)),Box::new(Table::new(Vec::new()))],
vec![Box::new(Table::new(Vec::new())),Box::new(Number(0.5))],
]);
let mut buf = String::new();
table.to_str(&mut buf);
println!("{}",buf)
}
error[E0597]: `table` does not live long enough
--> src/main.rs:98:5
|
92 | let table = Table::new(vec![
| ____________________________-
93 | | vec![Box::new(Number(0.5)),Box::new(Table::new(Vec::new()))],
94 | | vec![Box::new(Table::new(Vec::new())),Box::new(Number(0.5))],
95 | | ]);
| |_____- cast requires that `table` is borrowed for `'static`
...
98 | table.to_str(&mut buf);
| ^^^^^ borrowed value does not live long enough
99 | println!("{}",buf)
100 | }
| - `table` dropped here while still borrowed
I did as suggested trentcl, noting that my pointers would be invalidated on move, I refactored my code so that any Row methods would take the parent table's reference as the first parameter. Then I removed all the <'a> followed the compiler errors until it worked.
So the whole problem was allowing the lifetimes to get out of control and using unsafe methods to get parent references. Go figure...
Related
I am new to Rust and trying to implement a simple TCP-Server that sends me a live image. Therefore I am using this crate https://crates.io/crates/tinybmp for the RawBmp-type. Now I am struggling a bit to understand some of Rusts functionality. Or let's say I am still trying to figure out how to accomplish this in the 'Rust way'.
pub struct TCPBasePackage {
haspayload: bool,
pkgbytes: Vec<u8>,
}
impl TCPBasePackage {
pub const CMD: i32 = 0;
pub fn new() -> TCPBasePackage {
TCPBasePackage { haspayload: false, pkgbytes: Vec::new()}
}
}
impl TCPPackage for TCPBasePackage {
fn serialize(&mut self) -> &[u8] {
self.pkgbytes.append(&mut self.get_command().to_be_bytes().to_vec());
self.pkgbytes.append(&mut [u8::from(self.haspayload)].to_vec());
self.pkgbytes.as_slice()
}
fn read_from_stream(&mut self, mut stream: &TcpStream) -> Vec<u8> {
let mut vec: Vec<u8> = Vec::new();
let mut buf_pkg: [u8; 5] = [0; 5];
stream.read_exact(&mut buf_pkg).unwrap();
vec.append(&mut buf_pkg.to_vec());
self.pkgbytes = vec.to_vec();
return self.pkgbytes.clone();
}
fn deserialize(&mut self, buff: &[u8]) -> i32 {
self.haspayload = buff[4] != 0;
return 5;
}
fn get_command(&self) -> i32 {
TCPBasePackage::CMD
}
}
pub struct MsgLiveImage<'a> {
tcppackage: TCPBasePackage,
imgbytes: Vec::<u8>,
img: RawBmp<'a>,
}
impl<'a> MsgLiveImage<'a> {
pub const CMD: i32 = 2;
pub fn new() -> MsgLiveImage<'static> {
MsgLiveImage {
tcppackage: TCPBasePackage { haspayload: false, pkgbytes: Vec::new() },
imgbytes: Vec::new(),
img: RawBmp::from_slice(&[]).unwrap()
}
}
pub fn set_image_data(&'a mut self, data: Vec::<u8>) {
self.imgbytes = data;
self.img = RawBmp::from_slice(&self.imgbytes.as_slice()).unwrap();
}
}
In some function of my application I want to do this:
fn write_response_to_stream(
mut request: &Box<dyn mytcp::TCPPackage>,
mut stream: &TcpStream,
raspicam: &Arc<Mutex<camserv::CamServ>>,
) -> Result<Box<dyn mytcp::TCPPackage>, &'static str> {
match request.get_command() {
mytcp::MsgLiveImage::CMD => {
let mut pkg = mytcp::MsgLiveImage::new();
{
{
let tmpcam = raspicam.lock().unwrap(); // lock the camera server
let tmpbuff = tmpcam.imgbuff.lock().unwrap(); // lock the buffer of last image
pkg.set_image_data((*tmpbuff).clone()); // make copy of my image and transfer ownership of that copy to the package struct
}
{
let sbuff = pkg.serialize();
stream.write(&sbuff);
}
stream.flush();
}
Ok(Box::new(pkg)) // return the response package
}
_ => Err("Unknown request package"),
}
}
But the problem is, that the compiler is complaining about multiple borrowing the pkg-variable. As well as moving the pkg out of scope (which i learned should be possible by wrapping it in a Box.
Can someone explain me why the first mutable borrow still lasts even after the method returns? How can i achieve to call multiple methods on my struct without getting these borrowing conflicts?
Errors from rustc:
error[E0499]: cannot borrow `pkg` as mutable more than once at a time
--> src/raspiserv/mod.rs:90:33
|
87 | pkg.set_image_data((*tmpbuff).clone());
| -------------------------------------- first mutable borrow occurs here
...
90 | let sbuff = pkg.serialize();
| ^^^^^^^^^^^^^^^ second mutable borrow occurs here
...
95 | Ok(Box::new(pkg))
| ----------------- returning this value requires that `pkg` is borrowed for `'static`
error[E0515]: cannot return value referencing local variable `pkg`
--> src/raspiserv/mod.rs:95:13
|
87 | pkg.set_image_data((*tmpbuff).clone());
| -------------------------------------- `pkg` is borrowed here
...
95 | Ok(Box::new(pkg))
| ^^^^^^^^^^^^^^^^^ returns a value referencing data owned by the current function
error[E0505]: cannot move out of `pkg` because it is borrowed
--> src/raspiserv/mod.rs:95:25
|
87 | pkg.set_image_data((*tmpbuff).clone());
| -------------------------------------- borrow of `pkg` occurs here
...
95 | Ok(Box::new(pkg))
| ------------^^^--
| | |
| | move out of `pkg` occurs here
| returning this value requires that `pkg` is borrowed for `'static`
The problem is with your set_image_data
pub fn set_image_data(&'a mut self, data: Vec::<u8>) {
self.imgbytes = data;
self.img = RawBmp::from_slice(&self.imgbytes.as_slice()).unwrap();
}
you borrow self for the lifetime of 'a which is as long as the RawBmp inside of it is valid.
Thus the borrow lasts as long as the struct.
What you have is a self referential struct see this question on them for suggestions how to reslove this.
You can probably just drop the img from MsgLiveImage and replace it with a method:
impl MsgLiveImage {
pub fn as_img<'a>(&'a self) -> Result<RawBmp<'a>, ParseError> {
RawBmp::from_slice(&self.imgbytes.as_slice())
}
}
Might even be able to omit the lifetimes from that.
I haven't found an answer to this in other questions.
I have reduced my problem to the following:
use std::sync::RwLock;
pub fn main() {
iter_lock().for_each(|v| {
println!("{}", v);
});
}
fn get_lock<'a>() -> &'a RwLock<Vec<u32>> {
static mut lock: RwLock<Vec<u32>> = RwLock::new(Vec::new());
unsafe { &lock }
}
fn iter_lock<'a>() -> impl std::iter::Iterator<Item = &'a u32> {
get_lock().read().unwrap().iter()
}
playground
The code above will not compile and give the following error:
error[E0515]: cannot return reference to temporary value
--> src/main.rs:15:5
|
15 | get_lock().read().unwrap().iter()
| --------------------------^^^^^^^
| |
| returns a reference to data owned by the current function
| temporary value created here
|
= help: use `.collect()` to allocate the iterator
Note that the static mut is not necessary in the code above, but I need it because I need to define a variable inside of an impl block.
I need to return an iterator, not a Vec because I am trying to avoid any allocations, and this function will always be used to iterate.
How can I solve this issue? I'm not afraid of using unsafe code, so unsafe suggestions are also welcome.
You can try something like this:
use std::sync::{RwLock, RwLockReadGuard};
pub fn main() {
let data = Data::new(&[1, 2, 3]);
data.iter().for_each(|x| println!("{:?}", x));
}
struct Data {
inner: RwLock<Vec<u32>>,
}
impl Data {
fn new(vec: &[u32]) -> Self {
Self {
inner: RwLock::new(vec.to_vec()),
}
}
fn iter(&self) -> Iter<'_> {
let d = self.inner.read().unwrap();
Iter::new(d)
}
}
struct Iter<'a> {
inner: RwLockReadGuard<'a, Vec<u32>>,
current_index: usize,
}
impl<'a> Iter<'a> {
pub fn new(inner: RwLockReadGuard<'a, Vec<u32>>) -> Iter<'a> {
Self {
inner,
current_index: 0,
}
}
}
impl Iterator for Iter<'_> {
type Item = u32;
fn next(&mut self) -> Option<Self::Item> {
if self.current_index >= self.inner.len() {
return None;
}
let item = &self.inner[self.current_index];
self.current_index += 1;
Some(*item)
}
}
I feel like I am totally lost; what does "'1" mean in the error message?
error[E0597]: `o` does not live long enough
--> src/main.rs:32:19
|
31 | async fn foo6(&mut self, mut o: &'a mut Outer) {
| --------- lifetime `'1` appears in the type of `self`
32 | self.foo5(&mut o).await
| ----------^^^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `o` is borrowed for `'1`
33 | }
| - `o` dropped here while still borrowed
What can I do to make o live long enough? I think I'm using futures 03.
use futures::future::BoxFuture;
use futures::FutureExt;
use futures::stream::FuturesOrdered;
struct Inner {}
impl Inner {
async fn foo3(&mut self) -> Result<u32, ()> {
Ok(8)
}
}
// --- 1
struct Outer {
i: Inner,
}
impl Outer {
fn foo4(&mut self) -> BoxFuture<'_, Result<u32, ()>> {
self.i.foo3().boxed()
}
}
/// --- 2
struct Outer2<'a> {
futures_list: FuturesOrdered<BoxFuture<'a, Result<u32, ()>>>,
}
impl <'a> Outer2<'a> {
async fn foo6(&mut self, mut o: &'a mut Outer) {
self.foo5(&mut o).await
}
async fn foo5(&mut self, o: &'a mut Outer) {
self.futures_list.push(o.foo4());
}
}
#[tokio::main]
async fn main() {
let mut o = Outer { i: Inner {} };
let mut o2 = Outer2 { futures_list: FuturesOrdered::new() };
o2.foo5(&mut o).await;
}
playground
The '1 represents the anonymous lifetime of the self parameter for function foo5.
Note that in foo6, o is already a mutable reference to Outer, so writing &mut o actually gives you a mutable reference to a mutable reference to Outer with one too many indirections. You can fix your code by removing the extra reference:
async fn foo6(&mut self, mut o: &'a mut Outer) {
self.foo5(o).await
}
Playground
I have a trait with a function that takes a reference to an iterator:
#[derive(Clone)]
struct Dog {
name: &'static str,
}
trait DogListAction<'a, I>
where
I: Iterator<Item = &'a Dog>,
{
fn on_dog_list(&mut self, dog_list: I);
}
struct DogListActionExample {}
impl<'a, I> DogListAction<'a, I> for DogListActionExample
where
I: Iterator<Item = &'a Dog>,
{
fn on_dog_list(&mut self, dog_list: I) {
for dog in dog_list {
println!("{}", dog.name);
}
}
}
fn main() {
let dogs = vec![Dog { name: "Pluto" }, Dog { name: "Lilly" }];
let mut action_example = DogListActionExample {};
let mut dog_list_actions: Vec<Box<DogListAction<_>>> = vec![Box::new(action_example)];
loop {
let dog_clone = dogs.clone();
for dog_list_action in &mut dog_list_actions {
dog_list_action.on_dog_list(dog_clone.iter());
}
}
}
playground
It does not take any reference to the elements, so there is no need for it to last for more time than the function call.
Due to my limited understanding of lifetimes, I do not know yet how to express this. Calling this function causes a compilation error:
error[E0597]: `dog_clone` does not live long enough
--> src/main.rs:33:41
|
33 | dog_list_action.on_dog_list(dog_clone.iter());
| ^^^^^^^^^ borrowed value does not live long enough
34 | }
35 | }
| - `dog_clone` dropped here while still borrowed
36 | }
| - borrowed value needs to live until here
I guess the borrow checker thinks that data in dog_clone may be referenced after the function ends, but this is not the case.
The problem here is that the code can potentially save short-lived references to dog_clone elements in the longer-lived dog_list_actions. We need to tell the compiler that we will not be saving references the iterator produces. It can be done like this:
trait DogListAction {
fn on_dog_list<'a, I>(&'a mut self, dog_list: I)
where
I: Iterator<Item = &'a Dog>;
}
Now Item's can live for the duration of on_dog_list call. In the original code they must live for longer duration.
But this code creates another problem: we can't box the trait DogListAction anymore as it includes generic function. The usual approach here is to use a trait object:
trait DogListAction {
fn on_dog_list<'a>(&'a mut self, dog_list: Box<dyn Iterator<Item = &'a Dog> + 'a>);
}
Note the second 'a in Box<dyn Iterator<Item = &'a Dog> + 'a>. Rust adds a 'static trait bound to boxed trait objects by default and we don't want 'static here.
#[derive(Clone)]
struct Dog {
name: &'static str,
}
trait DogListAction {
fn on_dog_list<'a>(&'a mut self, dog_list: Box<dyn Iterator<Item = &'a Dog> + 'a>);
}
struct DogListActionExample {}
impl DogListAction for DogListActionExample {
fn on_dog_list<'a>(&'a mut self, dog_list: Box<dyn Iterator<Item = &'a Dog> + 'a>) {
for dog in dog_list {
println!("{}", dog.name);
}
}
}
fn main() {
let dogs = vec![Dog { name: "Pluto" }, Dog { name: "Lilly" }];
let action_example = DogListActionExample {};
let mut dog_list_actions: Vec<Box<DogListAction>> = vec![Box::new(action_example)];
{
let dogs_clone = dogs.clone();
for dog_list_action in &mut dog_list_actions {
dog_list_action.on_dog_list(Box::new(dogs_clone.iter()));
}
}
}
Playground
I have a data structure like this:
struct R {
hmhs: HashMap<i64, HashSet<i64>>,
}
impl R {
fn hs_for_hmhs(&mut self) -> &mut HashSet<i64> {
if let None = self.hmhs.get(&0) {
self.hmhs.insert(0, HashSet::new());
}
self.hmhs.get_mut(&0).unwrap()
}
fn iter_for_hmhs<'a>(&'a mut self) -> impl Iterator<Item = &'a i64> {
self.hs_for_hmhs().iter()
}
fn insert_for_hmhs(&mut self, i: i64) -> bool {
self.hs_for_hmhs().insert(i)
}
}
This seems to work, but all the methods require a mutable
reference to self which is unfortunate. I tried to give
interior mutability a go:
struct S {
hmhs: RefCell<HashMap<i64, HashSet<i64>>>,
}
impl S {
fn hs_for_hmhs(&self) -> &HashSet<i64> {
if let None = self.hmhs.borrow().get(&0) {
self.hmhs.borrow_mut().insert(0, HashSet::new());
}
self.hmhs.borrow_mut().get_mut(&0).unwrap()
}
fn iter_for_hmhs(&mut self) -> impl Iterator<Item = &i64> {
self.hs_for_hmhs().iter()
}
fn insert_for_hmhs(&mut self, i: i64) -> bool {
self.hs_for_hmhs().insert(i)
}
}
However, I constantly seem to hit problems. Mostly some variety of How do I return a reference to something inside a RefCell without breaking encapsulation?
I have tried lots of variants here, but I am missing something
fundamental in my understanding. Is there a way of achieving what I
want?
Complete Code:
use std::cell::RefCell;
use std::collections::{HashMap, HashSet};
struct R {
hmhs: HashMap<i64, HashSet<i64>>,
}
impl R {
fn hs_for_hmhs(&mut self) -> &mut HashSet<i64> {
if let None = self.hmhs.get(&0) {
self.hmhs.insert(0, HashSet::new());
}
self.hmhs.get_mut(&0).unwrap()
}
fn iter_for_hmhs<'a>(&'a mut self) -> impl Iterator<Item = &'a i64> {
self.hs_for_hmhs().iter()
}
fn insert_for_hmhs(&mut self, i: i64) -> bool {
self.hs_for_hmhs().insert(i)
}
}
struct S {
hmhs: RefCell<HashMap<i64, HashSet<i64>>>,
}
impl S {
fn hs_for_hmhs(&self) -> &mut HashSet<i64> {
if let None = self.hmhs.borrow().get(&0) {
self.hmhs.borrow_mut().insert(0, HashSet::new());
}
self.hmhs.borrow_mut().get_mut(&0).unwrap()
}
fn iter_for_hmhs(&self) -> impl Iterator<Item = &i64> {
self.hs_for_hmhs().iter()
}
fn insert_for_hmhs(&self, i: i64) -> bool {
self.hs_for_hmhs().insert(i)
}
}
fn main() {}
Compiler Message:
error[E0597]: borrowed value does not live long enough
--> src/main.rs:36:9
|
36 | self.hmhs.borrow_mut().get_mut(&0).unwrap()
| ^^^^^^^^^^^^^^^^^^^^^^ temporary value does not live long enough
37 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the method body at 31:5...
--> src/main.rs:31:5
|
31 | / fn hs_for_hmhs(&self) -> &mut HashSet<i64> {
32 | | if let None = self.hmhs.borrow().get(&0) {
33 | | self.hmhs.borrow_mut().insert(0, HashSet::new());
34 | | }
35 | |
36 | | self.hmhs.borrow_mut().get_mut(&0).unwrap()
37 | | }
| |_____^
I found a solution -- extract the HashMap as a raw pointer. This in turn means that I can get to the HashSet without shenanigans including returning a iterator.
I'm happy enough with this as a solution. The unsafe code is small and contained and if I understand the reason why the compiler is complaining without unsafe, it cannot occur in this code, since neither the HashMap nor the HashSet are ever removed or replaced after construction.
That was a lot of effort.
use std::cell::RefCell;
use std::collections::{HashMap, HashSet};
struct R {
hmhs: HashMap<i64, HashSet<i64>>,
}
impl R {
fn hs_for_hmhs(&mut self) -> &mut HashSet<i64> {
if let None = self.hmhs.get(&0) {
self.hmhs.insert(0, HashSet::new());
}
self.hmhs.get_mut(&0).unwrap()
}
fn iter_for_hmhs<'a>(&'a mut self) -> impl Iterator<Item = &'a i64> {
self.hs_for_hmhs().iter()
}
fn insert_for_hmhs(&mut self, i: i64) -> bool {
self.hs_for_hmhs().insert(i)
}
}
struct S {
hmhs: RefCell<HashMap<i64, HashSet<i64>>>,
}
impl S {
fn hs_as_ptr(&self) -> *mut HashMap<i64, HashSet<i64>> {
self.hmhs.borrow_mut().entry(0).or_insert(HashSet::new());
self.hmhs.as_ptr()
}
fn mut_hs_for_hmhs(&mut self) -> &mut HashSet<i64> {
unsafe { (*self.hs_as_ptr()).get_mut(&0).unwrap() }
}
fn hs_for_hmhs(&self) -> &HashSet<i64> {
unsafe { (*self.hs_as_ptr()).get(&0).unwrap() }
}
fn iter_for_hmhs<'a>(&'a self) -> impl Iterator<Item = &'a i64> + 'a {
self.hs_for_hmhs().iter()
}
fn insert_for_hmhs(&mut self, i: i64) -> bool {
self.mut_hs_for_hmhs().insert(i)
}
}
fn main() {
let mut r = R {
hmhs: HashMap::new(),
};
let mut s = S {
hmhs: RefCell::new(HashMap::new()),
};
r.insert_for_hmhs(10);
s.insert_for_hmhs(20);
println!("r next: {:?}", r.iter_for_hmhs().next());
println!("s next: {:?}", s.iter_for_hmhs().next());
}
https://play.rust-lang.org/?gist=3ed1977bdd5f9f82d144fe128f618979&version=stable&mode=debug&edition=2015