Nested async function call causes lifetime issue - rust

I feel like I am totally lost; what does "'1" mean in the error message?
error[E0597]: `o` does not live long enough
--> src/main.rs:32:19
|
31 | async fn foo6(&mut self, mut o: &'a mut Outer) {
| --------- lifetime `'1` appears in the type of `self`
32 | self.foo5(&mut o).await
| ----------^^^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `o` is borrowed for `'1`
33 | }
| - `o` dropped here while still borrowed
What can I do to make o live long enough? I think I'm using futures 03.
use futures::future::BoxFuture;
use futures::FutureExt;
use futures::stream::FuturesOrdered;
struct Inner {}
impl Inner {
async fn foo3(&mut self) -> Result<u32, ()> {
Ok(8)
}
}
// --- 1
struct Outer {
i: Inner,
}
impl Outer {
fn foo4(&mut self) -> BoxFuture<'_, Result<u32, ()>> {
self.i.foo3().boxed()
}
}
/// --- 2
struct Outer2<'a> {
futures_list: FuturesOrdered<BoxFuture<'a, Result<u32, ()>>>,
}
impl <'a> Outer2<'a> {
async fn foo6(&mut self, mut o: &'a mut Outer) {
self.foo5(&mut o).await
}
async fn foo5(&mut self, o: &'a mut Outer) {
self.futures_list.push(o.foo4());
}
}
#[tokio::main]
async fn main() {
let mut o = Outer { i: Inner {} };
let mut o2 = Outer2 { futures_list: FuturesOrdered::new() };
o2.foo5(&mut o).await;
}
playground

The '1 represents the anonymous lifetime of the self parameter for function foo5.
Note that in foo6, o is already a mutable reference to Outer, so writing &mut o actually gives you a mutable reference to a mutable reference to Outer with one too many indirections. You can fix your code by removing the extra reference:
async fn foo6(&mut self, mut o: &'a mut Outer) {
self.foo5(o).await
}
Playground

Related

Rust multiple borrowing struct

I am new to Rust and trying to implement a simple TCP-Server that sends me a live image. Therefore I am using this crate https://crates.io/crates/tinybmp for the RawBmp-type. Now I am struggling a bit to understand some of Rusts functionality. Or let's say I am still trying to figure out how to accomplish this in the 'Rust way'.
pub struct TCPBasePackage {
haspayload: bool,
pkgbytes: Vec<u8>,
}
impl TCPBasePackage {
pub const CMD: i32 = 0;
pub fn new() -> TCPBasePackage {
TCPBasePackage { haspayload: false, pkgbytes: Vec::new()}
}
}
impl TCPPackage for TCPBasePackage {
fn serialize(&mut self) -> &[u8] {
self.pkgbytes.append(&mut self.get_command().to_be_bytes().to_vec());
self.pkgbytes.append(&mut [u8::from(self.haspayload)].to_vec());
self.pkgbytes.as_slice()
}
fn read_from_stream(&mut self, mut stream: &TcpStream) -> Vec<u8> {
let mut vec: Vec<u8> = Vec::new();
let mut buf_pkg: [u8; 5] = [0; 5];
stream.read_exact(&mut buf_pkg).unwrap();
vec.append(&mut buf_pkg.to_vec());
self.pkgbytes = vec.to_vec();
return self.pkgbytes.clone();
}
fn deserialize(&mut self, buff: &[u8]) -> i32 {
self.haspayload = buff[4] != 0;
return 5;
}
fn get_command(&self) -> i32 {
TCPBasePackage::CMD
}
}
pub struct MsgLiveImage<'a> {
tcppackage: TCPBasePackage,
imgbytes: Vec::<u8>,
img: RawBmp<'a>,
}
impl<'a> MsgLiveImage<'a> {
pub const CMD: i32 = 2;
pub fn new() -> MsgLiveImage<'static> {
MsgLiveImage {
tcppackage: TCPBasePackage { haspayload: false, pkgbytes: Vec::new() },
imgbytes: Vec::new(),
img: RawBmp::from_slice(&[]).unwrap()
}
}
pub fn set_image_data(&'a mut self, data: Vec::<u8>) {
self.imgbytes = data;
self.img = RawBmp::from_slice(&self.imgbytes.as_slice()).unwrap();
}
}
In some function of my application I want to do this:
fn write_response_to_stream(
mut request: &Box<dyn mytcp::TCPPackage>,
mut stream: &TcpStream,
raspicam: &Arc<Mutex<camserv::CamServ>>,
) -> Result<Box<dyn mytcp::TCPPackage>, &'static str> {
match request.get_command() {
mytcp::MsgLiveImage::CMD => {
let mut pkg = mytcp::MsgLiveImage::new();
{
{
let tmpcam = raspicam.lock().unwrap(); // lock the camera server
let tmpbuff = tmpcam.imgbuff.lock().unwrap(); // lock the buffer of last image
pkg.set_image_data((*tmpbuff).clone()); // make copy of my image and transfer ownership of that copy to the package struct
}
{
let sbuff = pkg.serialize();
stream.write(&sbuff);
}
stream.flush();
}
Ok(Box::new(pkg)) // return the response package
}
_ => Err("Unknown request package"),
}
}
But the problem is, that the compiler is complaining about multiple borrowing the pkg-variable. As well as moving the pkg out of scope (which i learned should be possible by wrapping it in a Box.
Can someone explain me why the first mutable borrow still lasts even after the method returns? How can i achieve to call multiple methods on my struct without getting these borrowing conflicts?
Errors from rustc:
error[E0499]: cannot borrow `pkg` as mutable more than once at a time
--> src/raspiserv/mod.rs:90:33
|
87 | pkg.set_image_data((*tmpbuff).clone());
| -------------------------------------- first mutable borrow occurs here
...
90 | let sbuff = pkg.serialize();
| ^^^^^^^^^^^^^^^ second mutable borrow occurs here
...
95 | Ok(Box::new(pkg))
| ----------------- returning this value requires that `pkg` is borrowed for `'static`
error[E0515]: cannot return value referencing local variable `pkg`
--> src/raspiserv/mod.rs:95:13
|
87 | pkg.set_image_data((*tmpbuff).clone());
| -------------------------------------- `pkg` is borrowed here
...
95 | Ok(Box::new(pkg))
| ^^^^^^^^^^^^^^^^^ returns a value referencing data owned by the current function
error[E0505]: cannot move out of `pkg` because it is borrowed
--> src/raspiserv/mod.rs:95:25
|
87 | pkg.set_image_data((*tmpbuff).clone());
| -------------------------------------- borrow of `pkg` occurs here
...
95 | Ok(Box::new(pkg))
| ------------^^^--
| | |
| | move out of `pkg` occurs here
| returning this value requires that `pkg` is borrowed for `'static`
The problem is with your set_image_data
pub fn set_image_data(&'a mut self, data: Vec::<u8>) {
self.imgbytes = data;
self.img = RawBmp::from_slice(&self.imgbytes.as_slice()).unwrap();
}
you borrow self for the lifetime of 'a which is as long as the RawBmp inside of it is valid.
Thus the borrow lasts as long as the struct.
What you have is a self referential struct see this question on them for suggestions how to reslove this.
You can probably just drop the img from MsgLiveImage and replace it with a method:
impl MsgLiveImage {
pub fn as_img<'a>(&'a self) -> Result<RawBmp<'a>, ParseError> {
RawBmp::from_slice(&self.imgbytes.as_slice())
}
}
Might even be able to omit the lifetimes from that.

Rust borrow checker and early returns

Rust-lang Playground link
struct Foo {
val: i32
}
impl Foo {
pub fn maybe_get(&mut self) -> Option<&mut i32> {
Some(&mut self.val)
}
pub fn definitely_get(&mut self) -> &mut i32 {
{ // Add closure to ensure things have a chance to get dropped
if let Some(val) = self.maybe_get() {
// Explicit return to avoid potential scope sharing with an else block or a match arms.
return val;
}
}
// One would think any mutable references would not longer be at play at this point
&mut self.val
}
}
I have some code that's similar but more complicated than what is provided above that I've been fighting with for quite a while. The borrow checker is unhappy with the implementation of definitely_get and has the following error
error[E0499]: cannot borrow `self.val` as mutable more than once at a time
--> src/main.rs:19:9
|
10 | pub fn definitely_get(&mut self) -> &mut i32 {
| - let's call the lifetime of this reference `'1`
11 | {
12 | if let Some(val) = self.maybe_get() {
| ---------------- first mutable borrow occurs here
13 | return val;
| --- returning this value requires that `*self` is borrowed for `'1`
...
19 | &mut self.val
| ^^^^^^^^^^^^^ second mutable borrow occurs here
It seems unreasonable for there to be no way to implement fallback logic with a mutable reference in Rust so I can't imagine there isn't a way.
I've managed to fix this with an unfortunately expensive alternative implementation due to how maybe_get is implemented in my non-trivial example.
impl Foo {
pub fn has_maybe_val(&self) -> bool {
// Non-trivial lookup...
true
}
pub fn maybe_get(&mut self) -> Option<&mut i32> {
// Same non-trivial lookup...
Some(&mut self.val)
}
pub fn definitely_get(&mut self) -> &mut i32 {
if self.has_maybe_val() {
self.maybe_get().unwrap() // Ouch!
} else {
&mut self.val
}
}
}

Multiple mutable borrows in Rust

I'm playing around with building a very simple stack based evaluator in Rust. I want the user to be able to define functions later, so I'm storing all operators in a HashMap with closures as values.
use std::collections::HashMap;
pub type Value = i32;
pub struct Evaluator<'a> {
stack: Vec<Value>,
ops: HashMap<String, &'a dyn FnMut(&'a mut Vec<Value>)>,
}
impl<'a> Evaluator<'a> {
pub fn new() -> Evaluator<'a> {
let stack: Vec<Value> = vec![];
let mut ops: HashMap<String, &'a dyn FnMut(&'a mut Vec<Value>)> = HashMap::new();
ops.insert("+".to_string(), &|stack: &'a mut Vec<Value>| {
if let (Some(x), Some(y)) = (stack.pop(), stack.pop()) {
stack.push(y + x);
}
});
Evaluator { stack, ops }
}
pub fn stack(&self) -> &[Value] {
&self.stack
}
pub fn eval(&'a mut self, input: &str) {
let symbols = input
.split_ascii_whitespace()
.collect::<Vec<_>>();
for sym in symbols {
if let Ok(n) = sym.parse::<i32>() {
self.stack.push(n);
} else {
let s = sym.to_ascii_lowercase();
if let Some(f) = self.ops.get(&s) {
f(&mut self.stack);
} else {
println!("error");
}
}
}
}
}
fn main() {
let mut e = Evaluator::new();
e.eval("1 2 +")
}
I'm currently getting two errors:
error[E0499]: cannot borrow `self.stack` as mutable more than once at a time
--> src/sample.rs:34:17
|
10 | impl<'a> Evaluator<'a> {
| -- lifetime `'a` defined here
...
34 | self.stack.push(n);
| ^^^^^^^^^^ second mutable borrow occurs here
...
38 | f(&mut self.stack);
| ------------------
| | |
| | first mutable borrow occurs here
| argument requires that `self.stack` is borrowed for `'a`
error[E0596]: cannot borrow `**f` as mutable, as it is behind a `&` reference
--> src/sample.rs:38:21
|
38 | f(&mut self.stack);
| ^ cannot borrow as mutable
error[E0499]: cannot borrow `self.stack` as mutable more than once at a time
--> src/sample.rs:38:23
|
10 | impl<'a> Evaluator<'a> {
| -- lifetime `'a` defined here
...
38 | f(&mut self.stack);
| --^^^^^^^^^^^^^^^-
| | |
| | `self.stack` was mutably borrowed here in the previous iteration of the loop
| argument requires that `self.stack` is borrowed for `'a`
error: aborting due to 3 previous errors
Some errors have detailed explanations: E0499, E0596.
For more information about an error, try `rustc --explain E0499`.
My concern is the first one. I'm not sure what I'm doing wrong as I'm not borrowing them at the same time. Can I tell Rust the previous borrow (self.stack.pop()) is done? Any help appreciated.
I think I solved my problem. The thing I kept coming back to is, "What owns the closures?" In this case I'm using references, but nothing is taking ownership of the data. When I refactored (below) with Box to take ownership, it worked.
I'm curious if there is a way to do this with with just references and/or if my explanation is wrong?
Working code:
use std::collections::HashMap;
pub type Value = i32;
pub struct Evaluator {
stack: Vec<Value>,
ops: HashMap<String, Box<dyn FnMut(&mut Vec<Value>)>>,
}
impl Evaluator {
pub fn new() -> Evaluator {
let stack: Vec<Value> = vec![];
let mut ops: HashMap<String, Box<dyn FnMut(&mut Vec<Value>)>> = HashMap::new();
ops.insert("+".to_string(), Box::new(|stack: &mut Vec<Value>| {
if let (Some(x), Some(y)) = (stack.pop(), stack.pop()) {
stack.push(y + x);
}
}));
Evaluator { stack, ops }
}
pub fn stack(&self) -> &[Value] {
&self.stack
}
pub fn eval(&mut self, input: &str) {
let symbols = input
.split_ascii_whitespace()
.collect::<Vec<_>>();
for sym in symbols {
if let Ok(n) = sym.parse::<i32>() {
self.stack.push(n);
} else {
let s = sym.to_ascii_lowercase();
if let Some(f) = self.ops.get_mut(&s) {
f(&mut self.stack);
} else {
println!("error");
}
}
}
}
}
fn main() {
let mut e = Evaluator::new();
e.eval("1 2 +")
}
You have borrows with conflicting lifetimes:
You are defining a lifetime 'a for the struct in line 5: pub struct Evaluator<'a> {
In line 7, you are stating that ops is a HashMap that holds functions that receive mutable borrows for the whole duration of 'a
Then, in line 28, you are defining an eval method that holds a mutable reference to self for the whole duration of the struct ('a)
The conflict can be solved if you use two different lifetimes, since the time that an operation borrows self should be inherently shorter than the lifetime for the whole evaluation, since in eval you are running a loop and multiple invocations to the operations.
This should fix the issues mentioned above:
pub struct Evaluator<'a, 'b> {
stack: Vec<Value>,
ops: HashMap<String, &'b dyn FnMut(&'b mut Vec<Value>)>,
}
impl<'a, 'b> Evaluator<'a, 'b> {
pub fn new() -> Evaluator<'a> {
let stack: Vec<Value> = vec![];
let mut ops: HashMap<String, &'b dyn FnMut(&'b mut Vec<Value>)> = HashMap::new();

Cannot borrow as mutable in loop

I have the following code:
pub fn read_packet<'a>(buf: &'a mut [u8]) -> &'a [u8] {
loop {
read_exact(buf);
if let Some(packet) = to_packet(buf) {
return packet;
}
}
}
fn read_exact(_: &mut [u8]) {
todo!()
}
fn to_packet<'a>(_: &'a [u8]) -> Option<&'a [u8]> {
todo!()
}
I get the following error:
error[E0502]: cannot borrow `*buf` as mutable because it is also borrowed as immutable
--> src/lib.rs:3:9
|
1 | pub fn read_packet<'a>(buf: &'a mut [u8]) -> &'a [u8] {
| -- lifetime `'a` defined here
2 | loop {
3 | read_exact(buf);
| ^^^^^^^^^^^^^^^ mutable borrow occurs here
4 |
5 | if let Some(packet) = to_packet(buf) {
| --- immutable borrow occurs here
6 | return packet;
| ------ returning this value requires that `*buf` is borrowed for `'a`
I think it should work because:
The mutable borrow in read_exact completes on line 3.
If to_packet returns Some then the value is returned to the caller.
If not, the immutable borrow of to_packet is over at the end of the loop. So it is free to be taken mutable borrow of in the next iteration.
Can somebody please let me know why this doesn't work?
EDIT:
It seems like a current borrow checker limitation. I tried using Polonius in the nightly and it works fine with
RUSTFLAGS=-Zpolonius cargo +nightly check
It is a compiler limitation atm.
You could refactor to something like:
pub fn read_packet<'a>(buf: &'a mut [u8]) {
loop {
if read_exact(buf) {
break;
}
}
}
fn is_packet(a: &[u8]) -> bool {
true
}
fn read_exact<'a>(a: &'a mut [u8]) -> bool {
is_packet(a)
}
fn to_packet<'a>(_: &'a [u8]) -> Option<&'a [u8]> {
todo!()
}
fn process_packet<'a>(buf: &'a mut [u8]) {
read_packet(buf);
let _packet = to_packet(buf);
}
Playground
pub fn read_packet<'a>(buffer: &'a mut [u8]) -> &'a [u8] { |
let buf = buffer; |
loop { |
read_exact(buf); \/
if let Some(packet) = to_packet(buf) { 'a _
return packet; |
} |
} |
}
fn read_exact(_: &mut [u8]) {
todo!()
}
fn to_packet<'b>(_from: &'b [u8]) -> Option<&'b [u8]> {
todo!()
}
Compile error:
|
1 | pub fn read_packet<'a>(buffer: &'a mut [u8]) -> &'a [u8] {
| -- lifetime `'a` defined here
...
4 | read_exact(buf);
| ^^^^^^^^^^^^^^^ mutable borrow occurs here
5 | if let Some(packet) = to_packet(buf) {
| --- immutable borrow occurs here
6 | return packet;
| ------ returning this value requires that `*buf` is borrowed for `'a`
With Non Lexical Lifetime (NLL):
The return statement constrains the packet lifetime to be 'a.
If packet is 'a, so buf (to_packet) has to be also 'a.
'a is valid for the entire function. The loop makes the shared reference lifetime conflicts with the exclusive reference lifetime at the next iteration.
The conflict can be reproduced without the loop. This snippet doesn't compile for the same reason that buff is 'a. And we see that the root cause is the conditional return. Again 'a has to be valid for the entire function.
pub fn read_packet<'a>(buf: &'a mut [u8]) -> &'a [u8] {
if let Some(packet) = to_packet(buf) { 'a _
return packet; |
} |
read_exact(buf); |
return &[0]; \/
}
fn read_exact(_: &mut [u8]) {
todo!()
}
fn to_packet<'b>(_: &'b [u8]) -> Option<&'b [u8]> {
todo!()
}
Compile error:
|
1 | pub fn read_packet<'a>(buf: &'a mut [u8]) -> &'a [u8] {
| -- lifetime `'a` defined here
2 | if let Some(packet) = to_packet(buf) {
| --- immutable borrow occurs here
3 | return packet;
| ------ returning this value requires that `*buf` is borrowed for `'a`
4 | }
5 | read_exact(buf);
| ^^^^^^^^^^^^^^^ mutable borrow occurs here
With NLL, the lifetime is infered like you thought it would be.
pub fn read_packet<'a>(buffer: &'a mut [u8]) -> &'a [u8] {
let buf = buffer;
loop {
read_exact(buf);
if let Some(packet) = to_packet(buf) { 'x _
return packet; |
} \/
}
}
fn read_exact(_: &mut [u8]) {
todo!()
}
fn to_packet<'b>(_from: &'b [u8]) -> Option<&'b [u8]> {
todo!()
}

Store async callback in map

I'm trying to store multiple async callbacks in a single hash map which I later iterate over. This is my closest attempt to getting it working however, I'm getting a compilation error because the lifetime of the callback.
use std::boxed::Box;
use std::collections::HashMap;
use std::future::Future;
use std::pin::Pin;
type AsyncCallback = Box<dyn Fn() -> Pin<Box<dyn Future<Output = ()>>>>;
#[derive(Default)]
pub struct CallbackMap {
callbacks: HashMap<&'static str, AsyncCallback>,
}
impl CallbackMap {
pub fn add<C, F>(&mut self, name: &'static str, callback: C)
where
C: Fn() -> F,
C: 'static,
F: Future<Output = ()> + 'static,
{
self.callbacks
.insert(name, Box::new(|| Box::pin(callback())));
}
pub async fn execute(&self) {
for (_key, value) in &self.callbacks {
value().await;
}
}
}
async fn async_callback() {
println!("Callback 2");
}
fn main() {
let mut callbacks = CallbackMap::default();
callbacks.add("test1", || async {
println!("Callback 1");
});
callbacks.add("test2", async_callback);
callbacks.execute();
}
Here is the error I'm getting:
error[E0597]: `callback` does not live long enough
--> src/main.rs:21:48
|
21 | .insert(name, Box::new(|| Box::pin(callback())));
| ---------------------^^^^^^^^----
| | | |
| | | borrowed value does not live long enough
| | value captured here
| cast requires that `callback` is borrowed for `'static`
22 | }
| - `callback` dropped here while still borrowed
Use the move keyword to move callback into the closure:
self.callbacks
.insert(name, Box::new(move || Box::pin(callback())));
// ^^^^

Resources