Conflicting lifetimes inside closure - rust

I'm trying to create a trait for a Decoder that can return a packet that contains a reference inside. That is, I don't need to copy the packet to an owned object just so it's 'static.
Here's the sketch:
use std::sync::{Arc, Mutex};
pub trait DecodedPacket<'a>: Send {}
pub type OnConsume = Arc<dyn for<'a> Fn(&'a mut Option<Box<dyn DecodedPacket<'a>>>) + Send + Sync>;
pub trait Decoder: Send{
fn receive_ref(&self,on_packet: Arc<dyn for<'a> FnMut(Box<dyn DecodedPacket<'a>>)>);
}
pub struct DummyDecoder {}
impl Decoder for DummyDecoder {
fn receive_ref(&self,_: Arc<dyn for<'a> FnMut(Box<dyn DecodedPacket<'a>>)>){
unimplemented!();
}
}
struct DummyRenderer {
on_consume: OnConsume
}
impl DummyRenderer {
pub fn render(&self) {
let mut decoded_packet: Option<Box<dyn DecodedPacket<'_>>> = None;
let packet = (self.on_consume)(&mut decoded_packet);
//Render packet somehow here
}
}
fn main() {
let decoder: Box<dyn Decoder + Sync> = Box::new(DummyDecoder{});
let renderer_on_consume: OnConsume =
Arc::new(move |p: &mut Option<Box<dyn DecodedPacket>>| {
let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> =
Arc::new(Mutex::new(None));
let on_packet = Arc::new(move |packet: Box<dyn DecodedPacket>|{
decoded_packet.lock().unwrap().replace(packet);
});
decoder.receive_ref(on_packet);
if let Some(packet) = *decoded_packet.lock().unwrap() {
p.replace(packet);
}
});
let dummy_renderer = DummyRenderer{
on_consume: renderer_on_consume
};
dummy_renderer.render();
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=89a181ae763d220199be91c9cb552e80
You can see that receive_ref expects a function that can deal with all lifetimes, so it should work for a small non 'static lifetime. However, the compiler is making all the lifetimes 'static:
Error:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
--> src/main.rs:35:58
|
35 | let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> =
| ^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 34:18...
--> src/main.rs:34:18
|
34 | Arc::new(move |p: &mut Option<Box<dyn DecodedPacket>>| {
| __________________^
35 | | let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> =
36 | | Arc::new(Mutex::new(None));
37 | | let on_packet = Arc::new(move |packet: Box<dyn DecodedPacket>|{
... |
43 | | }
44 | | });
| |_________^
note: ...but the lifetime must also be valid for the anonymous lifetime #1 defined on the body at 37:38...
--> src/main.rs:37:38
|
37 | let on_packet = Arc::new(move |packet: Box<dyn DecodedPacket>|{
| ______________________________________^
38 | | decoded_packet.lock().unwrap().replace(packet);
39 | | });
| |_____________^
note: ...so that the types are compatible
--> src/main.rs:42:27
|
42 | p.replace(packet);
| ^^^^^^
= note: expected `dyn DecodedPacket<'_>`
found `dyn DecodedPacket<'_>`
ps: I don't like the let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> = Arc::new(Mutex::new(None)); but I had to do it otherwise decoded_packet was getting borrowed by the closure and thus forcing the closure lifetime to be 'static

Related

Lifetime for Cursor in Iteration

use std::io::Cursor;
use bytes::BytesMut;
struct Parser<'a> {
stream: &'a mut Cursor<&'a [u8]>
}
fn parse<'a>(buf:&'a mut Cursor<&'a [u8]>) -> Option<(usize, &'a str)> {
None
}
impl<'a> Iterator for Parser<'a> {
type Item = (usize, &'a str);
fn next(&mut self) -> Option<Self::Item> {
parse(self.stream)
}
}
fn main() {
let mut buf = BytesMut::with_capacity(1024);
let mut cursor = Cursor::new(&buf[..]);
let mut parser = Parser {stream: &mut cursor};
}
Playground link
Basically, I want to parse a buf without copy but failed to compile. Could anyone help? Much appreciated!
Error message:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
--> src/main.rs:15:9
|
15 | parse(self.stream)
| ^^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 14:5...
--> src/main.rs:14:5
|
14 | fn next(&mut self) -> Option<Self::Item> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:15:15
|
15 | parse(self.stream)
| ^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 12:6...
--> src/main.rs:12:6
|
12 | impl<'a> Iterator for Parser<'a> {
| ^^
note: ...so that the expression is assignable
--> src/main.rs:15:15
|
15 | parse(self.stream)
| ^^^^^^^^^^^
= note: expected `&mut std::io::Cursor<&[u8]>`
found `&mut std::io::Cursor<&'a [u8]>`
You need a different lifetime for what is inside the cursor:
struct Parser<'a, 'b> {
stream: &'a mut Cursor<&'b [u8]>,
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0c36650015dc7fb4d425794fca934242
The reason is to do with the fact that your stream is a mutable reference. If you make it immutable, everything works.
I could not explain why better than Ryan Levick did in his video he made on lifetimes.
That video should be one of the recommended learning materials 😃

Desugared recursive async method

I'm trying to write desugared recursive async method. Here is my attempt:
use std::future::*;
use std::pin::*;
struct TestRecursiveAsync;
impl TestRecursiveAsync {
pub fn foo(&self, value: u32) -> Pin<Box<(dyn Future<Output = u32>)>> {
let v: Pin<Box<(dyn Future<Output = u32> + 'static)>> = Box::pin(async move {
if value > 0 {
self.foo(value - 100).await
} else {
value
}
});
v
}
}
The compiler complains with this message (Playground):
Compiling playground v0.0.1 (/playground)
error[E0759]: cannot infer an appropriate lifetime
--> src/lib.rs:8:85
|
7 | pub fn foo(&self, value: u32) -> Pin<Box<(dyn Future<Output = u32>)>> {
| ----- this data with an anonymous lifetime `'_`...
8 | let v: Pin<Box<(dyn Future<Output = u32> + 'static)>> = Box::pin(async move {
| _____________________________________________________________________________________^
9 | | if value > 0 {
10 | | self.foo(value - 100).await
11 | | } else {
12 | | value
13 | | }
14 | | });
| |_________^ ...is captured here, requiring it to live as long as `'static`
|
help: to declare that the trait object captures data from argument `self`, you can add an explicit `'_` lifetime bound
|
7 | pub fn foo(&self, value: u32) -> Pin<Box<(dyn Future<Output = u32> + '_)>> {
| ^^^^
So the compiler requires &self to have 'static lifetime, but it has an anonymous. Is there a way to fix that?

Can I reassign a mutable slice reference to a sub-slice of itself?

I'm implementing a stack-like structure, where the structure holds a mutable reference to a slice.
struct StackLike<'a, X> {
data: &'a mut [X],
}
I'd like to be able to pop the last element off this stack, something like:
impl<'a, X> StackLike<'a, X> {
pub fn pop(&mut self) -> Option<&'a X> {
if self.data.is_empty() {
return None;
}
let n = self.data.len();
let result = &self.data[n - 1];
self.data = &mut self.data[0..n - 1];
Some(result)
}
}
This fails:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> src/lib.rs:11:23
|
11 | let result = &self.data[n - 1];
| ^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 6:5...
--> src/lib.rs:6:5
|
6 | / pub fn pop(&mut self) -> Option<&'a X> {
7 | | if self.data.is_empty() {
8 | | return None;
9 | | }
... |
13 | | Some(result)
14 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:11:23
|
11 | let result = &self.data[n - 1];
| ^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 5:6...
--> src/lib.rs:5:6
|
5 | impl<'a, X> StackLike<'a, X> {
| ^^
note: ...so that the expression is assignable
--> src/lib.rs:13:9
|
13 | Some(result)
| ^^^^^^^^^^^^
= note: expected `std::option::Option<&'a X>`
found `std::option::Option<&X>`
Even a simplified version of pop that doesn't return a value and only shrinks the slice doesn't work.
impl<'a, X> StackLike<'a, X> {
pub fn pop_no_return(&mut self) {
if self.data.is_empty() {
return;
}
let n = self.data.len();
self.data = &mut self.data[0..n - 1];
}
}
which gives
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> src/lib.rs:11:26
|
11 | self.data = &mut self.data[0..n - 1];
| ^^^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 6:5...
--> src/lib.rs:6:5
|
6 | / pub fn pop_no_return(&mut self) {
7 | | if self.data.is_empty() {
8 | | return;
9 | | }
10 | | let n = self.data.len();
11 | | self.data = &mut self.data[0..n - 1];
12 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:11:26
|
11 | self.data = &mut self.data[0..n - 1];
| ^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 5:6...
--> src/lib.rs:5:6
|
5 | impl<'a, X> StackLike<'a, X> {
| ^^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:11:21
|
11 | self.data = &mut self.data[0..n - 1];
| ^^^^^^^^^^^^^^^^^^^^^^^^
Is there a way to make this work, or do I need to track the bounds of the slice I'm interested in more explicitly?
I modified Masklinn's code slightly to allow multiple .pop()s to be called on the same stack:
struct StackLike<'a, X> {
data: &'a mut [X],
}
impl<'a, X> StackLike<'a, X> {
pub fn pop(&mut self) -> Option<&'a mut X> {
let data = std::mem::replace(&mut self.data, &mut []);
if let Some((last, subslice)) = data.split_last_mut() {
self.data = subslice;
Some(last)
} else {
None
}
}
}
fn main() {
let mut data = [1, 2, 3, 4, 5];
let mut stack = StackLike { data: &mut data };
let x = stack.pop().unwrap();
let y = stack.pop().unwrap();
println!("X: {}, Y: {}", x, y);
}
The tricky part here is this line (I added a type annotation for explicitness):
let data: &'a mut [X] = std::mem::replace(&mut self.data, &mut []);
We replace self.data with an empty slice temporarily so that we can split the slice. If you were to write simply
let data: &'a mut [X] = self.data;
the compiler would be unhappy:
error[E0312]: lifetime of reference outlives lifetime of borrowed content...
--> src/main.rs:7:33
|
7 | let data: &'a mut [X] = self.data;
| ^^^^^^^^^
|
note: ...the reference is valid for the lifetime `'a` as defined on the impl at 5:6...
--> src/main.rs:5:6
|
5 | impl<'a, X> StackLike<'a, X> {
| ^^
note: ...but the borrowed content is only valid for the anonymous lifetime #1 defined on the method body at 6:5
--> src/main.rs:6:5
|
6 | / pub fn pop(&mut self) -> Option<&'a mut X> {
7 | | let data: &'a mut [X] = self.data;
8 | | if let Some((last, subslice)) = data.split_last_mut() {
9 | | self.data = subslice;
... |
13 | | }
14 | | }
| |_____^
As far as I understand it, the problem is that self.data is a mutable reference, and mutable references are not Copy (remember, you can only have one at a time). And you cannot move out of self.data since self is a mutable reference, not an owner. So what the compiler tries to do is to reborrow self.data, which "infects" it with with the lifetime of &mut self. This is a dead end: we want the reference to live for 'a, but it is actually valid only for the lifetime of &mut self, and these lifetimes are generally unrelated (and they don't need to be related), which leaves the compiler perplexed.
To aid the compiler, we use std::mem::replace to explicitly move the slice out of self.data and temporarily replace it with an empty slice, which can be any lifetime. Now we can do anything with data without tangling it with the lifetime of &mut self.
For sub-question 2, you need to indicate the relation between &mut self and 'a otherwise they're considered unrelated. I don't know if there's a shortcut via lifetime elision but if you specify that self lives for 'a you're fine.
For sub-question 1, the compiler doesn't "see through" function calls (including indexing which desugars to a function call), so it does not know that &self.data[n - 1] and &mut self.data[0..n-1] are non-overlapping. You need to use split_mut_last.
struct StackLike<'a, X> {
data: &'a mut [X],
}
impl<'a, X> StackLike<'a, X> {
pub fn pop(&'a mut self) -> Option<&'a X> {
if let Some((last, subslice)) = self.data.split_last_mut() {
self.data = subslice;
Some(last)
} else {
None
}
}
}
playground

Create a struct pointing to another

How can I create the Item instance pointing to the database received as parameter?
struct Something {}
struct Database<'a> {
something: &'a Something,
}
struct Item<'a> {
database: &'a mut Database<'a>,
}
impl<'a> Item<'a> {
fn new(database: &'a mut Database) -> Self {
let mut obj = Self { database };
obj
}
}
This produces the error:
error[E0308]: mismatched types
--> src/main.rs:16:13
|
16 | database
| ^^^^^^^^ lifetime mismatch
|
= note: expected type `&'a mut Database<'a>`
found type `&'a mut Database<'_>`
note: the lifetime 'a as defined on the impl at 13:1...
--> src/main.rs:13:1
|
13 | / impl<'a> Item<'a> {
14 | | fn new(database: &'a mut Database) -> Self {
15 | | let mut obj = Self {
16 | | database
... |
20 | | }
21 | | }
| |_^
note: ...does not necessarily outlive the anonymous lifetime #1 defined on the method body at 14:5
--> src/main.rs:14:5
|
14 | / fn new(database: &'a mut Database) -> Self {
15 | | let mut obj = Self {
16 | | database
17 | | };
18 | |
19 | | obj
20 | | }
| |_____^
I do not understand if the error is in the return type (Self) or another thing.
You defined Database to require a lifetime parameter. That means you also have to give it one. The error message is really helpful here and your problem is fixed by simply doing what it says:
expected type `&'a mut Database<'a>`
found type `&'a mut Database<'_>`
It suffices to simply add <'a> to Database in the signature of new.
N.B.: Always read error messages from top to bottom. Usually by resolving the first error all the successive ones disappear.
struct Something {}
struct Database<'a> {
something: &'a Something
}
struct Item<'a> {
database: &'a mut Database<'a>
}
impl<'a> Item<'a> {
fn new(database: &'a mut Database<'a>) -> Self {
let mut obj = Self {
database
};
obj
}
}
fn main() {
let something = &Something {};
let mut database = Database { something };
let item = Item::new(&mut database);
}
Playground

Questions about Rust lifetime

I'm trying to implement a memory pool based on TypedArena. Here's a simplified version of my original code:
#![feature(rustc_private)]
extern crate arena;
use arena::TypedArena;
pub struct MemoryPool {
arena: TypedArena<Vec<u8>>,
bytes_allocated: usize,
}
impl MemoryPool {
pub fn consume(&mut self, buf: Vec<u8>) -> &[u8] {
self.bytes_allocated += buf.capacity();
self.arena.alloc(buf)
}
}
pub struct ByteArray<'a> {
data: &'a [u8],
}
impl<'a> ByteArray<'a> {
pub fn set_data(&mut self, data: &'a [u8]) {
self.data = data;
}
}
pub struct S<'a> {
pool: &'a mut MemoryPool,
}
impl<'a> S<'a> {
pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
let v = vec!();
let data = self.pool.consume(v);
buffer.set_data(data);
}
}
However, the compiler complains about the line: let data = self.pool.consume(v);:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> <anon>:34:26
|
34 | let data = self.pool.consume(v);
| ^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 32:54...
--> <anon>:32:55
|
32 | pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
| _______________________________________________________^ starting here...
33 | | let v = vec!();
34 | | let data = self.pool.consume(v);
35 | | buffer.set_data(data);
36 | | }
| |___^ ...ending here
note: ...so that reference does not outlive borrowed content
--> <anon>:34:16
|
34 | let data = self.pool.consume(v);
| ^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the body at 32:54...
--> <anon>:32:55
|
32 | pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
| _______________________________________________________^ starting here...
33 | | let v = vec!();
34 | | let data = self.pool.consume(v);
35 | | buffer.set_data(data);
36 | | }
| |___^ ...ending here
note: ...so that types are compatible (expected &mut ByteArray<'_>, found &mut ByteArray<'a>)
--> <anon>:35:12
|
35 | buffer.set_data(data);
| ^^^^^^^^
My question is:
Why data does not have lifetime 'a? I'm thinking that since pool has lifetime a and consume returns the same lifetime as self, it should have lifetime 'a.
What's the best way to make this code work as intended? Basically I want to allocate new bytes and adjust their lifetime to be the same as the memory pool. I know I can use TypedArena directly since alloc does not take a mut reference. However I really want to track other information such as bytes_allocated.
Let's tackle this step by step:
cannot infer an appropriate lifetime for autoref
"autoref" describes the process of building the right reference for the self argument of a method. The compiler is unable to find a reference with the right lifetime to call consume(). Why is it unable?
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 32:54...
--> <anon>:32:55
|
32 | pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
| _______________________________________________________^ starting here...
33 | | let v = vec!();
34 | | let data = self.pool.consume(v);
35 | | buffer.set_data(data);
36 | | }
| |___^ ...ending here
note: ...so that reference does not outlive borrowed content
--> <anon>:34:16
|
34 | let data = self.pool.consume(v);
| ^^^^^^^^^
The "anonymous lifetime #1" refers to the lifetime of &mut self. This note is just saying: we can't pass a reference with a lifetime greater than the lifetime of self into consume(): then consume() would think that its self argument lives longer than it actually does.
note: but, the lifetime must be valid for the lifetime 'a
This is the rule you already expected to be applied. But where is the problem now? Well: the lifetime of &mut self (anonymous lifetime #1) could life shorter than 'a! That's all! And we can fix it quite easily:
impl<'a> S<'a> {
pub fn write<'b: 'a>(&'b mut self, buffer: &mut ByteArray<'a>) {
// ^^^^^^^^ ^^
...
}
}
Here we just name the previously anonymous lifetime #1 to be able to bound it, saying that it has to outlive 'a (live longer than 'a).

Resources