Questions about Rust lifetime - rust

I'm trying to implement a memory pool based on TypedArena. Here's a simplified version of my original code:
#![feature(rustc_private)]
extern crate arena;
use arena::TypedArena;
pub struct MemoryPool {
arena: TypedArena<Vec<u8>>,
bytes_allocated: usize,
}
impl MemoryPool {
pub fn consume(&mut self, buf: Vec<u8>) -> &[u8] {
self.bytes_allocated += buf.capacity();
self.arena.alloc(buf)
}
}
pub struct ByteArray<'a> {
data: &'a [u8],
}
impl<'a> ByteArray<'a> {
pub fn set_data(&mut self, data: &'a [u8]) {
self.data = data;
}
}
pub struct S<'a> {
pool: &'a mut MemoryPool,
}
impl<'a> S<'a> {
pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
let v = vec!();
let data = self.pool.consume(v);
buffer.set_data(data);
}
}
However, the compiler complains about the line: let data = self.pool.consume(v);:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> <anon>:34:26
|
34 | let data = self.pool.consume(v);
| ^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 32:54...
--> <anon>:32:55
|
32 | pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
| _______________________________________________________^ starting here...
33 | | let v = vec!();
34 | | let data = self.pool.consume(v);
35 | | buffer.set_data(data);
36 | | }
| |___^ ...ending here
note: ...so that reference does not outlive borrowed content
--> <anon>:34:16
|
34 | let data = self.pool.consume(v);
| ^^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the body at 32:54...
--> <anon>:32:55
|
32 | pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
| _______________________________________________________^ starting here...
33 | | let v = vec!();
34 | | let data = self.pool.consume(v);
35 | | buffer.set_data(data);
36 | | }
| |___^ ...ending here
note: ...so that types are compatible (expected &mut ByteArray<'_>, found &mut ByteArray<'a>)
--> <anon>:35:12
|
35 | buffer.set_data(data);
| ^^^^^^^^
My question is:
Why data does not have lifetime 'a? I'm thinking that since pool has lifetime a and consume returns the same lifetime as self, it should have lifetime 'a.
What's the best way to make this code work as intended? Basically I want to allocate new bytes and adjust their lifetime to be the same as the memory pool. I know I can use TypedArena directly since alloc does not take a mut reference. However I really want to track other information such as bytes_allocated.

Let's tackle this step by step:
cannot infer an appropriate lifetime for autoref
"autoref" describes the process of building the right reference for the self argument of a method. The compiler is unable to find a reference with the right lifetime to call consume(). Why is it unable?
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 32:54...
--> <anon>:32:55
|
32 | pub fn write(&mut self, buffer: &mut ByteArray<'a>) {
| _______________________________________________________^ starting here...
33 | | let v = vec!();
34 | | let data = self.pool.consume(v);
35 | | buffer.set_data(data);
36 | | }
| |___^ ...ending here
note: ...so that reference does not outlive borrowed content
--> <anon>:34:16
|
34 | let data = self.pool.consume(v);
| ^^^^^^^^^
The "anonymous lifetime #1" refers to the lifetime of &mut self. This note is just saying: we can't pass a reference with a lifetime greater than the lifetime of self into consume(): then consume() would think that its self argument lives longer than it actually does.
note: but, the lifetime must be valid for the lifetime 'a
This is the rule you already expected to be applied. But where is the problem now? Well: the lifetime of &mut self (anonymous lifetime #1) could life shorter than 'a! That's all! And we can fix it quite easily:
impl<'a> S<'a> {
pub fn write<'b: 'a>(&'b mut self, buffer: &mut ByteArray<'a>) {
// ^^^^^^^^ ^^
...
}
}
Here we just name the previously anonymous lifetime #1 to be able to bound it, saying that it has to outlive 'a (live longer than 'a).

Related

Conflicting lifetimes inside closure

I'm trying to create a trait for a Decoder that can return a packet that contains a reference inside. That is, I don't need to copy the packet to an owned object just so it's 'static.
Here's the sketch:
use std::sync::{Arc, Mutex};
pub trait DecodedPacket<'a>: Send {}
pub type OnConsume = Arc<dyn for<'a> Fn(&'a mut Option<Box<dyn DecodedPacket<'a>>>) + Send + Sync>;
pub trait Decoder: Send{
fn receive_ref(&self,on_packet: Arc<dyn for<'a> FnMut(Box<dyn DecodedPacket<'a>>)>);
}
pub struct DummyDecoder {}
impl Decoder for DummyDecoder {
fn receive_ref(&self,_: Arc<dyn for<'a> FnMut(Box<dyn DecodedPacket<'a>>)>){
unimplemented!();
}
}
struct DummyRenderer {
on_consume: OnConsume
}
impl DummyRenderer {
pub fn render(&self) {
let mut decoded_packet: Option<Box<dyn DecodedPacket<'_>>> = None;
let packet = (self.on_consume)(&mut decoded_packet);
//Render packet somehow here
}
}
fn main() {
let decoder: Box<dyn Decoder + Sync> = Box::new(DummyDecoder{});
let renderer_on_consume: OnConsume =
Arc::new(move |p: &mut Option<Box<dyn DecodedPacket>>| {
let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> =
Arc::new(Mutex::new(None));
let on_packet = Arc::new(move |packet: Box<dyn DecodedPacket>|{
decoded_packet.lock().unwrap().replace(packet);
});
decoder.receive_ref(on_packet);
if let Some(packet) = *decoded_packet.lock().unwrap() {
p.replace(packet);
}
});
let dummy_renderer = DummyRenderer{
on_consume: renderer_on_consume
};
dummy_renderer.render();
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=89a181ae763d220199be91c9cb552e80
You can see that receive_ref expects a function that can deal with all lifetimes, so it should work for a small non 'static lifetime. However, the compiler is making all the lifetimes 'static:
Error:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
--> src/main.rs:35:58
|
35 | let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> =
| ^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 34:18...
--> src/main.rs:34:18
|
34 | Arc::new(move |p: &mut Option<Box<dyn DecodedPacket>>| {
| __________________^
35 | | let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> =
36 | | Arc::new(Mutex::new(None));
37 | | let on_packet = Arc::new(move |packet: Box<dyn DecodedPacket>|{
... |
43 | | }
44 | | });
| |_________^
note: ...but the lifetime must also be valid for the anonymous lifetime #1 defined on the body at 37:38...
--> src/main.rs:37:38
|
37 | let on_packet = Arc::new(move |packet: Box<dyn DecodedPacket>|{
| ______________________________________^
38 | | decoded_packet.lock().unwrap().replace(packet);
39 | | });
| |_____________^
note: ...so that the types are compatible
--> src/main.rs:42:27
|
42 | p.replace(packet);
| ^^^^^^
= note: expected `dyn DecodedPacket<'_>`
found `dyn DecodedPacket<'_>`
ps: I don't like the let decoded_packet: Arc<Mutex<Option<Box<dyn DecodedPacket>>>> = Arc::new(Mutex::new(None)); but I had to do it otherwise decoded_packet was getting borrowed by the closure and thus forcing the closure lifetime to be 'static

Creating a recursive list of references errors with "cannot infer an appropriate lifetime for autoref due to conflicting requirements"

The basic idea of the code is to create a hierarchical struct Context which contains some information of symbols, and a Context will be provided to the lambda in a Statement struct to get the final result. A child context can be derived from the parent if needed:
use anyhow::Result; // 1.0.40
use std::{collections::HashMap, rc::Rc};
struct Context<'a> {
parent: Option<&'a mut Context<'a>>,
symbols: HashMap<String, i64>,
}
impl<'a> Context<'a> {
fn new() -> Self {
Context {
parent: None,
symbols: HashMap::new(),
}
}
fn derive(&'a mut self) -> Self {
Context {
parent: Some(self),
symbols: HashMap::new(),
}
}
}
#[derive(Clone)]
struct Statement {
execute: Rc<dyn Fn(&mut Context) -> Result<()>>,
}
struct Node {
cond: Statement,
stmts: Vec<Statement>,
}
impl Node {
fn get(&self) -> Statement {
let cond = self.cond.clone();
let stmts = self.stmts.clone();
Statement {
execute: Rc::new(move |ctx| {
(cond.execute)(ctx)?;
let mut cctx = ctx.derive();
for stmt in stmts {
(stmt.execute)(&mut cctx)?;
}
Ok(())
}),
}
}
}
When I compile this code, I get the error:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/lib.rs:42:36
|
42 | let mut cctx = ctx.derive();
| ^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 40:30...
--> src/lib.rs:40:30
|
40 | execute: Rc::new(move |ctx| {
| ______________________________^
41 | | (cond.execute)(&mut ctx)?;
42 | | let mut cctx = ctx.derive();
43 | | for stmt in stmts {
... |
46 | | Ok(())
47 | | }),
| |_____________^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:42:32
|
42 | let mut cctx = ctx.derive();
| ^^^
note: but, the lifetime must be valid for the anonymous lifetime #2 defined on the body at 40:30...
--> src/lib.rs:40:30
|
40 | execute: Rc::new(move |ctx| {
| ______________________________^
41 | | (cond.execute)(&mut ctx)?;
42 | | let mut cctx = ctx.derive();
43 | | for stmt in stmts {
... |
46 | | Ok(())
47 | | }),
| |_____________^
note: ...so that the types are compatible
--> src/lib.rs:42:36
|
42 | let mut cctx = ctx.derive();
| ^^^^^^
= note: expected `&mut Context<'_>`
found `&mut Context<'_>`
I find this error message is useless. How can I fix this error?
With your current definition of execute, the lifetimes of Context and the &mut parameter are implied to be different due to lifetime elision rules but your use of .derive() within the closure requires them to be the same.
dyn Fn(&'a mut Context<'b>) -> Result<()>
You can fix this by using a higher-ranked trait bound to introduce a named lifetime to link them together:
dyn for<'a> Fn(&'a mut Context<'a>) -> Result<()>
However, fixing that and making the for loop not consume stmts (playground), you still have lifetime problems:
error[E0499]: cannot borrow `*ctx` as mutable more than once at a time
--> src/lib.rs:42:32
|
40 | execute: Rc::new(move |ctx| {
| --- has type `&'1 mut Context<'1>`
41 | (cond.execute)(ctx)?;
| -------------------
| | |
| | first mutable borrow occurs here
| argument requires that `*ctx` is borrowed for `'1`
42 | let mut cctx = ctx.derive();
| ^^^ second mutable borrow occurs here
error[E0499]: cannot borrow `cctx` as mutable more than once at a time
--> src/lib.rs:44:36
|
40 | execute: Rc::new(move |ctx| {
| --- has type `&'1 mut Context<'1>`
...
44 | (stmt.execute)(&mut cctx)?;
| ---------------^^^^^^^^^-
| | |
| | `cctx` was mutably borrowed here in the previous iteration of the loop
| argument requires that `cctx` is borrowed for `'1`
Which is answered by Why does this mutable borrow live beyond its scope? The short of it is, don't do &'a mut Context<'a>, it wreaks havoc on the borrow checker trying to make distinct lifetimes all line up and will inevitably cause confusing errors.
Using mutable references in particular are more limited, so you may get away with it by using immutable references, &'a Context<'a>, but depending on what you intend to do that might not be an option. If you need mutability, you may have to resort to shared ownership and interior mutability via Rc and RefCell.
So the problem has to do with lifetimes, but that's by far not the only problem.
First, whenever I run into errors with closures and weird compiler messages, I replace the closure with an actual function, for the time being, as that tends to make certain error messages easier to parse.
Then I tried adding explicit lifetime parameters to lots of other parts in your code:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0f5bb5010ea26cff3f99237b4439eba2
use anyhow::Result; // 1.0.40
use std::{collections::HashMap, rc::Rc};
struct Context<'a> {
parent: Option<&'a mut Context<'a>>,
symbols: HashMap<String, i64>,
}
impl<'a> Context<'a> {
fn new() -> Self {
Context {
parent: None,
symbols: HashMap::new(),
}
}
fn derive(&'a mut self) -> Context<'a> {
Context {
parent: Some(self),
symbols: HashMap::new(),
}
}
}
#[derive(Clone)]
struct Statement<'a> {
execute: Rc<(dyn Fn(&'a mut Context<'a>) -> Result<()> + 'a)>,
}
struct Node<'a> {
cond: Statement<'a>,
stmts: Vec<Statement<'a>>,
}
impl<'a> Node<'a> {
fn get(&self) -> Statement<'a> {
let cond = self.cond.clone();
let stmts = self.stmts.clone();
Statement {
execute: Rc::new(move |ctx| {
(cond.execute)(ctx)?;
let mut cctx = ctx.derive();
for stmt in stmts {
(stmt.execute)(&mut cctx)?;
}
Ok(())
}),
}
}
}
That solves the lifetime issue, but there's a bunch of other issues, too: You're generously using mutable references (mutable borrows) and of course now the compiler complains that you are borrowing mutably more than once, because your lifetimes require that the references stay alive for long enough!
The problem here is that you'd need to keep multiple mutable references to a given context alive at the same time, which Rust doesn't allow.
The solution to this conundrum rather than trying to hack away at the current structure, is to rethink if you really need all those mutable references, and if you can rework the code in some other way, too.
From other object oriented languages it's quite common to have this "sea of objects" with lots of (mutable, of course) references pointing back and forth, but Rust doesn't quite like that.

Lifetime for Cursor in Iteration

use std::io::Cursor;
use bytes::BytesMut;
struct Parser<'a> {
stream: &'a mut Cursor<&'a [u8]>
}
fn parse<'a>(buf:&'a mut Cursor<&'a [u8]>) -> Option<(usize, &'a str)> {
None
}
impl<'a> Iterator for Parser<'a> {
type Item = (usize, &'a str);
fn next(&mut self) -> Option<Self::Item> {
parse(self.stream)
}
}
fn main() {
let mut buf = BytesMut::with_capacity(1024);
let mut cursor = Cursor::new(&buf[..]);
let mut parser = Parser {stream: &mut cursor};
}
Playground link
Basically, I want to parse a buf without copy but failed to compile. Could anyone help? Much appreciated!
Error message:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
--> src/main.rs:15:9
|
15 | parse(self.stream)
| ^^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 14:5...
--> src/main.rs:14:5
|
14 | fn next(&mut self) -> Option<Self::Item> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:15:15
|
15 | parse(self.stream)
| ^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 12:6...
--> src/main.rs:12:6
|
12 | impl<'a> Iterator for Parser<'a> {
| ^^
note: ...so that the expression is assignable
--> src/main.rs:15:15
|
15 | parse(self.stream)
| ^^^^^^^^^^^
= note: expected `&mut std::io::Cursor<&[u8]>`
found `&mut std::io::Cursor<&'a [u8]>`
You need a different lifetime for what is inside the cursor:
struct Parser<'a, 'b> {
stream: &'a mut Cursor<&'b [u8]>,
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0c36650015dc7fb4d425794fca934242
The reason is to do with the fact that your stream is a mutable reference. If you make it immutable, everything works.
I could not explain why better than Ryan Levick did in his video he made on lifetimes.
That video should be one of the recommended learning materials 😃

Why this rust FnMut closure code has lifetime errors?

I want to make a closure hold &mut Vec, but these simple few lines of code can't compile.
I know this can be solved with RefCell, I just can't figure the error out.
struct Server<'a> {
data: &'a mut Vec<i32>,
}
fn main() {
let mut data = vec![1, 2, 3];
let mut c = || {
Server{
data: &mut data,
}
};
let server = c();
}
Rust playground link
The error message is:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/main.rs:36:19
|
36 | data: &mut data,
| ^^^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime '_ as defined on the body at 34:17...
--> src/main.rs:34:17
|
34 | let mut c = || {
| ^^
note: ...so that closure can access `data`
--> src/main.rs:36:19
|
36 | data: &mut data,
| ^^^^^^^^^
note: but, the lifetime must be valid for the call at 39:18...
--> src/main.rs:39:18
|
39 | let server = c();
| ^^^
note: ...so type `Server<'_>` of expression is valid during the expression
--> src/main.rs:39:18
|
39 | let server = c();
| ^^^
error: aborting due to previous error
Update:
I found this post answered the question, but I could not understand some parts of it:
It turns out that, returning &'a mut i32 is not feasible for the shorter-live invocation of call_mut. What we’ve really wanted for the return type of call_mut, was something like:
impl<'a> FnMut<(usize,)> for Closure<'a> {
extern "rust-call"
fn<'b> call_mut(&'b mut self, (i,): (usize, )) -> &'b mut i32 {
self.inner.get_mut(i).unwrap()
}
}
Why the call_mut fn has to use a 'b lifetime instead of 'a?
I found the code would work if change data: &mut Vec<i32> to data: &Vec<i32>, which makes me more confused that why lifetime is related to mut?

Rust complains of conflicting lifetime requirements, no iterators or closures [duplicate]

This question already has answers here:
Cannot infer an appropriate lifetime when storing Peekable iterators in a vector
(1 answer)
How to fix: cannot infer an appropriate lifetime for automatic coercion
(1 answer)
Closed 5 years ago.
I'm trying to teach myself Rust. I'm familiar with C++ and Scala, but Rust is still a Perlish language to me.
I've been bickering with the borrow checker for two days over this code. It seems pretty clear to me, but I can't get Rust to agree.
This is about as simple as I can get the code and still produce the error:
use std::io;
fn main() {
let mut streams: StdStreams = StdStreams {
stderr: &mut io::stderr(),
};
let command = Command {};
let streams_ref: &mut StdStreams = &mut streams;
command.go(streams_ref);
}
pub struct StdStreams<'a> {
stderr: &'a mut io::Write,
}
pub struct Command {}
impl Command {
pub fn go(&self, streams: &mut ::StdStreams) {
let mut server = Server { streams };
}
}
pub struct Server<'a> {
pub streams: &'a mut StdStreams<'a>,
}
Here's the compiler's opinion:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
--> src/main.rs:20:26
|
20 | let mut server = Server { streams };
| ^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #2 defined on the method body at 19:5...
--> src/main.rs:19:5
|
19 | / pub fn go(&self, streams: &mut ::StdStreams) {
20 | | let mut server = Server { streams };
21 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:20:35
|
20 | let mut server = Server { streams };
| ^^^^^^^
note: but, the lifetime must be valid for the anonymous lifetime #3 defined on the method body at 19:5...
--> src/main.rs:19:5
|
19 | / pub fn go(&self, streams: &mut ::StdStreams) {
20 | | let mut server = Server { streams };
21 | | }
| |_____^
note: ...so that expression is assignable (expected &mut StdStreams<'_>, found &mut StdStreams<'_>)
--> src/main.rs:20:35
|
20 | let mut server = Server { streams };
| ^^^^^^^
It seems clear to me that the item I'm storing the offending reference in will go out of scope and die immediately, at the same moment its stored reference does, so there shouldn't be any unpleasantness.
I also find this needlessly sarcastic: expected &mut StdStreams<'_>, found &mut StdStreams<'_>.
The issue becomes clearer when we see how the compiler makes all lifetimes explicit in the go method:
pub fn go<'a, 'b, 'c>(&'a self, streams: &'b mut StdStreams<'c>) {
let mut server = Server {streams};
}
That's right, a lifetime parameter was missing. The compiler does not infer that the lifetime parameter 'a in StdStreams<'a> will be the same as the lifetime of a reference of type &mut StdStreams (as in, it won't produce &'a Foo<'a>. The compiler will also use these named lifetimes in its error messages where possible:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
--> src/main.rs:20:26
|
20 | let mut server = Server {streams};
| ^^^^^^
|
note: first, the lifetime cannot outlive the lifetime 'b as defined on the method body at 19:5...
--> src/main.rs:19:5
|
19 | / pub fn go<'a, 'b, 'c>(&'a self, streams: &'b mut StdStreams<'c>) {
20 | | let mut server = Server {streams};
21 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:20:34
|
20 | let mut server = Server {streams};
| ^^^^^^^
note: but, the lifetime must be valid for the lifetime 'c as defined on the method body at 19:5...
--> src/main.rs:19:5
|
19 | / pub fn go<'a, 'b, 'c>(&'a self, streams: &'b mut StdStreams<'c>) {
20 | | let mut server = Server {streams};
21 | | }
| |_____^
note: ...so that expression is assignable (expected &mut StdStreams<'_>, found &mut StdStreams<'c>)
--> src/main.rs:20:34
|
20 | let mut server = Server {streams};
| ^^^^^^^
The conflict should now be clear: 'b needs to outlive the lifetime 'c, but the method does not impose that constraint. So, we can either bind the lifetime parameter to the same lifetime as the reference:
pub fn go<'a>(&self, streams: &'a mut StdStreams<'a>) {
}
Or add a constraint to the lifetime parameter.
pub fn go<'a: 'b, 'b>(&self, streams: &'a mut StdStreams<'b>) {
}
I also find this needlessly sarcastic: expected &mut StdStreams<'_>, found &mut StdStreams<'_>
That is something that the compiler could probably improve on, but anonymous lifetimes are not that easy to express to the user.

Resources