Implementing a recursive cache in Rust [duplicate] - rust

This question already has answers here:
Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in?
(1 answer)
When is it necessary to circumvent Rust's borrow checker?
(1 answer)
Closed 1 year ago.
use std::collections::HashMap;
#[derive(Clone, Hash, Eq, PartialEq)]
struct HeavyCloneKey;
struct Map<V> {
map : HashMap<HeavyCloneKey, V>
}
impl<V> Map<V> {
pub fn get_mut(&mut self, key: &HeavyCloneKey) -> Option<&mut V> {
if let Some(v) = self.map.get_mut(key) {
return Some(v);
}
// some logic to decide if create a new value;
if self.map.len() > 64 {
None
} else {
let v = create_v_with_long_code();
let v = self.map.entry(key.clone()).or_insert_with(|| v);
Some(v)
}
}
}
fn create_v_with_long_code<V>() -> V {
unimplemented!()
}
fn main() {
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=19c45ebfb1a39271ff5bd2443399596a
I can't understand why this can't work:
error[E0502]: cannot borrow `self.map` as immutable because it is also borrowed as mutable
--> src/main.rs:18:12
|
12 | pub fn get_mut(&mut self, key: &HeavyCloneKey) -> Option<&mut V> {
| - let's call the lifetime of this reference `'1`
13 | if let Some(v) = self.map.get_mut(key) {
| -------- mutable borrow occurs here
14 | return Some(v);
| ------- returning this value requires that `self.map` is borrowed for `'1`
...
18 | if self.map.len() > 64 {
| ^^^^^^^^ immutable borrow occurs here
These two borrows won't appear at the same time, what's the problem? How can I do this?
Although the code can be compiled by using contains_key before get_mut, it pays more overhead;

It helps a little to desugar the lifetimes by providing your own annotations on the references. It doesn't really matter here though as lifetimes aren't actually the issue.
pub fn get_mut<'s, 'k>(&'s mut self, key: &'k HeavyCloneKey) -> Option<&'s mut V> {
This gets us a little better error message:
error[E0502]: cannot borrow `self.map` as immutable because it is also borrowed as mutable
--> src/main.rs:18:12
|
12 | pub fn get_mut<'s, 'k>(&'s mut self, key: &'k HeavyCloneKey) -> Option<&'s mut V> {
| -- lifetime `'s` defined here
13 | if let Some(v) = self.map.get_mut(key) {
| -------- mutable borrow occurs here
14 | return Some(v);
| ------- returning this value requires that `self.map` is borrowed for `'s`
...
18 | if self.map.len() > 64 {
| ^^^^^^^^ immutable borrow occurs here
error[E0499]: cannot borrow `self.map` as mutable more than once at a time
--> src/main.rs:22:21
|
12 | pub fn get_mut<'s, 'k>(&'s mut self, key: &'k HeavyCloneKey) -> Option<&'s mut V> {
| -- lifetime `'s` defined here
13 | if let Some(v) = self.map.get_mut(key) {
| -------- first mutable borrow occurs here
14 | return Some(v);
| ------- returning this value requires that `self.map` is borrowed for `'s`
...
22 | let v = self.map.entry(key.clone()).or_insert_with(|| v);
| ^^^^^^^^ second mutable borrow occurs here
We have two errors here. Both stem from the rust compiler not being able to tell that we don't use the mutable borrow from line 13 later in the function.
The first says "you grabbed a mutable reference to self here so we can't let you have an immutable reference there". The second says "you grabbed a mutable reference to self here so we can't let you have a second mutable reference there".
How do we fix this? By making it clearer to the compiler that we'll never hold that first mutable reference. We can do this by manually checking if that key exists first. Change this
if let Some(v) = self.map.get_mut(key) {
return Some(v);
}
to this
if self.map.contains_key(key) {
return self.map.get_mut(key);
}
and it will compile.

Related

How do I get the length of a Vec after it has been modified?

I have a function which takes a Vec and pushes zero or more elements to it. The Vec and the reference passed to the function are both mut. When I try to get the length of the Vec upon return (to see how many elements were pushed), I get the dreaded "cannot borrow 'foo' as immutable because it is also borrowed as mutable" error.
In general, I get why this error exists in general. I don't get why it's necessary here.
Surely there's a way to do this -- I can't be the first person who has tried something like this.
Here's a stripped-down version of what I'm trying to do, that replicates the error.
fn main() {
let mut stack = vec!["one"];
push(&mut stack, "two");
}
fn push<'a>(stack: &'a mut Vec<&'a str>, value: &'a str) {
stack.push(value);
}
#[test]
fn test_len() {
let mut stack = vec!["one"];
push(&mut stack, "two");
assert_eq!(stack.len(), 2);
}
error[E0502]: cannot borrow `stack` as immutable because it is also borrowed as mutable
--> src/main.rs:14:16
|
13 | push(&mut stack, "two");
| ---------- mutable borrow occurs here
14 | assert_eq!(stack.len(), 2);
| ^^^^^^^^^^^
| |
| immutable borrow occurs here
| mutable borrow later used here
Don't force the references lifetime and the lifetime of the mutable borrow to be the same.
fn push<'a>(stack: &mut Vec<&'a str>, value: &'a str) {
stack.push(value);
}

How too loop over a function that requires a mutable refrence?

I am quite new to rust and I have a problem with the borrow checker.
I have a function like this:
impl<'a> DFA<'a> {
fn solve_next(&mut self, c: Option<char>) -> Option<TokenType> {
let node = self.state.unwrap_or_else(|| &self.start_node);
let checks: Vec<char> = node.connections.iter().map(|x| x.1).collect();
let mut position = None;
if let Some(c) = c {
position = checks.iter().position(|x| *x == c);
}
if let Some(i) = position {
let node = &node.connections[i].0;
self.state = Some(node);
None
} else {
if node.is_end {
return Some(TokenType::Keyword);
}
panic!("Invalid charter for current set");
}
}
}
This function either returns a TokenType (which is a enum) when the loop is complete or None when it is not.
its is on this struct:
struct DFA<'a> {
state: Option<&'a Node>,
start_node: Node,
}
Node is:
struct Node {
connections: Vec<(Node, char)>,
is_end: bool,
token_type: TokenType,
}
with the method
fn insert(&mut self, _match: char, end: bool) -> &mut Node {
which creates and return a new node and adds this node into its own connections.
So when I wanted to loop over the solve next function I tried:
impl<'a> Lexer<'a> {
fn next_char(&self) -> Option<char> {
let mut r: Option<char> = None;
for c in &self.chars {
match c {
' ' | '\n' | '\r' => {
continue;
}
_ => {
r = Some(c.clone());
break;
}
}
}
return r;
}
fn next_token(&'a mut self) {
let somechar = 'c';
let mut x = self.dfa.solve_next(self.next_char());
while let None = x {
x = self.dfa.solve_next(self.next_char());
}
}
}
which is a method of
struct Lexer<'a> {
//the output of the lexer
pub tokens: Vec<Token>,
chars: Vec<char>,
dfa: DFA<'a>,
}
the compile error's with
error[E0499]: cannot borrow `self.dfa` as mutable more than once at a time
--> src/main.rs:177:17
|
146 | impl<'a> Lexer<'a> {
| -- lifetime `'a` defined here
...
175 | let mut x = self.dfa.solve_next(self.next_char());
| -------------------------------------
| |
| first mutable borrow occurs here
| argument requires that `self.dfa` is borrowed for `'a`
176 | while let None = x {
177 | x = self.dfa.solve_next(self.next_char());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ second mutable borrow occurs here
error[E0502]: cannot borrow `*self` as immutable because it is also borrowed as mutable
--> src/main.rs:177:37
|
146 | impl<'a> Lexer<'a> {
| -- lifetime `'a` defined here
...
175 | let mut x = self.dfa.solve_next(self.next_char());
| -------------------------------------
| |
| mutable borrow occurs here
| argument requires that `self.dfa` is borrowed for `'a`
176 | while let None = x {
177 | x = self.dfa.solve_next(self.next_char());
| ^^^^^^^^^^^^^^^^ immutable borrow occurs here
So is their any way I can use solve_next in this loop? as I would not know how to make a function that can be used like this without having it take a mutable refrence.
The actual error you get with your code is the following (playground):
error: lifetime may not live long enough
--> src/lib.rs:27:49
|
25 | impl<'a> DFA<'a> {
| -- lifetime `'a` defined here
26 | fn solve_next(&mut self, c: Option<char>) -> Option<TokenType> {
| - let's call the lifetime of this reference `'1`
27 | let node = self.state.unwrap_or_else(|| &self.start_node);
| ^^^^^^^^^^^^^^^^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`
This is actually complaining about the reference to self.start_node not living long enough, that is for at least 'a.
The reason for that is that your local variable node, that is of type &'x Node, being the lifetime 'x the shorter of that of self.state.unwrap() and &self.start_node. The first one is 'a and the second one is the unnamed lifetime of &self (from now on 's).
But by the rules of Rust lifetimes, 'a outlives DFA<'a> and Self outlives 's, and Self is equal to DFA<'a>, then we conclude that 'a outlives 's. So the lifetime of the local node is actually 's.
Now, the other key line is this one:
self.state = Some(node);
being self.state of type Option<&'a Node>, that requires that the lifetime of node, that is 's, outlives 'a. But that cannot be, we already determined that it is the other way around: 'a outlives 's. Thus a compiler error.
In a nutshell, you are trying to write a self-referential struct, storing in one field a reference to the other field. And that is well known to be impossible.
In the original version of your question, you tried to solve it adding an extra lifetime to the solve_next() function:
fn solve_next(&'a mut self, c: Option<char>) -> Option<TokenType>
Doing that you force the 's from my explanation above to be exactly equal to 'a, so the body of this function is actually correct. Unfortunately if you try to call it normally with:
fn next_token(&mut self) {
let mut x = self.dfa.solve_next(self.next_char());
}
It fails with:
error: lifetime may not live long enough
--> src/lib.rs:63:21
|
46 | impl<'a> Lexer<'a> {
| -- lifetime `'a` defined here
...
62 | fn next_token(&mut self) {
| - let's call the lifetime of this reference `'1`
63 | let mut x = self.dfa.solve_next(self.next_char());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`
Exactly for the same reason as above: the anonymous lifetime of self must outlive 'a but the compiler deduces it the other way around.
Naturally you can fix it the same way:
fn next_token(&'a mut self) {
And that will compile... as long as you do not try to call solve_next() twice.
I'm not sure why calling solve_next() once it compiles, but calling it twice fails with. But it does not actually matter because even if this function worked, with this extra &'a mut self you'd still have the same problem calling this function from the external code.
What about the solution? Well, I think you need to refactor the code so that you never try to add a reference start_node in the same struct.
For example, you can store the start_node outside this struct:
struct DFA<'a> {
state: Option<&'a Node>,
start_node: &'a Node,
}
This, and remove all the lifetime annotations from &'a self and it will just compile (playground).

Creating a recursive list of references errors with "cannot infer an appropriate lifetime for autoref due to conflicting requirements"

The basic idea of the code is to create a hierarchical struct Context which contains some information of symbols, and a Context will be provided to the lambda in a Statement struct to get the final result. A child context can be derived from the parent if needed:
use anyhow::Result; // 1.0.40
use std::{collections::HashMap, rc::Rc};
struct Context<'a> {
parent: Option<&'a mut Context<'a>>,
symbols: HashMap<String, i64>,
}
impl<'a> Context<'a> {
fn new() -> Self {
Context {
parent: None,
symbols: HashMap::new(),
}
}
fn derive(&'a mut self) -> Self {
Context {
parent: Some(self),
symbols: HashMap::new(),
}
}
}
#[derive(Clone)]
struct Statement {
execute: Rc<dyn Fn(&mut Context) -> Result<()>>,
}
struct Node {
cond: Statement,
stmts: Vec<Statement>,
}
impl Node {
fn get(&self) -> Statement {
let cond = self.cond.clone();
let stmts = self.stmts.clone();
Statement {
execute: Rc::new(move |ctx| {
(cond.execute)(ctx)?;
let mut cctx = ctx.derive();
for stmt in stmts {
(stmt.execute)(&mut cctx)?;
}
Ok(())
}),
}
}
}
When I compile this code, I get the error:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/lib.rs:42:36
|
42 | let mut cctx = ctx.derive();
| ^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 40:30...
--> src/lib.rs:40:30
|
40 | execute: Rc::new(move |ctx| {
| ______________________________^
41 | | (cond.execute)(&mut ctx)?;
42 | | let mut cctx = ctx.derive();
43 | | for stmt in stmts {
... |
46 | | Ok(())
47 | | }),
| |_____________^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:42:32
|
42 | let mut cctx = ctx.derive();
| ^^^
note: but, the lifetime must be valid for the anonymous lifetime #2 defined on the body at 40:30...
--> src/lib.rs:40:30
|
40 | execute: Rc::new(move |ctx| {
| ______________________________^
41 | | (cond.execute)(&mut ctx)?;
42 | | let mut cctx = ctx.derive();
43 | | for stmt in stmts {
... |
46 | | Ok(())
47 | | }),
| |_____________^
note: ...so that the types are compatible
--> src/lib.rs:42:36
|
42 | let mut cctx = ctx.derive();
| ^^^^^^
= note: expected `&mut Context<'_>`
found `&mut Context<'_>`
I find this error message is useless. How can I fix this error?
With your current definition of execute, the lifetimes of Context and the &mut parameter are implied to be different due to lifetime elision rules but your use of .derive() within the closure requires them to be the same.
dyn Fn(&'a mut Context<'b>) -> Result<()>
You can fix this by using a higher-ranked trait bound to introduce a named lifetime to link them together:
dyn for<'a> Fn(&'a mut Context<'a>) -> Result<()>
However, fixing that and making the for loop not consume stmts (playground), you still have lifetime problems:
error[E0499]: cannot borrow `*ctx` as mutable more than once at a time
--> src/lib.rs:42:32
|
40 | execute: Rc::new(move |ctx| {
| --- has type `&'1 mut Context<'1>`
41 | (cond.execute)(ctx)?;
| -------------------
| | |
| | first mutable borrow occurs here
| argument requires that `*ctx` is borrowed for `'1`
42 | let mut cctx = ctx.derive();
| ^^^ second mutable borrow occurs here
error[E0499]: cannot borrow `cctx` as mutable more than once at a time
--> src/lib.rs:44:36
|
40 | execute: Rc::new(move |ctx| {
| --- has type `&'1 mut Context<'1>`
...
44 | (stmt.execute)(&mut cctx)?;
| ---------------^^^^^^^^^-
| | |
| | `cctx` was mutably borrowed here in the previous iteration of the loop
| argument requires that `cctx` is borrowed for `'1`
Which is answered by Why does this mutable borrow live beyond its scope? The short of it is, don't do &'a mut Context<'a>, it wreaks havoc on the borrow checker trying to make distinct lifetimes all line up and will inevitably cause confusing errors.
Using mutable references in particular are more limited, so you may get away with it by using immutable references, &'a Context<'a>, but depending on what you intend to do that might not be an option. If you need mutability, you may have to resort to shared ownership and interior mutability via Rc and RefCell.
So the problem has to do with lifetimes, but that's by far not the only problem.
First, whenever I run into errors with closures and weird compiler messages, I replace the closure with an actual function, for the time being, as that tends to make certain error messages easier to parse.
Then I tried adding explicit lifetime parameters to lots of other parts in your code:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0f5bb5010ea26cff3f99237b4439eba2
use anyhow::Result; // 1.0.40
use std::{collections::HashMap, rc::Rc};
struct Context<'a> {
parent: Option<&'a mut Context<'a>>,
symbols: HashMap<String, i64>,
}
impl<'a> Context<'a> {
fn new() -> Self {
Context {
parent: None,
symbols: HashMap::new(),
}
}
fn derive(&'a mut self) -> Context<'a> {
Context {
parent: Some(self),
symbols: HashMap::new(),
}
}
}
#[derive(Clone)]
struct Statement<'a> {
execute: Rc<(dyn Fn(&'a mut Context<'a>) -> Result<()> + 'a)>,
}
struct Node<'a> {
cond: Statement<'a>,
stmts: Vec<Statement<'a>>,
}
impl<'a> Node<'a> {
fn get(&self) -> Statement<'a> {
let cond = self.cond.clone();
let stmts = self.stmts.clone();
Statement {
execute: Rc::new(move |ctx| {
(cond.execute)(ctx)?;
let mut cctx = ctx.derive();
for stmt in stmts {
(stmt.execute)(&mut cctx)?;
}
Ok(())
}),
}
}
}
That solves the lifetime issue, but there's a bunch of other issues, too: You're generously using mutable references (mutable borrows) and of course now the compiler complains that you are borrowing mutably more than once, because your lifetimes require that the references stay alive for long enough!
The problem here is that you'd need to keep multiple mutable references to a given context alive at the same time, which Rust doesn't allow.
The solution to this conundrum rather than trying to hack away at the current structure, is to rethink if you really need all those mutable references, and if you can rework the code in some other way, too.
From other object oriented languages it's quite common to have this "sea of objects" with lots of (mutable, of course) references pointing back and forth, but Rust doesn't quite like that.

What's the difference between lifetimes of immutable and mutable references? [duplicate]

This question already has answers here:
Why does linking lifetimes matter only with mutable references?
(2 answers)
Cannot borrow as mutable more than once at a time in one code - but can in another very similar
(2 answers)
Closed 2 years ago.
There is a struct definition as follows.
struct Test<'a> {
t: &'a str,
}
impl<'a> Test<'a> {
pub fn shared_borrow(&'a self) {
println!("{}", self.t);
}
pub fn mutable_borrow(&'a mut self) {
println!("{}", self.t);
}
}
Case 1. It can pass through the borrowing check, if shared_borrow() is called before mutable_borrow().
fn main() {
let mut x = Test { t: "test" };
x.shared_borrow();
x.mutable_borrow();
}
Case 2. It fails if I exchange the order of the method calls as follows.
fn main() {
let mut x = Test { t: "test" };
x.mutable_borrow();
x.shared_borrow();
}
compiler's output:
error[E0502]: cannot borrow `x` as immutable because it is also borrowed as mutable
--> src/main.rs:16:5
|
15 | x.mutable_borrow();
| - mutable borrow occurs here
16 | x.shared_borrow();
| ^
| |
| immutable borrow occurs here
| mutable borrow later used here
error: aborting due to previous error
As I know, shared_borrow() and mutable_borrow() both borrow x for the entire lifetime of x, because the lifetimes are explicitly declared as 'a, which is the lifetime of x.
In Case 1, x.shared_borrow() will borrow x for the entire lifetime of x. So It shouldn't be allowed to call x.mutable_borrow() after that. I'm confused why the compiler doesn't report an error as what it does in Case 2.

Cannot borrow immutable borrowed content as mutable

I'm trying to develop a message routing app. I've read the official Rust docs and some articles and thought that I got how pointers, owning, and borrowing stuff works but realized that I didn't.
use std::collections::HashMap;
use std::vec::Vec;
struct Component {
address: &'static str,
available_workers: i32,
lang: i32
}
struct Components {
data: HashMap<i32, Vec<Component>>
}
impl Components {
fn new() -> Components {
Components {data: HashMap::new() }
}
fn addOrUpdate(&mut self, component: Component) -> &Components {
if !self.data.contains_key(&component.lang) {
self.data.insert(component.lang, vec![component]);
} else {
let mut q = self.data.get(&component.lang); // this extra line is required because of the error: borrowed value does not live long enough
let mut queue = q.as_mut().unwrap();
queue.remove(0);
queue.push(component);
}
self
}
}
(Also available on the playground)
Produces the error:
error: cannot borrow immutable borrowed content `**queue` as mutable
--> src/main.rs:26:13
|
26 | queue.remove(0);
| ^^^^^ cannot borrow as mutable
error: cannot borrow immutable borrowed content `**queue` as mutable
--> src/main.rs:27:13
|
27 | queue.push(component);
| ^^^^^ cannot borrow as mutable
Could you please explain the error and it would be great if you can give me the right implementation.
Here is an MCVE of your problem:
use std::collections::HashMap;
struct Components {
data: HashMap<u8, Vec<u8>>,
}
impl Components {
fn add_or_update(&mut self, component: u8) {
let mut q = self.data.get(&component);
let mut queue = q.as_mut().unwrap();
queue.remove(0);
}
}
Before NLL
error[E0596]: cannot borrow immutable borrowed content `**queue` as mutable
--> src/lib.rs:11:9
|
11 | queue.remove(0);
| ^^^^^ cannot borrow as mutable
After NLL
error[E0596]: cannot borrow `**queue` as mutable, as it is behind a `&` reference
--> src/lib.rs:11:9
|
11 | queue.remove(0);
| ^^^^^ cannot borrow as mutable
Many times, when something seems surprising like this, it's useful to print out the types involved. Let's print out the type of queue:
let mut queue: () = q.as_mut().unwrap();
error[E0308]: mismatched types
--> src/lib.rs:10:29
|
10 | let mut queue: () = q.as_mut().unwrap();
| ^^^^^^^^^^^^^^^^^^^ expected (), found mutable reference
|
= note: expected type `()`
found type `&mut &std::vec::Vec<u8>`
We have a mutable reference to an immutable reference to a Vec<u8>. Because we have an immutable reference to the Vec, we cannot modify it! Changing self.data.get to self.data.get_mut changes the type to &mut &mut collections::vec::Vec<u8> and the code compiles.
If you want to implement the concept of "insert or update", you should check into the entry API, which is more efficient and concise.
Beyond that, Rust uses snake_case for method naming, not camelCase.

Resources