How to properly solve lifetime inferences in struct? - rust

Question

The Rust playground code is here.
I have a struct of Token which has lifetime 'tok, and a scanner has lifetime 'lexer. I'm using both of them in another struct Parser, then I had a problem:
pub struct Token<'tok> {
pub value: Cow<'tok, str>,
pub line: usize,
}
pub struct Scanner {
pub source: Vec<char>,
pub current: usize,
pub line: usize,
}
pub struct Parser<'lexer> {
pub curr: &'lexer Token<'lexer>,
pub prev: &'lexer Token<'lexer>,
scanner: &'lexer mut Scanner,
}
impl <'lexer> Parser<'lexer> {
pub fn advance(&mut self) {
self.prev = self.curr;
self.curr = &self.scanner.next(); // cannot inference lifetime
}
}
I think the problem is Token has lifetime 'tok, and the borrow checker doesn't know the relation between 'tok and 'lexer so it can't inference proper lifetime.
However, I can avoid the problem by modifying it into the updated code:
pub struct Parser<'lexer> {
pub curr: Token<'lexer>,
pub prev: Token<'lexer>,
scanner: &'lexer mut Scanner,
}
impl <'lexer> Parser<'lexer> {
pub fn advance(&mut self) {
let prev = std::mem::replace(&mut self.curr, self.scanner.next());
self.prev = prev;
}
}
And with Token produced by next() is static:
impl Scanner {
pub fn next(&mut self) -> Token<'static> {
Token {
value: Cow::from(""),
line: 0,
}
}
}
It does compile but I think it's not ideal because all tokens are cloned from scanner into parser(they are not references anymore) and living until end of Parser's life. So memory usage is doubled. Is there a more proper way to deal with this?

Answer 1

Actualy your code structure is not ideal to deal with the borrow checker here why:
Token struct should be owned, the struct itself do not require any allocations (as the token is owned some indrection are required to allow prev <-> next switching)
None of the Parser or Lexer should own the base data so it will be easy to bound proper lifetime
In our case Vec<Char> is not a friendly type to work with (we do not need to own the data and this will be harder to make the comipler understand lifetimes), instead we're going to use an &'str but you could reproduce the exact behaviours with an &[char])
Here is an example that compile just fine
pub struct Token<'source> {
pub value: Cow<'source, str>,
pub line: usize,
}
pub struct Scanner<'source> {
pub source: &'source str,
pub current: usize,
pub line: usize,
}
pub struct Parser<'source> {
pub curr: Option<Token<'source>>,
pub prev: Option<Token<'source>>,
scanner: Scanner<'source>,
}
impl <'source>Scanner<'source> {
pub fn next(&'source /* DONT Forget to bound 'source to `self` */ self) -> Token<'source> {
Token {
value: Cow::from(self.source), /* `self.source` is bound to `'source` so the compiler understand that the token lifetime is the same than the source's one */
line: 0,
}
}
}
impl <'lexer> Parser<'lexer> {
pub fn advance(&'lexer mut self) {
self.prev = self.curr.take();
self.curr = Some(self.scanner.next());
}
}