Lifetime problem when working code is extracted into a separate function - rust

Question

I'm writing a program to extract information from log files (which are in text format). The overall flow is
Read the file line-by-line into a String
Create a ParsedLine structure which borrows several string slices from that line (some using Cow)
Use the ParsedLine to write a CSV record.
This has been going very well so far, but I have run into a problem I do not understand, I think it is with lifetimes or data-flow analysis. The problem is with a small refactor I am trying to make.
I have this function which works:
fn process_line(columns: &[Column], line: String, writer: &mut Writer<File>) {
let parsed_line = ParsedLine::new(&line);
if parsed_line.is_err() {
let data = vec![""];
writer.write_record(&data).expect("Writing a CSV record should always succeed.");
return;
}
let parsed_line = parsed_line.unwrap();
// let data = output::make_output_record(&parsed_line, columns);
// The below code works. But if I try to pull it out into a separate function
// Rust will not compile it.
let mut data = Vec::new();
for column in columns {
match column.name.as_str() {
config::LOG_DATE => data.push(parsed_line.log_date),
config::LOG_LEVEL => data.push(parsed_line.log_level),
config::MESSAGE => data.push(&parsed_line.message),
_ => {
let ci_comparer = UniCase::new(column.name.as_str());
match parsed_line.kvps.get(&ci_comparer) {
Some(val) => {
let x = val.as_ref();
data.push(x);
},
None => data.push(""),
}
},
}
}
writer.write_record(&data).expect("Writing a CSV record should always succeed.");
}
But I want to pull out the bit of code that constructs data into a separate function so that I can test it more easily. Here's the function:
pub fn make_output_record<'p, 't, 'c>(parsed_line: &'p ParsedLine<'t>, columns: &'c [Column]) -> Vec<&'t str> {
let mut data = Vec::new();
for column in columns {
match column.name.as_str() {
config::LOG_DATE => data.push(parsed_line.log_date),
config::LOG_LEVEL => data.push(parsed_line.log_level),
config::MESSAGE => data.push(&parsed_line.message),
_ => {
let ci_comparer = UniCase::new(column.name.as_str());
match parsed_line.kvps.get(&ci_comparer) {
// This is the problem here. To make it explicit:
// val is a "&'t Cow<'t, str>" and x is "&'t str"
Some(val) => {
let x = val.as_ref();
data.push(x);
},
None => data.push(""),
}
},
}
}
data
}
And the error I get and do not understand is:
error[E0623]: lifetime mismatch
--> src/main.rs:201:5
|
177 | pub fn make_output_record<'p, 't, 'c>(parsed_line: &'p ParsedLine<'t>, columns: &'c [Column]) -> Vec<&'t str> {
| ------------ ------------
| |
| this parameter and the return type are declared with different lifetimes...
...
201 | data
| ^^^^ ...but data from `columns` is returned here
The compiler thinks that the returned vector contains information from Columns, but Columns is actually only used to get the name of the column, which is then used to lookup a value in the kvps HashMap (UniCase is used to make the lookup case-insensitive). If a value is found, we add the &str to data.
So I don't understand why the compiler thinks that something from Columns ends up in data, because to my mind Columns is just a bit of metadata used to drive the final contents of data, but does not in and of itself appear in data. Once the kvps lookup is done and we have the value Columns might as well not exist.
I've tried various ways of fixing this (including adding explicit lifetimes to everything, removing some lifetimes, and adding various outlives lifetime specifications) but no combination appears to be able to tell the compiler that Columns is not used in data.
For reference, here is the definition of ParsedLine:
#[derive(Debug, Default, PartialEq, Eq)]
pub struct ParsedLine<'t> {
pub line: &'t str,
pub log_date: &'t str,
pub log_level: &'t str,
pub message: Cow<'t, str>,
pub kvps: HashMap<UniCase<&'t str>, Cow<'t, str>>
}
Note that I am resisting getting rid of the Cows: I assume this would fix the problem, but the number of String allocations would probably rise by a factor of 20 and I'd like to avoid that. The current program is impressively fast!
I suspect the problem is actually with that UniCase<&'t str> and I need to give the key it's own lifetime. Not sure how though.
So my question is
Why can't I easily move this code into a new function?
How do I fix it?
I appreciate this is a rather long question. It may be easier to fiddle with the code locally. It is on Github and the error should be reproducable with:
git clone https://github.com/PhilipDaniels/log-file-processor
git checkout 80158b3
cargo build

Answer 1

The call for make_output_record from process_line will infer the lifetime parameter of make_output_record.
pub fn make_output_record<'p>(parsed_line: &'p ParsedLine, columns: &'p [Column]) -> Vec<&'p str> {
This means 'p is a lifetime which the owner will be alive in process_line's scope(because of the inference). According to your code parsed_line and columnslives in 'p. The 'p is the common lifetime for your return value and the arguments. That's why your code was not working because 'p, 't ,'c is not common for arguments and your return value.
I simplified your code in here, this is the working version, you can have your error back if you add other life time parameters back to make_output_record.