Iterating through a Window of a String without collect - string

Question

I need to iterate through and compare a window of unknown length of a string. My current implementation works, however I've done performance tests against it, and it is very inefficient. The method needs to be guaranteed to be safe against Unicode.
fn foo(line: &str, patt: &str) {
for window in line.chars().collect::<Vec<char>>().windows(patt.len()) {
let mut bar = String::new();
for ch in window {
bar.push(*ch);
}
// perform various comparison checks
}
}

Answer 1

An improvement on Shepmaster's final solution, which significantly lowers overhead (by a factor of ~1.5), is
fn foo(line: &str, pattern: &str) -> bool {
let pattern_len = pattern.chars().count();
let starts = line.char_indices().map(|(i, _)| i);
let mut ends = line.char_indices().map(|(i, _)| i);
// Itertools::dropping
if pattern_len != 0 { ends.nth(pattern_len - 1); }
for (start, end) in starts.zip(ends.chain(Some(line.len()))) {
let bar = &line[start..end];
if bar == pattern { return true }
}
false
}
That said, your code from the Github page is a little odd. For instance, you try to deal with different length open and close tags with a wordier version of
let length = cmp::max(comment.len(), comment_end.len());
but your check
if window.contains(comment)
could then trigger multiple times!
Much better would be to just iterate over shrinking slices. In the mini example this would be
fn foo(line: &str, pattern: &str) -> bool {
let mut chars = line.chars();
loop {
let bar = chars.as_str();
if bar.starts_with(pattern) { return true }
if chars.next().is_none() { break }
}
false
}
(Note that this once again ends up again improving performance by another factor of ~1.5.)
and in a larger example this would be something like
let mut is_in_comments = 0u64;
let start = match line.find(comment) {
Some(start) => start,
None => return false,
};
let end = match line.rfind(comment_end) {
Some(end) => end,
None => return true,
};
let mut chars = line[start..end + comment_end.len()].chars();
loop {
let window = chars.as_str();
if window.starts_with(comment) {
if nested {
is_in_comments += 1;
} else {
is_in_comments = 1;
}
} else if window.starts_with(comment_end) {
is_in_comments = is_in_comments.saturating_sub(1);
}
if chars.next().is_none() { break }
}
Note that this still counts overlaps, so /*/ might count as an opening /* immediately followed by a closing */.

Answer 2

The method needs to be guaranteed to be safe against Unicode.
pattern.len() returns the number of bytes that the string requires, so it's already possible that your code is doing the wrong thing. I might suggest you check out tools like QuickCheck to produce arbitrary strings that include Unicode.
Here's my test harness:
use std::iter;
fn main() {
let mut haystack: String = iter::repeat('a').take(1024*1024*100).collect();
haystack.push('b');
println!("{}", haystack.len());
}
And I'm compiling and timing via cargo build --release && time ./target/release/x. Creating the string by itself takes 0.274s.
I used this version of your original code just to have some kind of comparison:
fn foo(line: &str, pattern: &str) -> bool {
for window in line.chars().collect::<Vec<char>>().windows(pattern.len()) {
let mut bar = String::new();
for ch in window {
bar.push(*ch);
}
if bar == pattern { return true }
}
false
}
This takes 4.565s, or 4.291s for just foo.
The first thing I see is that there is a lot of allocation happening on the inner loop. The code creates, allocates, and destroys the String for each iteration. Let's reuse the String allocation:
fn foo_mem(line: &str, pattern: &str) -> bool {
let mut bar = String::new();
for window in line.chars().collect::<Vec<char>>().windows(pattern.len()) {
bar.clear();
bar.extend(window.iter().cloned());
if bar == pattern { return true }
}
false
}
This takes 2.155s or 1.881s for just foo_mem.
Continuing on, another extraneous allocation is the one for the String at all. We already have bytes that look like the right thing, so let's reuse them:
fn foo_no_string(line: &str, pattern: &str) -> bool {
let indices: Vec<_> = line.char_indices().map(|(i, _c)| i).collect();
let l = pattern.chars().count();
for window in indices.windows(l + 1) {
let first_idx = *window.first().unwrap();
let last_idx = *window.last().unwrap();
let bar = &line[first_idx..last_idx];
if bar == pattern { return true }
}
// Do the last pair
{
let last_idx = indices[indices.len() - l];
let bar = &line[last_idx..];
if bar == pattern { return true }
}
false
}
This code is ugly and unidiomatic. I'm pretty sure some thinking (that I'm currently too lazy to do) would make it look a lot better.
This takes 1.409s or 1.135s for just foo_mem.
As this is ~25% of the original time, Amdahl's Law suggests this is a reasonable stopping point.