Missed optimization based on using a reference to a counter variable - rust

Question

In the following example I'm reading a 2.7GiB file from my desktop, count the bytes and do something with the bytes that prevents optimizing the loop away.
use std::fs::File;
use std::io::Read;
use std::io::{BufReader, ErrorKind};
fn main() {
let file = File::open("/home/seb/Desktop/ubuntu-20.04.1-desktop-amd64.iso")
.expect("Cannot read file.");
let buf = BufReader::new(file);
let mut x = 0u8;
let mut num_bytes = 0usize;
read(buf, &mut num_bytes, &mut x);
print(num_bytes, x);
}
fn read(mut buf: BufReader<File>, num_bytes: &mut usize, x: &mut u8) {
let mut bytes = [0; 512];
loop {
match buf.read(&mut bytes) {
Ok(0) => break,
Ok(n) => {
for i in 0..n {
*num_bytes += n;
*x += bytes[i];
}
}
Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => panic!("{:?}", e),
};
}
}
fn print(num_bytes: usize, x: u8) {
println!("{}", num_bytes);
println!("{}", x);
}
I get roughly 20% differences in throughput based on whether num_bytes is passed as a reference to print or by value. Unscientifically testing through time shows consistent differences, always in the same ballpark.
fn print(num_bytes: &usize, x: u8) {
println!("{}", num_bytes);
println!("{}", x);
}
# passing num_bytes by value
cargo build --release && time target/release/test_read
Compiling test_read v0.1.0 (/data/rust_projects/test_read)
Finished release [optimized] target(s) in 0.23s
2785017856
101
target/release/test_read 0,67s user 0,29s system 99% cpu 0,966 total
As soon as a reference to num_bytes is used later, the program is about 20% slower.
fn print(num_bytes: usize, x: u8) {
println!("{}", num_bytes);
println!("{}", x);
}
# passing num_bytes by reference
cargo build --release && time target/release/test_read
Compiling test_read v0.1.0 (/data/rust_projects/test_read)
Finished release [optimized] target(s) in 0.22s
2785017856
101
target/release/test_read 1,00s user 0,26s system 99% cpu 1,258 total
I'm running this on Ubuntu 20.04 and with Rust 1.50.
The example is mostly taken from this question: What is a faster way to iterate through the bytes of a file in Rust?
Is there any explanation for using a reference to a counter variable to cause this?

Answer 1

After narrowing down the example, it turns out that using the reference to the counter variable stops the compiler from optimizing away additions on every loop iteration. by_ref and by_value two benches here have a ten-fold difference in run time, while by_ref_use_length matches by_value:
// benches/bench.rs
pub fn by_ref(buf: &[u8]) {
let mut x = 0u8;
let mut num_bytes = 0usize;
for &byte in buf {
num_bytes += 1;
x += byte;
}
black_box((&num_bytes, x));
}
pub fn by_value(buf: &[u8]) {
let mut x = 0u8;
let mut num_bytes = 0usize;
for &byte in buf {
num_bytes += 1;
x += byte;
}
black_box((num_bytes, x));
}
pub fn by_ref_use_length(buf: &[u8]) {
let mut x = 0u8;
let num_bytes = buf.len();
for &byte in buf {
x += byte;
}
black_box((&num_bytes, x));
}
fn criterion_benchmark(c: &mut Criterion) {
let buf = vec![0; 1024 * 1024 * 64];
c.bench_function("len num_bytes by ref", |b| b.iter(|| by_ref_use_length(&buf)));
c.bench_function("num_bytes by ref", |b| b.iter(|| by_ref(&buf)));
c.bench_function("num_bytes by value", |b| b.iter(|| by_value(&buf)));
}
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
# Cargo.toml
[package]
name = "test_read"
version = "0.1.0"
authors = [""]
edition = "2018"
[dev-dependencies]
criterion = "0.3"
[[bench]]
name = "bench"
harness = false
cargo bench --all
num_bytes by ref time: [723.42 us 726.62 us 729.54 us]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) low mild
len num_bytes by ref time: [696.26 us 699.16 us 701.87 us]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) low mild
num_bytes by value time: [721.68 us 725.48 us 729.55 us]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low mild
https://rust.godbolt.org/z/zqfn7n shows no instructions related to the counter in the loop when the counter variable is not passed by reference, whereas passing by reference prevents such optimizations. I'm not familiar with reading assembly, but I'm assuming that the length of the input slice is used rather than actually incrementing.
In the original example, the differences were only about 20%, which is probably due to the dependency on IO which restricts the optimization to adding n on every read iteration rather than simply returning the length of the input array. In fact, changing the counter in the loop to a single addition outside the inner loop matches the performance of both implementations:
let mut bytes = [0; 512];
loop {
match buf.read(&mut bytes) {
Ok(0) => break,
Ok(n) => {
// pulling the num_bytes addition out of the inner loop
// seems to imitate the optimization done by the compiler
// when num_bytes is not passed anywhere by reference.
num_bytes += n;
for i in 0..n {
x += bytes[i];
}
},
Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => panic!("{:?}", e),
};
}
This should answer what happens, but I have no clue why the optimization depends on a reference being used somewhere downstream.