I've been stepping through the Programming Rust book and wanted to observe the two's complement wrapping, so simple code of:
fn main() {
let mut x: u8 = 255;
println!("the value of x is {}", x) ;
x = 255 + 1 ;
println!("The value of x now is {}",x) ;
}
when I try and compile this with Cargo as per the guide, I run
cargo build --release
which in the book says will let it compile without overflow protection, but it won't compile. I get the protection error
|
6 | x = 255 + 1 ;
| ^^^^^^^^^^^ attempt to compute u8::MAX + 1_u8, which would overflow
Can you explain what I'm doing wrong please ?
I believe the value is not checked dynamically during run-time (it wont panic and would overflow) but still statically checked for (if possible) during compile time.
In this case the compiler is able to determine at compile time what you're trying to do and prevents you from doing it.
That being said if you look at the compiler output you can see the following message:
note: #[deny(arithmetic_overflow)] on by default
You'll see this message regardless of the optimization level.
If you'd like to observe the overflow put the following inner attribute at the top of your source file.
#![allow(arithmetic_overflow)]
Or, if you're compiling with rustc directly you can pass the following flags:
-O -A arithmetic_overflow
The rustc docs show that the following lints are on by default (regardless of optimization level)
ambiguous_associated_items
arithmetic_overflow
conflicting_repr_hints
const_err
ill_formed_attribute_input
incomplete_include
invalid_type_param_default
macro_expanded_macro_exports_accessed_by_absolute_paths
missing_fragment_specifier
mutable_transmutes
no_mangle_const_items
order_dependent_trait_objects
overflowing_literals
patterns_in_fns_without_body
pub_use_of_private_extern_crate
soft_unstable
unconditional_panic
unknown_crate_types
useless_deprecated
When you write a literal 255+1 in your code, the compiler evaluates the expression at compile-time and sees the overflow immediately, whether in debug or release mode. When the book says that --release disables overflow protection, it's talking about runtime checks. You can see the difference with this code:
fn increment (x: u8) -> u8 { x + 1 }
fn main() {
let x = 255;
println!("x: {}, x+1: {}", x, increment (x));
}
Playground
If you run this code in debug mode, you get:
thread 'main' panicked at 'attempt to add with overflow', src/main.rs:1:30
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
But if you run it in release mode, you get:
x: 255, x+1: 0
Related
I have this piece of simple code:
let val: u8 = 255 + 1;
println!("{}", val);
It is said here that such a code will compile normally if run with the --release flag.
I am running this code via cargo run --release, and I still see the checks:
error: this arithmetic operation will overflow
--> src/main.rs:2:19
|
2 | let val: u8 = 255 + 1;
| ^^^^^^^ attempt to compute `u8::MAX + 1_u8`, which would overflow
|
= note: `#[deny(arithmetic_overflow)]` on by default
error: could not compile `rust-bin` due to previous error
Am I missing something?
The book is slightly imprecise. Overflow is disallowed in both debug and release modes, it's just that release mode omits runtime checks for performance reasons (replacing them with overflow, which CPUs typically do anyway). Static checks are not removed because they don't compromise on performance of generated code. This prints 0 in release mode and panics in debug1:
let x: u8 = "255".parse().unwrap();
let val: u8 = x + 1;
println!("{}", val);
You can disable the compile-time checks using #[allow(arithmetic_overflow)]. This also prints 0 in release mode and panics in debug:
#[allow(arithmetic_overflow)]
let val: u8 = 255 + 1;
println!("{}", val);
The correct approach is to not depend on this behavior of release mode, but to tell the compiler what you want. This prints 0 in both debug and release mode:
let val: u8 = 255u8.wrapping_add(1);
println!("{}", val);
1
The example uses "255".parse() because, to my surprise, let x = 255u8; let val = x + 1; doesn't compile - in other words, rustc doesn't just prevent overflow in constant arithmetic, but also wherever else it can prove it happens. The change was apparently made in Rust 1.45, because it compiled in Rust 1.44 and older. Since it broke code that previously compiled, the change was technically backward-incompatible, but presumably broke sufficiently few actual crates that it was deemed worth it. Surprising as it is, it's quite possible that "255".parse::<u8>() + 1 will become a compile-time error in a later release.
In your code, the compiler is able to detect the problem. That's why it prevents it even in release mode. In many cases it's not possible or feasible for the compiler to detect or prevent an error.
Just as an example, imagine you have code like this:
let a = b + 5;
Let's say b's value comes from a database, user input or some other external source. It is literally impossible to prevent overflows in cases like that.
How can I hide project info (like source code paths) from panic messages occurred while running a release build?
Assume following code as a Minima Reproducible Example
fn main() {
println!("Hello!");
let v = vec![0];
println!("{:?}", v[1]); // panic - index out of bounds
}
> cargo build --release
> target/release/hello-rust
Getting:
Hello!
thread 'main' panicked at 'index out of bounds: the len is 1 but the index is 1', src/main.rs:5:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Revealing "src/main.rs:5:22" is very unwanted. I would prefer not to reveal even that the executable build with Rust.
I would like just to exit the process with "1" error code on every panic situation, like index out of bounds or .unwrap() on error.
Tried:
[profile.release]
panic = "abort"
Didn't help.
Tried to look in Profile Settings, didn't find something useful.
I'm trying to get the max number of digits in an i32 array. I'm using log10(n) + 1 as my formula to calculate how many digits an i32 has and I thought I would just be able to use that within a max_by_key, but I am getting
thread 'main' panicked at 'attempt to add with overflow', /mnt/f/Personal-Docs/Repos/radix_sort_rs/src/lib/lib.rs:9:60
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Here's my code:
fn get_max_digits<const N: usize>(arr: [i32; N]) -> i32 {
return *arr.iter().max_by_key(|a| a.log10() + 1).unwrap();
}
I'm still new to rust so sorry if this is a simple question.
Figured out the answer to my question not too long after posting. I was trying to do a logarithm of 0 causing undefined. I believe the attempt to add with overflow is the current interaction with 0i32.log10() as it is a nightly only feature.
I thought that once an object is moved, the memory occupied by it on the stack can be reused for other purpose. However, the minimal example below shows the opposite.
#[inline(never)]
fn consume_string(s: String) {
drop(s);
}
fn main() {
println!(
"String occupies {} bytes on the stack.",
std::mem::size_of::<String>()
);
let s = String::from("hello");
println!("s at {:p}", &s);
consume_string(s);
let r = String::from("world");
println!("r at {:p}", &r);
consume_string(r);
}
After compiling the code with --release flag, it gives the following output on my computer.
String occupies 24 bytes on the stack.
s at 0x7ffee3b011b0
r at 0x7ffee3b011c8
It is pretty clear that even if s is moved, r does not reuse the 24-byte chunk on the stack that originally belonged to s. I suppose that reusing the stack memory of a moved object is safe, but why does the Rust compiler not do it? Am I missing any corner case?
Update:
If I enclose s by curly brackets, r can reuse the 24-byte chunk on the stack.
#[inline(never)]
fn consume_string(s: String) {
drop(s);
}
fn main() {
println!(
"String occupies {} bytes on the stack.",
std::mem::size_of::<String>()
);
{
let s = String::from("hello");
println!("s at {:p}", &s);
consume_string(s);
}
let r = String::from("world");
println!("r at {:p}", &r);
consume_string(r);
}
The code above gives the output below.
String occupies 24 bytes on the stack.
s at 0x7ffee2ca31f8
r at 0x7ffee2ca31f8
I thought that the curly brackets should not make any difference, because the lifetime of s ends after calling comsume_string(s) and its drop handler is called within comsume_string(). Why does adding the curly brackets enable the optimization?
The version of the Rust compiler I am using is given below.
rustc 1.54.0-nightly (5c0292654 2021-05-11)
binary: rustc
commit-hash: 5c029265465301fe9cb3960ce2a5da6c99b8dcf2
commit-date: 2021-05-11
host: x86_64-apple-darwin
release: 1.54.0-nightly
LLVM version: 12.0.1
Update 2:
I would like to clarify my focus of this question. I want to know the proposed "stack reuse optimization" lies in which category.
This is an invalid optimization. Under certain cases the compiled code may fail if we perform the "optimization".
This is a valid optimization, but the compiler (including both rustc frontend and llvm) is not capable of performing it.
This is a valid optimization, but is temporarily turned off, like this.
This is a valid optimization, but is missed. It will be added in the future.
My TLDR conclusion: A missed optimization opportunity.
So the first thing I did was look into whether your consume_string function actually makes a difference. To do this I created the following (a bit more) minimal example:
struct Obj([u8; 8]);
fn main()
{
println!(
"Obj occupies {} bytes on the stack.",
std::mem::size_of::<Obj>()
);
let s = Obj([1,2,3,4,5,6,7,8]);
println!("{:p}", &s);
std::mem::drop(s);
let r = Obj([11,12,13,14,15,16,17,18]);
println!("{:p}", &r);
std::mem::drop(r);
}
Instead of consume_string I use std::mem::drop which is dedicated to simply consuming an object. This code behaves just like yours:
Obj occupies 8 bytes on the stack.
0x7ffe81a43fa0
0x7ffe81a43fa8
Removing the drop doesn't affect the result.
So the question is then why rustc doesn't notice that s is dead before r goes live. As your second example shows, enclosing s in a scope will allow the optimization.
Why does this work? Because the Rust semantics dictate that an object is dropped at the end of its scope. Since s is in an inner scope, it is dropped before the scope exits. Without the scope, s is alive until the main function exits.
Why doesn't it work when moving s into a function, where it should be dropped on exit?
Probably because rust doesn't correctly flag the memory location used by s as free after the function call. As has been mentioned in the comments, it is LLVM that actually handles this optimization (called 'Stack Coloring' as far as I can tell), which means rustc must correctly tell it when the memory is no longer in use. Clearly, from your last example, rustc does it on scope exit, but apparently not when an object is moved.
i think the fn drop do not free the memory of S, just call the fn drop.
in first case the s still use the stack memory, rust can not be reused.
in second case, because the {} scope, the memory is free. so the stack memory reused
I tried a recursive factorial algorithm in Rust. I use this version of the compiler:
rustc 1.12.0 (3191fbae9 2016-09-23)
cargo 0.13.0-nightly (109cb7c 2016-08-19)
Code:
extern crate num_bigint;
extern crate num_traits;
use num_bigint::{BigUint, ToBigUint};
use num_traits::One;
fn factorial(num: u64) -> BigUint {
let current: BigUint = num.to_biguint().unwrap();
if num <= 1 {
return One::one();
}
return current * factorial(num - 1);
}
fn main() {
let num: u64 = 100000;
println!("Factorial {}! = {}", num, factorial(num))
}
I got this error:
$ cargo run
thread 'main' has overflowed its stack
fatal runtime error: stack overflow
error: Process didn't exit successfully
How to fix that? And why do I see this error when using Rust?
Rust doesn't have tail call elimination, so your recursion is limited by your stack size. It may be a feature for Rust in the future (you can read more about it at the Rust FAQ), but in the meantime you will have to either not recurse so deep or use loops.
Why?
This is a stack overflow which occurs whenever there is no stack memory left. For example, stack memory is used by
local variables
function arguments
return values
Recursion uses a lot of stack memory, because for every recursive call, the memory for all local variables, function arguments, ... has to be allocated on the stack.
How to fix that?
The obvious solution is to write your algorithm in a non-recursive manner (you should do this when you want to use the algorithm in production!). But you can also just increase the stack size. While the stack size of the main thread can't be modified, you can create a new thread and set a specific stack size:
fn main() {
let num: u64 = 100_000;
// Size of one stack frame for `factorial()` was measured experimentally
thread::Builder::new().stack_size(num as usize * 0xFF).spawn(move || {
println!("Factorial {}! = {}", num, factorial(num));
}).unwrap().join();
}
This code works and, when executed via cargo run --release (with optimization!), outputs the solution after only a couple of seconds calculation.
Measuring stack frame size
In case you want to know how the stack frame size (memory requirement for one call) for factorial() was measured: I printed the address of the function argument num on each factorial() call:
fn factorial(num: u64) -> BigUint {
println!("{:p}", &num);
// ...
}
The difference between two successive call's addresses is (more or less) the stack frame size. On my machine, the difference was slightly less than 0xFF (255), so I just used that as size.
In case you're wondering why the stack frame size isn't smaller: the Rust compiler doesn't really optimize for this metric. Usually it's really not important, so optimizers tend to sacrifice this memory requirement for better execution speed. I took a look at the assembly and in this case many BigUint methods were inlined. This means that the local variables of other functions are using stack space as well!
Just as an alternative.. (I do not recommend)
Matts answer is true to an extent. There is a crate called stacker (here) that can artificially increase the stack size for usage in recursive algorithms. It does this by allocating some heap memory to overflow into.
As a word of warning... this takes a very long time to run ... but, it runs, and it doesn't blow the stack. Compiling with optimizations brings it down but its still pretty slow. You're likely to get better perf from a loop as Matt suggests. I thought I would throw this out there anyway.
extern crate num_bigint;
extern crate num_traits;
extern crate stacker;
use num_bigint::{BigUint, ToBigUint};
use num_traits::One;
fn factorial(num: u64) -> BigUint {
// println!("Called with: {}", num);
let current: BigUint = num.to_biguint().unwrap();
if num <= 1 {
// println!("Returning...");
return One::one();
}
stacker::maybe_grow(1024 * 1024, 1024 * 1024, || {
current * factorial(num - 1)
})
}
fn main() {
let num: u64 = 100000;
println!("Factorial {}! = {}", num, factorial(num));
}
I have commented out the debug printlns.. you can uncomment them if you like.