I was looking at the rust String standard library, and there was so much unsafe code like this one:
#[inline]
#[stable(feature = "rust1", since = "1.0.0")]
pub fn remove(&mut self, idx: usize) -> char {
let ch = match self[idx..].chars().next() {
Some(ch) => ch,
None => panic!("cannot remove a char from the end of a string"),
};
let next = idx + ch.len_utf8();
let len = self.len();
unsafe {
ptr::copy(self.vec.as_ptr().add(next), self.vec.as_mut_ptr().add(idx), len - next);
self.vec.set_len(len - (next - idx));
}
ch
}
why there is so much unsafe code in the standard library?
and how is the language still safe?
There is a misconception here that using unsafe is automatically unsound and will cause memory errors. It does not. In fact, you are not allowed to cause memory errors even in unsafe code blocks; if you do, then the code will exhibit undefined behavior and the whole program is ill-defined. The point of unsafe is to allow things that the compiler cannot ensure are actually safe. That responsibility falls to the developer to ensure the code does not invoke undefined behavior by understanding the safety requirements required to use unsafe syntax, functions, and other items.
The design philosophy for writing and using unsafe functions is if some set of parameters or circumstances may cause a function to exhibit undefined behavior, then it must be marked unsafe and should be documented what the safe parameters and circumstances are. The caller must then abide by this documentation within an unsafe block. The flip side of this design philosophy is that if a function is not marked unsafe, then no possible parameters or circumstances may cause undefined behavior.
In this situation, shifting bytes around in memory is not always safe so you must use unsafe to call ptr::copy. However, the method .remove() is not marked unsafe so whatever happens in the unsafe block must be safe if the developers of the Rust standard library have done their job, and I'm sure they have. You can see that any possible input is bounds-checked and what is being copy'd is within the already allocated block. The only way this could cause undefined behavior is if there was already undefined behavior or broken invariants before calling this function.
You cannot build the Rust standard library without using unsafe. The underlying manual memory management that computers are based on is inherently fraught with memory foot-guns, however you can build off of these "unsafe" operations with guarantees that make them safe.
Some of the unsafe'ty is required, but other instances are simply for performance reasons. Safe abstractions may require many checks to ensure they are safe, especially if any kind of dynamicism is involved, but if your existing invariants are encoded correctly, then using unsafe can avoid those checks while still being safe. In this function, it probably could have been done entirely safely by just relying on other self.vec methods (which would have unsafe internally at some point), but it may include additional bounds checks that would be entirely unnecessary.
The standard library is expected to operate with as little overhead as possible, while staying safe (unless the function is marked unsafe of course).
Related
Rust disallows this kind of code because it is unsafe:
fn main() {
let mut i = 42;
let ref_to_i_1 = unsafe { &mut *(&mut i as *mut i32) };
let ref_to_i_2 = unsafe { &mut *(&mut i as *mut i32) };
*ref_to_i_1 = 1;
*ref_to_i_2 = 2;
}
How can I do do something bad (e.g. segmentation fault, undefined behavior, etc.) with multiple mutable references to the same thing?
The only possible issues I can see come from the lifetime of the data. Here, if i is alive, each mutable reference to it should be ok.
I can see how there might be problems when threads are introduced, but why is it prevented even if I do everything in one thread?
A really common pitfall in C++ programs, and even in Java programs, is modifying a collection while iterating over it, like this:
for (it: collection) {
if (predicate(*it)) {
collection.remove(it);
}
}
For C++ standard library collections, this causes undefined behaviour. Maybe the iteration will work until you get to the last entry, but the last entry will dereference a dangling pointer or read off the end of an array. Maybe the whole array underlying the collection will be relocated, and it'll fail immediately. Maybe it works most of the time but fails if a reallocation happens at the wrong time. In most Java standard collections, it's also undefined behaviour according to the language specification, but the collections tend to throw ConcurrentModificationException - a check which causes a runtime cost even when your code is correct. Neither language can detect the error during compilation.
This is a common example of a data race caused by concurrency, even in a single-threaded environment. Concurrency doesn't just mean parallelism: it can also mean nested computation. In Rust, this kind of mistake is detected during compilation because the iterator has an immutable borrow of the collection, so you can't mutate the collection while the iterator is alive.
An easier-to-understand but less common example is pointer aliasing when you pass multiple pointers (or references) to a function. A concrete example would be passing overlapping memory ranges to memcpy instead of memmove. Actually, Rust's memcpy equivalent is unsafe too, but that's because it takes pointers instead of references. The linked page shows how you can make a safe swap function using the guarantee that mutable references never alias.
A more contrived example of reference aliasing is like this:
int f(int *x, int *y) { return (*x)++ + (*y)++; }
int i = 3;
f(&i, &i); // result is undefined
You couldn't write a function call like that in Rust because you'd have to take two mutable borrows of the same variable.
How can I do do something bad (e.g. segmentation fault, undefined behavior, etc.) with multiple mutable references to the same thing?
I believe that although you trigger 'undefined behavior' by doing this, technically the noalias flag is not used by the Rust compiler for &mut references, so practically speaking, right now, you probably can't actually trigger undefined behavior this way. What you're triggering is 'implementation specific behavior', which is 'behaves like C++ according to LLVM'.
See Why does the Rust compiler not optimize code assuming that two mutable references cannot alias? for more information.
I can see how there might be problems when threads are introduced, but why is it prevented even if I do everything in one thread?
Have a read of this series of blog articles about undefined behavior
In my opinion, race conditions (like iterators) aren't really a good example of what you're talking about; in a single threaded environment you can avoid that sort of problem if you're careful. This is no different to creating an arbitrary pointer to invalid memory and writing to it; just don't do it. You're no worse off than using C.
To understand the issue here, consider when compiling in release mode the compiler may or may not reorder statements when optimizations are performed; that means that although your code may run in the linear sequence:
a; b; c;
There is no guarantee the compiler will execute them in that sequence when it runs, if (according to what the compiler knows), there is no logical reason that the statements must be performed in a specific atomic sequence. Part 3 of the blog I've linked to above demonstrates how this can cause undefined behavior.
tl;dr: Basically, the compiler may perform various optimizations; these are guaranteed to continue to make your program behave in a deterministic fashion if and only if your program does not trigger undefined behavior.
As far as I'm aware the Rust compiler currently doesn't use many 'advanced optimizations' that may cause this kind of failure, but there is no guarantee that it won't in the future. It is not a 'breaking change' to introduce new compiler optimizations.
So... it's actually probably quite unlikely you'll be able to trigger actual undefined behavior just via mutable aliasing right now; but the restriction allows the possibility of future performance optimizations.
Pertinent quote:
The C FAQ defines “undefined behavior” like this:
Anything at all can happen; the Standard imposes no requirements. The program may fail to compile, or it may execute incorrectly (either crashing or silently generating incorrect results), or it may fortuitously do exactly what the programmer intended.
Author's Note: The following answer was originally written for How do intertwined scopes create a "data race"?
The compiler is allowed to optimize &mut pointers under the assumption that they are exclusive (not aliased). Your code breaks this assumption.
The example in the question is a little too trivial to exhibit any kind of interesting wrong behavior, but consider passing ref_to_i_1 and ref_to_i_2 to a function that modifies both and then does something with them:
fn main() {
let mut i = 42;
let ref_to_i_1 = unsafe { &mut *(&mut i as *mut i32) };
let ref_to_i_2 = unsafe { &mut *(&mut i as *mut i32) };
foo(ref_to_i_1, ref_to_i_2);
}
fn foo(r1: &mut i32, r2: &mut i32) {
*r1 = 1;
*r2 = 2;
println!("{}", r1);
println!("{}", r2);
}
The compiler may (or may not) decide to de-interleave the accesses to r1 and r2, because they are not allowed to alias:
// The following is an illustration of how the compiler might rearrange
// side effects in a function to optimize it. Optimization passes in the
// compiler actually work on (MIR and) LLVM IR, not on raw Rust code.
fn foo(r1: &mut i32, r2: &mut i32) {
*r1 = 1;
println!("{}", r1);
*r2 = 2;
println!("{}", r2);
}
It might even realize that the println!s always print the same value and take advantage of that fact to further rearrange foo:
fn foo(r1: &mut i32, r2: &mut i32) {
println!("{}", 1);
println!("{}", 2);
*r1 = 1;
*r2 = 2;
}
It's good that a compiler can do this optimization! (Even if Rust's currently doesn't, as Doug's answer mentions.) Optimizing compilers are great because they can use transformations like those above to make code run faster (for instance, by better pipelining the code through the CPU, or by enabling the compiler to do more aggressive optimizations in a later pass). All else being equal, everybody likes their code to run fast, right?
You might say "Well, that's an invalid optimization because it doesn't do the same thing." But you'd be wrong: the whole point of &mut references is that they do not alias. There is no way to make r1 and r2 alias without breaking the rules†, which is what makes this optimization valid to do.
You might also think that this is a problem that only appears in more complicated code, and the compiler should therefore allow the simple examples. But bear in mind that these transformations are part of a long multi-step optimization process. It's important to uphold the properties of &mut references everywhere, so that the compiler can make minor optimizations to one section of code without needing to understand all the code.
One more thing to consider: it is your job as the programmer to choose and apply the appropriate types for your problem; asking the compiler for occasional exceptions to the &mut aliasing rule is basically asking it to do your job for you.
If you want shared mutability and to forego those optimizations, it's simple: don't use &mut. In the example, you can use &Cell<i32> instead of &mut i32, as the comments mentioned:
fn main() {
let mut i = std::cell::Cell::new(42);
let ref_to_i_1 = &i;
let ref_to_i_2 = &i;
foo(ref_to_i_1, ref_to_i_2);
}
fn foo(r1: &Cell<i32>, r2: &Cell<i32>) {
r1.set(1);
r2.set(2);
println!("{}", r1.get()); // prints 2, guaranteed
println!("{}", r2.get()); // also prints 2
}
The types in std::cell provide interior mutability, which is jargon for "disallow certain optimizations because & references may mutate things". They aren't always quite as convenient as using &mut, but that's because using them gives you more flexibility to write code like the above.
Also read
The Problem With Single-threaded Shared Mutability describes how having multiple mutable references can cause soundness issues even in the absence of multiple threads and data races.
Dan Hulme's answer illustrates how aliased mutation of more complex data can also cause undefined behavior (even before compiler optimizations).
† Be aware that using unsafe by itself does not count as "breaking the rules". &mut references cannot be aliased, even when using unsafe, in order for your code to have defined behavior.
The simplest example I know of is trying to push into a Vec that's borrowed:
let mut v = vec!['a'];
let c = &v[0];
v.push('b');
dbg!(c);
This is a compiler error:
error[E0502]: cannot borrow `v` as mutable because it is also borrowed as immutable
--> src/main.rs:4:5
|
3 | let c = &v[0];
| - immutable borrow occurs here
4 | v.push('b');
| ^^^^^^^^^^^ mutable borrow occurs here
5 | dbg!(c);
| - immutable borrow later used here
It's good that this is a compiler error, because otherwise it would be a use-after-free. push reallocates the Vec's heap storage and invalidates our c reference. Rust doesn't actually know what push does; all Rust knows is that push takes &mut self, and here that violates the aliasing rule.
Many other single-threaded examples of undefined behavior involve destroying objects on the heap like this. But if we play around a bit with references and enums, we can express something similar without heap allocation:
enum MyEnum<'a> {
Ptr(&'a i32),
Usize(usize),
}
let my_int = 42;
let mut my_enum = MyEnum::Ptr(&my_int);
let my_int_ptr_ptr: &&i32 = match &my_enum {
MyEnum::Ptr(i) => i,
MyEnum::Usize(_) => unreachable!(),
};
my_enum = MyEnum::Usize(0xdeadbeefdeadbeef);
dbg!(**my_int_ptr_ptr);
Here we've taken a pointer to my_int, stored that pointer in my_enum, and made my_int_ptr_ptr point into my_enum. If we could then reassign my_enum, we could trash the bits that my_int_ptr_ptr was pointing to. A double dereference of my_int_ptr_ptr would be a wild pointer read, which would probably segfault. Luckily, this it another violation of the aliasing rule, and it won't compile:
error[E0506]: cannot assign to `my_enum` because it is borrowed
--> src/main.rs:12:1
|
8 | let my_int_ptr_ptr: &&i32 = match &my_enum {
| -------- borrow of `my_enum` occurs here
...
12 | my_enum = MyEnum::Usize(0xdeadbeefdeadbeef);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ assignment to borrowed `my_enum` occurs here
13 | dbg!(**my_int_ptr_ptr);
| ---------------- borrow later used here
The term "aliasing" is typically used to identify situations where changing the order of operations involving different references would change the effect of those operations. If multiple references to an object are stored in different places, but the object is not modified during the lifetime of those references, compilers may usefully hoist, defer, or consolidate operations using those references without affecting program behavior.
For example, if a compiler sees that code reads the contents of an object referenced by x, then does something with an object referenced by y, and again reads the contents of the object referenced by x, and if the compiler knows that the action on y cannot have modified the object referenced by x, the compiler may consolidate both reads of x into a single read.
Determining in all cases whether an action on one reference might affect another is would be an intractable problem if programmers had unlimited freedom to use and store references however they saw fit. Rust, however, seeks to handle the two easy cases:
If an object will never be modified during the lifetime of a reference, machine code using the reference won't have to worry about what operations might change it during that lifetime, since it would be impossible for any operations to do so.
If during the lifetime of a reference, an object will only be modified by references that are visibly based upon that reference, machine code using that reference won't have to worry about whether any operations using that reference will interact with operations involving references that appear to be unrelated, because no seemingly-unrelated references will identify the same object.
Allowing for the possibility of aliasing between mutable references would make things much more complicated, since many optimizations which could be performed interchangeably with unshared references to mutable objects or shareable references to immutable ones could no longer do so. Once a language supports situations where operations involving seemingly independent references need to be processed in precisely ordered fashion, it's hard for compilers to know when such precise sequencing is required.
This question already has an answer here:
replace a value behind a mutable reference by moving and mapping the original
(1 answer)
Closed 1 year ago.
Is a function that modifies a &mut T in place by a function FnOnce(T) -> T safe to have in rust, or can it lead to undefined behavior? Is it included in the standard library somewhere, or a well-known crate?
If you additionally assume T: Default, that looks like
fn modify<T, F: FnOnce(T) -> T>(x: &mut T, f: F) -> ()
where
T: Default
{
let val = std::mem::take(x);
let val = f(val);
*x = val;
}
(See also
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f015812bac6f527fe663fe4e0b7a3188)
My question is about doing the same but dropping the where T: Default clause (and no T: Clone either). This requires a different implementation, since you can't use std::mem::take.
I'm not sure how to implement the unconstrained version, but it should be possible using unsafe Rust.
I'm learning Rust from a background of linear types and sub-structural logic. Rust's mutable borrow seems very similar to moving a resource in and then back out of a function, but I don't know if it is actually safe to take temporary ownership of the contents of a mutable borrow like this.
It is safe, and there are even crates for that (can't find them now).
HOWEVER.
When writing unsafe code, you have to be very careful. If you don't know exactly what you're doing, it can easily lead to UB.
Here, for example, there is something you maybe haven't thought of: panic safety.
Suppose we implement that trivially:
pub fn modify<T, F: FnOnce(T) -> T>(v: &mut T, f: F) {
let prev = unsafe { std::ptr::read(v) };
let new = f(prev);
unsafe { std::ptr::write(v, new) };
}
Trivially right.
Or is it?
fn main() {
struct MyStruct(pub i32);
impl Drop for MyStruct {
fn drop(&mut self) {
println!("MyStruct({}) dropped", self.0);
}
}
let mut v = MyStruct(123);
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
modify(&mut v, |_prev| {
// `prev` is dropped here.
panic!("Haha, evil panic!");
})
}))
.unwrap_err();
v.0 = 456; // Writing to an uninitialized memory!
// `v` is dropped here, double drop!
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6f7312a8be70cd43cf5cf7a9816be56a
I used a custom type that its destructor does nothing but to print, but imagine what could happen if this was a Vec that freed the memory and we were writing into freed memory (then, as a bonus, get a double-free).
It is correct, like #Kendas said, that when there are no interruption point it is valid to leave memory in an uninitialized state in Rust. The problem is, that much more places than you wish are actually interruption points. In fact, when writing unsafe code, you have to consider any call to external code (i.e. not yours code neither code that you trust to not do bad things, for example std) to be an interruption point.
Unsafe code is hard. Better stay in the safe land.
Edit: You may wonder what the AssertUnwindSafe is. Maybe you even tried to remove it and noticed it doesn't compiler. Well, UnwindSafe is a protection against this, and AssertUnwindSafe is a way to bypass the protection.
You may ask, what's the point? The point is, this protection is really not accurate. So much not accurate, that bypassing it does not even require unsafe. But it still exists, so we have a lower chance of accidental UB.
It doesn't matter to you as the writer of the API - you should act like this protection doesn't exist, because it is safe to bypass it and easy to do so by mistake. The Rust standard library itself had bugs like that in the past (#86443, #81740, ... - It is not an accident that they're both in the same code - those issues tend to appear in chunks. But there're more).
Well, you are replacing the contents of the borrowed memory location with the default value. This would mean that the memory is indeed correct at every point. So there should not be any undefined behavior.
Basically from the perspective of the mutable reference x, you are mutating it to the default value, then mutating it again to a different new value.
In general, if there is a chance of undefined behavior, you will need to use the unsafe keyword. Or somebody has made a mistake while using the unsafe keyword further down the stack. It is relatively rare for these things to happen in the standard library.
Go ahead and look at the safety remarks in the code if you must: https://doc.rust-lang.org/src/core/mem/mod.rs.html#756
I have a function that takes a usize equivalent to a pointer, and aligns it up to the next alignment point.
It doesn't require any unsafe as it's side effect free, but the alignment must be a power of two with this implementation. This means that if you use the function with bad parameters, you might get undefined behaviour later down the line. I can't check for this inside the function itself with assert! as it's supposed to be very fast.
/// Align the given address `addr` upwards to alignment `align`.
///
/// Unsafe as `align` must be a power of two.
unsafe fn align_next_unsafe(addr: usize, align: usize) -> usize {
(addr + align - 1) & !(align - 1)
}
Currently, I've made this unsafe for the above reasons, but I'm not sure if that's best practice. Should I only define a function as unsafe if it has side effects? Or is this a valid time to require an unsafe block?
I'll preface this by saying this is a fairly opinion-heavy answer, and represents a point of view, rather than "the truth".
Consider this code taken from the Vec docs:
let x = vec![1, 2, 4];
let x_ptr = x.as_ptr();
unsafe {
for i in 0..x.len() {
assert_eq!(*x_ptr.add(i), 1 << i);
}
}
The function you're describing seems to have a similar safety profile to Vec::as_ptr. Vec::as_ptr is not unsafe, and does nothing particularly bad on its own; having an invalid *const T isn't bad until you dereference it. That's why dereferencing the raw pointer requires unsafe.
Similarly, I'd argue that align_next doesn't do anything particularly bad unless that value is then passed into some unsafe context. As with any question of unsafe, it's a tradeoff between safety/risk and ergonomics.
In Vec::as_ptr's case, the risk is relatively low; the stdlib has lots of eyes on it, and is well "battle-tested". Moreover, it is a single function with a single implementation.
If your align_next was a function on a trait, I'd be much more tempted to make it unsafe, since someone in the future could implement it badly, and you might have other code whose safety relies on a correct implementation of align_next.
However, in your case, I'd say the pattern is similar to Vec::as_ptr, and you should make sure that any functions that consume this value are marked unsafe if they can cause UB.
I'd also second Martin Gallagher's point about creating a Result returning variant and benchmarking (you could also try an Option<usize>-returning API to make use of null-pointer optimizations).
Rust disallows this kind of code because it is unsafe:
fn main() {
let mut i = 42;
let ref_to_i_1 = unsafe { &mut *(&mut i as *mut i32) };
let ref_to_i_2 = unsafe { &mut *(&mut i as *mut i32) };
*ref_to_i_1 = 1;
*ref_to_i_2 = 2;
}
How can I do do something bad (e.g. segmentation fault, undefined behavior, etc.) with multiple mutable references to the same thing?
The only possible issues I can see come from the lifetime of the data. Here, if i is alive, each mutable reference to it should be ok.
I can see how there might be problems when threads are introduced, but why is it prevented even if I do everything in one thread?
A really common pitfall in C++ programs, and even in Java programs, is modifying a collection while iterating over it, like this:
for (it: collection) {
if (predicate(*it)) {
collection.remove(it);
}
}
For C++ standard library collections, this causes undefined behaviour. Maybe the iteration will work until you get to the last entry, but the last entry will dereference a dangling pointer or read off the end of an array. Maybe the whole array underlying the collection will be relocated, and it'll fail immediately. Maybe it works most of the time but fails if a reallocation happens at the wrong time. In most Java standard collections, it's also undefined behaviour according to the language specification, but the collections tend to throw ConcurrentModificationException - a check which causes a runtime cost even when your code is correct. Neither language can detect the error during compilation.
This is a common example of a data race caused by concurrency, even in a single-threaded environment. Concurrency doesn't just mean parallelism: it can also mean nested computation. In Rust, this kind of mistake is detected during compilation because the iterator has an immutable borrow of the collection, so you can't mutate the collection while the iterator is alive.
An easier-to-understand but less common example is pointer aliasing when you pass multiple pointers (or references) to a function. A concrete example would be passing overlapping memory ranges to memcpy instead of memmove. Actually, Rust's memcpy equivalent is unsafe too, but that's because it takes pointers instead of references. The linked page shows how you can make a safe swap function using the guarantee that mutable references never alias.
A more contrived example of reference aliasing is like this:
int f(int *x, int *y) { return (*x)++ + (*y)++; }
int i = 3;
f(&i, &i); // result is undefined
You couldn't write a function call like that in Rust because you'd have to take two mutable borrows of the same variable.
How can I do do something bad (e.g. segmentation fault, undefined behavior, etc.) with multiple mutable references to the same thing?
I believe that although you trigger 'undefined behavior' by doing this, technically the noalias flag is not used by the Rust compiler for &mut references, so practically speaking, right now, you probably can't actually trigger undefined behavior this way. What you're triggering is 'implementation specific behavior', which is 'behaves like C++ according to LLVM'.
See Why does the Rust compiler not optimize code assuming that two mutable references cannot alias? for more information.
I can see how there might be problems when threads are introduced, but why is it prevented even if I do everything in one thread?
Have a read of this series of blog articles about undefined behavior
In my opinion, race conditions (like iterators) aren't really a good example of what you're talking about; in a single threaded environment you can avoid that sort of problem if you're careful. This is no different to creating an arbitrary pointer to invalid memory and writing to it; just don't do it. You're no worse off than using C.
To understand the issue here, consider when compiling in release mode the compiler may or may not reorder statements when optimizations are performed; that means that although your code may run in the linear sequence:
a; b; c;
There is no guarantee the compiler will execute them in that sequence when it runs, if (according to what the compiler knows), there is no logical reason that the statements must be performed in a specific atomic sequence. Part 3 of the blog I've linked to above demonstrates how this can cause undefined behavior.
tl;dr: Basically, the compiler may perform various optimizations; these are guaranteed to continue to make your program behave in a deterministic fashion if and only if your program does not trigger undefined behavior.
As far as I'm aware the Rust compiler currently doesn't use many 'advanced optimizations' that may cause this kind of failure, but there is no guarantee that it won't in the future. It is not a 'breaking change' to introduce new compiler optimizations.
So... it's actually probably quite unlikely you'll be able to trigger actual undefined behavior just via mutable aliasing right now; but the restriction allows the possibility of future performance optimizations.
Pertinent quote:
The C FAQ defines “undefined behavior” like this:
Anything at all can happen; the Standard imposes no requirements. The program may fail to compile, or it may execute incorrectly (either crashing or silently generating incorrect results), or it may fortuitously do exactly what the programmer intended.
Author's Note: The following answer was originally written for How do intertwined scopes create a "data race"?
The compiler is allowed to optimize &mut pointers under the assumption that they are exclusive (not aliased). Your code breaks this assumption.
The example in the question is a little too trivial to exhibit any kind of interesting wrong behavior, but consider passing ref_to_i_1 and ref_to_i_2 to a function that modifies both and then does something with them:
fn main() {
let mut i = 42;
let ref_to_i_1 = unsafe { &mut *(&mut i as *mut i32) };
let ref_to_i_2 = unsafe { &mut *(&mut i as *mut i32) };
foo(ref_to_i_1, ref_to_i_2);
}
fn foo(r1: &mut i32, r2: &mut i32) {
*r1 = 1;
*r2 = 2;
println!("{}", r1);
println!("{}", r2);
}
The compiler may (or may not) decide to de-interleave the accesses to r1 and r2, because they are not allowed to alias:
// The following is an illustration of how the compiler might rearrange
// side effects in a function to optimize it. Optimization passes in the
// compiler actually work on (MIR and) LLVM IR, not on raw Rust code.
fn foo(r1: &mut i32, r2: &mut i32) {
*r1 = 1;
println!("{}", r1);
*r2 = 2;
println!("{}", r2);
}
It might even realize that the println!s always print the same value and take advantage of that fact to further rearrange foo:
fn foo(r1: &mut i32, r2: &mut i32) {
println!("{}", 1);
println!("{}", 2);
*r1 = 1;
*r2 = 2;
}
It's good that a compiler can do this optimization! (Even if Rust's currently doesn't, as Doug's answer mentions.) Optimizing compilers are great because they can use transformations like those above to make code run faster (for instance, by better pipelining the code through the CPU, or by enabling the compiler to do more aggressive optimizations in a later pass). All else being equal, everybody likes their code to run fast, right?
You might say "Well, that's an invalid optimization because it doesn't do the same thing." But you'd be wrong: the whole point of &mut references is that they do not alias. There is no way to make r1 and r2 alias without breaking the rules†, which is what makes this optimization valid to do.
You might also think that this is a problem that only appears in more complicated code, and the compiler should therefore allow the simple examples. But bear in mind that these transformations are part of a long multi-step optimization process. It's important to uphold the properties of &mut references everywhere, so that the compiler can make minor optimizations to one section of code without needing to understand all the code.
One more thing to consider: it is your job as the programmer to choose and apply the appropriate types for your problem; asking the compiler for occasional exceptions to the &mut aliasing rule is basically asking it to do your job for you.
If you want shared mutability and to forego those optimizations, it's simple: don't use &mut. In the example, you can use &Cell<i32> instead of &mut i32, as the comments mentioned:
fn main() {
let mut i = std::cell::Cell::new(42);
let ref_to_i_1 = &i;
let ref_to_i_2 = &i;
foo(ref_to_i_1, ref_to_i_2);
}
fn foo(r1: &Cell<i32>, r2: &Cell<i32>) {
r1.set(1);
r2.set(2);
println!("{}", r1.get()); // prints 2, guaranteed
println!("{}", r2.get()); // also prints 2
}
The types in std::cell provide interior mutability, which is jargon for "disallow certain optimizations because & references may mutate things". They aren't always quite as convenient as using &mut, but that's because using them gives you more flexibility to write code like the above.
Also read
The Problem With Single-threaded Shared Mutability describes how having multiple mutable references can cause soundness issues even in the absence of multiple threads and data races.
Dan Hulme's answer illustrates how aliased mutation of more complex data can also cause undefined behavior (even before compiler optimizations).
† Be aware that using unsafe by itself does not count as "breaking the rules". &mut references cannot be aliased, even when using unsafe, in order for your code to have defined behavior.
The simplest example I know of is trying to push into a Vec that's borrowed:
let mut v = vec!['a'];
let c = &v[0];
v.push('b');
dbg!(c);
This is a compiler error:
error[E0502]: cannot borrow `v` as mutable because it is also borrowed as immutable
--> src/main.rs:4:5
|
3 | let c = &v[0];
| - immutable borrow occurs here
4 | v.push('b');
| ^^^^^^^^^^^ mutable borrow occurs here
5 | dbg!(c);
| - immutable borrow later used here
It's good that this is a compiler error, because otherwise it would be a use-after-free. push reallocates the Vec's heap storage and invalidates our c reference. Rust doesn't actually know what push does; all Rust knows is that push takes &mut self, and here that violates the aliasing rule.
Many other single-threaded examples of undefined behavior involve destroying objects on the heap like this. But if we play around a bit with references and enums, we can express something similar without heap allocation:
enum MyEnum<'a> {
Ptr(&'a i32),
Usize(usize),
}
let my_int = 42;
let mut my_enum = MyEnum::Ptr(&my_int);
let my_int_ptr_ptr: &&i32 = match &my_enum {
MyEnum::Ptr(i) => i,
MyEnum::Usize(_) => unreachable!(),
};
my_enum = MyEnum::Usize(0xdeadbeefdeadbeef);
dbg!(**my_int_ptr_ptr);
Here we've taken a pointer to my_int, stored that pointer in my_enum, and made my_int_ptr_ptr point into my_enum. If we could then reassign my_enum, we could trash the bits that my_int_ptr_ptr was pointing to. A double dereference of my_int_ptr_ptr would be a wild pointer read, which would probably segfault. Luckily, this it another violation of the aliasing rule, and it won't compile:
error[E0506]: cannot assign to `my_enum` because it is borrowed
--> src/main.rs:12:1
|
8 | let my_int_ptr_ptr: &&i32 = match &my_enum {
| -------- borrow of `my_enum` occurs here
...
12 | my_enum = MyEnum::Usize(0xdeadbeefdeadbeef);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ assignment to borrowed `my_enum` occurs here
13 | dbg!(**my_int_ptr_ptr);
| ---------------- borrow later used here
The term "aliasing" is typically used to identify situations where changing the order of operations involving different references would change the effect of those operations. If multiple references to an object are stored in different places, but the object is not modified during the lifetime of those references, compilers may usefully hoist, defer, or consolidate operations using those references without affecting program behavior.
For example, if a compiler sees that code reads the contents of an object referenced by x, then does something with an object referenced by y, and again reads the contents of the object referenced by x, and if the compiler knows that the action on y cannot have modified the object referenced by x, the compiler may consolidate both reads of x into a single read.
Determining in all cases whether an action on one reference might affect another is would be an intractable problem if programmers had unlimited freedom to use and store references however they saw fit. Rust, however, seeks to handle the two easy cases:
If an object will never be modified during the lifetime of a reference, machine code using the reference won't have to worry about what operations might change it during that lifetime, since it would be impossible for any operations to do so.
If during the lifetime of a reference, an object will only be modified by references that are visibly based upon that reference, machine code using that reference won't have to worry about whether any operations using that reference will interact with operations involving references that appear to be unrelated, because no seemingly-unrelated references will identify the same object.
Allowing for the possibility of aliasing between mutable references would make things much more complicated, since many optimizations which could be performed interchangeably with unshared references to mutable objects or shareable references to immutable ones could no longer do so. Once a language supports situations where operations involving seemingly independent references need to be processed in precisely ordered fashion, it's hard for compilers to know when such precise sequencing is required.
Saying the same thing in code, when would I pick either of the following examples?
unsafe trait MyCoolTrait {
fn method(&self) -> u8;
}
trait MyCoolTrait {
unsafe fn method(&self) -> u8;
}
The opt-in builtin traits (OIBIT) RFC states:
An unsafe trait is a trait that is unsafe to implement, because it represents some kind of trusted assertion. Note that unsafe traits are perfectly safe to use. Send and Share (note: now called Sync) are examples of unsafe traits: implementing these traits is effectively an assertion that your type is safe for threading.
There's another example of an unsafe trait in the standard library, Searcher. It says:
The trait is marked unsafe because the indices returned by the next() methods are required to lie on valid utf8 boundaries in the haystack. This enables consumers of this trait to slice the haystack without additional runtime checks.
Unfortunately, neither of these paragraphs really help my understanding of when it is correct to mark the entire trait unsafe instead of some or all of the methods.
I've asked about marking a function as unsafe before, but this seems different.
A function is marked unsafe to indicate that it is possible to violate memory safety by calling it. A trait is marked unsafe to indicate that it is possible to violate memory safety by implementing it at all. This is commonly because the trait has invariants that other unsafe code relies on being upheld, and that these invariants cannot be expressed any other way.
In the case of Searcher, the methods themselves should be safe to call. That is, users should not have to worry about whether or not they're using a Searcher correctly; the interface contract says all calls are safe. There's nothing you can do that will cause the methods to violate memory safety.
However, unsafe code will be calling the methods of a Searcher, and such unsafe code will be relying on a given Searcher implementation to return offsets that are on valid UTF-8 code point boundaries. If this assumption is violated, then the unsafe code could end up causing a memory safety violation itself.
To put it another way: the correctness of unsafe code using Searchers depends on every single Searcher implementation also being correct. Or: implementing this trait incorrectly allows for safe code to induce a memory safety violation is unrelated unsafe code.
So why not just mark the methods unsafe? Because they aren't unsafe at all! They don't do anything that could violate memory safety in and of themselves. next_match just scans for and returns an Option<(usize, usize)>. The danger only exists when unsafe code assumes that these usizes are valid indices into the string being searched.
So why not just check the result? Because that'd be slower. The searching code wants to be fast, which means it wants to avoid redundant checks. But those checks can't be expressed in the Searcher interface... so instead, the whole trait is flagged as unsafe to warn anyone implementing it that there are extra conditions not stated or enforced in the code that must be respected.
There's also Send and Sync: implementing those when you shouldn't violates the expectations of (among other things) code that has to deal with threads. The code that lets you create threads is safe, but only so long as Send and Sync are only implemented on types for which they're appropriate.
The rule of thumb will be like this:
Use unsafe fn method() if the method user need to wrap method call in unsafe block.
Use unsafe trait MyTrait if the trait implementor need to unsafe impl MyTrait.
unsafe is a tip to Rust user: unsafe code must be written carefully.
The key point is that unsafe should be used as dual: when an author declare a trait/function as unsafe, the implementor/user need to implement/use it with unsafe.
When function is marked as unsafe, it means the user needs to use the function carefully. The function author is making assumption that the function user must keep.
When trait is marked as unsafe, it means the trait implementor needs to implement carefully. The trait requires implementor keeps certain assumption. But users of the unsafe trait can insouciantly call methods defined in the trait.
For concrete example, unsafe trait Searcher requires all Searcher implementation should return valid utf8 boundary when calling next. And all implementation are marked as unsafe impl Searcher, indicating implementation code might be unsafe. But as a user of Searcher, one can call searcher.next() without wrapping it in unsafe block.