I'm strugglin' with the borrow checker - wonder o' wonder.
While I found a solution by adding a block, I am curious if there are other ways to end a mutable borrow so the next statement can access a binding afterwards.
Here's what I did so far:
let mut canvas: Canvas = Canvas {
width: 5,
height: 5,
array: vec!['x'; 5*5],
};
{
let mut renderer: CanvasRenderer = CanvasRenderer::new(&mut canvas);
renderer.render_point('x', 3, 3);
}
println!("The Value in the array is: {}", canvas.array[9]);
I wrap a block around the binding of a CanvasRenderer object and after mutating the canvas and the scope ends, the CanvasRenderer dies and my mutable borrowed canvas is available to be read or whatever.
This works - but now I'd like to see other solutions!
I heard about drop(stuff) but it did not work as I thought it should.
There is no other way; using blocks is the way to do it. Before Rust 2018 (available in Rust 1.31) all borrows are lexical, that is, they always correspond to some lexical scope. The only scope which is larger than a single statement is that of a block, so blocks are your only tool to limit borrow scopes.
drop() would not work for two reasons: first, because it would require non-lexical scope which is not supported before Rust 2018, and second, it cannot be a general-purpose tool for managing borrows: for example, it wouldn't be able to end an immutable borrow simply because immutable references are Copy and can't be "dropped".
See also:
Moved variable still borrowing after calling `drop`?
What are non-lexical lifetimes?
Related
I'm having difficulty understanding why Rust's borrow checker does not allow multiple mutable borrows when it is safe to do so.
Let's give an example:
fn borrow_mut(s : &mut String) {
s.push_str(" world!");
println!("{}", s);
}
fn main() {
let mut s = String::from("hello");
let rs : &mut String = &mut s;
// second mutable borrow
borrow_mut(&mut s);
println!("{rs}");
}
This code fails to compile with the following message:
error[E0499]: cannot borrow `s` as mutable more than once at a time
--> main.rs:11:16
|
8 | let rs : &mut String = &mut s;
| ------ first mutable borrow occurs here
...
11 | borrow_mut(&mut s);
| ^^^^^^ second mutable borrow occurs here
12 |
13 | println!("{rs}");
| -- first borrow later used here
rs points on the variable of type String in the stack frame. String contains pointer on memory in the heap. So even if the string reallocates its data in borrow_mut(), both pointers are still valid, so this code should be safe.
Could someone explain the reason of why the borrow checker prevents multiple mutable borrows even when it's safe?
It's for thread safety, to avoid data races. If two such mutable borrowings can exist, then two threads of execution can both attempt to modify the original data. If they do, all sorts of nasty race conditions can arise, e.g. if both threads try to append to the string:
The underlying array holding the data can get reallocated twice, with one of them leaked
The appended data could end up writing out of bounds due to time-of-check/time-of-use issues
You could end up with inconsistent definitions of the length and capacity
On some architectures and data sizes, tearing could mean a single logical value is read half as the old version and half as the updated value (producing something that could easily be unrelated to either the old or new value)
etc.
Borrows as a language feature mean that the function can temporarily hand off its unique mutable-ownership to some other function; while that other function holds the borrow, the original object can't be accessed through anything but that mutable borrow. It also means that for non-mutable borrows, it can prevent mutable borrows that might causes races between reads through the non-mutable borrow and writes through the mutable borrow. The borrow checker is preventing you from launching a thread that modifies s, then calling borrow_mut from the main thread, and the two threads producing garbage or crashing the program when they modify s simultaneously.
To be clear, with an advanced borrow-checker in some future version of Rust, this code could be made to work (the code you wrote does nothing inherently unsafe). But fully analyzing deep code paths to ensure nothing evil could possibly occur is hard, and it's relatively easy to impose stricter rules (which might be loosened in the future if they're sure it won't impose restrictions on the language design that bite them later). Your code would work just fine if you passed the single mutable borrow you already had into borrow_mut after all; your code is not made worse by doing things The Rust Way™.
So I'm pursuing my Rust adventures (loving it) and I'm exploring threads. As usual I stumbled upon an error that I do not understand.
Here is a minimal example:
use std::thread;
pub fn compute_something(input: &Vec<&usize>) -> usize {
input.iter().map(|v| *v).sum()
}
pub fn main() {
let items = vec![0, 1, 2, 3, 4, 5];
let mut slice: Vec<&usize> = Vec::new();
slice.push(&items[1]); // borrowed value does not live long enough
// argument requires that `items` is borrowed for `'static`
slice.push(&items[2]); // borrowed value does not live long enough
// argument requires that `items` is borrowed for `'static`
assert_eq!(3, compute_something(&slice));
let h = thread::spawn(move || compute_something(&slice));
match h.join() {
Ok(result) => println!("Result: {:?}", result),
Err(e) => println!("Nope: {:?}", e)
}
} // `items` dropped here while still borrowed
I have of course made a playground to illustrate.
If I drop the thread part (everything after the assert_eq! line) and just call compute_something(&slice) it compiles fine.
There are three main things I don't understand here:
Why is it a problem to drop items while borrowed at the end of the program, shouldn't the runtime clean-up the memory just fine? It's not like I'm gonna be able to access slice outside of main.
What is still borrowing items at the end of the program? slice? If so, why does that same program compile by just removing everything after the assert_eq! line? I can't see how it changes the borrowing pattern.
Why is calling compute_something from inside the thread's closure creating the issue and how do I solve it?
You move slice into the closure that you pass to thread::spawn(). Since the closure passed to thread::spawn() must be 'static, this implies that the vector being moved into the closure must not borrow anything that isn't 'static either. The compiler therefore deduces the type of slice to be Vec<&'static usize>.
But it does borrow something that's not 'static -- the values that you try to push into it borrow from a variable local to main(), and so the compiler complains about this.
The simplest way to fix this case is to have slice be a Vec<usize> and not borrow from items at all.
Another option is to use scoped threads from the crossbeam crate, which know how to borrow from local variables safely by enforcing that all threads are joined before a scope ends.
To directly answer the questions you posed:
Why is it a problem to drop items while borrowed at the end of the program, shouldn't the runtime clean-up the memory just fine?
When main() terminates, all threads are also terminated -- however, there is a brief window of time during which the values local to main() have been destroyed but before the threads are terminated. There can exist dangling references during this window, and that violates Rust's memory safety model. This is why thread::spawn() requires a 'static closure.
Even though you join the thread yourself, the borrow checker doesn't know that joining the thread ends the borrow. (This is the problem that crossbeam's scoped threads solve.)
What is still borrowing items at the end of the program?
The vector that was moved into the closure is still borrowing items.
Why is calling compute_something from inside the thread's closure creating the issue and how do I solve it?
Calling this function isn't creating the issue. Moving slice into the closure is creating the issue.
Here is the way I solved this issue.
I used Box::Leak: https://doc.rust-lang.org/std/boxed/struct.Box.html#method.leak
let boxed_data = data.into_boxed_slice();
let boxed_data_static_ref = Box::leak(boxed_data);
let compressed_data = &boxed_data_static_ref[start_of_data..start_of_data+zfr.compressed_size as usize];
let handles = (0..NUM_THREADS).map(|thread_nr| {
thread::spawn(move || {
main2(thread_nr, compressed_data, zfr.crc);
})
}).collect::<Vec<_>>();
for h in handles {
h.join().unwrap();
}
I've read What are non-lexical lifetimes?. With the non-lexical borrow checker, the following code compiles:
fn main() {
let mut scores = vec![1, 2, 3];
let score = &scores[0]; // borrows `scores`, but never used
// its lifetime can end here
scores.push(4); // borrows `scores` mutably, and succeeds
}
It seems reasonable in the case above, but when it comes to a mutex lock, we don't want it to be released prematurely.
In the following code, I would like to lock a shared structure first and then execute a closure, mainly to avoid deadlock. However, I'm not sure if the lock will be released prematurely.
use lazy_static::lazy_static; // 1.3.0
use std::sync::Mutex;
struct Something;
lazy_static! {
static ref SHARED: Mutex<Something> = Mutex::new(Something);
}
pub fn lock_and_execute(f: Box<Fn()>) {
let _locked = SHARED.lock(); // `_locked` is never used.
// does its lifetime end here?
f();
}
Does Rust treat locks specially, so that their lifetimes are guaranteed to extend to the end of their scope? Must we use that variable explicitly to avoid premature dropping of the lock, like in the following code?
pub fn lock_and_execute(f: Box<Fn()>) {
let locked = SHARED.lock(); // - lifetime begins
f(); // |
drop(locked); // - lifetime ends
}
There is a misunderstanding here: NLL (non-lexical lifetimes) affects the borrow-checks, not the actual lifetime of the objects.
Rust uses RAII1 extensively, and thus the Drop implementation of a number of objects, such as locks, has side-effects which have to occur at a well-determined and predictable point in the flow of execution.
NLL did NOT change the lifetime of such objects, and therefore their destructor is executed at exactly the same point that it was before: at the end of their lexical scope, in reverse order of creation.
NLL did change the understanding of the compiler of the use of lifetimes for the purpose of borrow-checking. This does not, actually, cause any code change; this is purely analysis. This analysis was made more clever, to better recognize the actual scope in which a reference is used:
Prior to NLL, a reference was considered "in use" from the moment it was created to the moment it was dropped, generally its lexical scope (hence the name).
NLL, instead:
Tries to defer the start of the "in use" span, if possible.
Ends the "in use" span with the last use of the reference.
In the case of a Ref<'a> (from RefCell), the Ref<'a> will be dropped at the end of the lexical scope, at which point it will use the reference to RefCell to decrement the counter.
NLL does not peel away layers of abstractions, so must consider that any object containing a reference (such as Ref<'a>) may access said reference in its Drop implementation. As a result, any object that contains a reference, such as a lock, will force NLL to consider that the "in use" span of the reference extends until they are dropped.
1 Resource Acquisition Is Initialization, whose original meaning is that once a variable constructor has been executed it has acquired the resources it needed and is not in a half-baked state, and which is generally used to mean that the destruction of said variable will release any resources it owned.
Does Rust treat locks specially, so that their lifetimes are guaranteed to extend to the end of their scope?
No. This is the default for every type, and has nothing to do with the borrow checker.
Must we use that variable explicitly to avoid premature dropping of the lock
No.
All you need to do is ensure that the lock guard is bound to a variable. Your example does this (let _lock = ...), so the lock will be dropped at the end of scope. If you had used the _ pattern instead, the lock would have been dropped immediately:
You can prove this for yourself by testing if the lock has indeed been dropped:
pub fn lock_and_execute() {
let shared = Mutex::new(Something);
println!("A");
let _locked = shared.lock().unwrap();
// If `_locked` was dropped, then we can re-lock it:
println!("B");
shared.lock().unwrap();
println!("C");
}
fn main() {
lock_and_execute();
}
This code will deadlock, as the same thread attempts to acquire the lock twice.
You could also attempt to use a method that requires &mut self to see that the immutable borrow is still held by the guard, which has not been dropped:
pub fn lock_and_execute() {
let mut shared = Mutex::new(Something);
println!("A");
let _locked = shared.lock().unwrap();
// If `_locked` was dropped, then we can re-lock it:
println!("B");
shared.get_mut().unwrap();
println!("C");
}
error[E0502]: cannot borrow `shared` as mutable because it is also borrowed as immutable
--> src/main.rs:13:5
|
9 | let _locked = shared.lock().unwrap();
| ------ immutable borrow occurs here
...
13 | shared.get_mut().unwrap();
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
...
16 | }
| - immutable borrow might be used here, when `_locked` is dropped and runs the `Drop` code for type `std::sync::MutexGuard`
See also:
Where is a MutexGuard if I never assign it to a variable?
How to lock a Rust struct the way a struct is locked in Go?
Why does _ destroy at the end of statement?
I am learning Rust from The Rust Programming Language book available from No Starch Press but ran into an issue where the compiler did not behave as explained in the book in chapter 4 on p. 77.
Chapter 4 of the book is discussing ownership, and the example on p. 77 is similar to this without the final println!() in main() (I've also added comments and the function from p. 76 to create an MCVE). I also created a playground.
fn main() {
let mut s = String::from("Hello world!");
let word = first_word(&s);
// according to book, compiler should not allow this mutable borrow
// since I'm already borrowing as immutable, but it does allow it
s.clear();
// but of course I do get error here about immutable borrow later being
// used here, but shouldn't it have errored on the clear() operation before
// it got here?
println!("First word of s is \"{}\"", word);
}
// return string slice reference to first word in string or entire string if
// no space found
fn first_word(s: &String) -> &str {
let bytes = s.as_bytes();
for (i, &item) in bytes.iter().enumerate() {
if item == b' ' {
return &s[..i];
}
}
&s[..]
}
I understand why the compiler throws an error where it currently does. But my understanding from the book is that it should have caused a compiler error when I tried to clear the string because I cannot borrow s as mutable because it is also borrowed as immutable, thus eliminating the possibility of the error I received (i.e., it should not compile even without my final println!()). But it compiles fine for me so long as I don't try to use the reference to word after the clear() operation.
The book is using Rust 1.21.0 (see p. 2) whereas I am using Rust 1.31.0—so this is likely a change that has been introduced to the compiler, but I am trying to understand why. Why is it better to error as it currently does versus where the book said it would error?
To be clear, I understand the errors themselves. I'm trying to understand why it doesn't throw a compiler error in the location the book says it should (i.e., why the change in compiler behavior?).
This is a change due to the non-lexical lifetimes, the update made in the latest versions of Rust (stabilized in the 2018 edition introduced with Rust 1.31, if I'm not mistaken).
In earlier versions of Rust (including the one on which the book is based), any reference was supposed to be alive for the whole scope, where it was created (that is, until the end of the enclosing braces). If you removed the line using word and try to compile the code on the old version, it would issue the same error — "borrowed as mutable while borrowed as immutable".
Now, the borrow checker tracks whether the reference is really used. If you weren't using word after s.clear(), it would be assumed that the immutable reference to s can be safely dropped before s.clear() takes a mutable one, so, as you've mentioned, this code will be safely compiled. When the println! is there, the borrow checker sees that the immutable and mutable borrows' scoped are intersecting, and tells you exactly that — note that the error is divided in three parts:
start of immutable borrow,
start of mutable borrow,
usage of immutable borrow.
I understand you're not allowed to create two mutable references to an object at once in Rust. I don't entirely understand why the following code works:
fn main() {
let mut string = String::from("test");
let mutable_reference: &mut String = &mut string;
mutable_reference.push_str(" test");
// as I understand it, this creates a new mutable reference (2nd?)
test(&mut *mutable_reference);
println!("{}", mutable_reference);
}
fn test(s: &mut String) {
s.push_str(" test");
}
The rule
There shall only be one usable mutable reference to a particular value at any point in time.
This is NOT a spatial exclusion (there CAN be multiple references to the same piece) but a temporal exclusion.
The mechanism
In order to enforce this, &mut T is NOT Copy; therefore calling:
test(mutable_reference);
should move the reference into test.
Actually doing this would make it unusable later on and not be very ergonomic, so the Rust compiler inserts an automatic reborrowing, much like you did yourself:
test(&mut *mutable_reference);
You can force the move if you wanted to:
test({ let x = mutable_reference; x });
The effect
Re-borrowing is, in essence, just borrowing:
mutable_reference is borrowed for as long as the unnamed temporary mutable reference exists (or anything that borrows from it),
the unnamed temporary mutable reference is moved into test,
at the of expression, the unnamed temporary mutable reference is destroyed, and therefore the borrow of mutable_reference ends.
Is there more than one mutable pointer somewhere in memory referring to the same location? Yes.
Is there more than one mutable pointer in the code to the same location which is usable? No.
Re-borrowing a mutable pointer locks out the one you're re-borrowing from.
I analyzed the differences in MIR between test(mutable_reference) and test(&mut *mutable_reference). It appears that the latter only introduces an additional level of indirection:
When you are using an extra dereference in the function call, it doesn't create a mutable reference that holds; in other words, it doesn't cause any actual difference outside the function call.