I'm hitting a lifetime error when compiling a change I made for Firecracker (on aarch64, but I doubt the issue is architecture-dependent):
error[E0716]: temporary value dropped while borrowed
--> src/vmm/src/device_manager/mmio.rs:174:24
|
174 | let int_evt = &serial
| ________________________^
175 | | .lock()
176 | | .expect("Poisoned legacy serial lock")
| |__________________________________________________^ creates a temporary which is freed while still in use
177 | .interrupt_evt();
| - temporary value is freed at the end of this statement
178 | vm.register_irqfd(int_evt, self.irq)
| ------- borrow later used here
|
= note: consider using a `let` binding to create a longer lived value
The original code (which compiles fine) is:
vm.register_irqfd(&serial
.lock()
.expect("Poisoned legacy serial lock")
.interrupt_evt(), self.irq)
.map_err(Error::RegisterIrqFd)?;
I don't understand the difference. The error message seems to state that expect() is returning a temporary and that I'm taking a const reference to it, in C++ this would extend the lifetime of the temporary, does it not in Rust? Either way, why does it work in the original code but not after I bind to an l-value (C++ parlance, I'm not sure if it is the same for Rust)?
I tried creating a SSCE here, but it worked as expected!
A simple, reproducible example of the problem (playground):
// create array inside mutex
let mutex = Mutex::new([ 0i32 ]);
// get reference to item inside array
let item: &i32 = mutex.lock().unwrap().get(0).unwrap();
// use reference for something
println!("item = {:?}", item);
mutex.lock().unwrap() returns a MutexGuard<'_, Option<i32>>, which borrows the data inside the mutex. It also owns a lock on the data, that is released when the guard is dropped, which means that noone else may borrow the data at the same time.
When you call a method of the inner type on that guard (like .get in the above example, or .interrupt_evt in your code), it will borrow with the lifetime of the guard, since you can only access the data safely while the guard exist. But the guard isn't stored in any variable, so it only exists temporarily for that statement, and is immediately dropped at the end of it. So you cannot get a reference to the data outside of the statement.
To solve this problem is very simple: first store the guard in a variable, and then borrow from it. That will ensure that the guard lives longer than the references you get from it (playground):
// create array inside mutex
let mutex = Mutex::new([ 0i32 ]);
// get reference to item inside array
let guard = mutex.lock().unwrap();
let item: &i32 = guard.get(0).unwrap();
// use reference for something
println!("item = {:?}", item);
// guard is now destroyed at end of scope
// and mutex lock is released here
Related
Writing a simple interpreter has lead me to this battle with the borrow checker.
#[derive(Clone, Debug)]
struct Context<'a> {
display_name: &'a str,
parent: Option<Box<Context<'a>>>,
parent_entry_pos: Position<'a>,
}
// --snip--
#[derive(Copy, Clone, Debug)]
pub enum BASICVal<'a> {
Float(f64, Position<'a>, Position<'a>, &'a Context<'a>),
Int(i64, Position<'a>, Position<'a>, &'a Context<'a>),
Nothing(Position<'a>, Position<'a>, &'a Context<'a>),
}
// --snip--
pub fn run<'a>(text: &'a String, filename: &'a String) -> Result<(Context<'a>, BASICVal<'a>), BASICError<'a>> {
// generate tokens
let mut lexer = Lexer::new(text, filename);
let tokens = lexer.make_tokens()?;
// parse program to AST
let mut parser = Parser::new(tokens);
let ast = parser.parse();
// run the program
let context: Context<'static> = Context {
display_name: "<program>",
parent: None,
parent_entry_pos: Position::default(),
};
Ok((context, interpreter_visit(&ast?, &context)?))
}
The error is "cannot return value referencing local variable `context`" and (secondary) the "borrow of moved value: `context`":
error[E0515]: cannot return value referencing local variable `context`
--> src\basic.rs:732:2
|
732 | Ok((context, interpreter_visit(&ast?, &context)?))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^--------^^^^
| | |
| | `context` is borrowed here
| returns a value referencing data owned by the current function
error[E0382]: borrow of moved value: `context`
--> src\basic.rs:732:40
|
727 | let context: Context<'static> = Context {
| ------- move occurs because `context` has type `basic::Context<'_>`, which does not implement the `Copy` trait
...
732 | Ok((context, interpreter_visit(&ast?, &context)?))
| ------- value moved here ^^^^^^^^ value borrowed here after move
As far as I understand it: The context references several lifetime-dependent structs. The values of these structs are static in this case, as I can see by explicitly setting the lifetime parameter to 'static and the compiler not complaining. The interpreter_visit function needs to borrow the context because it gets passed to several independent functions, including itself recursively. In addition, the interpreter_visit returns BASICVals that reference the context themselves. For this reason, the context needs to outlive the run return. I try to achieve that by passing the context itself as part of the return value, thereby giving the caller control over its life. But now, I move the context to the return value before actually using it? This makes no sense. I should be able to reference one part of the return value in another part of the return value because both values make it out of the function "alive". I have tried:
boxing the context, thereby forcing it off the stack onto the heap, but that seems to only complicate things.
switching the order of the tuple, but that doesn't help.
storing interpreter_visit's result in an intermediate variable, which as expected doesn't help.
cloning the interpreter_visit result or the context itself
The issue may lie with the result and the error. The error doesn't reference a context but giving it a separate lifetime in interpreter_visit breaks the entire careful balance I have been able to achieve until now.
Answering this so that people don't have to read the comment thread.
This is a problem apparently not solvable by Rust's borrow checker. The borrow checker cannot understand that a Box of context will live on the heap and therefore last longer than the function return, therefore being "legally" referencable by the return value of interpreter_visit which itself escapes the function. The solution in this case is to circumvent borrow checking via unsafe, namely a raw pointer. Like this:
let context = Box::new(Context {
display_name: "<program>",
parent: None,
parent_entry_pos: Position::default(),
});
// Obtain a pointer to a location on the heap
let context_ptr: *const Context = &*context;
// outsmart the borrow checker
let result = interpreter_visit(&ast?, unsafe { &*context_ptr })?;
// The original box is passed back, so it is destroyed safely.
// Because the result lives as long as the context as required by the lifetime,
// we cannot get a pointer to uninitialized memory through the value and its context.
Ok((context, result))
I store a raw pointer to the context in context_ptr. The borrowed value passed to interpreter_visit is then piped through a (completely memory-safe) raw pointer dereference and borrow. This will (for some reason, only the Rust gods know) disable the borrow check, so the context data given to interpreter_visit is considered to have a legal lifetime. As I am however still passing back the very safe Box around the context data, I can avoid creating memory leaks by leaving the context with no owner. It might be possible now to pass around the interpreter_visit return value with having the context destroyed, but because both values are printed and discarded immediately, I see no issues arising from this in the future.
If you have a deeper understanding of Rust's borrow checker and would consider this a fixable edge case that doesn't have more "safe" solutions I couldn't come up with, please do comment and I will report this to the Rust team. I'm however not that certain especially because my experience with and knowledge of Rust is limited.
I have a circular ring buffer (implemented as a vector) where I want one thread to periodically write to the ring buffer and another to periodically read from the ring buffer. Is it possible to create a vector that can be read and written at the same time so long as threads accessing the vector are not at the same index?
What I am hoping to achieve:
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
fn main() {
let vec = Arc::new(vec![Mutex::new(1), Mutex::new(2),Mutex::new(3)]);
{
let vec = vec.clone();
thread::spawn(move|| {
let mut s2 = *vec.get_mut(2).unwrap().lock().unwrap();
s2 = 7;
});
}
println!("{}", vec[2].lock().unwrap());
}
Compiler output is:
Compiling playground v0.0.1 (/playground)
warning: variable `s2` is assigned to, but never used
--> src/main.rs:12:21
|
12 | let mut s2 = *vec.get_mut(2).unwrap().lock().unwrap();
| ^^
|
= note: `#[warn(unused_variables)]` on by default
= note: consider using `_s2` instead
warning: value assigned to `s2` is never read
--> src/main.rs:13:13
|
13 | s2 = 7;
| ^^
|
= note: `#[warn(unused_assignments)]` on by default
= help: maybe it is overwritten before being read?
error[E0596]: cannot borrow data in an `Arc` as mutable
--> src/main.rs:12:27
|
12 | let mut s2 = *vec.get_mut(2).unwrap().lock().unwrap();
| ^^^ cannot borrow as mutable
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `std::sync::Arc<std::vec::Vec<std::sync::Mutex<i32>>>`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0596`.
error: could not compile `playground`.
To learn more, run the command again with --verbose.
Foiled by the rust type system trying to prevent a race condition :(
What I don't want
An implementation that involves having the lock scope including the vector.
An atomic read and write to the vector is not an option since the vector will contain images.
Link to playground:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5b5efe91bdd45c658d11f1cefb16045e
First, I would recommend you to use std::sync::RwLock, because it allows multiple readers to read data simultaneously.
Second, spawning threads can lead to performance bottlenecks in your code. Try to use thread pool.
Of course, the exact choice will vary depending on the result of benchmarks, but those are general recommendations.
Your code is mostly correct, except one crucial part. You are using Mutex which implements interior mutability pattern and also provides thread-safety.
Interior mutability moves compiletime checks of XOR borrowing rule (either N immutable borrows or just one mutable) to the run-time. So, Mutex ensures that any time there exists only one reader or only one writer.
When you try to get mutable reference from vec, like this
vec.get_mut(..)
You are essentially ignoring benefits provided by interior mutability. Compiler can't guarantee that XOR rule is not broken, because you borrow vec as mutable.
Obvious solution is to borrow vec as immutable and using Mutex to safeguard against race condition and don't utilize compiler borrowing rules.
let mut s2 = vec
.get(2) // Get immutable reference to second item
.unwrap() // Ensure that it exists
.lock() // Lock mutex.
.unwrap(); // Ensure mutex isn't poisoned.
// s2 is now `std::sync::MutexGuard<i32>`, which implements `std::ops::DerefMut`,
// so it can get us mutable reference to data.
*s2 = 7;
I've read What are non-lexical lifetimes?. With the non-lexical borrow checker, the following code compiles:
fn main() {
let mut scores = vec![1, 2, 3];
let score = &scores[0]; // borrows `scores`, but never used
// its lifetime can end here
scores.push(4); // borrows `scores` mutably, and succeeds
}
It seems reasonable in the case above, but when it comes to a mutex lock, we don't want it to be released prematurely.
In the following code, I would like to lock a shared structure first and then execute a closure, mainly to avoid deadlock. However, I'm not sure if the lock will be released prematurely.
use lazy_static::lazy_static; // 1.3.0
use std::sync::Mutex;
struct Something;
lazy_static! {
static ref SHARED: Mutex<Something> = Mutex::new(Something);
}
pub fn lock_and_execute(f: Box<Fn()>) {
let _locked = SHARED.lock(); // `_locked` is never used.
// does its lifetime end here?
f();
}
Does Rust treat locks specially, so that their lifetimes are guaranteed to extend to the end of their scope? Must we use that variable explicitly to avoid premature dropping of the lock, like in the following code?
pub fn lock_and_execute(f: Box<Fn()>) {
let locked = SHARED.lock(); // - lifetime begins
f(); // |
drop(locked); // - lifetime ends
}
There is a misunderstanding here: NLL (non-lexical lifetimes) affects the borrow-checks, not the actual lifetime of the objects.
Rust uses RAII1 extensively, and thus the Drop implementation of a number of objects, such as locks, has side-effects which have to occur at a well-determined and predictable point in the flow of execution.
NLL did NOT change the lifetime of such objects, and therefore their destructor is executed at exactly the same point that it was before: at the end of their lexical scope, in reverse order of creation.
NLL did change the understanding of the compiler of the use of lifetimes for the purpose of borrow-checking. This does not, actually, cause any code change; this is purely analysis. This analysis was made more clever, to better recognize the actual scope in which a reference is used:
Prior to NLL, a reference was considered "in use" from the moment it was created to the moment it was dropped, generally its lexical scope (hence the name).
NLL, instead:
Tries to defer the start of the "in use" span, if possible.
Ends the "in use" span with the last use of the reference.
In the case of a Ref<'a> (from RefCell), the Ref<'a> will be dropped at the end of the lexical scope, at which point it will use the reference to RefCell to decrement the counter.
NLL does not peel away layers of abstractions, so must consider that any object containing a reference (such as Ref<'a>) may access said reference in its Drop implementation. As a result, any object that contains a reference, such as a lock, will force NLL to consider that the "in use" span of the reference extends until they are dropped.
1 Resource Acquisition Is Initialization, whose original meaning is that once a variable constructor has been executed it has acquired the resources it needed and is not in a half-baked state, and which is generally used to mean that the destruction of said variable will release any resources it owned.
Does Rust treat locks specially, so that their lifetimes are guaranteed to extend to the end of their scope?
No. This is the default for every type, and has nothing to do with the borrow checker.
Must we use that variable explicitly to avoid premature dropping of the lock
No.
All you need to do is ensure that the lock guard is bound to a variable. Your example does this (let _lock = ...), so the lock will be dropped at the end of scope. If you had used the _ pattern instead, the lock would have been dropped immediately:
You can prove this for yourself by testing if the lock has indeed been dropped:
pub fn lock_and_execute() {
let shared = Mutex::new(Something);
println!("A");
let _locked = shared.lock().unwrap();
// If `_locked` was dropped, then we can re-lock it:
println!("B");
shared.lock().unwrap();
println!("C");
}
fn main() {
lock_and_execute();
}
This code will deadlock, as the same thread attempts to acquire the lock twice.
You could also attempt to use a method that requires &mut self to see that the immutable borrow is still held by the guard, which has not been dropped:
pub fn lock_and_execute() {
let mut shared = Mutex::new(Something);
println!("A");
let _locked = shared.lock().unwrap();
// If `_locked` was dropped, then we can re-lock it:
println!("B");
shared.get_mut().unwrap();
println!("C");
}
error[E0502]: cannot borrow `shared` as mutable because it is also borrowed as immutable
--> src/main.rs:13:5
|
9 | let _locked = shared.lock().unwrap();
| ------ immutable borrow occurs here
...
13 | shared.get_mut().unwrap();
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
...
16 | }
| - immutable borrow might be used here, when `_locked` is dropped and runs the `Drop` code for type `std::sync::MutexGuard`
See also:
Where is a MutexGuard if I never assign it to a variable?
How to lock a Rust struct the way a struct is locked in Go?
Why does _ destroy at the end of statement?
For reasons related to code organization, I need the compiler to accept the following (simplified) code:
fn f() {
let mut vec = Vec::new();
let a = 0;
vec.push(&a);
let b = 0;
vec.push(&b);
// Use `vec`
}
The compiler complains
error: `a` does not live long enough
--> src/main.rs:8:1
|
4 | vec.push(&a);
| - borrow occurs here
...
8 | }
| ^ `a` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
error: `b` does not live long enough
--> src/main.rs:8:1
|
6 | vec.push(&b);
| - borrow occurs here
7 | // Use `vec`
8 | }
| ^ `b` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
However, I'm having a hard time convincing the compiler to drop the vector before the variables it references. vec.clear() doesn't work, and neither does drop(vec). mem::transmute() doesn't work either (to force vec to be 'static).
The only solution I found was to transmute the reference into &'static _. Is there any other way? Is it even possible to compile this in safe Rust?
Is it even possible to compile this in safe Rust?
No. What you are trying to do is inherently unsafe in the general case.
The collection contains a reference to a variable that will be dropped before the collection itself is dropped. This means that the destructor of the collection has access to references that are no longer valid. The destructor could choose to dereference one of those values, breaking Rust's memory safety guarantees.
note: values in a scope are dropped in the opposite order they are created
As the compiler tells you, you need to reorder your code. You didn't actually say what the limitations are for "reasons related to code organization", but the straight fix is:
fn f() {
let a = 0;
let b = 0;
let mut vec = Vec::new();
vec.push(&a);
vec.push(&b);
}
A less obvious one is:
fn f() {
let a;
let b;
let mut vec = Vec::new();
a = 0;
vec.push(&a);
b = 0;
vec.push(&b);
}
That all being said, once non-lexical lifetimes are enabled, your original code will work! The borrow checker becomes more granular about how long a value needs to live.
But wait; I just said that the collection might access invalid memory if a value inside it is dropped before the collection, and now the compiler is allowing that to happen? What gives?
This is because the standard library pulls a sneaky trick on us. Collections like Vec or HashSet guarantee that they do not access their generic parameters in the destructor. They communicate this to the compiler using the unstable #[may_dangle] feature.
See also:
Moved variable still borrowing after calling `drop`?
"cannot move out of variable because it is borrowed" when rotating variables
For reasons related to code organization, I need the compiler to accept the following (simplified) code:
fn f() {
let mut vec = Vec::new();
let a = 0;
vec.push(&a);
let b = 0;
vec.push(&b);
// Use `vec`
}
The compiler complains
error: `a` does not live long enough
--> src/main.rs:8:1
|
4 | vec.push(&a);
| - borrow occurs here
...
8 | }
| ^ `a` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
error: `b` does not live long enough
--> src/main.rs:8:1
|
6 | vec.push(&b);
| - borrow occurs here
7 | // Use `vec`
8 | }
| ^ `b` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
However, I'm having a hard time convincing the compiler to drop the vector before the variables it references. vec.clear() doesn't work, and neither does drop(vec). mem::transmute() doesn't work either (to force vec to be 'static).
The only solution I found was to transmute the reference into &'static _. Is there any other way? Is it even possible to compile this in safe Rust?
Is it even possible to compile this in safe Rust?
No. What you are trying to do is inherently unsafe in the general case.
The collection contains a reference to a variable that will be dropped before the collection itself is dropped. This means that the destructor of the collection has access to references that are no longer valid. The destructor could choose to dereference one of those values, breaking Rust's memory safety guarantees.
note: values in a scope are dropped in the opposite order they are created
As the compiler tells you, you need to reorder your code. You didn't actually say what the limitations are for "reasons related to code organization", but the straight fix is:
fn f() {
let a = 0;
let b = 0;
let mut vec = Vec::new();
vec.push(&a);
vec.push(&b);
}
A less obvious one is:
fn f() {
let a;
let b;
let mut vec = Vec::new();
a = 0;
vec.push(&a);
b = 0;
vec.push(&b);
}
That all being said, once non-lexical lifetimes are enabled, your original code will work! The borrow checker becomes more granular about how long a value needs to live.
But wait; I just said that the collection might access invalid memory if a value inside it is dropped before the collection, and now the compiler is allowing that to happen? What gives?
This is because the standard library pulls a sneaky trick on us. Collections like Vec or HashSet guarantee that they do not access their generic parameters in the destructor. They communicate this to the compiler using the unstable #[may_dangle] feature.
See also:
Moved variable still borrowing after calling `drop`?
"cannot move out of variable because it is borrowed" when rotating variables