The code below compiles only if dyn Fn is UnwindSafe + RefUnwindSafe, because panic::catch_unwind requires it to be able to catch the panic.
use std::panic;
use std::panic::{UnwindSafe, RefUnwindSafe};
fn launch_closure(f: Box<dyn Fn() + UnwindSafe + RefUnwindSafe>) {
let result = panic::catch_unwind(|| {
f();
});
}
However, std::thread::JoinHandle::join function is able to catch the panic even if the thread closure is not UnwindSafe + RefUnwindSafe:
If the child thread panics, Err is returned with the parameter given
to panic!.
How?
I'd like to be able to know if my closure panicked, but UnwindSafe + RefUnwindSafe is too restrictive, I cannot use CondVar for example.
Playground
thread::spawn wraps the closure in an AssertUnwindSafe to tell the compiler that it knows that the given closure is unwind-safe:
let try_result = panic::catch_unwind(panic::AssertUnwindSafe(|| {
crate::sys_common::backtrace::__rust_begin_short_backtrace(f)
}));
So, what is unwind safety and how can thread::spawn make that assertion?
From the documentation for UnwindSafe:
In Rust a function can “return” early if it either panics or calls a function which transitively panics. This sort of control flow is not always anticipated, and has the possibility of causing subtle bugs through a combination of two critical components:
A data structure is in a temporarily invalid state when the thread panics.
This broken invariant is then later observed.
A type is not unwind safe if both of these can be true.
Types like Mutex and RwLock are unwind safe because they use poisoning to protect you from broken invariants. If a panic occurs in another thread that has a lock on a Mutex then it becomes poisoned and you must explicitly call PoisonError::into_inner to access the possibly-inconsistent data. If you cause a bug by making assumptions about a poisoned mutex, then that's your own responsibility and the Rust type system can't help you there.
Mutable references and non-sharable types with interior mutability like RefCell are not unwind safe because they don't offer such protection. However, they are also not Sync, so you can't get into a situation where you use them after another thread has panicked while holding a reference.
The final piece of the puzzle is that thread::spawn creates a new thread stack. That means it can guarantee that the closure is called first in the stack so nothing in the same thread as the closure can access its environment after a panic is caught.
While thread::spawn can't guarantee that the closure is unwind safe in the general case, it knows that:
The closure is Send (by its own bounds), so it cannot contain references to non-Sync types.
The non-unwind safe types in std (mutable references and cell types) are also not Sync, which means nothing can access a non-unwind safe type from outside the thread
Nothing in the same thread as the call to the closure can access its environment after the panic is caught.
So it is safe for the closure to be unwound because there is no possibility of broken invariants being unintentionally observed after a panic.
It is of course possible for a closure to make use of a user-defined type that is not unwind safe but is Sync, in which case this assumption would turn out to be incorrect. However, this would require unsafe code either from a third party crate or by the same author as the closure itself. It is always the responsibility of the author of unsafe code to ensure memory safety. It is not sound for a type to be Sync if a panic in another thread could cause UB. It is up to the author to decide if logical bugs are acceptable, but memory unsafety never is.
So... can you use the same trick in your code? Unfortunately, you probably cannot. Since you don't have control over the caller of launch_closure, you can't make the guarantee that a panic won't cause invalid state to be observed by callers in the same thread.
Related
I'llI was thinking about the rust async infrastructure and at the heart of the API lies the Future trait which is:
pub trait Future {
type Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}
According to the docs
The core method of future, poll, attempts to resolve the future into a final value. This method does not block if the value is not ready. Instead, the current task is scheduled to be woken up when it’s possible to make further progress by polling again. The context passed to the poll method can provide a Waker, which is a handle for waking up the current task.
This implies that the Context value passed to .poll() (particularly the waker) needs some way to refer to the pinned future &mut self in order to wake it up. However &mut implies that the reference is not aliased. Am I misunderstanding something or is this a "special case" where aliasing &mut is allowed? If the latter, are there other such special cases besides UnsafeCell?
needs some way to refer to the pinned future &mut self in order to wake it up.
No, it doesn't. The key thing to understand is that a “task” is not the future: it is the future and what the executor knows about it. What exactly the waker mutates is up to the executor, but it must be something that isn't the Future. I say “must” not just because of Rust mutability rules, but because Futures don't contain any state that says whether they have been woken. So, there isn't anything there to usefully mutate; 100% of the bytes of the future's memory are dedicated to the specific Future implementation and none of the executor's business.
Well on the very next page if the book you will notice that task contains a boxed future and the waker is created from a reference to task. So there is a reference to future held from task albeit indirect.
OK, let's look at those data structures. Condensed:
struct Executor {
ready_queue: Receiver<Arc<Task>>,
}
struct Task {
future: Mutex<Option<BoxFuture<'static, ()>>>,
task_sender: SyncSender<Arc<Task>>,
}
The reference to the task is an Arc<Task>, and the future itself is inside a Mutex (interior mutability) in the task. Therefore,
It is not possible to get an &mut Task from the Arc<Task>, because Arc doesn't allow that.
The future is in a Mutex which does run-time checking that there is at most one mutable reference to it.
The only things you can do with an Arc<Task> are
clone it and send it
get & access to the future in a Mutex (which allows requesting run-time-checked mutation access to the Future)
get & access to the task_sender (which allows sending things to ready_queue).
So, in this case, when the waker is called, it sort-of doesn't even mutate anything specific to the Task at all: it makes a clone of the Arc<Task> (which increments an atomic reference count stored next to the Task) and puts it on the ready_queue (which mutates storage shared between the Sender and Receiver).
Another executor might indeed have task-specific state in the Task that is mutated, such as a flag marking that the task is already woken and doesn't need to be woken again. That flag might be stored in an AtomicBoolean field in the task. But still, it does not alias with any &mut of the Future because it's not part of the Future, but the task.
All that said, there actually is something special about Futures and noalias — but it's not about executors, it's about Pin. Pinning explicitly allows the pinned type to contain “self-referential” pointers into itself, so Rust does not declare noalias for Pin<&mut T>. However, exactly what the language rules around this are is still not quite rigorously specified; the current situation is just considered a kludge so that async functions can be correctly compiled, I think.
There is no such special case. Judging from your comment about the Rust executor, you are misunderstanding how interior mutability works.
The example in the Rust book uses an Arc wrapped Task structure, with the future contained in a Mutex. When the task is run, it locks the mutex and obtains the singular &mut reference that's allowed to exist.
Now look at how the example implements wake_by_ref, and notice how it never touches the future at all. Indeed, that function would not be able to lock the future at all, as the upper level already has the lock. It would not be able to safely get a &mut reference, and so it prevents you from doing so, therefore, no issue.
The restriction for UnsafeCell and its wrappers is that only one &mut reference may exist for an object at any point in time. However, multiple & immutable references may exist to the UnsafeCell or structures containing it just fine - that is the point of interior mutability.
The docs for parking_lot say:
Mutex and RwLock allow raw unlocking without a RAII guard object.
Mutex<()> and RwLock<()> allow raw locking without a RAII guard object.
There is no further mention of these features, what they mean and how to use them. What are some pointers or sample uses?
The Mutex API controls access to its data via a guard, which unlocks the Mutex when it goes out of scope. The Mutex owns its data and can enforce that it is only accessible through a MutexGuard when it is locked. Both std::sync::Mutex and parking_lot::Mutex are the same in this regard.
However, parking_lot::Mutex also exposes its internals, which are a raw pointer to the data and a RawMutex. A RawMutex is just a lock, which does not control access to the data, but just tracks the state of the lock.
One reason to use RawMutex might be for times when it is very inconvenient to keep a MutexGuard in scope, and you are prepared to manage the lock status yourself. This is more likely in a library that defines new synchronization primitives or smart pointers rather than in application code, but you might also find it useful if you were mechanically translating existing C/C++ code to Rust.
By way of a simple example, these functions do the same thing as each other, but one uses the unsafe RawMutex:
use parking_lot::{Mutex, lock_api::RawMutex as _};
fn update_mutex(mutex: &Mutex<i32>) {
let mut guard = mutex.lock();
*guard = 2;
// guard goes out of scope here, causing the Mutex to be unlocked
}
fn update_mutex_raw(mutex: &Mutex<i32>) {
let raw_mutex = unsafe { mutex.raw() };
let data = mutex.data_ptr();
raw_mutex.lock();
unsafe {
*data = 2;
// need to manually unlock the RawMutex
raw_mutex.unlock();
};
}
RawMutex.unlock() is unsafe because it would trigger Undefined Behaviour to call it when the murex is not locked. Dereferencing the data pointer twice at once would be Undefined Behaviour too, so it's up to you to make sure you don't, by honouring the lock state of the RawMutex.
As always, when using unsafe functions, read the documentation carefully and make sure you thoroughly undersand the invariants that you must preserve in order to avoid Undefined Behaviour.
Rust guarantees that safe code will not contain data races (concurrent mutable access to the same object). Mutexes allow for threads to have mutually exclusive access to read/write an object, thereby avoiding the race.
In other languages, (Java and C++ come to mind) mutexes aren't explicitly associated with data. It is up to programmers to make sure they lock and unlock them appropriately, accessing the data only within the critical section between the lock and unlock. In Rust, this would mean that safe code could contain data races if things were written incorrectly.
The solution to this is the RAII guard. The mutex "owns" the associated object and only allows read/write access through a RAII guard that represents a lock on the mutex. This is the MutexGuard type returned by std's Mutex::lock()
Parking_lot is claiming to allow locking/unlocking without creation of a RAII guard, which can be useful when writing unsafe code doing fancy things with mutexes for speed reasons. This differentiates it from std's sync::Mutex, which does not provide these methods.
The primary limitation of RAII guards is that by the nature of RAII they only last as long as the enclosing scope. Further, they hold a reference to the mutex, so "storing" the "lock" past its scope is hard due to Rust's borrowing rules.
The referenced parking_lot methods are the unsafe raw_unlock and the safe raw_lock. Since a raw_lock() needs an associated raw_unlock() to finish the critical section, use of these features means delving into unsafe code, which is usually ill-advised and unnecessary unless you have good reason to believe its the only way to accomplish what you need.
I encountered a deadlock while developing with Rust and WebAssembly.
Due to the use of some globally accessed variables, I choose lazy_static and a Mutex (using thread_local callbacks would have caused nesting problems). I have declared a lot of Rust functions are used by JavaScript through #[wasm_bindgen]. They read and write the lazy_static variables.
After one of the functions panics, the mutex lock cannot be released, causing other functions to panic if they need to use the same mutex.
I know that the panic problem is unexpected and needs to be fixed, but these functions are relatively independent of each other. Although the reading and writing of the lazy_static variables intersect, a certain bug may not necessarily affect other parts.
How do I trigger the release of the Mutex after a panic in Wasm to allow other calls to be ok? Is there any better practice for this kind of problem?
Rust:
use std::sync::Mutex;
use std::sync::PoisonError;
use wasm_bindgen::prelude::*;
pub struct CurrentStatus {
pub index: i32,
}
impl CurrentStatus {
fn new() -> Self {
CurrentStatus { index: 1 }
}
fn get_index(&mut self) -> i32 {
self.index += 1;
self.index.clone()
}
fn add_index(&mut self) {
self.index += 2;
}
}
lazy_static! {
pub static ref FOO: Mutex<CurrentStatus> = Mutex::new(CurrentStatus::new());
}
unsafe impl Send for CurrentStatus {}
#[wasm_bindgen]
pub fn add_index() {
FOO.lock()
.unwrap_or_else(PoisonError::into_inner)
.add_index();
}
#[wasm_bindgen]
pub fn get_index() -> i32 {
let mut foo = FOO.lock().unwrap_or_else(PoisonError::into_inner);
if foo.get_index() == 6 {
panic!();
}
return foo.get_index();
}
JavaScript:
const js = import("../pkg/hello_wasm.js");
js.then(js => {
window.js = js;
console.log(js.get_index());
js.add_index();
console.log(js.get_index());
js.add_index();
console.log(js.get_index());
js.add_index();
console.log(js.get_index());
js.add_index();
console.log(js.get_index());
js.add_index();
});
After the panic, I can not call the function at all and it is as if the Wasm is dead.
Before answering this question I should probably mention, that panic handling shouldn't be used as general error mechanism. They should be used for unrecoverable errors.
Citing documentation.
This allows a program to terminate immediately and provide feedback to the caller of the program. panic! should be used when a program reaches an unrecoverable state.
Panics in Rust are actually much more gentle than it may seem in the first place for people coming from C++ background (which I assume is the case for some people writing in the comments). Uncaught Rust panics by default terminate thread, while C++ exception terminate whole process.
Citing documentation
Fatal logic errors in Rust cause thread panic, during which a thread will unwind the stack, running destructors and freeing owned resources. While not meant as a 'try/catch' mechanism, panics in Rust can nonetheless be caught (unless compiling with panic=abort) with catch_unwind and recovered from, or alternatively be resumed with resume_unwind. If the panic is not caught the thread will exit, but the panic may optionally be detected from a different thread with join. If the main thread panics without the panic being caught, the application will exit with a non-zero exit code.
It is fine to catch_unwind and recover thread from panic, but you should know that catch_unwind isn't guaranteed to catch all panics.
Note that this function may not catch all panics in Rust. A panic in Rust is not always implemented via unwinding, but can be implemented by aborting the process as well. This function only catches unwinding panics, not those that abort the process.
So, we understood that recovering from panic is fine. The question is what to do when lock is poisoned.
Citing documentation
The mutexes in this module implement a strategy called "poisoning" where a mutex is considered poisoned whenever a thread panics while holding the mutex. Once a mutex is poisoned, all other threads are unable to access the data by default as it is likely tainted (some invariant is not being upheld).
There is a valid reason for poisoning, because invariants of your data may not be held. Consider panic! in the middle of some function. This is just an additional level of security, that you can bypass.
A poisoned mutex, however, does not prevent all access to the underlying data. The PoisonError type has an into_inner method which will return the guard that would have otherwise been returned on a successful lock. This allows access to the data, despite the lock being poisoned.
use std::sync::{Mutex, PoisonError};
fn main() {
let mutex = Mutex::new(1);
// We are prepared to face bugs if invariants are wrong
println!("{}", mutex.lock().unwrap_or_else(PoisonError::into_inner));
}
Playground link
Of course it's always better to fix panic, than do this.
I had a problem where I was running several integration tests in parallel, and they had a mutex (that was accessed globally), but if one test failed, all subsequent tests would fail as well. This was a problem when I wanted to debug where a test failed, simply because the output would have a lot of other failed tests (due to poison error).
The solution was to simply use the parking_lot crate (a library for using mutexes), which seems to clear the mutex if the thread crashes.
Sadly I didn't find anything in the documentation that explains how it works, or if it even guarantees this behavior in future versions. But the current version works well for me, and if it works for you, it's easier than the accepted answer, since all you have to do is replace sync::Mutex for parking_lot::Mutex and you are ready to go (and also remove the .unwrap to the lock(), since it doesn't return a Result)
This question already has an answer here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed 5 years ago.
I have the following code:
fn main() {
let message = "Can't shoot yourself in the foot if you ain't got no gun";
let t1 = std::thread::spawn(|| {
println!("{}", message);
});
t1.join();
}
rustc gives me the compilation error:
closure may outlive the current function, but it borrows message, which is owned by the current function
This is wrong since:
The function it's referring to here is (I believe) main. The threads will be killed or enter in UB once main is finished executing.
The function it's referring to clearly invokes .join() on said thread.
Is the previous code unsafe in any way? If so, why? If not, how can I get the compiler to understand that?
Edit: Yes I am aware I can just move the message in this case, my question is specifically asking how can I pass a reference to it (preferably without having to heap allocate it, similarly to how this code would do it:
std::thread([&message]() -> void {/* etc */});
(Just to clarify, what I'm actually trying to do is access a thread safe data structure from two threads... other solutions to the problem that don't involve making the copy work would also help).
Edit2: The question this has been marked as a duplicate of is 5 pages long and as such I'd consider it and invalid question in it's own right.
Is the previous code 'unsafe' in any way ? If so, why ?
The goal of Rust's type-checking and borrow-checking system is to disallow unsafe programs, but that does not mean that all programs that fail to compile are unsafe. In this specific case, your code is not unsafe, but it does not satisfy the type constraints of the functions you are using.
The function it's referring to clearly invokes .join() on said thread.
But there is nothing from a type-checker standpoint that requires the call the .join. A type-checking system (on its own) can't enforce that a function has or has not been called on a given object. You could just as easily imagine an example like
let message = "Can't shoot yourself in the foot if you ain't got no gun";
let mut handles = vec![];
for i in 0..3 {
let t1 = std::thread::spawn(|| {
println!("{} {}", message, i);
});
handles.push(t1);
}
for t1 in handles {
t1.join();
}
where a human can tell that each thread is joined before main exits. But a typechecker has no way to know that.
The function it's referring to here is (I believe) main. So presumably those threads will be killed when main exists anyway (and them running after main exists is ub).
From the standpoint of the checkers, main is just another function. There is no special knowledge that this specific function can have extra behavior. If this were any other function, the thread would not be auto-killed. Expanding on that, even for main there is no guarantee that the child threads will be killed instantly. If it takes 5ms for the child threads to be killed, that is still 5ms where the child threads could be accessing the content of a variable that has gone out of scope.
To gain the behavior that you are looking for with this specific snippet (as-is), the lifetime of the closure would have to be tied to the lifetime of the t1 object, such that the closure was guaranteed to never be used after the handles have been cleaned up. While that is certainly an option, it is significantly less flexible in the general case. Because it would be enforced at the type level, there would be no way to opt out of this behavior.
You could consider using crossbeam, specifically crossbeam::scope's .spawn, which enforces this lifetime requirement where the standard library does not, meaning a thread must stop execution before the scope is finished.
In your specific case, your code works fine as long as you transfer ownership of message to the child thread instead of borrowing it from the main function, because there is no risk of unsafe code with or without your call to .join. Your code works fine if you change
let t1 = std::thread::spawn(|| {
to
let t1 = std::thread::spawn(move || {
I'm wrapping a Rust object to be used from Lua. I need the object to be destroyed when neither Rust code nor Lua still has a reference to it, so the obvious (to me) solution is to use Rc<T>, stored in Lua-managed memory.
The Lua API (I'm using rust-lua53 for now) lets you allocate a chunk of memory and attach methods and a finalizer to it, so I want to store an Rc<T> into that chunk of memory.
My current attempt looks like. First, creating an object:
/* Allocate a block of uninitialized memory to use */
let p = state.new_userdata(mem::size_of::<Rc<T>>() as size_t) as *mut Rc<T>;
/* Make a ref-counted pointer to a Rust object */
let rc = Rc::<T>::new(...);
/* Store the Rc */
unsafe { ptr::write(p, rc) };
And in the finaliser:
let p: *mut Rc<T> = ...; /* Get a pointer to the item to finalize */
unsafe { ptr::drop_in_place(p) }; /* Release the object */
Now this seems to work (as briefly tested by adding a println!() to the drop method). But is it correct and safe (as long as I make sure it's not accessed after finalization)? I don't feel confident enough in unsafe Rust to be sure that it's ok to ptr::write an Rc<T>.
I'm also wondering about, rather than storing an Rc<T> directly, storing an Option<Rc<T>>; then instead of drop_in_place() I would ptr::swap() it with None. This would make it easy to handle any use after finalization.
Now this seems to work (as briefly tested by adding a println!() to the drop method). But is it correct and safe (as long as I make sure it's not accessed after finalisation)? I don't feel confident enough in unsafe Rust to be sure that it's ok to ptr::write an Rc<T>.
Yes, you may ptr::write any Rust type to any memory location. This "leaks" the Rc<T> object, but writes a bit-equivalent to the target location.
When using it, you need to guarantee that no one modified it outside of Rust code and that you are still in the same thread as the one where it was created. If you want to be able to move across threads, you need to use Arc.
Rust's thread safety cannot protect you here, because you are using raw pointers.
I'm also wondering about, rather than storing an Rc<T> directly, storing an Option<Rc<T>>; then instead of drop_in_place() I would ptr::swap() it with None. This would make it easy to handle any use after finalisation.
The pendant to ptr::write is ptr::read. So if you can guarantee that no one ever tries to ptr::read or drop_in_place() the object, then you can just call ptr::read (which returns the object) and use that object as you would use any other Rc<T> object. You don't need to care about dropping or anything, because now it's back in Rust's control.
You should also be using new_userdata_typed instead of new_userdata, since that takes the memory handling off your hands. There are other convenience wrapper functions ending with the postfix _typed for most userdata needs.
Your code will work; of course, note that the drop_in_place(p) will just decrease the counter of the Rc and only drop the contained T if and only if it was the last reference, which is the correct action.