I'm developing for a single-core embedded chip. In C & C++ it's common to statically-define mutable values that can be used globally. The Rust equivalent is roughly this:
static mut MY_VALUE: usize = 0;
pub fn set_value(val: usize) {
unsafe { MY_VALUE = val }
}
pub fn get_value() -> usize {
unsafe { MY_VALUE }
}
Now anywhere can call the free functions get_value and set_value.
I think that this should be entirely safe in single-threaded embedded Rust, but I've not been able to find a definitive answer. I'm only interested in types that don't require allocation or destruction (like the primitive in the example here).
The only gotcha I can see is with the compiler or processor reordering accesses in unexpected ways (which could be solves using the volatile access methods), but is that unsafe per se?
Edit:
The book suggests that this is safe so long as we can guarantee no multi-threaded data races (obviously the case here)
With mutable data that is globally accessible, it’s difficult to ensure there are no data races, which is why Rust considers mutable static variables to be unsafe.
The docs are phrased less definitively, suggesting that data races are only one way this can be unsafe but not expanding on other examples
accessing mutable statics can cause undefined behavior in a number of ways, for example due to data races in a multithreaded context
The nomicon suggests that this should be safe so long as you don't somehow dereference a bad pointer.
Be aware as there is no such thing as single-threaded code as long as interrupts are enabled. So even for microcontrollers, mutable statics are unsafe.
If you really can guarantee single-threaded access, your assumption is correct that accessing primitive types should be safe. That's why the Cell type exists, which allows mutability of primitive types with the exception that it is not Sync (meaning it explicitely prevents threaded access).
That said, to create a safe static variable, it needs to implement Sync for exactly the reason mentioned above; which Cell doesn't do, for obvious reasons.
To actually have a mutable global variable with a primitive type without using an unsafe block, I personally would use an Atomic. Atomics do not allocate and are available in the core library, meaning they work on microcontrollers.
use core::sync::atomic::{AtomicUsize, Ordering};
static MY_VALUE: AtomicUsize = AtomicUsize::new(0);
pub fn set_value(val: usize) {
MY_VALUE.store(val, Ordering::Relaxed)
}
pub fn get_value() -> usize {
MY_VALUE.load(Ordering::Relaxed)
}
fn main() {
println!("{}", get_value());
set_value(42);
println!("{}", get_value());
}
Atomics with Relaxed are zero-overhead on almost all architectures.
In this case it's not unsound, but you still should avoid it because it is too easy to misuse it in a way that is UB.
Instead, use a wrapper around UnsafeCell that is Sync:
pub struct SyncCell<T>(UnsafeCell<T>);
unsafe impl<T> Sync for SyncCell<T> {}
impl<T> SyncCell<T> {
pub const fn new(v: T) -> Self { Self(UnsafeCell::new(v)); }
pub unsafe fn set(&self, v: T) { *self.0.get() = v; }
}
impl<T: Copy> SyncCell<T> {
pub unsafe fn get(&self) -> T { *self.0.get() }
}
If you use nightly, you can use SyncUnsafeCell.
Mutable statics are unsafe in general because they circumvent the normal borrow checker rules that enforce either exactly 1 mutable borrow exists or any number of immutable borrows exist (including 0), which allows you to write code which causes undefined behavior. For instance, the following compiles and prints 2 2:
static mut COUNTER: i32 = 0;
fn main() {
unsafe {
let mut_ref1 = &mut COUNTER;
let mut_ref2 = &mut COUNTER;
*mut_ref1 += 1;
*mut_ref2 += 1;
println!("{mut_ref1} {mut_ref2}");
}
}
However we have two mutable references to the same location in memory existing concurrently, which is UB.
I believe the code that you posted there is safe, but I generally would not recommend using static mut. Use an atomic, SyncUnsafeCell/UnsafeCell, a wrapper around a Cell that implements Sync which is safe since your environment is single-threaded, or honestly just about anything else. static mut is wildly unsafe and its use is highly discouraged.
In order to sidestep the issue of exactly how mutable statics can be used safely in single-threaded code, another option is to use thread-local storage:
use std::cell::Cell;
thread_local! (static MY_VALUE: Cell<usize> = {
Cell::new(0)
});
pub fn set_value(val: usize) {
MY_VALUE.with(|cell| cell.set(val))
}
pub fn get_value() -> usize {
MY_VALUE.with(|cell| cell.get())
}
Related
I've got a struct that contains some locked data. The real world is complex, but here's a minimal example (or as minimal as I can make it):
use std::fmt::Display;
use std::ops::{Index, IndexMut};
use std::sync::Mutex;
struct LockedVector<T> {
stuff: Mutex<Vec<T>>,
}
impl<T> LockedVector<T> {
pub fn new(v: Vec<T>) -> Self {
LockedVector {
stuff: Mutex::new(v),
}
}
}
impl<T> Index<usize> for LockedVector<T> {
type Output = T;
fn index(&self, index: usize) -> &Self::Output {
todo!()
}
}
impl<T> IndexMut<usize> for LockedVector<T> {
fn index_mut(&mut self, index: usize) -> &mut Self::Output {
let thing = self.stuff.get_mut().unwrap();
&mut thing[index]
}
}
impl<T: Display> Display for LockedVector<T> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let strings: Vec<String> = self
.stuff
.lock()
.unwrap()
.iter()
.map(|s| format!("{}", s))
.collect();
write!(f, "{}", strings.join(", "))
}
}
fn main() {
let mut my_stuff = LockedVector::new(vec![0, 1, 2, 3, 4]);
println!("initially: {}", my_stuff);
my_stuff[2] = 5;
println!("then: {}", my_stuff);
let a_mut_var: &mut usize = &mut my_stuff[3];
*a_mut_var = 54;
println!("Still working: {}", my_stuff);
}
What I'm trying to do here is implement the Index and IndexMut traits on a struct, where the data being indexed is behind a Mutex lock. My very fuzzy reasoning for why this should be possible is that the result of locking a mutex is sort-of like a reference, and it seems like you could map a reference onto another reference, or somehow make a sort of reference that wraps the entire lock but only de-references the specific index.
My much less fuzzy reasoning is that the code above compiles and runs (note the todo!) - I'm able to get back mutable references, and I assume I haven't somehow snuck past the mutex in an unthread-safe way. (I made an attempt to test the threaded behavior, but ran into other issues trying to get a mutable reference into another thread at all.)
The weird issue is, I can't do the same for Index - there is no get_immut() I can use, and I haven't found another approach. I can get a mutable reference out of Mutex, but not an immutable one (and of course, I can't get the mutable one if I only have an immutable reference to begin with)
My expectation is that indexing would acquire a lock, and the returned reference (in both mutable and immutable cases) would maintain the lock for their lifetimes. As a bonus, it would be nice if RwLock-ed things could only grab/hold the read lock for the immutable cases, and the write lock for mutable ones.
For context as to why I'd do this: I have a Grid trait that is used by a bunch of different code, but backed by different implementations, some of which are thread-safe. I was hoping to put the Index and IndexMut traits on it for the nice syntax. Threads don't generally have mutable references to the thread-safe Grids at all, so the IndexMut trait would see little use there, but I could see it being valuable during setup or for the non-thread-safe cases. The immutable Index behavior seems like it would be useful everywhere.
Bonus question: I absolutely hate that Display code, how can I make it less hideous?
If you look at the documentation of get_mut you'll see it's only possible precisely because a mutable reference ensures that there is no other reference or a lock to it, unfortunately for you that means that a get_ref for Mutex would only be possible by taking a mutable reference, that's just an artificially limited get_mut though.
Unfortunately for you since Index only gives you a shared reference you can't safely get a shared reference to it's contents, so you can't implement an Index so that it indexes into something behind a Mutex.
Given the code snippet below:
use std::{io::BufWriter, pin::Pin};
pub struct SelfReferential {
pub writer: BufWriter<&'static mut [u8]>, // borrowed from buffer
pub buffer: Pin<Box<[u8]>>,
}
#[cfg(test)]
mod tests {
use std::io::Write;
use super::*;
fn init() -> SelfReferential {
let mut buffer = Pin::new(vec![0; 12].into_boxed_slice());
let writer = unsafe { buffer.as_mut().get_unchecked_mut() };
let writer = unsafe { (writer as *mut [u8]).as_mut().unwrap() };
let writer = BufWriter::new(writer);
SelfReferential { writer, buffer }
}
#[test]
fn move_works() {
let mut sr = init();
sr.writer.write(b"hello ").unwrap();
sr.writer.flush().unwrap();
let mut slice = &mut sr.buffer[6..];
slice.write(b"world!").unwrap();
assert_eq!(&sr.buffer[..], b"hello world!".as_ref());
let mut sr_moved = sr;
sr_moved.writer.write(b"W").unwrap();
sr_moved.writer.flush().unwrap();
assert_eq!(&sr_moved.buffer[..], b"hello World!".as_ref());
}
}
The first question: is it OK to assign 'static lifetime to mutable slice reference in BufWriter? As technically speaking, it's bound to the lifetime of struct instances themselves, and AFAIK there's no safe way to invalidate it.
The second question: besides the fact that unsafe instantiation of this type, in test example, creates two mutable references into the underlying buffer, is there any other potential dangers associated with such an "unidiomatic" (for the lack of better word) type?
is it OK to assign 'static lifetime to mutable slice reference in BufWriter?
Sort of, but there's a bigger problem. The lifetime itself is not worse than any other choice, because there is no lifetime that you can use here which is really accurate. But it is not safe to expose that reference, because then it can be taken:
let w = BufWriter<&'static mut [u8]> = {
let sr = init();
sr.writer
};
// `sr.buffer` has now been dropped, so `w` has a dangling reference
is there any other potential dangers associated with such an "unidiomatic" (for the lack of better word) type?
Yes, it's undefined behavior. Box isn't just managing an allocation; it also (currently) signals a claim of unique, non-aliasing access to the contents. You violate that non-aliasing by creating the writer and then moving the buffer — even though the heap memory is not actually touched, the move of buffer is counted invalidating all references into it.
This is an area of Rust semantics which is not yet fully nailed down, but as far as the current compiler is concerned, this is UB. You can see this if you run your test code under the Miri interpreter.
The good news is, what you're trying to do is a very common desire and people have worked on the problem. I personally recommend using ouroboros — with the help of a macro, it allows you to create the struct you want without writing any new unsafe code. There will be some restrictions on how you use the writer, but nothing you can't tidy out of the way by writing an impl io::Write for SelfReferential. Another, newer library in this space is yoke; I haven't tried it.
I'm considering making Rust my primary development language instead of Go, so I've been reading the docs.
One thing that my work requires a lot of is reading and writing from multiple threads into a custom database that is stored in memory as a single massive array and could be 32GB in size. The database functions are designed to avoid race conditions, and mutexes or atomic primitives are used where necessary.
The Rust doc implies that an array can only be either mutable (writeable) on a single thread, or non-mutable (read only) by many threads, and cannot be writable on one and readable on another simultaneously. How then can an in-memory database be used...? It doesn't make sense!
Do I have this wrong?
Forgive me that I can't give any specific Rust example, because I'm still learning the Rust syntax, and to be honest I need to know the answer to this question before using all my time learning a language I will be unable to use.
There is an unsafe way to do it, namely using the UnsafeCell, which returns mutable raw pointers to its interior data. These are not tracked by the Borrow-Checker and so you have to make sure the invariants are upheld
pub struct UnsafeVec<T> {
data: UnsafeCell<Vec<T>>
}
impl<T> UnsafeVec<T> {
pub fn new() -> Self {
UnsafeVec { data: UnsafeCell::new(Vec::new()) }
}
pub fn push(&mut self, arg: T) {
self.data.get_mut().push(arg)
}
pub unsafe fn index_mut(&self, index: usize) -> &mut T {
&mut (*self.data.get())[index]
}
}
unsafe impl<T> Sync for UnsafeVec<T> {}
unsafe impl<T> Send for UnsafeVec<T> {}
which allows you to write
fn main() {
let mut unsafe_vec = UnsafeVec::<i32>::new();
unsafe_vec.push(15);
unsafe {
*unsafe_vec.index_mut(0) += 1;
}
}
The method index_mut allows to modify the interior vector with an immutable reference.
The Sync and Send traits signal the compiler that the type can be safely shared across threads, which is only true if you prevent possible data races manually!.
Again, this is an unsafe option that requires you to uphold the invariants yourself.
I have a struct that is not Send because it contains Rc. Lets say that Arc has too big overhead, so I want to keep using Rc. I would still like to occasionally Send this struct between threads, but only when I can verify that the Rc has strong_count 1 and weak_count 0.
Here is (hopefully safe) abstraction that I have in mind:
mod my_struct {
use std::rc::Rc;
#[derive(Debug)]
pub struct MyStruct {
reference_counted: Rc<String>,
// more fields...
}
impl MyStruct {
pub fn new() -> Self {
MyStruct {
reference_counted: Rc::new("test".to_string())
}
}
pub fn pack_for_sending(self) -> Result<Sendable, Self> {
if Rc::strong_count(&self.reference_counted) == 1 &&
Rc::weak_count(&self.reference_counted) == 0
{
Ok(Sendable(self))
} else {
Err(self)
}
}
// There are more methods, some may clone the `Rc`!
}
/// `Send`able wrapper for `MyStruct` that does not allow you to access it,
/// only unpack it.
pub struct Sendable(MyStruct);
// Safety: `MyStruct` is not `Send` because of `Rc`. `Sendable` can be
// only created when the `Rc` has strong count 1 and weak count 0.
unsafe impl Send for Sendable {}
impl Sendable {
/// Retrieve the inner `MyStruct`, making it not-sendable again.
pub fn unpack(self) -> MyStruct {
self.0
}
}
}
use crate::my_struct::MyStruct;
fn main() {
let handle = std::thread::spawn(|| {
let my_struct = MyStruct::new();
dbg!(&my_struct);
// Do something with `my_struct`, but at the end the inner `Rc` should
// not be shared with anybody.
my_struct.pack_for_sending().expect("Some Rc was still shared!")
});
let my_struct = handle.join().unwrap().unpack();
dbg!(&my_struct);
}
I did a demo on the Rust playground.
It works. My question is, is it actually safe?
I know that the Rc is owned only by a single onwer and nobody can change that under my hands, because it can't be accessed by other threads and we wrap it into Sendable which does not allow access to the contained value.
But in some crazy world Rc could for example internally use thread local storage and this would not be safe... So is there some guarantee that I can do this?
I know that I must be extremely careful to not introduce some additional reason for the MyStruct to not be Send.
No.
There are multiple points that need to be verified to be able to send Rc across threads:
There can be no other handle (Rc or Weak) sharing ownership.
The content of Rc must be Send.
The implementation of Rc must use a thread-safe strategy.
Let's review them in order!
Guaranteeing the absence of aliasing
While your algorithm -- checking the counts yourself -- works for now, it would be better to simply ask Rc whether it is aliased or not.
fn is_aliased<T>(t: &mut Rc<T>) -> bool { Rc::get_mut(t).is_some() }
The implementation of get_mut will be adjusted should the implementation of Rc change in ways you have not foreseen.
Sendable content
While your implementation of MyStruct currently puts String (which is Send) into Rc, it could tomorrow change to Rc<str>, and then all bets are off.
Therefore, the sendable check needs to be implemented at the Rc level itself, otherwise you need to audit any change to whatever Rc holds.
fn sendable<T: Send>(mut t: Rc<T>) -> Result<Rc<T>, ...> {
if !is_aliased(&mut t) {
Ok(t)
} else {
...
}
}
Thread-safe Rc internals
And that... cannot be guaranteed.
Since Rc is not Send, its implementation can be optimized in a variety of ways:
The entire memory could be allocated using a thread-local arena.
The counters could be allocated using a thread-local arena, separately, so as to seamlessly convert to/from Box.
...
This is not the case at the moment, AFAIK, however the API allows it, so the next release could definitely take advantage of this.
What should you do?
You could make pack_for_sending unsafe, and dutifully document all assumptions that are counted on -- I suggest using get_mut to remove one of them. Then, on each new release of Rust, you'd have to double-check each assumption to ensure that your usage if still safe.
Or, if you do not mind making an allocation, you could write a conversion to Arc<T> yourself (see Playground):
fn into_arc<T>(this: Rc<T>) -> Result<Arc<T>, Rc<T>> {
Rc::try_unwrap(this).map(|t| Arc::new(t))
}
Or, you could write a RFC proposing a Rc <-> Arc conversion!
The API would be:
fn Rc<T: Send>::into_arc(this: Self) -> Result<Arc<T>, Rc<T>>
fn Arc<T>::into_rc(this: Self) -> Result<Rc<T>, Arc<T>>
This could be made very efficiently inside std, and could be of use to others.
Then, you'd convert from MyStruct to MySendableStruct, just moving the fields and converting Rc to Arc as you go, send to another thread, then convert back to MyStruct.
And you would not need any unsafe...
The only difference between Arc and Rc is that Arc uses atomic counters. The counters are only accessed when the pointer is cloned or dropped, so the difference between the two is negligible in applications which just share pointers between long-lived threads.
If you have never cloned the Rc, it is safe to send between threads. However, if you can guarantee that the pointer is unique then you can make the same guarantee about a raw value, without using a smart pointer at all!
This all seems quite fragile, for little benefit; future changes to the code might not meet your assumptions, and you will end up with Undefined Behaviour. I suggest that you at least try making some benchmarks with Arc. Only consider approaches like this when you measure a performance problem.
You might also consider using the archery crate, which provides a reference-counted pointer that abstracts over atomicity.
I notice that Rust's Atomic* structs have functions which modify the value, such as fetch_add. For instance, I can write this program:
use std::sync::atomic::{AtomicUsize, Ordering};
struct Tester {
counter: AtomicUsize
}
impl Tester {
fn run(&self) {
let counter = self.counter.fetch_add(1, Ordering::Relaxed);
println!("Hi there, this has been run {} times", counter);
}
}
fn main() {
let t = Tester { counter: AtomicUsize::new(0) };
t.run();
t.run();
}
This compiles and runs fine, but if I change the AtomicUsize to a normal integer, it will (correctly) fail to compile due to mutability concerns:
struct Tester {
counter: u64
}
impl Tester {
fn run(&self) {
self.counter = self.counter + 1;
println!("Hi there, this has been run {} times", self.counter);
}
}
It wouldn’t be very useful if it didn’t work this way. With &mut references, only one can exist at a time, and no & references at that time, so the whole question of atomicity of operation would be moot.
Another way of looking at it is &mut are unique references, and & aliasable references. For normal types, mutation can only safely occur if you have a unique reference, but atomic types are all about mutation (via replacement) without needing a unique reference.
The naming of & and &mut has been a fraught matter, with much fear, uncertainty and doubt in the community and documents like Focusing on Ownership explaining how things actually are. The language has ended up staying with & and &mut, but &mut is actually about uniqueness rather than mutability (it’s just that most of the time the two are equivalent).