Array of structs with individual RwLocks for each block [duplicate] - rust

This question already has answers here:
Initialize a large, fixed-size array with non-Copy types
(8 answers)
Why is the Copy trait needed for default (struct valued) array initialization?
(2 answers)
Closed 4 years ago.
I am trying to create a struct which holds multiple blocks of bytes where each block of bytes is held in an individual RwLock.
This is for a voxel engine where performance is very important. Each block of bytes will need to be read/written by multiple threads. Multiple RwLocks are needed so that only a specific block will be locked leaving the rest of the blocks free to be read/written by other threads; locking the full struct would cause locking of all threads that are performing work.
Most other structs will be assigned a specific slot, the block of bytes needs to be stack allocated.
The compiler complains that it cannot copy the RwLocks as there is no Copy trait on RwLock, which I don't want to copy but instance multiple RwLocks.
mod constants {
pub const CHUNK_BASE_SIZE: usize = 7;
pub const CHUNK_ALLOC_COUNT: usize = 11;
}
mod example {
use std::sync::RwLock;
use super::constants;
// =====================================================================
struct BU8 {
bytes: [[u8; constants::CHUNK_BASE_SIZE]; constants::CHUNK_BASE_SIZE],
}
impl BU8 {
pub fn new() -> BU8 {
BU8 {
bytes: [[0; constants::CHUNK_BASE_SIZE]; constants::CHUNK_BASE_SIZE],
}
}
}
// =====================================================================
pub struct Bytes {
block: RwLock<BU8>,
}
impl Bytes {
pub fn new() -> Bytes {
Bytes {
block: RwLock::new(BU8::new()),
}
}
pub fn read(&self, y: usize, xz: usize) -> u8 {
self.block.read().unwrap().bytes[y][xz]
}
pub fn write(&mut self, y: usize, xz: usize, value: u8) {
self.block.write().unwrap().bytes[y][xz] = value;
}
}
// =====================================================================
pub struct Stacks {
slots: [Bytes; constants::CHUNK_ALLOC_COUNT],
}
impl Stacks {
pub fn new() -> Stacks {
Stacks {
// Cannot Copy, I dont want a Copy I want to instance a fixed sized array of RwLocks<BU8> blocks
slots: [Bytes::new(); constants::CHUNK_ALLOC_COUNT],
}
}
}
}
fn main() {}
error[E0277]: the trait bound `example::Bytes: std::marker::Copy` is not satisfied
--> src/main.rs:53:24
|
53 | slots: [Bytes::new(); constants::CHUNK_ALLOC_COUNT],
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::marker::Copy` is not implemented for `example::Bytes`
|
= note: the `Copy` trait is required because the repeated element will be copied
The main thread spawns a child thread that will hold all game/core data on and will be accessible through each worker thread the child thread will spawn.
Each worker thread will be assigned workloads to read/write data on an assigned Block on the Stacks, but I don't know how to instance multiple RwLocks this way or any other way without using the collections.
I tried adding #[derive(Copy, Clone)] for each struct but the Bytes struct error is:
error[E0204]: the trait `Copy` may not be implemented for this type
--> src/main.rs:24:18
|
24 | #[derive(Copy, Clone)]
| ^^^^
25 | pub struct Bytes {
26 | block: RwLock<BU8>,
| ------------------ this field does not implement `Copy`
I'm guessing RwLock size cannot known at compile time so would a vector of pointers to each block of bytes work? If so how do I implement that safely?

Related

Rust bindgen use of pointer addresse `*mut i8` for allocating memory for camera, getting "cannot be sent between threads safely" [duplicate]

I have a scenario where Rust will call C to malloc a buffer and stash the resulting pointer into a struct. Later on, the struct will be moved to a thread and passed to a C function which mutates it.
The naive approach to my problem looks like this (playground):
extern crate libc;
use libc::{c_void, malloc, size_t};
use std::thread;
const INITIAL_CAPACITY: size_t = 8;
extern "C" {
fn mutate(s: *mut Storage);
}
#[repr(C)]
struct Storage {
#[allow(dead_code)]
buf: *mut c_void,
capacity: usize,
}
fn main() {
let buf = unsafe { malloc(INITIAL_CAPACITY) };
let mut s = Storage {
buf: buf,
capacity: INITIAL_CAPACITY,
};
thread::spawn(move || {
unsafe {
mutate(&mut s); // mutates s.val, maybe reallocates it, updating s.capacity if so.
}
}).join()
.unwrap();
}
Gives:
error[E0277]: the trait bound `*mut libc::c_void: std::marker::Send` is not satisfied in `[closure#src/main.rs:26:19: 30:6 s:Storage]`
--> src/main.rs:26:5
|
26 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut libc::c_void` cannot be sent between threads safely
|
= help: within `[closure#src/main.rs:26:19: 30:6 s:Storage]`, the trait `std::marker::Send` is not implemented for `*mut libc::c_void`
= note: required because it appears within the type `Storage`
= note: required because it appears within the type `[closure#src/main.rs:26:19: 30:6 s:Storage]`
= note: required by `std::thread::spawn`
Which is the compiler's way of saying that because a *mut c_void doesn't implement Send, neither does Storage so you can't move it into the thread closure.
I thought that using a Unique pointer might solve this. Let's try it (playground):
#![feature(ptr_internals)]
extern crate libc;
use libc::{c_void, malloc, size_t};
use std::ptr::Unique;
use std::thread;
const INITIAL_CAPACITY: size_t = 8;
extern "C" {
fn mutate(s: *mut Storage);
}
#[repr(C)]
struct Storage {
#[allow(dead_code)]
buf: Unique<c_void>,
capacity: usize,
}
fn main() {
let buf = Unique::new(unsafe { malloc(INITIAL_CAPACITY) }).unwrap();
let mut s = Storage {
buf: buf,
capacity: INITIAL_CAPACITY,
};
thread::spawn(move || {
unsafe {
mutate(&mut s); // mutates s.val, maybe reallocates it, updating s.capacity if so.
}
}).join()
.unwrap();
}
But this gives:
warning: `extern` block uses type `std::ptr::Unique<libc::c_void>` which is not FFI-safe: this struct has unspecified layout
--> src/main.rs:11:18
|
11 | fn mutate(s: *mut Storage);
| ^^^^^^^^^^^^
|
= note: #[warn(improper_ctypes)] on by default
= help: consider adding a #[repr(C)] or #[repr(transparent)] attribute to this struct
Is there a way to have the Storage struct both implement Send and have mutable pointers to its instances be FFI safe?
By default Rust assumes *mut T is not safe to send between threads, and this means structs containing it are not safe either.
You can tell Rust that it is safe indeed:
unsafe impl Send for Storage {}
It relies entirely on your knowledge of how C uses data behind this pointer. Implementing Send means C won't rely on thread-local storage or thread-specific locks when using the object behind this pointer (paradoxically, that's true for most "thread-unsafe" C code).
It doesn't require C to handle access from multiple threads at once — that's what Sync is for.

How do I convert a concrete, statically dispatched `T: Trait` to a dynamically dispatched `dyn Trait`?

In this code, I have the skeleton of an observable system. The documentation on implementing Observable and other decoupling patterns are usually missing a final step on allowing access to the listener while also having it be &mut during the notification call, but that's what RefCell is intended to handle. So I have this code all worked out, but I'm having a last piece of trouble getting my concrete T into the &dyn Trait.
use std::{cell::RefCell, rc::Rc};
pub trait Observer {
fn notify(&mut self);
}
#[derive(Default)]
pub struct Subject<'s> {
listeners: Vec<Rc<RefCell<Box<dyn Observer + 's>>>>,
}
pub fn wrap<T>(t: T) -> Rc<RefCell<Box<T>>> {
Rc::new(RefCell::new(Box::new(t)))
}
impl<'s> Subject<'s> {
pub fn add_observer(&mut self, observer: Rc<RefCell<Box<dyn Observer + 's>>>) {
self.listeners.push(observer)
}
pub fn notify(&mut self) {
for listener in &mut self.listeners {
listener.borrow_mut().notify();
}
}
}
#[cfg(test)]
mod test {
use super::{wrap, Observer, Subject};
#[derive(Default)]
pub struct Counter {
count: usize,
}
impl Observer for Counter {
fn notify(&mut self) {
self.count += 1;
}
}
#[test]
fn it_observes() {
let mut subject = Subject::default();
let counter = wrap(Counter::default());
subject.add_observer(counter); // mismatched types
for i in 1..5 {
subject.notify();
subject.notify();
assert_eq!(counter.borrow().count, i * 2);
}
}
}
The full error is
error[E0308]: mismatched types
--> src/observer.rs:48:30
|
48 | subject.add_observer(counter);
| ^^^^^^^ expected trait object `dyn Observer`, found struct `Counter`
|
= note: expected struct `Rc<RefCell<Box<(dyn Observer + 'static)>>>`
found struct `Rc<RefCell<Box<Counter>>>
I've seen (what looks like) this pattern used in a number of contexts, but I can't tell what either I'm missing or doing differently.
How do I get the T: Trait out of its static dispatch and into dynamic dispatch?
You can take to boxing outside of your wrap function, and box the counter itself with the proper box type:
pub fn wrap<T>(t: T) -> Rc<RefCell<T>> {
Rc::new(RefCell::new(t))
}
...
let counter: Box<dyn Observer> = Box::new(Counter::default());
let counter = wrap(counter);
Playground
Btw, once you have the dynamic dispatch, you have an dyn Observer so you lose access to the Counter itself. You will have to take ownership of it and downcast the pointer to the concrete type again.

Store allocated data alongside allocator on struct

I want to use the memory allocator M to allocate an Allocated which is gonna live together with M inside a struct. However, I don't want to pollute UseM with its lifetime. Is there a way to store Allocated inside UseM?
use std::marker::PhantomData;
pub struct Allocated<'r, T> {
_phantom: PhantomData<&'r T>
}
pub trait MemPool<'s, 'r, T> {
fn allocate_default(
&'s self,
size: usize,
) -> Allocated<'r, T>;
}
struct M<T> {
_phantom: PhantomData<T>
}
impl<'s, T> MemPool<'s, 's, T> for M<T>
{
fn allocate_default(
&'s self,
size: usize,
) -> Allocated<'s, T>{
todo!()
}
}
struct UseM<T> {
//I wanted to allocated with m
m: M<T>,
//and store here, withotu `UseM` having to have a lifetime
allocated: Allocated<'r, T>
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=8bf2af2f7202a49c196cda98189f6971
error[E0261]: use of undeclared lifetime name `'r`
--> src/lib.rs:32:26
|
28 | struct UseM<T> {
| - help: consider introducing lifetime `'r` here: `'r,`
...
32 | allocated: Allocated<'r, T>
| ^^ undeclared lifetime
This is a “self-referential struct”, a pattern that can't be implemented in safe Rust directly, for two reasons:
There is no way to write lifetime annotations that express it, and
Structs are allowed to be moved, which would invalidate pointers into them, and there's no way to declare “this won't move because it is in a separate heap allocation” (except for the Pin mechanism which is not usable with only safe code).
The crate ouroboros uses macros and internal unsafe code to provide a safe mechanism to create this type of structure. However, you will have some additional verbosity in accessing the struct, and some things are not possible. Declaring your struct using ouroboros would look like this:
#[ouroboros::self_referencing]
struct UseM<T> {
m: M<T>,
#[borrows(m)]
allocated: Allocated<'this, T>,
}

Is it possible to call a parent struct's methods from a child struct? [duplicate]

This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
How do I express mutually recursive data structures in safe Rust?
(4 answers)
Pass self reference to contained object's function
(2 answers)
Passing mutable self reference to method of owned object
(2 answers)
Closed 3 years ago.
I am trying to pass a reference to self down to a child struct that is a field variable
use std::cell::RefCell;
use std::rc::Rc;
pub struct BUS {
pub processor: CPU,
pub ram: [u8; 65536],
}
impl BUS {
pub fn new() -> BUS {
BUS {
processor: CPU::new(),
ram: [0; 65536],
}
}
pub fn setup(&mut self) {
self.processor.connect_bus(Rc::new(RefCell::new(self)));
self.ram.iter_mut().for_each(|x| *x = 00);
}
pub fn write(&mut self, addr: u16, data: u8) {
self.ram[addr as usize] = data;
}
pub fn read(&mut self, addr: u16, _read_only: bool) -> u8 {
self.ram[addr as usize]
}
}
pub struct CPU {
bus_ptr: Rc<RefCell<BUS>>,
}
impl CPU {
pub fn new() -> CPU {
CPU {
bus_ptr: Rc::new(RefCell::new(BUS::new())),
}
}
pub fn connect_bus(&mut self, r: Rc<RefCell<BUS>>) {
self.bus_ptr = r;
}
pub fn read_ram(&self, addr: u16, _read_only: bool) -> u8 {
(self.bus_ptr.borrow_mut().read(addr, false))
}
pub fn write_ram(&mut self, addr: u16, data: u8) {
(self.bus_ptr.borrow_mut().write(addr, data))
}
}
fn main() {
let comp = BUS::new();
comp.setup();
}
Rust Playground
This errors:
error[E0308]: mismatched types
--> src/main.rs:18:57
|
18 | self.processor.connect_bus(Rc::new(RefCell::new(self)));
| ^^^^ expected struct `BUS`, found &mut BUS
|
= note: expected type `BUS`
found type `&mut BUS`
I can't pass in self to the RefCell as it is a second mutable borrow. I got around this by moving my functions around but want to know how possible this structure is.
I achieved this in C++ by passing in this from BUS and then using *bus in connect_bus so that read_ram can be *bus->read(...).
Is it possible to call the BUS struct's read and write functions from a method on the CPU struct?
The short answer is no.
RefCell owns its inner object. This means it has the only copy of that object, so that it can fully control all access to it and not allow any other access from outside. An object can't exist in RefCell and outside of RefCell at the same time.
Your setup could take an existing Rc<RefCell<BUS>> instead and pass that around. &mut BUS without the wrapper won't do.
The borrow checker can't ensure safety of mutual parent-child relationships. It wants program data structured as trees or DAGs. Otherwise you're forced to use wrapper types or C-like raw pointers.
The borrow checker checks against interfaces, not implementation. If your setter borrows &mut self, that's exclusive borrow of the entire object, and for borrow checking that's the most restrictive and most inflexible option. You will need to peel some layers of abstraction to make this work, e.g. pass RAM down to the CPU. Alternatively, make RAM use Cell<u8> type, so that it can be mutated via shared references.

Add new member variable to struct [duplicate]

This question already has answers here:
Is it possible for one struct to extend an existing struct, keeping all the fields?
(5 answers)
Closed 3 years ago.
I'm trying to add new member variables to a struct in an impl block.
I'm using protoc-rust to procedurally generate Rust code from protocol buffers. In particular, it generates structs and methods for the messages defined in the protos. I need to add new members to these structs to initialize some WebAssembly::Instances off of a bytes object in the struct
The struct:
pub struct Module {
// message fields
pub module_name: ::std::string::String,
pub init_script: ::std::string::String,
pub JS_boiler_plate: ::std::string::String,
pub functions: ::protobuf::RepeatedField<Function>,
pub meta_data: ::std::string::String,
pub packager: ::std::string::String,
pub pure_wasm: bool,
pub wasm_binary: ::std::vec::Vec<u8>,
// special fields
pub unknown_fields: ::protobuf::UnknownFields,
pub cached_size: ::protobuf::CachedSize,
}
What I want to do:
impl RPC_Module::Module {
self.wasm_instance: WebAssembly::Instance;
pub fn init(&mut self) -> Result<(), &'static str> {
// Init the instance based off of wasm_binary
let self.wasm_instance = WebAssembly::Instance::new()
}
}
What I get:
Compiling wRPC v0.1.0 (/Users/swarfield/Code/wRPC-Core/w-rpc-core)
error: expected one of `async`, `const`, `crate`, `default`, `existential`, `extern`, `fn`, `pub`, `type`, `unsafe`, or `}`, found `let`
--> src/protos/mod.rs:12:5
|
11 | impl RPC_Module::Module {
| - expected one of 11 possible tokens here
12 | let self.wasm_instance: WebAssembly::Instance;
| ^^^ unexpected token
I'm trying to add new member variables to a struct in an impl block.
Rust does not allow that.
What you can do is define another struct:
struct ModuleInstance {
wasm_instance: WebAssembly::Instance,
module: RPC_Module::Module
}
.. and return it from your init function.

Resources