I am looking to use a member function of a struct as a mutable callback. The standard way I would do this is by wrapping my struct instance in a mutex and then in an Arc. In this use case however, my callback is a real-time audio callback which needs to be real-time safe and is triggered on a dedicated thread. This means no locking can happen that might delay the callback from occurring.
My actual member callback method must take a mutable reference to the struct (I think this is where things fall apart). The struct does contain members but these can all be atomic variables.
Here is an example of the kind of thing I'm looking for:
use std::sync::Arc;
use clap::error;
use cpal::{traits::{HostTrait, DeviceTrait}, StreamConfig, OutputCallbackInfo, StreamError};
use std::sync::atomic::AtomicI16;
pub struct AudioProcessor {
var: AtomicI16,
}
impl AudioProcessor {
fn audio_block_f32(&mut self, audio: &mut [f32], _info: &OutputCallbackInfo) {
}
fn audio_error(&mut self, error: StreamError) {
}
}
fn initAudio()
{
let out_dev = cpal::default_host().default_output_device().expect("No available output device found");
let mut supported_configs_range = out_dev.supported_output_configs().expect("Could not obtain device configs");
let config = supported_configs_range.next().expect("No available configs").with_max_sample_rate();
let proc = Arc::new(AudioProcessor{var: AtomicI16::new(5)});
let audio_callback_instance = proc.clone();
let error_callback_instance = proc.clone();
out_dev.build_output_stream(&StreamConfig::from(config), move |audio: &mut [f32], info: &OutputCallbackInfo| audio_callback_instance.audio_block_f32(audio, info), move |stream_error| error_callback_instance.clone().audio_error(stream_error));
}
Currently this code fails in the build_output_stream() function with this error message:
cannot borrow data in an Arc as mutable
trait DerefMut is required to modify through a dereference, but it is not implemented for Arc<AudioProcessor>
Are there any standard ways of dealing with problems like this?
Related
I have a struct that contains a vector of functions and a method that iterates over each and calls it.
struct App {
functions: Vec<Box<dyn FnOnce(i32) -> i32>>
}
impl App {
pub fn run(&mut self) {
for function in &mut self.functions {
println!("{}", (function)(1));
}
}
}
However, I this moves the vector and I have not found a clean way to copy it.
cannot move out of `*function` which is behind a mutable reference
move occurs because `*function` has type `Box<dyn FnOnce(i32) -> i32>`, which does not
implement the `Copy` trait (rustc E0507)
So is there any way to call the function without moving it or a clean way to copy it.
The way function traits in Rust are structured are a little confusing at the beginning, but follow the same pattern of ownership.
There are three function ownership types:
FnOnce - similar to self. Requires the caller to own this function object, and the object will be consumed. This is the strongest guarantee a caller can make to a function, and every function is FnOnce.
FnMut - similar to &mut self. Requires the caller to hold a mutable reference to the function. Can be called multiple times. All the functions that might have side effects require at least FnMut ownership. Every function that is executable through an FnMut is also compatible with FnOnce, as FnOnce has stronger ownership guarantees.
Fn - similar to &self. Allows the caller to reference this immutably, meaning, this object can be referenced and called from multiple points simultaneously (important: not talking multi-threading here, multi-threading needs Sync additionally). This is the weakest ownership requirement, functions that allow owners to call them through this usually have no side effects and just compute an output from an input. All functions that allow being called through Fn automatically also can be called through FnOnce and FnMut, as those give stronger ownership guarantees.
With that out of the way, in your case your functions are FnOnce, meaning they require owning. Calling them will consume them. That last part especially is the reason why you get errors here.
The way you own self in your run method is too weak. You can't call FnOnce, which require ownership, on a self object that you don't own. A &mut self object only allows you to call FnMut members.
So there are two ways to fix your code:
Change the type of the self parameter to owned. Note, however, that this will make run a consuming function that will destroy self in the end.
Change the type of your function objects to FnMut. This is of course only possible if your functions allow being called through FnMut.
Here is an example of how it would look like if you changed FnOnce to Fn:
struct App {
functions: Vec<Box<dyn Fn(i32) -> i32>>,
}
impl App {
pub fn run(&self) {
for function in &self.functions {
println!("{}", (function)(1));
}
}
}
fn main() {
let app = App {
functions: vec![Box::new(|x| 2 * x), Box::new(|x| 3 * x)],
};
app.run();
app.run();
}
2
3
Note that the closure |x| 2 * x here is compatible with Fn, because it has zero side effects. This means that our self variable can now be an immutable reference, because calling those functions is guaranteed to not mutate self.
We can also call app.run() multiple times, and app doesn't have to be mutable.
Now here is an example of a function that requires FnMut:
struct App {
functions: Vec<Box<dyn FnMut(i32) -> i32>>,
}
impl App {
pub fn run(&mut self) {
for function in &mut self.functions {
println!("{}", (function)(1));
}
}
}
fn main() {
let mut add = {
let mut sum = 0;
move |x| {
sum += x;
println!("sum is now {}!", sum);
sum
}
};
add(5);
add(10);
let mut app = App {
functions: vec![Box::new(add)],
};
app.run();
app.run();
}
sum is now 5!
sum is now 15!
sum is now 16!
16
sum is now 17!
17
This function now has a side effect - it changes the value of sum every time it is called. It is therefore no longer compatible with Fn, and using it in the previous code would cause an error.
Our function objects now have to be FnMut, our self has to become &mut self and our app object is required to be mutable.
We are, however, still able to call app.run() multiple times.
Now, lets see what would be necessary to use an FnOnce:
struct App {
functions: Vec<Box<dyn FnOnce(i32) -> i32>>,
}
impl App {
pub fn run(self) {
for function in self.functions {
println!("{}", (function)(1));
}
}
}
fn main() {
let add = {
let s = String::new();
move |x| {
drop(s);
x
}
};
let app = App {
functions: vec![Box::new(add)],
};
app.run();
}
1
The reason this closure is FnOnce is because of the drop. It destroys the s variable when called, and that can for obvious reasons only happen once.
Therefore, to store it in the App, App has to change its function object type to FnOnce. Further, self in the run() method now also needs to be taken as an owned object, because parts of it will get destroyed in the process.
This now means that we can call app.run() only once. There is no need to mark app as mut, because it won't get mutated in the process, it will get completely consumed, so we don't need to worry about mutability. It simply can't get accessed afterwards any more.
It makes sense that we can now call app.run() only once, because the function it contains can also only be called once.
I hope this helped somehow to increase your understanding of your situation.
I know there are hundreds of questions just like this one, but i'm having trouble wrapping my head around how to do the thing I'm trying to do.
I want an http server that accepts and processes events. On receiving/processing an event, i want the EventManager to send an update to an ApplicationMonitor that is tracking how many events have been accepted/processed. The ApplicationMonitor would also (eventually) handle things like tracking number of concurrent connections, but in this example I just want my EventManager to send an Inc('event_accepted') update to my ApplicationMonitor.
To be useful, I need the ApplicationMonitor to be able to return a snapshot of the stats when the requested through a /stats route.
So I have an ApplicationMonitor which spawns a thread and listens on a channel for incoming Stat events. When it receives a Stat event it updates the stats HashMap. The stats hashmap must be mutable within both ApplicationMonitor as well as the spawned thread.
use std::sync::mpsc;
use std::sync::mpsc::Sender;
use std::thread;
use std::thread::JoinHandle;
use std::collections::HashMap;
pub enum Stat {
Inc(&'static str),
Dec(&'static str),
Set(&'static str, i32)
}
pub struct ApplicationMonitor {
pub tx: Sender<Stat>,
pub join_handle: JoinHandle<()>
}
impl ApplicationMonitor {
pub fn new() -> ApplicationMonitor {
let (tx, rx) = mpsc::channel::<Stat>();
let mut stats: HashMap<&'static str, i32> = HashMap::new();
let join_handle = thread::spawn(move || {
for stat in rx.recv() {
match stat {
Stat::Inc(nm) => {
let current_val = stats.entry(nm).or_insert(0);
stats.insert(nm, *current_val + 1);
},
Stat::Dec(nm) => {
let current_val = stats.entry(nm).or_insert(0);
stats.insert(nm, *current_val - 1);
},
Stat::Set(nm, val) => {
stats.insert(nm, val);
}
}
}
});
let am = ApplicationMonitor {
tx,
join_handle
};
am
}
pub fn get_snapshot(&self) -> HashMap<&'static str, i32> {
self.stats.clone()
}
}
Because rx cannot be cloned, I must move the references into the closure. When I do this, I am no longer able to access stats outside of the thread.
I thought maybe I needed a second channel so the thread could communicate it's internals back out, but this doesn't work as i would need another thread to listen for that in a non-blocking way.
Is this where I'd use Arc?
How can I have stats live inside and out of the thread context?
Yes, this is a place where you'd wrap your stats in an Arc so that you can have multiple references to it from different threads. But just wrapping in an Arc will only give you a read-only view of the HashMap - if you need to be able to modify it, you'll also need to wrap it in something which guarantees that only one thing can modify it at a time. So you'll probably end up with either an Arc<Mutex<HashMap<&'static str, i32>>> or a Arc<RwLock<HashMap<&'static str, i32>>>.
Alternatively, if you're just changing the values, and not adding or removing values, you could potentially use an Arc<HashMap<&static str, AtomicU32>>, which would allow you to read and modify different values in parallel without needing to take out a Map-wide lock, but Atomics can be a little more fiddly to understand and use correctly than locks.
I want my method of struct to perform in a synchronized way. I wanted to do this by using Mutex (Playground):
use std::sync::Mutex;
use std::collections::BTreeMap;
pub struct A {
map: BTreeMap<String, String>,
mutex: Mutex<()>,
}
impl A {
pub fn new() -> A {
A {
map: BTreeMap::new(),
mutex: Mutex::new(()),
}
}
}
impl A {
fn synchronized_call(&mut self) {
let mutex_guard_res = self.mutex.try_lock();
if mutex_guard_res.is_err() {
return
}
let mut _mutex_guard = mutex_guard_res.unwrap(); // safe because of check above
let mut lambda = |text: String| {
let _ = self.map.insert("hello".to_owned(),
"d".to_owned());
};
lambda("dd".to_owned());
}
}
Error message:
error[E0500]: closure requires unique access to `self` but `self.mutex` is already borrowed
--> <anon>:23:26
|
18 | let mutex_guard_res = self.mutex.try_lock();
| ---------- borrow occurs here
...
23 | let mut lambda = |text: String| {
| ^^^^^^^^^^^^^^ closure construction occurs here
24 | if let Some(m) = self.map.get(&text) {
| ---- borrow occurs due to use of `self` in closure
...
31 | }
| - borrow ends here
As I understand when we borrow anything from the struct we are unable to use other struct's fields till our borrow is finished. But how can I do method synchronization then?
The closure needs a mutable reference to the self.map in order to insert something into it. But closure capturing works with whole bindings only. This means, that if you say self.map, the closure attempts to capture self, not self.map. And self can't be mutably borrowed/captured, because parts of self are already immutably borrowed.
We can solve this closure-capturing problem by introducing a new binding for the map alone such that the closure is able to capture it (Playground):
let mm = &mut self.map;
let mut lambda = |text: String| {
let _ = mm.insert("hello".to_owned(), text);
};
lambda("dd".to_owned());
However, there is something you overlooked: since synchronized_call() accepts &mut self, you don't need the mutex! Why? Mutable references are also called exclusive references, because the compiler can assure at compile time that there is only one such mutable reference at any given time.
Therefore you statically know, that there is at most one instance of synchronized_call() running on one specific object at any given time, if the function is not recursive (calls itself).
If you have mutable access to a mutex, you know that the mutex is unlocked. See the Mutex::get_mut() method for more explanation. Isn't that amazing?
Rust mutexes do not work the way you are trying to use them. In Rust, a mutex protects specific data relying on the borrow-checking mechanism used elsewhere in the language. As a consequence, declaring a field Mutex<()> doesn't make sense, because it is protecting read-write access to the () unit object that has no values to mutate.
As Lukas explained, your call_synchronized as declared doesn't need to do synchronization because its signature already requests an exclusive (mutable) reference to self, which prevents it from being invoked from multiple threads on the same object. In other words, you need to change the signature of call_synchronized because the current one does not match the functionality it is intended to provide.
call_synchronized needs to accept a shared reference to self, which will signal to Rust that it can be called from multiple threads in the first place. Inside call_synchronized a call to Mutex::lock will simultaneously lock the mutex and provide a mutable reference to the underlying data, carefully scoped so that the lock is held for the duration of the reference:
use std::sync::Mutex;
use std::collections::BTreeMap;
pub struct A {
synced_map: Mutex<BTreeMap<String, String>>,
}
impl A {
pub fn new() -> A {
A {
synced_map: Mutex::new(BTreeMap::new()),
}
}
}
impl A {
fn synchronized_call(&self) {
let mut map = self.synced_map.lock().unwrap();
// omitting the lambda for brevity, but it would also work
// (as long as it refers to map rather than self.map)
map.insert("hello".to_owned(), "d".to_owned());
}
}
I'm trying to put in one structure information about inotify events and hashmap with inotify watch id as key and name of the file as value.
extern crate inotify;
use inotify::INotify;
use std::sync::{Arc, Mutex};
use std::collections::HashMap;
struct Notificator {
inotify: INotify,
watch_service: Arc<Mutex<HashMap<inotify::wrapper::Watch, Arc<String>>>>,
}
impl Notificator {
pub fn new() -> Notificator {
Notificator {
inotify: INotify::init().unwrap(),
watch_service: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn resolving_events(&mut self) {
{
let mut events = self.check_for_events();
self.events_execution(events);
}
}
fn check_for_events(&mut self) -> &[inotify::wrapper::Event] {
self.inotify.available_events().unwrap()
}
fn events_execution(&self, events: &[inotify::wrapper::Event]) {
for event in events.iter() {
}
}
}
During compilation I am receiving an error
src/main.rs:204:13: 204:17 error: cannot borrow `*self` as immutable because it is also borrowed as mutable [E0502]
src/main.rs:204 self.events_execution(events);
I thought the best solution would be to separate somehow inotify variable in Notificator structure with watch_service, but I can't dereference self.check_for_events(); because I receive
src/main.rs:203:17: 203:27 error: the trait bound `[inotify::wrapper::Event]: std::marker::Sized` is not satisfied [E0277]
src/main.rs:203 let mut events = *self.check_for_events();
I understand the core of the problem: I'm trying to borrow reference by check_for_events and then using it as parameter in events_execution which also requires self as parameter, but I have no idea how to resolve it.
One thing you could do, although it's not very elegant, is to have your mutable method consume its mutable borrow and return an immutable one that you can then use:
pub fn resolving_events(&mut self) {
let (slf, events) = self.check_for_events();
slf.events_execution(events);
}
fn check_for_events(&mut self) -> (&Self, &[inotify::wrapper::Event]) {
let events = self.inotify.available_events().unwrap();
(&self, events)
}
I've made a small proof-of-concept on the playground (using vecs of u64 as the mutable state, but the principle is similar). It might be cleaner to refactor your code so that some external client can (mutably) borrow the Notifier, produce the events, release the borrow, and borrow it (immutably) to process them...
This is a known issue in the borrow checker (more discussion on internals). You cannot have a function taking a &mut T to an object and returning a &T without losing the ability to access the object again until the &T goes out of scope. You can't work around it due to the way inotify is implemented.
But you can ask the inotify authors to create a get_available_notifications method that doesn't fetch new data. This way you can call available_notifications once and drop the returned value. Then call get_available_notifications (which doesn't take &mut INotify, but just &INotify) and work from there.
I have a struct:
struct MyData {
x: i32
}
I want to asynchronously start a long operation on this struct.
My first attempt was this:
fn foo(&self) { //should return immediately
std::thread::Thread::spawn(move || {
println!("{:?}",self.x); //consider a very long operation
});
}
Clearly the compiler cannot infer an appropriate lifetime due to conflicting requirements because self may be on the stack frame and thus cannot be guaranteed to exist by the time the operation is running on a different stack frame.
To solve this, I attempted to make a copy of self and provide that copy to the new thread:
fn foo(&self) { //should return immediately
let clone = self.clone();
std::thread::Thread::spawn(move || {
println!("{:?}",clone.x); //consider a very long operation
});
}
I think that does not compile because now clone is on the stack frame which is similar to before. I also tried to do the clone inside the thread, and that does not compile either, I think for similar reasons.
Then I decided maybe I could use a channel to push the copied data into the thread, on the theory that perhaps channel can magically move (copy?) stack-allocated data between threads, which is suggested by this example in the documentation. However the compiler cannot infer a lifetime for this either:
fn foo(&self) { //should return immediately
let (tx, rx) = std::sync::mpsc::channel();
tx.send(self.clone());
std::thread::Thread::spawn(move || {
println!("{:?}",rx.recv().unwrap().x); //consider a very long operation
});
}
Finally, I decided to just copy my struct onto the heap explicitly, and pass an Arc into the thread. But not even here can the compiler figure out a lifetime:
fn foo(&self) { //should return immediately
let arc = std::sync::Arc::new(self.clone());
std::thread::Thread::spawn(move || {
println!("{:?}",arc.clone().x); //consider a very long operation
});
}
Okay borrow checker, I give up. How do I get a copy of self onto my new thread?
I think your issue is simply because your structure does not derive the Clone trait. You can get your second example to compile and run by adding a #[derive(Clone)] before your struct's definition.
What I don't understand in the compiler behaviour here is what .clone() function it tried to use here. Your structure indeed did not implement the Clone trait so should not by default have a .clone() function.
playpen
You may also want to consider in your function taking self by value, and let your caller decide whether it should make a clone, or just a move.
As an alternative solution, you could use thread::scoped and maintain a handle to the thread. This allows the thread to hold a reference, without the need to copy it in:
#![feature(old_io,std_misc)]
use std::thread::{self,JoinGuard};
use std::old_io::timer;
use std::time::duration::Duration;
struct MyData {
x: i32,
}
// returns immediately
impl MyData {
fn foo(&self) -> JoinGuard<()> {
thread::scoped(move || {
timer::sleep(Duration::milliseconds(300));
println!("{:?}", self.x); //consider a very long operation
timer::sleep(Duration::milliseconds(300));
})
}
}
fn main() {
let d = MyData { x: 42 };
let _thread = d.foo();
println!("I'm so fast!");
}