I have this static element:
pub static CREDENTIALS: Mutex<DatabaseCredentials> = Mutex::new(DatabaseCredentials::new());
and I want to take the content out of the Mutex:
let creds = CREDENTIALS.into_inner().unwrap().clone();
and I am getting this error:
let creds = CREDENTIALS.into_inner().unwrap();
| ^^^^^^^^^^^ move occurs because `CREDENTIALS` has type `std::sync::Mutex<connection::credentials::DatabaseCredentials<'_>>`, which does not implement the `Copy` trait
Where my DatabaseCredentials looks as follows:
#[derive(Clone, Copy)]
pub struct DatabaseCredentials<'a> {
pub username: &'a str,
pub password: &'a str,
pub host: &'a str,
pub db_name: &'a str
}
unsafe impl Send for DatabaseCredentials<'_> {}
unsafe impl Sync for DatabaseCredentials<'_> {}
// impl block...
How can I get rid out of the error, and take the underlying value inside the Mutex?
How can I get rid out of the error, and take the underlying value inside the Mutex?
I would suggest reworking your system so you don't have to, because it's odd, why would you have a mutex and want to move things out of it rather than just use the protected object in-place?
But if you really need to, given your mutex is static you have three options:
Copy or Clone the stored DatabaseCredentials, no need to into_inner anything, just deref' the MutexGuard. That will create an owned copy of the credentials.
Add an indirection through an Option, this way you can Option::take the DatabaseCredentials out of the Mutex. This will leave the mutex valid but containing a None.
std::mem::replace the contents of the mutex by a newly created DatabaseCredentials, you'll get the one which was stored in the mutex.
In all three cases, you have to lock the mutex first.
Related
I am looking to use a member function of a struct as a mutable callback. The standard way I would do this is by wrapping my struct instance in a mutex and then in an Arc. In this use case however, my callback is a real-time audio callback which needs to be real-time safe and is triggered on a dedicated thread. This means no locking can happen that might delay the callback from occurring.
My actual member callback method must take a mutable reference to the struct (I think this is where things fall apart). The struct does contain members but these can all be atomic variables.
Here is an example of the kind of thing I'm looking for:
use std::sync::Arc;
use clap::error;
use cpal::{traits::{HostTrait, DeviceTrait}, StreamConfig, OutputCallbackInfo, StreamError};
use std::sync::atomic::AtomicI16;
pub struct AudioProcessor {
var: AtomicI16,
}
impl AudioProcessor {
fn audio_block_f32(&mut self, audio: &mut [f32], _info: &OutputCallbackInfo) {
}
fn audio_error(&mut self, error: StreamError) {
}
}
fn initAudio()
{
let out_dev = cpal::default_host().default_output_device().expect("No available output device found");
let mut supported_configs_range = out_dev.supported_output_configs().expect("Could not obtain device configs");
let config = supported_configs_range.next().expect("No available configs").with_max_sample_rate();
let proc = Arc::new(AudioProcessor{var: AtomicI16::new(5)});
let audio_callback_instance = proc.clone();
let error_callback_instance = proc.clone();
out_dev.build_output_stream(&StreamConfig::from(config), move |audio: &mut [f32], info: &OutputCallbackInfo| audio_callback_instance.audio_block_f32(audio, info), move |stream_error| error_callback_instance.clone().audio_error(stream_error));
}
Currently this code fails in the build_output_stream() function with this error message:
cannot borrow data in an Arc as mutable
trait DerefMut is required to modify through a dereference, but it is not implemented for Arc<AudioProcessor>
Are there any standard ways of dealing with problems like this?
I'm trying to write this following code for a server:
use std::io::{BufReader, BufWriter};
use std::net::TcpStream;
struct User<'a> {
stream: Arc<TcpStream>,
reader: BufReader<&'a TcpStream>,
writer: BufWriter<&'a TcpStream>,
}
fn accept_socket(users: &mut Vec<User>, stream: Arc<TcpStream>) {
let stream_clone = stream.clone();
let user = User {
stream: stream_clone,
reader: BufReader::new(stream_clone.as_ref()),
writer: BufWriter::new(stream_clone.as_ref()),
};
users.push(user);
}
The stream is behind an Arc because it is shared across threads. The BufReader and BufWriter point to the User's own Arc, but the compiler complains that the reference stream_clone.as_ref() does not live long enough, even though it obviously does (it points to the Arc, which isn't dropped as long as the User is alive). How do I get the compiler to accept this code?
Self-referential structs are a no-go. Rust has no way of updating the address in the references if the struct is moved since moving is always a simple bit copy. Unlike C++ with its move constructors, there's no way to attach behavior to moves.
What you can do instead is store Arcs inside the reader and writer so they share ownership of the TcpStream.
struct User {
stream: Arc<TcpStream>,
reader: BufReader<IoArc<TcpStream>>,
writer: BufWriter<IoArc<TcpStream>>,
}
The tricky part is that Arc doesn't implement Read and Write. You'll need a newtype that does (IoArc, above). Yoshua Wuyts wrote about this problem:
One of those patterns is perhaps lesser known but integral to std’s functioning: impl Read/Write for &Type. What this means is that if you have a reference to an IO type, such as File or TcpStream, you’re still able to call Read and Write methods thanks to some interior mutability tricks.
The implication of this is also that if you want to share a std::fs::File between multiple threads you don’t need to use an expensive Arc<Mutex<File>> because an Arc<File> suffices.
You might expect that if we wrap an IO type T in an Arc that it would implement Clone + Read + Write. But in reality it only implements Clone + Deref<T>... However, there's an escape hatch here: we can create a wrapper type around Arc<T> that implements Read + Write by dereferencing &T internally.
Here is his solution:
/// A variant of `Arc` that delegates IO traits if available on `&T`.
#[derive(Debug)]
pub struct IoArc<T>(Arc<T>);
impl<T> IoArc<T> {
/// Create a new instance of IoArc.
pub fn new(data: T) -> Self {
Self(Arc::new(data))
}
}
impl<T> Read for IoArc<T>
where
for<'a> &'a T: Read,
{
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
(&mut &*self.0).read(buf)
}
}
impl<T> Write for IoArc<T>
where
for<'a> &'a T: Write,
{
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
(&mut &*self.0).write(buf)
}
fn flush(&mut self) -> io::Result<()> {
(&mut &*self.0).flush()
}
}
MIT license
IoArc is available in the io_arc crate, though it is short enough to implement yourself if you don't want to pull in the dependency.
Simple answer: You can't.
In Rust, every type is implicitly movable by memcpy. So if your type stores references to itself, it would break as soon as the move happens; the references would be dangling.
More complex answer: You can't, unless you use Pin, unsafe and raw pointers.
But I'm pretty sure that using Arc for everything is the way to go instead.
Arc<TcpStream> does not implement Read or Write
You could just write a very thin wrapper struct around Arc<TcpStream> which implements Read and Write. It should be fairly easy.
Edit: Take a look at #JohnKugelman's anwser for such a wrapper.
I am trying to access a static hashmap for reading and writing but I am always getting error:
use std::collections::HashMap;
use std::sync::Mutex;
pub struct ModuleItem {
pub absolute_path: String,
}
lazy_static! {
static ref MODULE_MAP: Mutex<HashMap<i32, ModuleItem>> = Mutex::new(HashMap::new());
}
pub fn insert(identity_hash: i32, module_item: ModuleItem) {
MODULE_MAP
.lock()
.unwrap()
.insert(identity_hash, module_item);
}
pub fn get(identity_hash: i32) -> Option<&'static ModuleItem> {
MODULE_MAP.lock().unwrap().get(&identity_hash).clone()
}
But I am getting an error on the get function cannot return value referencing temporary value
I tried with .cloned(), .clone() or even nothing but I don't manage to get it to work. Can you help me?
I tried with .cloned(), .clone() or even nothing but I don't manage to get it to work. Can you help me?
All Option::clone does is clone the underlying structure, which in this case is an &ModuleItem so it just clones the reference, and you still have a reference, which you can't return because you only have access to the hashmap's contents while you hold the lock (otherwise it could not work).
Option::cloned actually clones the object being held by reference, but doesn't compile here because ModuleItem can't be cloned.
First you have to return a Option<ModuleItem>, you can not return a reference to the map contents since the lock is going to be released at the end of the function, and you can't keep a handle on hashmap contents across mutex boundaries as they could go away at any moment (e.g. an other thread could move them, or even clear the map entirely).
Then copy the ModuleItem, either by deriving Clone on ModuleItem (then calling Option::cloned) or by creating a new ModuleItem "by hand" e.g.
pub fn get(identity_hash: i32) -> Option<ModuleItem> {
MODULE_MAP.lock().unwrap().get(&identity_hash).map(|m|
ModuleItem { absolute_path: m.absolute_path.clone() }
)
}
If you need to get keys out a lot and are worried about performances, you could always store Arc<ModuleItem>. That has something of a cost (as it's a pointer so your string is now behind two pointers) however cloning an Arc is very cheap.
To avoid the double pointer you could make ModuleItem into an unsized type and have it store an str but... that's pretty difficult to work with so I wouldn't recommend it.
The function get cannot use a static lifetime because the data does not live for the entire life of the program (from the Rust book):
As a reference lifetime 'static indicates that the data pointed to by the reference lives for the entire lifetime of the running program. It can still be coerced to a shorter lifetime.
So you have to return either a none-static reference or a copy of the value of the HashMap. Reference is not possible because MODULE_MAP.lock().unwrap() returns a MutexGuard which is a local and therefore a temporary variable that hold the HashMap. And get() of the HashMap returns a reference.
Due to the fact that the temporary MutexGuard will be destroy at the end of the function, the reference returned by get would point to a temporary value.
To fix this, you could make ModuleItem clonable and return a copy of the value:
use std::collections::HashMap;
use std::sync::Mutex;
#[derive(Clone)]
pub struct ModuleItem {
pub absolute_path: String,
}
lazy_static::lazy_static! {
static ref MODULE_MAP: Mutex<HashMap<i32, ModuleItem>> = Mutex::new(HashMap::new());
}
pub fn insert(identity_hash: i32, module_item: ModuleItem) {
MODULE_MAP
.lock()
.unwrap()
.insert(identity_hash, module_item);
}
pub fn get(identity_hash: i32) -> Option<ModuleItem> {
MODULE_MAP.lock().unwrap().get(&identity_hash).cloned()
}
I am unable to share a reference between threads.
trait Facade { /*some functions*/ }
struct Client<'a> {
facade: &'a mut Facade,
join_grd: thread::JoinGuard<'a()>,
}
impl<'a> Client<'a> {
pub fn new(my_facade: &'a mut Facade) -> Client<'a> {
Client {
facade: my_facade,
join_grd: thread::scoped(|| Client::start(my_facade)),
}
}
fn start(my_facade: &'a mut Facade) { unimplemented!() }
}
Given my newbie status in Rust, I'm getting confused with concepts and errors. How do I achieve the above ?
I'm pretty sure you can't do this due to the mutable aliasing guarantees in Rust. In Rust you can't have two mutable references to the same thing at the same time, but this is exactly what happens in your code: you store my_facade to the field of Client and at the same time you are trying to pass it to start() method in another thread. This would require having two mutable references to the same Facade which is disallowed.
The actual errors which compiler emits on your code are caused by that you're using a non-moving closure. If you change thread::scoped() instantiation to this:
join_grd: thread::scoped(move || Client::start(my_facade))
the error would be more sensible:
test.rs:16:60: 16:69 error: cannot move `my_facade` into closure because it is borrowed
test.rs:16 join_grd: thread::scoped(move || Client::start(my_facade))
^~~~~~~~~
test.rs:15:21: 15:30 note: borrow of `*my_facade` occurs here
test.rs:15 facade: my_facade,
^~~~~~~~~
This essentially means that since &mut references are unique and are moved instead of copied, you can't duplicate them. Similar code with the regular & reference instead of &mut (and an additional Sync parent trait on Facade) works fine.
You have to rethink your architecture to fix this error. It is difficult to understand what you want from this piece of code alone, so I can't give any exact advices, but you may consider using Arc and Mutex if you want to share mutable state between threads.
Naive usage of Arc/Mutex like this:
fn start(my_facade: Arc<Mutex<Facade>>)
won't work because Facade is a trait, not a regular type. When you use traits as types, you're in fact opting into dynamic dispatch in form of trait objects. In short, trait objects can't be used directly; they should always be behind a pointer. Your original program also used trait objects (&'a mut Facade is a trait object). Ideally we should be able to form trait objects with any kind of smart pointer, and ideally Arc<Mutex<Facade>> should work, but unfortunately for now trait objects can only be created with &, &mut or Box:
fn start(my_facade: Arc<Mutex<Box<Facade>>>)
This is the reason of the error about Sized that you observe.
However, you should also consider not using trait objects at all and just use generics:
trait Facade: Send { fn f(&self); }
struct Client<'a, F: Facade> { // '
facade: Arc<Mutex<F>>,
join_grd: thread::JoinGuard<'a, ()>, // '
}
impl<'a, F: Facade+'a> Client<'a, F> { // '
pub fn new(my_facade: Arc<Mutex<F>>) -> Client<'a, F> { // '
let my_facade_2 = my_facade.clone(); // clone the Arc pointer
Client {
facade: my_facade,
join_grd: thread::scoped(move || Client::start(my_facade_2)),
}
}
fn start(my_facade: Arc<Mutex<F>>) { unimplemented!() }
}
You also need to add Send bound either on the trait itself (as in the example above) or on F type variable (as in F: Facade+Send+'a) because only Send data may be transferred between threads safely, so you need to specify that F is Send, either directly or as a supertrait constraint on Facade.
I have the following problem: I have a have a data structure that is parsed from a buffer and contains some references into this buffer, so the parsing function looks something like
fn parse_bar<'a>(buf: &'a [u8]) -> Bar<'a>
So far, so good. However, to avoid certain lifetime issues I'd like to put the data structure and the underlying buffer into a struct as follows:
struct BarWithBuf<'a> {bar: Bar<'a>, buf: Box<[u8]>}
// not even sure if these lifetime annotations here make sense,
// but it won't compile unless I add some lifetime to Bar
However, now I don't know how to actually construct a BarWithBuf value.
fn make_bar_with_buf<'a>(buf: Box<[u8]>) -> BarWithBuf<'a> {
let my_bar = parse_bar(&*buf);
BarWithBuf {buf: buf, bar: my_bar}
}
doesn't work, since buf is moved in the construction of the BarWithBuf value, but we borrowed it for parsing.
I feel like it should be possible to do something along the lines of
fn make_bar_with_buf<'a>(buf: Box<[u8]>) -> BarWithBuf<'a> {
let mut bwb = BarWithBuf {buf: buf};
bwb.bar = parse_bar(&*bwb.buf);
bwb
}
to avoid moving the buffer after parsing the Bar, but I can't do that because the whole BarWithBuf struct has to be initalised in one go.
Now I suspect that I could use unsafe code to partially construct the struct, but I'd rather not do that.
What would be the best way to solve this problem? Do I need unsafe code? If I do, would it be safe do to this here? Or am I completely on the wrong track here and there is a better way to tie a data structure and its underlying buffer together?
I think you're right in that it's not possible to do this without unsafe code. I would consider the following two options:
Change the reference in Bar to an index. The contents of the box won't be protected by a borrow, so the index might become invalid if you're not careful. However, an index might convey the meaning of the reference in a clearer way.
Move Box<[u8]> into Bar, and add a function buf() -> &[u8] to the implementation of Bar; instead of references, store indices in Bar. Now Bar is the owner of the buffer, so it can control its modification and keep the indices valid (thereby avoiding the problem of option #1).
As per DK's suggestion below, store indices in BarWithBuf (or in a helper struct BarInternal) and add a function fn bar(&self) -> Bar to the implementation of BarWithBuf, which constructs a Bar on-the-fly.
Which of these options is the most appropriate one depends on the actual problem context. I agree that some form of "member-by-member construction" of structs would be immensely helpful in Rust.
Here's an approach that will work through a little bit of unsafe code. This approach requires that you are okay with putting the referred-to thing (here, your [u8]) on the heap, so it won't work for direct reference of a sibling field.
Let's start with a toy Bar<'a> implementation:
struct Bar<'a> {
refs: Vec<&'a [u8]>,
}
impl<'a> Bar<'a> {
pub fn parse(src: &'a [u8]) -> Self {
// placeholder for actually parsing goes here
Self { refs: vec![src] }
}
}
We'll make BarWithBuf that uses a Bar<'static>, as 'static is the only lifetime with an an accessible name. The buffer we store things in can be anything that doesn't move the target data around on us. I'm going to go with a Vec<u8>, but Box, Pin, whatever will work fine.
struct BarWithBuf {
buf: Vec<u8>,
bar: Bar<'static>,
}
The implementation requires a tiny bit of unsafe code.
impl BarWithBuf {
pub fn new(buf: Vec<u8>) -> Self {
// The `&'static [u8]` is inferred, but writing it here for demo
let buf_slice: &'static [u8] = unsafe {
// Going through a pointer is a "good" way to get around lifetime checks
std::slice::from_raw_parts(&buf[0], buf.len())
};
let bar = Bar::parse(buf_slice);
Self { buf, bar }
}
/// Access to Bar should always come through this function.
pub fn bar(&self) -> &Bar {
&self.bar
}
}
The BarWithBuf::bar is an important function to re-associate the proper lifetimes to the references. Rust's lifetime elision rules make the function equivalent to pub fn bar<'a>(&'a self) -> &'a Bar<'a>, which turns out to be exactly what we want. The lifetime of the slices in BarWithBuf::bar::refs are tied to the lifetime of BarWithBuf.
WARNING: You have to be very careful with your implementation here. You cannot make #[derive(Clone)] for BarWithBuf, since the default clone implementation will clone buf, but the elements of bar.refs will still point to the original. It is only one line of unsafe code, but the safety is still off in the "safe" bits.
For larger bits of self-referencing structures, there's the ouroboros crate, which wraps up a lot of unsafe bits for you. The techniques are similar to the one I described above, but they live behind macros, which is a more pleasant experience if you find yourself making a number of self-references.