I've been trying to separate all my Redis operations in a separate implementation, but I got this error when I passed the Redis reference to the implementation.
error[E0596]: cannot borrow `*self.redis_client` as mutable, as it is behind a `&` reference
--> src/main.rs:30:21
|
30 | let _: () = self.redis_client.set("key", "value").unwrap();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
Code:
use redis::{Commands};
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
let redis_client_uri = String::from("redis://127.0.0.1:6379");
let redis_client = redis::Client::open(redis_client_uri)?;
let mut redis = redis_client
.get_connection()
.expect("Failed to connect redis");
let batch = vec!["Sample value".to_string()];
CacheOperations::new(&redis, batch);
Ok(())
}
struct CacheOperations<'a> {
redis_client: &'a redis::Connection,
batch: Vec<String>,
}
impl<'a> CacheOperations<'a> {
fn new(redis_client: &'a redis::Connection, batch: Vec<String>) -> Self {
Self {
redis_client: &redis_client,
batch: batch,
}
}
fn insert_batch(self) {
let _: () = self.redis_client.set("key", "value").unwrap();
}
}
You just need to use the mut keyword to indicate that a reference or value is mutable. When you use it with a lifetime, the mut goes after the lifetime usage (Ex: &'abc mut x).
use redis::{Commands};
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
let redis_client_uri = String::from("redis://127.0.0.1:6379");
let redis_client = redis::Client::open(redis_client_uri)?;
let mut redis = redis_client
.get_connection()
.expect("Failed to connect redis");
let batch = vec!["Sample value".to_string()];
// Use a mutable reference when creating the CacheOperations
CacheOperations::new(&mut redis, batch);
Ok(())
}
struct CacheOperations<'a> {
// Store a mutable reference
redis_client: &'a mut redis::Connection,
batch: Vec<String>,
}
impl<'a> CacheOperations<'a> {
// We need to consume a mutable reference
fn new(redis_client: &'a mut redis::Connection, batch: Vec<String>) -> Self {
// No need to reference the reference, we can store it as-is
Self { redis_client, batch }
}
// While not required here, you can use mut self to state that self is mutable
// when a function consumes self as an owned value.
fn insert_batch(mut self) {
let _: () = self.redis_client.set("key", "value").unwrap();
}
}
Related
I'm trying to implement a StateMachine in Rust, but I encountered some problems while trying to fire the callback of StateMachine in a spawn thread.
Here is my StateMachine struct. The state is a generic T because I want to use it in many different scenerios, and I use a Vec to store all the callbacks those registered into this StateMachine.
At the very begining, I didn't use the lifetime 'a, but it will run into some lifetime problems, so I add the lifetime 'a by this suggestion: Idiomatic callbacks in Rust
pub struct StateMachine<'a, T> where T:Clone+Eq+'a {
state: RwLock<T>,
listeners2: Vec<Arc<Mutex<ListenerCallback<'a, T>>>>,
}
pub type ListenerCallback<'a, T> = dyn FnMut(T) -> Result<()> + Send + Sync + 'a ;
When the state is changed, the StateMachine will fire all the callbacks, as follows.
pub async fn try_set(&mut self, new_state:T) -> Result<()> {
if (block_on(self.state.read()).deref().eq(&new_state)) {
return Ok(())
}
// todo change the state
// fire every listener in spawn
let mut fire_results = vec![];
for listener in &mut self.listeners2 {
let state = new_state.clone();
let fire_listener = listener.clone();
fire_results.push(tokio::spawn(async move {
let mut guard = fire_listener.lock().unwrap();
guard.deref_mut()(state);
}));
}
// if fire result return Err, return it
for fire_result in fire_results {
fire_result.await?;
}
Ok(())
}
But it will cause a compilation error.
error[E0521]: borrowed data escapes outside of associated function
--> src/taf/taf-core/src/execution/state_machine.rs:54:33
|
15 | impl<'a,T> StateMachine<'a,T> where T:Clone+Eq+Send {
| -- lifetime `'a` defined here
...
34 | pub async fn try_set(&mut self, new_state:T) -> Result<()> {
| --------- `self` is a reference that is only valid in the associated function body
...
54 | let fire_listener = listener.clone();
| ^^^^^^^^^^^^^^^^
| |
| `self` escapes the associated function body here
| argument requires that `'a` must outlive `'static`
##########################################################
The full code is coupled with a lot of business logic, so I rewrite 2 demos as follows, the problems is the same. The first demo fire callback synchronously and it works, the second demo try to fire callback asynchronously, it encounter the same problem: self escapes the associated function body here.
First demo(it works):
use std::alloc::alloc;
use std::ops::DerefMut;
use std::sync::{Arc, Mutex, RwLock};
use anyhow::Result;
use dashmap::DashMap;
struct StateMachine<'a,T> where T:Clone+Eq+'a {
state: T,
listeners: Vec<Box<Callback<'a, T>>>,
}
type Callback<'a, T> = dyn FnMut(T) -> Result<()> + Send + Sync + 'a;
impl<'a, T> StateMachine<'a,T> where T:Clone+Eq+'a {
pub fn new(init_state: T) -> Self {
StateMachine {
state: init_state,
listeners: vec![]
}
}
pub fn add_listener(&mut self, listener: Box<Callback<'a, T>>) -> Result<()> {
self.listeners.push(listener);
Ok(())
}
pub fn set(&mut self, new_state: T) -> Result<()> {
self.state = new_state.clone();
for listener in &mut self.listeners {
listener(new_state.clone());
}
Ok(())
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
enum ExeState {
Waiting,
Running,
Finished,
Failed,
}
struct Execution<'a> {
exec_id: String,
pub state_machine: StateMachine<'a, ExeState>,
}
struct ExecManager<'a> {
all_jobs: Arc<RwLock<DashMap<String, Execution<'a>>>>,
finished_jobs: Arc<RwLock<Vec<String>>>,
}
impl<'a> ExecManager<'a> {
pub fn new() -> Self {
ExecManager {
all_jobs: Arc::new(RwLock::new(DashMap::new())),
finished_jobs: Arc::new(RwLock::new(vec![]))
}
}
fn add_job(&mut self, job_id: String) {
let mut execution = Execution {
exec_id: job_id.clone(),
state_machine: StateMachine::new(ExeState::Waiting)
};
// add listener
let callback_finished_jobs = self.finished_jobs.clone();
let callback_job_id = job_id.clone();
execution.state_machine.add_listener( Box::new(move |new_state| {
println!("listener fired!, job_id {}", callback_job_id.clone());
if new_state == ExeState::Finished || new_state == ExeState::Failed {
let mut guard = callback_finished_jobs.write().unwrap();
guard.deref_mut().push(callback_job_id.clone());
}
Ok(())
}));
let mut guard = self.all_jobs.write().unwrap();
guard.deref_mut().insert(job_id, execution);
}
fn mock_exec(&mut self, job_id: String) {
let mut guard = self.all_jobs.write().unwrap();
let mut exec = guard.deref_mut().get_mut(&job_id).unwrap();
exec.state_machine.set(ExeState::Finished);
}
}
#[test]
fn test() {
let mut manager = ExecManager::new();
manager.add_job(String::from("job_id1"));
manager.add_job(String::from("job_id2"));
manager.mock_exec(String::from("job_id1"));
manager.mock_exec(String::from("job_id2"));
}
Second demo:
use std::alloc::alloc;
use std::ops::DerefMut;
use std::sync::{Arc, Mutex, RwLock};
use anyhow::Result;
use dashmap::DashMap;
use petgraph::algo::astar;
struct StateMachine<'a,T> where T:Clone+Eq+Send+'a {
state: T,
listeners: Vec<Arc<Mutex<Box<Callback<'a, T>>>>>,
}
type Callback<'a, T> = dyn FnMut(T) -> Result<()> + Send + Sync + 'a;
impl<'a, T> StateMachine<'a,T> where T:Clone+Eq+Send+'a {
pub fn new(init_state: T) -> Self {
StateMachine {
state: init_state,
listeners: vec![]
}
}
pub fn add_listener(&mut self, listener: Box<Callback<'a, T>>) -> Result<()> {
self.listeners.push(Arc::new(Mutex::new(listener)));
Ok(())
}
pub fn set(&mut self, new_state: T) -> Result<()> {
self.state = new_state.clone();
for listener in &mut self.listeners {
let spawn_listener = listener.clone();
tokio::spawn(async move {
let mut guard = spawn_listener.lock().unwrap();
guard.deref_mut()(new_state.clone());
});
}
Ok(())
}
}
#[derive(Clone, Eq, PartialEq, Hash)]
enum ExeState {
Waiting,
Running,
Finished,
Failed,
}
struct Execution<'a> {
exec_id: String,
pub state_machine: StateMachine<'a, ExeState>,
}
struct ExecManager<'a> {
all_jobs: Arc<RwLock<DashMap<String, Execution<'a>>>>,
finished_jobs: Arc<RwLock<Vec<String>>>,
}
impl<'a> ExecManager<'a> {
pub fn new() -> Self {
ExecManager {
all_jobs: Arc::new(RwLock::new(DashMap::new())),
finished_jobs: Arc::new(RwLock::new(vec![]))
}
}
fn add_job(&mut self, job_id: String) {
let mut execution = Execution {
exec_id: job_id.clone(),
state_machine: StateMachine::new(ExeState::Waiting)
};
// add listener
let callback_finished_jobs = self.finished_jobs.clone();
let callback_job_id = job_id.clone();
execution.state_machine.add_listener( Box::new(move |new_state| {
println!("listener fired!, job_id {}", callback_job_id.clone());
if new_state == ExeState::Finished || new_state == ExeState::Failed {
let mut guard = callback_finished_jobs.write().unwrap();
guard.deref_mut().push(callback_job_id.clone());
}
Ok(())
}));
let mut guard = self.all_jobs.write().unwrap();
guard.deref_mut().insert(job_id, execution);
}
fn mock_exec(&mut self, job_id: String) {
let mut guard = self.all_jobs.write().unwrap();
let mut exec = guard.deref_mut().get_mut(&job_id).unwrap();
exec.state_machine.set(ExeState::Finished);
}
}
#[test]
fn test() {
let mut manager = ExecManager::new();
manager.add_job(String::from("job_id1"));
manager.add_job(String::from("job_id2"));
manager.mock_exec(String::from("job_id1"));
manager.mock_exec(String::from("job_id2"));
}
Compile error of second demo:
error[E0521]: borrowed data escapes outside of associated function
--> generic/src/callback2.rs:34:34
|
15 | impl<'a, T> StateMachine<'a,T> where T:Clone+Eq+Send+'a {
| -- lifetime `'a` defined here
...
29 | pub fn set(&mut self, new_state: T) -> Result<()> {
| --------- `self` is a reference that is only valid in the associated function body
...
34 | let spawn_listener = listener.clone();
| ^^^^^^^^^^^^^^^^
| |
| `self` escapes the associated function body here
| argument requires that `'a` must outlive `'static`
|
= note: requirement occurs because of the type `std::sync::Mutex<Box<dyn FnMut(T) -> Result<(), anyhow::Error> + Send + Sync>>`, which makes the generic argument `Box<dyn FnMut(T) -> Result<(), anyhow::Error> + Send + Sync>` invariant
= note: the struct `std::sync::Mutex<T>` is invariant over the parameter `T`
= help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance
Tasks spawned with tokio::spawn() cannot use borrowed data (here, the data with lifetime 'a, whatever it may be). This is because there is not currently (and likely will never be) any way to guarantee that the borrowed data reliably outlives the spawned task.
You have two choices:
Fire the notifications without spawning. You can put the notification futures into a FuturesUnordered to run them all concurrently, but they will still all have to finish before try_set() does.
Remove the lifetime parameter; stop allowing callbacks that borrow data. Put 'static on your dyn types where necessary. Change the users of the StateMachine so they do not try to use borrowed data but use Arc instead, if necessary.
pub struct StateMachine<T> where T: Clone + Eq + 'static {
state: RwLock<T>,
listeners2: Vec<Arc<Mutex<ListenerCallback<T>>>>,
}
pub type ListenerCallback<T> = dyn FnMut(T) -> Result<()> + Send + Sync + 'static;
I got a Box<Rc<RefCell<T>>> from FFI. How can I get the &mut T based on it?
I can not compile it. Compiler tells me:
47 | let mut r: &mut Server = server.borrow_mut();
| ^^^^^^^^^^ the trait BorrowMut<Server> is not implemented for Box<Rc<RefCell<Server>>>
|
= help: the trait BorrowMut<T> is implemented for Box<T, A>
For more information about this error, try rustc --explain E0277.
#[derive(Debug)]
struct Server {
id: i32,
}
impl Server {
pub fn change_id(&mut self) {
self.id = self.id + 1;
}
}
#[no_mangle]
pub extern "C" fn server_change_id(server: *mut Rc<RefCell<Server>>) -> isize {
let server: Box<Rc<RefCell<Server>>> = unsafe { Box::from_raw(server) };
let mut r: &mut Server = server.borrow_mut();
r.change_id();
return 0;
}
Auto-deref will make borrow_mut() directly accessible.
use std::{cell::RefCell, cell::RefMut, ops::DerefMut, rc::Rc};
fn main() {
let a = Box::new(Rc::new(RefCell::new("aaa".to_owned())));
//
println!("{:?}", a);
{
let mut r: RefMut<String> = a.borrow_mut();
r.push_str("bbb"); // via RefMut
let r2: &mut String = r.deref_mut();
r2.push_str("ccc"); // via exclusive-reference
}
println!("{:?}", a);
}
/*
RefCell { value: "aaa" }
RefCell { value: "aaabbbccc" }
*/
In your code, let mut r = server.borrow_mut(); should be enough to invoke r.change_id().
let mut r = server.borrow_mut();
r.change_id();
If you absolutely want a &mut, then use let r2 = r.deref_mut() and invoke r2.change_id().
let mut r = server.borrow_mut();
let r2 = r.deref_mut();
r2.change_id();
T gets wrapped inside the RefCell<T> must implement the Deref or DerefMut trait in order to be borrowed mutably or reference borrowed. In your case, Server must implement a deref method.
use std::ops::DerefMut;
impl DerefMut for Server {
fn deref_mut(&mut self) -> &mut Self {
*self // this assumes your server has more than just one i32 field.
}
}
After this implementation, you should be able to call server.borrow_mut() should work perfectly and return you a mutable server object.
I am trying this code:
struct ByteIter<'a> {
remainder: &'a mut [u8],
index: usize,
}
impl<'a> ByteIter<'a> {
fn new(remainder: &'a mut [u8]) -> ByteIter<'a> {
ByteIter{remainder, index: 0}
}
fn next(&'a mut self) -> Option<&'a mut u8> {
if self.index >= self.remainder.len() {
None
} else {
let mut byte = &mut self.remainder[self.index];
self.index += 1;
Some(byte)
}
}
}
fn main() {
let mut a = [ 0x51, 0x52, 0x53, 0x54];
let mut bytes = ByteIter::new(&mut a);
{
let byte_1 = bytes.next();
}
let byte_2 = bytes.next();
}
Now, as I understand, the byte_1 was borrowed from bytes. But it's lifetime has already expired. Still when I compile this, I see the following error:
29 | let byte_2 = bytes.next();
| ^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
What is the first mutable borrow here? Shouldn't it be released when byte_1 goes out of scope?
Importantly, what should I do to fix this?
With impl<'a> ByteIter<'a>, you declare that 'a is the lifetime used in the structure, ie, it's the lifetime of the underlying slice.
Now, when you say fn next(&'a mut self) -> Option<&'a mut u8> {, you're reusing the same 'a, and you say it's the same than the returned mutable reference. You're saying that the returned lifetime is the same than the ByteIter struct content.
Remove the additional lifetime constraint and the compiler will be free to compute the appropriate lifetime:
fn next(& mut self) -> Option<& mut u8> {
if self.index >= self.remainder.len() {
None
} else {
let mut byte = &mut self.remainder[self.index];
self.index += 1;
Some(byte)
}
}
I have this trait and implementation:
#[async_trait]
pub trait AsyncKeyProvider {
async fn get_key_async(&mut self, key_id: &str) -> Result<Option<Jwk>, ()>;
fn as_any(&self) -> &dyn Any;
}
#[async_trait]
impl AsyncKeyProvider for GoogleKeyProvider {
async fn get_key_async(&mut self, key_id: &str) -> Result<Option<Jwk>, ()> {
{...}
}
fn as_any(&self) -> &dyn Any {
self
}
}
In order to pass it into my handler in actix-web, I'm passing through a GoogleKeyProvider like this:
let key_provider = web::Data::from(Arc::new(GoogleKeyProvider::default()));
let server = HttpServer::new(move || {
App::new()
.app_data(key_provider.clone())
.route("/validate", web::post().to(validate))
})
With the handler doing this:
pub async fn validate(jwt_body: web::Json<JwtBody>, provider: web::Data<Box<dyn AsyncKeyProvider>>) -> impl Responder {
let provider_object: &GoogleKeyProvider = provider.as_any().downcast_ref::<GoogleKeyProvider>().expect("Wasn't a GoogleKeyProvider");
match validate_jwt(&jwt_body.jwt, provider_object).await {
{...}
}
}
validate_jwt then tries to call a method on the provider struct like this:
async fn validate_jwt(jwt: &String, provider: &GoogleKeyProvider) -> Result<bool, Box<dyn std::error::Error>> {
let key_to_use = provider.get_key_async(<thing>).await.unwrap();
}
Which presents me with this error:
error[E0596]: cannot borrow `*provider` as mutable, as it is behind a `&` reference
--> src\routes\validate.rs:48:22
|
48 | let key_to_use = provider.get_key_async(<thing>).await.unwrap();
| ^^^^^^^^ `provider` is a `&` reference, so the data it refers to cannot be borrowed as mutable
As far as I can understand, this is happening because the result of my downcasting is a reference (due to downcast_ref), but I think I'd be wanting the plain GoogleKeyProvider type instead - I'm not sure on that though. I believe the provider needs to be mutable as the values inside it (see below) can change during the lifetime of the provider (it's intended to provide a temporary cache for some keys, and automatically update them if they're out of date)
#[derive(Clone)]
pub struct GoogleKeyProvider {
cached: Option<JwkSet>,
expiration_time: Instant,
}
I'm not sure how to get this working with downcasting, though. Is anyone able to help me see where I've gone wrong?
You have to choice if get_key_async update somme thing at the struct.
The simple code below show you the error
trait Atrait {
fn afn(&mut self) -> i32;
}
struct Astruct {}
impl Atrait for Astruct {
fn afn(&mut self) -> i32 {
2
}
}
fn main()
{
// test should be mutable
let test = Astruct{};
let value = test.afn();
println!("Value {}", value);
}
This work because afn(self) is not declared mutable afn(&mut self)
trait Atrait {
fn afn(&self) -> i32;
}
struct Astruct {}
impl Atrait for Astruct {
fn afn(&self) -> i32 {
2
}
}
fn main()
{
let test = Astruct{};
let value = test.afn();
println!("Value {}", value);
}
Problem Description
I have a Config struct that can store a FnMut callback function. The catch is: not all of my configurations require a callback function, so I would like to make adding a callback function optional. This requires the member variable to be initialized with a default function that will get used if no callback is set.
Existing Code
struct Config<'a>{
callback: &'a mut dyn (FnMut(&str))
}
fn default_fn(msg: &str){
println!("default_fn({})", msg);
}
impl<'a> Config<'a> {
pub fn new() -> Config<'a> {
Config{
callback: &default_fn // ERROR: types differ in mutability
}
}
pub fn set_callback(mut self, callback_fn: &'a mut dyn (FnMut(&str))) -> Config<'a> {
self.callback = callback_fn;
self
}
}
fn main() {
// Our FnMut callback
let mut msg_log: Vec<String> = vec![];
let mut callback_fn = |msg: &str| {
msg_log.push(msg.to_string());
};
{
let mut config = Config::new();
(config.callback)("Hello World!");
config = config.set_callback(&mut callback_fn);
(config.callback)("Hello World!");
}
// Demonstration that the callback actually works
println!("{:?}", msg_log);
}
error[E0308]: mismatched types
--> src/main.rs:13:23
|
13 | callback: &default_fn // ERROR: types differ in mutability
| ^^^^^^^^^^^ types differ in mutability
|
= note: expected type `&mut dyn for<'r> std::ops::FnMut(&'r str)`
found type `&for<'r> fn(&'r str) {default_fn}`
Does someone have any suggestions on how to solve that problem?
Things I already tried, without any success:
Initializing it with a closure: callback: &|_: &str|{}
Using a member function instead of a global function
Creating a mutable reference: callback: &mut default_fn
(causes: cannot return value referencing temporary value)
I'm running out of ideas, any help is appreciated. Even if the answer is that what I am trying to do is impossible for reasons I didn't realize yet.
You should really box the trait object function. That makes the whole code much easier to use:
struct Config<'a>{
callback: Box<dyn FnMut(&str) + 'a>,
}
fn default_fn(msg: &str){
println!("default_fn({})", msg);
}
impl<'a> Config<'a> {
pub fn new() -> Config<'a> {
Config{
callback: Box::new(default_fn)
}
}
pub fn set_callback(self, callback: &'a mut dyn (FnMut(&str))) -> Config<'a> {
Config {
callback: Box::new(callback),
..self
}
}
}
fn main() {
// Our FnMut callback
let mut msg_log = vec![];
let mut callback_fn = |msg: &str| {
msg_log.push(msg.to_string());
};
{
let mut config = Config::new();
(config.callback)("Hello World!");
config = config.set_callback(&mut callback_fn);
(config.callback)("Hello World!");
}
// Demonstration that the callback actually works
println!("{:?}", msg_log);
}
Note that it is difficult to use callbacks in idiomatic Rust. I would even say that they aren't idiomatic at all. You should use a channel, something like that:
use std::sync::mpsc::{channel, Sender, SendError};
struct Config {
sender: Sender<String>,
}
impl Config {
pub fn new(sender: Sender<String>) -> Config {
Config{
sender
}
}
pub fn send(&self, message: String) -> Result<(), SendError<String>> {
self.sender.send(message)
}
}
fn main() {
let (sender, receiver) = channel();
let config = Config::new(sender);
config.send("Hello world!".into()).unwrap();
println!("{:?}", receiver.recv().unwrap());
}
Just wanted to share the solution I found:
Single-threaded, callback-based.
While in my opinion this one really answers the question I had, I think you guys are still right about the problems I might encounter in the future with this programming style. I will definitely reconsider your advice about using channels.
struct Config<'a>{
callback: Option<&'a mut dyn (FnMut(&str))>
}
impl<'a> Config<'a> {
pub fn new() -> Config<'a> {
Config{
callback: None
}
}
pub fn set_callback(mut self, callback_fn: &'a mut dyn (FnMut(&str))) -> Config<'a> {
self.callback = Some(callback_fn);
self
}
pub fn run_callback(&mut self, msg: &str){
if let Some(callback) = &mut self.callback{
callback(msg);
} else {
// Default code
println!("default_fn({})", msg);
}
}
}
fn main() {
// Our FnMut callback
let mut msg_log: Vec<String> = vec![];
let mut callback_fn = |msg: &str| {
msg_log.push(msg.to_string());
};
let mut config = Config::new();
config.run_callback("Hello World!");
config = config.set_callback(&mut callback_fn);
config.run_callback("Hello World!");
// Demonstration that the callback actually works
println!("{:?}", msg_log);
}