I have this rust code where I am trying to implement observability. And, in my closure add_foo_listener I simply update the called variable and then assert it it's called.
This code compiles, and the closure is invoked, as is evidenced by the println, but the called variable doesn't stick. Everything I read says I should be using FnMut which, I believe I am. But, I don't understand why this boolean isn't being updated
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
struct Foo {
baz: u32
}
struct Holder {
pub foo_observers: Vec<Box<dyn FnMut(Foo)>>,
}
impl Holder {
pub fn add_foo_listener<CB: 'static + FnMut(Foo) + FnMut(Foo)>(&mut self, on_foo: CB) -> &mut Self {
self.foo_observers.push(Box::new(on_foo));
self
}
pub fn notify_foo_event(&mut self, foo: Foo) -> &mut Self {
self.foo_observers
.iter_mut()
.for_each(|notify| notify(foo.clone()));
self
}
}
#[test]
fn test_foo() -> Result<(), String> {
let mut holder = Holder{
foo_observers: Vec::new()
};
let mut called: bool = false;
let foo = Foo{
baz : 10
};
holder.add_foo_listener( move |f: Foo| {
assert_eq!(f,foo);
println!("Listener called");
called = true;
}).notify_foo_event(foo);
assert!(called); // called is still false
Ok(())
}
Output:
---- notifier::test::test_foo stdout ----
Listener called
thread 'notifier::test::test_foo' panicked at 'assertion failed: called', foo.rs
assert!(called);
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
In the closure, you are using the move keyword: this means that all the captured values are moved into the closure.
A non-Copy value would be unusable after the closure creation, but your called is of type bool that is Copy. This means that the closure has copy of called while your test_foo() has another copy. They are disconnected, so if you modify one the other will notice nothing.
If you want to use the same value you have to somehow capture in the closure a reference to the original value. If the closure function is simple you can just remove the move, but if it is more complex that may not be possible, then you can capture a reference explicitly:
let rcalled = &mut called;
holder.add_foo_listener(
move |f: Foo| {
println!("Listener called");
*rcalled = true;
}
).notify_foo_event(foo);
Unfortunately, this does not compile because this closure now holds a reference to a local variable, that makes it no 'static, which is required by add_foo_listener().
To get a reference-like value that is 'static you can use the Rust big-gun: Arc<RefCell<T>>. Although in your case, since it is a Copy value, you can use the simpler Arc<Cell<T>>:
let called = Arc::new(Cell::new(false));
holder.add_foo_listener({
let called = called.clone();
move |f: Foo| {
println!("Listener called");
called.set(true);
}
}).notify_foo_event(foo);
assert!(called.get()); // called is now true
Related
I am trying to get rate limiting to work with Warp using Governor crate. However, when I try using rate_limiter instance wrapped in Arc part of a closure, I keep getting "expected a closure that implements the Fn trait, but this closure only implements FnOnce"
I tried cloning the rate_limiter instance inside and outside the closure, but it still complains. Can someone help me out with this?
use crate::rejections::RateLimitFailure;
use dashmap::DashMap;
use governor::{
clock::{QuantaClock, QuantaInstant},
middleware::NoOpMiddleware,
state::InMemoryState,
Quota, RateLimiter,
};
use nonzero_ext::nonzero;
use std::collections::hash_map::RandomState;
use std::sync::Arc;
use warp::{Filter, Rejection};
const LIMIT: u32 = 50;
#[derive(Debug, Clone)]
pub struct FridayRateLimiter<'a> {
pub lim: Arc<
RateLimiter<
&'a str,
DashMap<&'a str, InMemoryState, RandomState>,
QuantaClock,
NoOpMiddleware<QuantaInstant>,
>,
>,
}
impl<'a> FridayRateLimiter<'a> {
pub fn new() -> Self {
let lim = Arc::new(RateLimiter::keyed(Quota::per_second(nonzero!(LIMIT))));
FridayRateLimiter { lim }
}
}
pub fn with_rate_limiter(
rate_limiter: FridayRateLimiter,
) -> impl Filter<Extract = (bool,), Error = Rejection> + Clone {
let addr = warp::header::<String>("HTTP_X_FORWARDED_FOR");
let rate_limiter = rate_limiter.clone();
addr.and_then(|ip: String| async move {
let rate_limiter = rate_limiter.clone();
if rate_limiter.lim.check_key(&ip.as_str()).is_err() {
return Err(warp::reject::custom(RateLimitFailure));
}
Ok(true)
})
}
From here: When does a closure implement Fn, FnMut and FnOnce?
Cloning the rate limiter inside the closure seems wrong, for starters.
Next, I don't see why you'd need to take ownership of anything, so I'd drop the move. You really only need an immutable borrow to call check_key on the rate limiter, don't you?
And that's the condition for implementing Fn: Something that won't mess up its environment because it only needs &-access to it.
I am using sdl2 crate and I am trying to create a simple "context" object with timer inside of it. I have the following code
extern crate sdl2;
struct Ctx<'a, 'b> {
timer: sdl2::timer::Timer<'a, 'b>,
}
impl<'a,'b> Ctx<'a, 'b> {
pub fn new() -> Ctx<'a,'b> {
let sdl_context = sdl2::init().unwrap();
let timer_subsystem = sdl_context.timer().unwrap();
let timer = timer_subsystem.add_timer(100, Box::new(|| -> u32 { 100 }));
Ctx { timer }
}
}
pub fn main() {
let _ = Ctx::new();
}
The problem is that add_timer() seem to borrow timer_subsystem object, and it prevents timer to be moved out of the function.
error[E0515]: cannot return value referencing local variable `timer_subsystem`
--> src/main.rs:13:9
|
11 | let timer = timer_subsystem.add_timer(100, Box::new(|| -> u32 { 100 }));
| ----------------------------------------------------------- `timer_subsystem` is borrowed here
12 |
13 | Ctx { timer }
| ^^^^^^^^^^^^^ returns a value referencing data owned by the current function
However, if I look into add_timer(), I cannot see why does the reference is considered still alive. In fact the self reference is not used in the function at all:
https://docs.rs/sdl2/latest/src/sdl2/timer.rs.html#17-32
pub fn add_timer<'b, 'c>(&'b self, delay: u32, callback: TimerCallback<'c>) -> Timer<'b, 'c> {
unsafe {
let callback = Box::new(callback);
let timer_id = sys::SDL_AddTimer(
delay,
Some(c_timer_callback),
mem::transmute_copy(&callback),
);
Timer {
callback: Some(callback),
raw: timer_id,
_marker: PhantomData,
}
}
}
I was trying to put both timer and timer_subsystem to Ctx (to make sure their lifetime is the same), but it didn't help. Neither wrapping them with Box<> (to prevent having a dangling pointer after timer_subsystem was moved), but neither worked.
Why of I have a reference to local data here and how can I properly return the timer object?
The secret is in the PhantomData. The full type as declared inside Timer is PhantomData<&'b ()>, and 'b is specified as &'b self in add_timer(). This ties the lifetime of the returned Timer to the lifetime of the TimerSubsystem. This is because when the subsystem is dropped it is shutted down, and it is not permitted to access the timer afterward.
The best option is probably to initialize the timers subsystem (and SDL) before calling Ctx::new() and passing it as an argument, as it is hard to store both the subsystem and the timer (a self-referential struct).
I am attempting to model some program state as Mutables from the futures-signals library, whose value I want to set generically from a serde_json Value identified by some string key.
For example, given I received some payload instructing me to update "my_int" with a Value, I want to be able to set the value of the Mutable that is known as "my_int".
My idea was to have a map from identifiers like "my_int" to a non-templated wrapper around a mutable's setter. It is important that said wrapper is non-templated, because otherwise I couldn't hold a collection of them in one map:
let my_int = Mutable::new(123);
let my_str = Mutable::new("asdf");
// ...
let setters = HashMap::from([
("my_int", /* magic wrapper around setter here somehow */),
("my_str", /* magic wrapper around setter here somehow */),
// ...
]);
let property_name = "my_int"; // constant for demo purposes
let value = Value::from(234); // constant for demo purposes
let setter = setters.get(property_name).unwrap();
(setter.deser_and_set)(value);
Right now said magic wrapper looks like this:
struct SetterWrapper<'a> {
name: &'static str,
deser_and_set: &'a dyn Fn(Value) -> Result<(), Error>,
// + some other unrelated fields
}
And I can create those inline, and it works:
let my_int_setter = SetterWrapper {
name: "my_int",
deser_and_set: &(|v: Value| {
my_int.set(serde_json::from_value(v)?);
Ok(())
}),
// + some other unrelated fields
};
But I have many mutables and don't want to repeat the above code for every one of them, so I attempted to put it into a function:
fn wrap_setter<'a, T>(name: &'static str, mutable: &'a Mutable<T>) -> SetterWrapper<'a>
where T: for<'de> Deserialize<'de>
{
let deser_and_set = |v: Value| {
mutable.set(serde_json::from_value::<T>(v)?);
Ok(())
};
SetterWrapper {
name,
deser_and_set: &deser_and_set,
}
}
which I intend to use like let my_int_setter = wrap_setter("my_int", &my_int);, however I am encountering the following error:
error[E0515]: cannot return value referencing local variable `deser_and_set`
--> src\main.rs:66:5
|
66 | / SetterWrapper {
67 | | name,
68 | | deser_and_set: &deser_and_set,
| | -------------- `deser_and_set` is borrowed here
69 | | }
| |_____^ returns a value referencing data owned by the current function
The error itself makes sense to me: of course I can't return references to local variables, as those would dangle. But I believe conceptually I could solve the issue by somehow marking the closure in the function to have the same lifetime as the mutable, namely 'a, but you cannot give variables lifetime annotations.
How can I solve this issue? Or is my approach already clumsy?
To work around the issue, one way I can think of is change the property deser_and_set to a Box from a reference. With that, the ownership of the Box can be moved out of the function. Give it a try.
struct SetterWrapper {
name: &'static str,
deser_and_set: Box<dyn Fn(Value) -> Result<(), Error>>,
// + some other unrelated fields
}
fn wrap_setter<T>(name: &'static str, mutable: &Mutable<T>) -> SetterWrapper
where T: for<'de> Deserialize<'de>
{
SetterWrapper {
name,
deser_and_set: Box::new(|v: Value| {
mutable.set(serde_json::from_value::<T>(v)?);
Ok(())
};),
}
}
Probably the answer from #Joe_Jingyu is cleaner but I want to point out a second way you could take:
make SetterWrapper a trait and implement it for Mutable:
trait SetterWrapper {
fn deser_and_set(&self, v: Value) -> Result<(), Error>;
}
impl<T> SetterWrapper for Mutable<T>
where
T: for<'de> serde::de::Deserialize<'de>,
{
fn deser_and_set(&self, v: Value) -> Result<(), Error> {
self.set(serde_json::from_value::<T>(v)?);
Ok(())
}
}
Now you can create the HashMap with the trait objects and set the value:
let setters = HashMap::from([
("my_int", &my_int as &dyn SetterWrapper),
("my_str", &my_str),
]);
let property_name = "my_int"; // constant for demo purposes
let value = Value::from(234); // constant for demo purposes
let setter = setters.get(property_name).unwrap();
// now the call can be direct
setter.deser_and_set(value).unwrap();
Playground link (Note: I have build a simple Mutable myself, just to make the example work)
Consider the following code:
enum Deferred<F, T> {
Fn(F),
Ready(T),
}
impl<F, T> Deferred<F, T>
where
F: FnOnce() -> T,
T: Clone,
{
fn get_clone(&mut self) -> T {
let clone;
*self = match *self {
Self::Fn(f) => { // `f` moved here
let data = f(); // this needs to consume `f`
clone = data.clone();
Self::Ready(data)
},
Self::Ready(data) => { // `data` moved here
clone = data.clone();
Self::Ready(data)
},
};
clone
}
}
The compiler complains about the *self = match *self {...}; statement because its right hand side takes ownership of contents of self. Is there a way to accomplish this behaviour with just a mutable reference to self?
I found a workaround using F: FnMut() -> T so that f doesn't have to be moved but this approach clearly has its limitations.
I also tried what the answer to Is there a safe way to temporarily retrieve an owned value from a mutable reference in Rust? suggested but it led to an issue with initialization of clone (the compiler could no longer reason that the match statement would initialize clone because the code was moved into a closure) so I had to use MaybeUninit with unsafe.
At that point it was better to read/write self through a raw pointer:
unsafe {
std::ptr::write(
self as *mut Self,
match std::ptr::read(self as *const Self) {
Self::Fn(f) => {
let data = f();
clone = data.clone();
Self::Ready(data)
},
Self::Ready(data) {
clone = data.clone();
Self::Ready(data)
},
}
);
}
It is not possible to do this in a straightforward safe fashion, because while f is being executed there is no possible valid value of Deferred as defined: you don't yet have a T to go in Deferred::Ready, and you're consuming the F so you can't have Deferred::Fn.
If you use the take_mut crate or similar, that accomplishes this by replacing panicking with aborting, so the invalid state can never be observed. But, in your case, I would recommend introducing a third state to the enum instead — this changes the semantics but in a way that, shall we say, respects the fact that f can fail.
enum Deferred<F, T> {
Fn(F),
Ready(T),
Failed,
}
impl<F, T> Deferred<F, T>
where
F: FnOnce() -> T,
T: Clone,
{
fn get_clone(&mut self) -> T {
match std::mem::replace(self, Self::Failed) {
Self::Ready(data) => {
let clone = data.clone();
*self = Self::Ready(data);
clone
},
Self::Fn(f) => {
let data = f();
*self = Self::Ready(data.clone());
data
}
Self::Failed => {
panic!("A previous call failed");
}
}
}
}
With this solution, the very first thing we do is swap out *self for Self::Failed, so we own the Deferred value and can freely move out the non-clonable F value. Notice that the expression being matched is not a borrow of self, so we aren't blocked from further modifying *self.
This particular solution does have a disadvantage: it's unnecessarily writing to *self on every call (which is unobservable, but could reduce performance). We can fix that, by separating the decision of what to do from doing it, but this requires writing a second pattern match to extract the value:
impl<F, T> Deferred<F, T>
where
F: FnOnce() -> T,
T: Clone,
{
fn get_clone(&mut self) -> T {
match self {
Self::Ready(data) => {
return data.clone();
},
Self::Failed => {
panic!("A previous call failed");
}
Self::Fn(_) => {
// Fall through below, relinquishing the borrow of self.
}
}
match std::mem::replace(self, Self::Failed) {
Self::Fn(f) => {
let data = f();
*self = Self::Ready(data.clone());
data
}
_ => unreachable!()
}
}
}
The fundamental problem here is trying to take ownership of the value in the enum while simultaneously having a live reference (via &mut self) to it. Safe Rust code will not let you do that, which is the reason for the compiler error. This is also a problem with your unsafe-solution: What if any code inside one of the match-arms panics? Then the owned value created via ptr::read gets dropped, the stack unwinds, and the caller might catch that panic and is then perfectly capable of observing the Deferred it owns (of which it gave a &mut to get_clone()) after it has been dropped, that is, in an invalid state and therefor causing Undefined Behaviour (see the docs for ptr::read for an example).
What must be achieved therefore is to hold &mut self in a valid/defined state while taking ownership. You can do that by having a cheap or possibly even free variant on Deferred, which is put into &mut self while the new value is being constructed.
For instance:
#[derive(Debug)]
enum FooBar {
Foo(String),
Bar(&'static str),
}
impl FooBar {
fn switch(&mut self) {
// notice here
*self = match std::mem::replace(self, FooBar::Bar("temporary")) {
FooBar::Foo(_) => FooBar::Bar("default"),
FooBar::Bar(s) => FooBar::Foo(s.to_owned()),
}
}
}
fn main() {
let mut s = FooBar::Bar("hello");
s.switch();
println!("{:?}", s);
}
Here, we use std::mem::replace to switch the value behind &mut self with a cheaply constructed temporary value; replace() returns ownership of the original value, which we match on to construct a new value; the value returned by the match-expression is then put into place via the *self-assignment. If everything goes well, the FooBar::Bar("temporary") is not ever observed; but it could be observed if the match panicked; but even then, the code is at least safe. In case your match can't unwind at all, the compiler might even be able to eliminate that useless store entirely.
Coming back to your original code, I don't see how to construct a Deferred without T being Default, as you can neither construct the Fn nor the Ready case in a safe way. Either that can be added via an additional bound, or you can add a third variant Empty to your enum.
As I see it, the difference between your problem and the one that you referenced is that you also want to return the inner T while replacing self. Which is something that the there proposed crate take_mut does not provide. However, I noticed that it seems a bit unmaintained. So, I had a short look around, and found the replace_with crate instead. It basically does the same as take_mut, but it also has a replace_with_or_abort_and_return function -- exactly what you need:
use replace_with::replace_with_or_abort_and_return;
impl<F, T> Deferred<F, T>
where
F: FnOnce() -> T,
T: Clone,
{
fn get_clone(&mut self) -> T {
replace_with_or_abort_and_return(self, |s| match s {
Self::Fn(f) => {
// `f` moved here
let data = f(); // this needs to consume `f`
let clone = data.clone();
(clone, Self::Ready(data))
}
Self::Ready(data) => {
// `data` moved here
let clone = data.clone();
(clone, Self::Ready(data))
}
})
}
}
Tho, notice, that taking a value out of a mutable borrow, has the potential of UB that is specifically if your f() panics, because self would be left in an uninitialized state. Therefore, we get this slightly cumbersome function name, which indicates that if f() panic, the program will be aborted (which is the safe thing to do). However, if you expect f() to panic and don't want to abort the program, you can instead provide a default for self via the replace_with_and_return or replace_with_or_default_and_return function.
I'm trying to select a function to call depending on a condition. I want to store that function in a variable so that I can call it again later without carrying the condition around. Here's a working minimal example:
fn foo() {
println! ("Foo");
}
fn bar() {
println! ("Bar");
}
fn main() {
let selector = 0;
let foo: &Fn() = &foo;
let bar: &Fn() = &bar;
let test = match selector {
0 => foo,
_ => bar
};
test();
}
My question is: is it possible to get rid of the intermediate variables? I've tried simply removing them:
fn foo() {
println! ("Foo");
}
fn bar() {
println! ("Bar");
}
fn main() {
let selector = 0;
let test = match selector {
0 => &foo as &Fn(),
_ => &bar as &Fn()
};
test();
}
but then the borrow checker complains that the borrowed values are only valid until the end of the match (btw, why? the functions are 'static anyway so should be valid to the end of times). I've also tried making the 'static lifetime explicit by using &foo as &'static Fn() but that doesn't work either.
The following works, if you only need to work with static functions and not closures:
fn foo() {
println!("Foo");
}
fn bar() {
println!("Bar");
}
fn main() {
let selector = 0;
let test: fn() = match selector {
0 => foo,
_ => bar
};
test();
}
(try on playground)
Here I've used function type instead of function trait.
The reason that the borrowed trait object doesn't work is probably the following. Any trait object is a fat pointer which consists of a pointer to some value and a pointer to a virtual table. When the trait object is created out of a closure, everything is clear - the value would be represented by the closure itself (internally being an instance of a structure containing all captured variables) and the virtual table would contain a pointer to the implementation of the corresponding Fn*() trait generated by the compiler whose body would be the closure body.
With functions, however, things are not so clear. There are no value to create a trait object from because the function itself should correspond to the implementation of Fn() trait. Therefore, rustc probably generates an empty structure and implements Fn() for it, and this implementation calls the static function directly (not actual Rust, but something close):
struct SomeGeneratedStructFoo;
impl Fn<()> for SomeGeneratedStructFoo {
type Output = ();
fn call(&self, args: ()) -> () {
foo();
}
}
Therefore, when a trait object is created out of fn foo(), a reference is taken in fact to a temporary value of type SomeGeneratedStructFoo. However, this value is created inside the match, and only a reference to it is returned from the match, thus this value does not live long enough, and that's what the error is about.
fn() is a function pointer type. It's already a pointer type. You can check this with std::mem::size_of::<fn()>(). It is not a zero-sized type.
When you do &foo, you take a pointer to a stack allocated pointer. This inner pointer does not survive very long, causing the error.
You can cast these to the generic fn() type as suggested. I would be interested in knowing why you can't cast fn() to &Fn(), though.