I tried to use FnMut for callback, store in struct with Box , but lifetime issues and mutable issues can't be resolved.
I'm a JavaScript programmer, used to using callback functions, the code structure may not be suitable.
struct S {
f: Box<dyn FnMut(i32)>,
}
impl S {
fn new<F: FnMut(i32) + 'static>(f: F) -> S {
S { f: Box::new(f) }
}
fn start(&mut self) {
(*self.f)(2);
(*self.f)(3);
}
}
fn main() {
let mut i: i32 = 1;
let mut s: S = S::new(|x| {
println!("{} + {} = {}", i, x, i + x);
i = i + x;
});
s.start();
println!("i = {}", i);
}
the compiler show the errors:
error[E0373]: closure may outlive the current function, but it borrows `i`, which is owned by the current function
--> src\main.rs:16:27
|
16 | let mut s: S = S::new(|x| {
| ^^^ may outlive borrowed value `i`
17 | println!("{} + {} = {}", i, x, i + x);
18 | i = i + x;
| - `i` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src\main.rs:16:20
|
16 | let mut s: S = S::new(|x| {
| ____________________^
17 | | println!("{} + {} = {}", i, x, i + x);
18 | | i = i + x;
19 | | });
| |______^
help: to force the closure to take ownership of `i` (and any other referenced variables), use the `move` keyword
|
16 | let mut s: S = S::new(move |x| {
| ++++
error[E0502]: cannot borrow `i` as immutable because it is also borrowed as mutable
--> src\main.rs:21:24
|
16 | let mut s: S = S::new(|x| {
| - --- mutable borrow occurs here
| ____________________|
| |
17 | | println!("{} + {} = {}", i, x, i + x);
18 | | i = i + x;
| | - first borrow occurs due to use of `i` in closure
19 | | });
| |______- argument requires that `i` is borrowed for `'static`
20 | s.start();
21 | println!("i = {}", i);
| ^ immutable borrow occurs here
|
= note: this error originates in the macro `$crate::format_args_nl` (in Nightly builds, run with -Z macro-backtrace for more info)
expect result
1 + 2 = 3
3 + 3 = 6
i = 6
fn new<F: FnMut(i32) + 'static>(f: F) -> S
In this context, the 'static lifetime annotation means that the actual type of F may not be declared to own any references, unless those references refer to values that have static lifetime.
However, the closure you pass captures the outer variable i by reference, which does not have static lifetime. Therefore, the generic lifetime bound 'static is not satisfied.
There's a few ways you can fix this. One would be to have S take a lifetime parameter, which will enable giving it closures that reference data with a non-static lifetime:
struct S<'a> {
f: Box<dyn FnMut(i32) + 'a>,
}
impl<'a> S<'a> {
fn new<F: FnMut(i32) + 'a>(f: F) -> S<'a> {
S { f: Box::new(f) }
}
fn start(&mut self) {
(*self.f)(2);
(*self.f)(3);
}
}
Note that you also have to drop s before you can print i because s owns a mutable borrow of i; the mutable borrow can't coexist with an immutable borrow. You can drop s using std::mem::drop() or by declaring it in a new scope that ends before you access i. For example, using the second approach:
fn main() {
let mut i: i32 = 1;
{
let mut s: S = S::new(|x| {
println!("{} + {} = {}", i, x, i + x);
i = i + x;
});
s.start();
} // s is dropped here, releasing the mutable borrow of i
println!("i = {}", i);
}
(Playground)
Where possible, this should be the preferred approach because it adds no runtime overhead.
A second approach that is less restrictive but has runtime overhead would be using Rc in combination with Cell. This approach explicitly gives you in Rust what you have implicitly in JavaScript: shared ownership of values.1 To accomplish this, the value exists outside of any of the Rc values (on the heap) which are reference-counted handles to the value.
To make this work, Cell is required to provide interior mutability, as you can't get a mutable reference out of an Rc unless only one Rc exists for the value it handles, which isn't the case here. Cell circumvents this, allowing you to set the contained value even if multiple Rcs exist.
Here's an example of how this would apply to this situation, using an Rc<Cell<i32>>. We clone the Rc once and move the clone into the closure. Now both the closure and main() have an Rc to the same value. The inner i32 value will be destroyed when the last Rc sharing ownership of it is destroyed.
(Note this uses the definition and implementation of S verbatim from your question, which works because an owned Rc doesn't count as a reference. This means the closure will satisfy the 'static constraint since it doesn't contain any references at all.)
use std::cell::Cell;
use std::rc::Rc;
// Definition and implementation of S omitted
fn main() {
let i = Rc::new(Cell::new(1i32));
let i2 = i.clone();
let mut s: S = S::new(move |x| {
println!("{} + {} = {}", i2.get(), x, i2.get() + x);
i2.set(i2.get() + x);
});
s.start();
// Dropping s isn't required in this case; both i and i2 can
// coexist without any problems.
println!("i = {}", i.get());
}
(Playground)
1 What you have in JavaScript would actually be closer to Rc<RefCell<T>> than Rc<Cell<T>>. Additionally, most JavaScript runtimes don't do reference counting, but instead some type of mark-and-sweep garbage collection. This is meant to be a comparison about ownership semantics and not a true one-to-one comparison of all possible aspects of JavaScript's object model.
Related
Here is trivial reproduction (nightly rustc as of 2-feb-23):
fn main() {
let closure = |_v| {};
// this one works fine
// let closure = |_v: &_| {};
{
let x = 1;
closure(&x);
}
{
let y = 1;
closure(&y);
}
}
The error is:
6 | let x = 1;
| - binding `x` declared here
7 | closure(&x);
| ^^ borrowed value does not live long enough
8 | }
| - `x` dropped here while still borrowed
...
11 | closure(&y);
| ------- borrow later used here
Which doesn't make sense, as variable x is not captured by the closure, but is just an argument passed by reference.
Providing an explicit reference for the closure parameter _v: &_ solves the issue, but shouldn't it be inferred automatically?
Is it some bug/limitation of the borrow checker? Or I'm missing something more fundamental here?
This looks like it could be a bug with type inference to me.
I think what is happening is
fn main() {
let closure = |_v| {};
{
let x = 1; // Lifetime `a begins
closure(&x); // closure is inferred to be fn(&'a i32)
// Lifetime 'a end
}
{
let y = 1; // Lifetime `b begins
closure(&y); // attempt to call closure fn(&'a i32) with &'b i32
// Lifetime 'b end
}
}
but defining the closure as
let closure = |_v: &'_ i32| {};
Stops the compiling from inferring its lifetime and uses hrtb as it should.
I want to write a FuncWrapper struct with a new function that takes a (Boxed) closure as parameter, and returns a decorated closure that just adds some boilerplate to the incoming one. But I also want the returned value to be "owned", to allow for the following (e.g.):
fn main() {
let a: FuncWrapper<u32>;
{
let foo = |x: u32| {print!("{}", x)};
let b = &Box::new(foo);
a = FuncWrapper::new(b);
}
let _c = (a.func)(42);
}
That is, I want for the return value of new to be an "owned" value.
Now I recently learned that all closures in Rust (being Trais) must have a lifetime associated with them (and it will default to 'static if not specified.) But I suppose its seems that any lifetime for the closure in the FuncWrapper of the return value of new will be wrong.
I don't want the lifetime to be that of the incoming reference, because this will be too short. E.g. the following will not work because it the lifetime of the return value is that of the parameter f_box_ref which does not live long enough in the case I provide above
struct FuncWrapper<'b, DomainType> {
func: Box<dyn Fn(DomainType) + 'b>
}
impl<'b, DomainType> FuncWrapper<'b, DomainType> {
fn new<F: Fn(DomainType) + 'static>(f_box_ref: &'b Box<F>) -> FuncWrapper<'b, DomainType> {
let new_f = move |a: DomainType| {
// ..add some boilerplate then
(**f_box_ref)(a)
};
let b = Box::new(new_f);
FuncWrapper {func: b}
}
}
results in
error[E0716]: temporary value dropped while borrowed
--> src/main.rs:22:18
|
22 | let b = &Box::new(foo);
| ^^^^^^^^^^^^^ creates a temporary which is freed while still in use
23 | a = FuncWrapper::new(b);
24 | }
| - temporary value is freed at the end of this statement
...
27 | }
| - borrow might be used here, when `a` is dropped and runs the destructor for type `FuncWrapper<'_, u32>`
|
= note: consider using a `let` binding to create a longer lived value
Changing the lifetime of the return parameter to 'static also seems wrong, since my goal is to create a new closure inside the new function`, (so how could it be static?) but regardless this:
struct FuncWrapper<DomainType> {
func: Box<dyn Fn(DomainType) + 'static>
}
impl<DomainType> FuncWrapper<DomainType> {
fn new<'b, F: Fn(DomainType) + 'static>(f_box_ref: &'b Box<F>) -> FuncWrapper<DomainType> {
let new_f = move |a: DomainType| {
// ..add some boilerplate then
(**f_box_ref)(a)
};
let b = Box::new(new_f);
FuncWrapper {func: b}
}
}
errors with
error[E0759]: `f_box_ref` has lifetime `'b` but it needs to satisfy a `'static` lifetime requirement
--> src/main.rs:7:21
|
6 | fn new<'b, F: Fn(DomainType) + 'static>(f_box_ref: &'b Box<F>) -> FuncWrapper<DomainType> {
| ---------- this data with lifetime `'b`...
7 | let new_f = move |a: DomainType| {
| _____________________^
8 | | // ..add some boilerplate then
9 | | (**f_box_ref)(a)
10 | | };
| |_________^ ...is used here...
11 | let b = Box::new(new_f);
12 | FuncWrapper {func: b}
| - ...and is required to live as long as `'static` here
The error surprised me since I thought the job of move was to capture by value. At first I thought that I need to dereference the f_box_ref first? But that
impl<DomainType> FuncWrapper<DomainType> {
fn new<'b, F: Fn(DomainType) + 'static>(f_box_ref: &'b Box<F>) -> FuncWrapper<DomainType> {
let f_box: Box<F> = *f_box_ref;
let new_f = move |a: DomainType| {
// ..add some boilerplate then
(*f_box)(a)
};
let b = Box::new(new_f);
FuncWrapper {func: b}
}
}
results in
error[E0507]: cannot move out of `*f_box_ref` which is behind a shared reference
--> src/main.rs:7:30
|
7 | let f_box: Box<F> = *f_box_ref;
| ^^^^^^^^^^
| |
| move occurs because `*f_box_ref` has type `Box<F>`, which does not implement the `Copy` trait
| help: consider borrowing here: `&*f_box_ref`
Alas, following the hint to "help: consider borrowing here: &*f_box_ref" doesn't help either
impl<DomainType> FuncWrapper<DomainType> {
fn new<'b, F: Fn(DomainType) + 'static>(f_box_ref: &'b Box<F>) -> FuncWrapper<DomainType> {
let f_box: &Box<F> = &*f_box_ref;
let new_f = move |a: DomainType| {
// ..add some boilerplate then
(*f_box)(a)
};
let b = Box::new(new_f);
FuncWrapper {func: b}
}
}
errors with
error[E0759]: `f_box_ref` has lifetime `'b` but it needs to satisfy a `'static` lifetime requirement
--> src/main.rs:7:31
|
6 | fn new<'b, F: Fn(DomainType) + 'static>(f_box_ref: &'b Box<F>) -> FuncWrapper<DomainType> {
| ---------- this data with lifetime `'b`...
7 | let f_box: &Box<F> = &*f_box_ref;
| ^^^^^^^^^^^ ...is used here...
...
13 | FuncWrapper {func: b}
| - ...and is required to live as long as `'static` here
so obviously we're still not returning anything like a new owned closure, and any choice for the lifetime of the returned closure seems arbitrary and wrong.
What I'm trying to do here seems like it should be common. What am I missing?
It's fairly straight forward, to get an owned type from a reference you have 3 options that come to mind immediately:
Copy, Clone or ToOwned.
ToOwned is a little more advanced since it can change the type as well and since Copy requires Cloned I'll focus on the later.
To get an owned Fn from a &Box<Fn> you can just call clone():
impl<DomainType> FuncWrapper<DomainType> {
fn new<F: Fn(DomainType) + Clone + 'static>(f_box_ref: &Box<F>) -> FuncWrapper<DomainType> {
let f_box = f_box_ref.clone();
let new_f = move |a: DomainType| {
// ..add some boilerplate then
(f_box)(a)
};
let b = Box::new(new_f);
FuncWrapper { func: b }
}
}
Since it is not FnMut, just Fn maybe you can use an Rc instead of Box? You can always clone an Rc
use std::rc::Rc;
impl<DomainType> FuncWrapper<DomainType> {
fn new<F: Fn(DomainType) + 'static>(f_rc: Rc<F>) -> FuncWrapper<DomainType> {
let new_f = move |a: DomainType| {
// ..add some boilerplate then
(f_rc)(a)
};
let b = Box::new(new_f);
FuncWrapper { func: b }
}
}
I am quite new to rust and I have a problem with the borrow checker.
I have a function like this:
impl<'a> DFA<'a> {
fn solve_next(&mut self, c: Option<char>) -> Option<TokenType> {
let node = self.state.unwrap_or_else(|| &self.start_node);
let checks: Vec<char> = node.connections.iter().map(|x| x.1).collect();
let mut position = None;
if let Some(c) = c {
position = checks.iter().position(|x| *x == c);
}
if let Some(i) = position {
let node = &node.connections[i].0;
self.state = Some(node);
None
} else {
if node.is_end {
return Some(TokenType::Keyword);
}
panic!("Invalid charter for current set");
}
}
}
This function either returns a TokenType (which is a enum) when the loop is complete or None when it is not.
its is on this struct:
struct DFA<'a> {
state: Option<&'a Node>,
start_node: Node,
}
Node is:
struct Node {
connections: Vec<(Node, char)>,
is_end: bool,
token_type: TokenType,
}
with the method
fn insert(&mut self, _match: char, end: bool) -> &mut Node {
which creates and return a new node and adds this node into its own connections.
So when I wanted to loop over the solve next function I tried:
impl<'a> Lexer<'a> {
fn next_char(&self) -> Option<char> {
let mut r: Option<char> = None;
for c in &self.chars {
match c {
' ' | '\n' | '\r' => {
continue;
}
_ => {
r = Some(c.clone());
break;
}
}
}
return r;
}
fn next_token(&'a mut self) {
let somechar = 'c';
let mut x = self.dfa.solve_next(self.next_char());
while let None = x {
x = self.dfa.solve_next(self.next_char());
}
}
}
which is a method of
struct Lexer<'a> {
//the output of the lexer
pub tokens: Vec<Token>,
chars: Vec<char>,
dfa: DFA<'a>,
}
the compile error's with
error[E0499]: cannot borrow `self.dfa` as mutable more than once at a time
--> src/main.rs:177:17
|
146 | impl<'a> Lexer<'a> {
| -- lifetime `'a` defined here
...
175 | let mut x = self.dfa.solve_next(self.next_char());
| -------------------------------------
| |
| first mutable borrow occurs here
| argument requires that `self.dfa` is borrowed for `'a`
176 | while let None = x {
177 | x = self.dfa.solve_next(self.next_char());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ second mutable borrow occurs here
error[E0502]: cannot borrow `*self` as immutable because it is also borrowed as mutable
--> src/main.rs:177:37
|
146 | impl<'a> Lexer<'a> {
| -- lifetime `'a` defined here
...
175 | let mut x = self.dfa.solve_next(self.next_char());
| -------------------------------------
| |
| mutable borrow occurs here
| argument requires that `self.dfa` is borrowed for `'a`
176 | while let None = x {
177 | x = self.dfa.solve_next(self.next_char());
| ^^^^^^^^^^^^^^^^ immutable borrow occurs here
So is their any way I can use solve_next in this loop? as I would not know how to make a function that can be used like this without having it take a mutable refrence.
The actual error you get with your code is the following (playground):
error: lifetime may not live long enough
--> src/lib.rs:27:49
|
25 | impl<'a> DFA<'a> {
| -- lifetime `'a` defined here
26 | fn solve_next(&mut self, c: Option<char>) -> Option<TokenType> {
| - let's call the lifetime of this reference `'1`
27 | let node = self.state.unwrap_or_else(|| &self.start_node);
| ^^^^^^^^^^^^^^^^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`
This is actually complaining about the reference to self.start_node not living long enough, that is for at least 'a.
The reason for that is that your local variable node, that is of type &'x Node, being the lifetime 'x the shorter of that of self.state.unwrap() and &self.start_node. The first one is 'a and the second one is the unnamed lifetime of &self (from now on 's).
But by the rules of Rust lifetimes, 'a outlives DFA<'a> and Self outlives 's, and Self is equal to DFA<'a>, then we conclude that 'a outlives 's. So the lifetime of the local node is actually 's.
Now, the other key line is this one:
self.state = Some(node);
being self.state of type Option<&'a Node>, that requires that the lifetime of node, that is 's, outlives 'a. But that cannot be, we already determined that it is the other way around: 'a outlives 's. Thus a compiler error.
In a nutshell, you are trying to write a self-referential struct, storing in one field a reference to the other field. And that is well known to be impossible.
In the original version of your question, you tried to solve it adding an extra lifetime to the solve_next() function:
fn solve_next(&'a mut self, c: Option<char>) -> Option<TokenType>
Doing that you force the 's from my explanation above to be exactly equal to 'a, so the body of this function is actually correct. Unfortunately if you try to call it normally with:
fn next_token(&mut self) {
let mut x = self.dfa.solve_next(self.next_char());
}
It fails with:
error: lifetime may not live long enough
--> src/lib.rs:63:21
|
46 | impl<'a> Lexer<'a> {
| -- lifetime `'a` defined here
...
62 | fn next_token(&mut self) {
| - let's call the lifetime of this reference `'1`
63 | let mut x = self.dfa.solve_next(self.next_char());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`
Exactly for the same reason as above: the anonymous lifetime of self must outlive 'a but the compiler deduces it the other way around.
Naturally you can fix it the same way:
fn next_token(&'a mut self) {
And that will compile... as long as you do not try to call solve_next() twice.
I'm not sure why calling solve_next() once it compiles, but calling it twice fails with. But it does not actually matter because even if this function worked, with this extra &'a mut self you'd still have the same problem calling this function from the external code.
What about the solution? Well, I think you need to refactor the code so that you never try to add a reference start_node in the same struct.
For example, you can store the start_node outside this struct:
struct DFA<'a> {
state: Option<&'a Node>,
start_node: &'a Node,
}
This, and remove all the lifetime annotations from &'a self and it will just compile (playground).
I have a snippet of code which fails to compile:
struct A {
x: [u32; 10],
}
impl A {
fn iter<'a>(&'a self) -> impl Iterator<Item = u32> + 'a {
(0..10).map(|i| self.x[i])
}
}
fn main() {}
(playground)
The compiler says:
error[E0373]: closure may outlive the current function, but it borrows `self`, which is owned by the current function
--> src/main.rs:7:21
|
7 | (0..10).map(|i| self.x[i])
| ^^^ ---- `self` is borrowed here
| |
| may outlive borrowed value `self`
help: to force the closure to take ownership of `self` (and any other referenced variables), use the `move` keyword
|
7 | (0..10).map(move |i| self.x[i])
| ^^^^^^^^
What should I do to make this work? I need self later so I cannot move it as the compiler suggests.
Edit: I believed that move will create issues in using self later. For example, see the code:
struct A {
x: [u32; 3],
}
impl A {
fn iter<'a>(&'a self) -> impl Iterator<Item=u32> + 'a {
(0..3).filter(move |&i| self.x[i] != 0).map(move |i| self.x[i])
}
}
fn main() {
let a = A { x : [0, 1, 2]};
for el in a.iter() {
println!("{}", el);
}
}
(playground)
Here &'a self is being moved twice, so in effect, both the closures got ownership of &'a self. The code actually compiles but I did not expect it as once moved, a variable should not be usable anymore. The book (relevant section) also gives an example corroborating my understanding:
fn main() {
let x = vec![1, 2, 3];
let equal_to_x = move |z| z == x;
println!("can't use x here: {:?}", x);
let y = vec![1, 2, 3];
assert!(equal_to_x(y));
}
This code does not compile. Why is it that my iterator code works with move?
It looks like the compiler is already telling you what to do:
help: to force the closure to take ownership of `self` (and any other referenced variables), use the `move` keyword
|
7 | (0..10).map(move |i| self.x[i])
| ^^^^^^^^
The move keyword gives ownership of any captured variable to the closure.
Here, you are capturing the self reference by move, not the object itself. You cannot consume the object by calling the iter method, because it doesn't accept your object by move, but by reference. You can still use your object after calling iter() on an instance.
use std::rc::Rc;
fn f1(cb: Box<Fn(i32) -> i32>) {
let res = cb(15);
println!("res {}", res);
}
fn main() {
let mut v2 = Rc::new(5_i32);
// 1
// f1(Box::new(move |x: i32| *v2 + x));
// 2
f1(Box::new(move |x: i32| {
let tmp = *v2;
*Rc::get_mut(&mut v2).unwrap() = tmp + 1;
x + *v2
}));
}
The code referenced as "1", if uncommented, compiles and runs just fine, but the code referenced as "2" does not compile, failing with the message:
error[E0596]: cannot borrow `v2` as mutable, as it is a captured variable in a `Fn` closure
How can I fix this, if I want keep the code structure as it is?
In my real code, I want connect two traits. One of them will call a callback on event, and the other has a function to handle the callback:
trait Foo {
fn add_callback(&mut self, cb: Box<Fn(i32)>);
}
trait Boo {
fn on_new_data(&mut self, data: i32);
}
I want to create a trait object with Boo, wrap it with Rc, and pass it to Foo::add_callback in the form of |x:i32| Rc::get_mut(&mut boo).unwrap().on_new_data(x)
The entire error message is mostly helpful:
error[E0596]: cannot borrow `v2` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:17:19
|
17 | *Rc::get_mut(&mut v2).unwrap() = tmp + 1;
| ^^^^^^^ cannot borrow as mutable
|
help: consider changing this to accept closures that implement `FnMut`
--> src/main.rs:15:17
|
15 | f1(Box::new(move |x: i32| {
| _________________^
16 | | let tmp = *v2;
17 | | *Rc::get_mut(&mut v2).unwrap() = tmp + 1;
18 | | x + *v2
19 | | }));
| |_____^
Changing f1 to accept a FnMut and making the variable mutable allows the code to compile:
fn f1(mut cb: Box<FnMut(i32) -> i32>) {
This is needed in order to mutate the captured variable v2, required by the &mut v2 argument to Rc::get_mut.