I just found out that the following code compiles in Rust 21 (used to not compile in 18)
fn get_func (i: &mut i32) -> impl Fn() -> i32 + '_ {
|| *i
}
Is there an implicit move of i involved ? If so, then why does the following code compile too?
fn get_func (i: &mut i32) -> impl Fn() -> i32 + '_ {
let f = || *i;
println!("{:?}", i); // was expecting here to give borrow of moved variable error, `&mut` doesnt implement `Copy` trait
f
}
Or instead is it implicitly moving (copying in this case) the value being pointed to? But then the following code should compile, which doesn't -- indicating it's moving the reference.
fn get_func (i: &mut i32) -> impl Fn() -> i32 {
|| *i
}
It's downgrading your &mut to a & and making as many copies as it needs to.
This is all a byproduct of the fact that &mut has all the permissions of &; in other words it's fine to downgrade from a mutable reference to a non-mutable reference, and it's fine to copy a non-mutable reference as many times as you want, so this is just doing both of those implicitly.
fn get_func (i: &mut i32) -> impl Fn() -> i32 + '_ {
*i += 1; // fine, still mutable
let f = || *i;
println!("{}", *i); // fine, no mutability required
*i += 1; // ERROR: cannot mutate *i while borrowed by f
f
}
It's worth noting that if you actually do try to capture it as mutable, this happens.
fn get_func (i: &mut i32) -> impl FnMut() -> i32 + '_ {
println!("{}", *i); // Fine
*i += 1; // Fine
let f = || {*i += 1; *i};
println!("{}", *i); // ERROR: cannot borrow *i as immutable
f
}
Also the lifetime doesn't refer to the return of the closure, but the closure itself. Because no memory is moved, the closure is only valid so long as the memory at i's address remains valid. That's why removing the lifetime makes the code not compile.
In any case, all you really have to remember is the classic: "one mutable reference OR multiple immutable references, not both".
Related
Is it possible to specify a generic lifetime on FnMut in Rust for a function like this?
fn f1<'a, FRet:'a + FnMut() -> ()>(v: &'a mut i32) -> FRet {
let mut fret = || {
let i = v;
println!("inside");
// lots of complicated logic on captured var
*i = 5;
};
return fret;
}
Right now I'm getting an error, that return is incompatible with FRet.
Your code has several issues. One is that you cannot make the function that returns a closure have a generic return type. f1 being generic over FRet means that the caller gets to choose the FRet, and that obviously can't work when you're returning a closure you implement, and not something the caller to specify. This is why the compiler complains that FRet is incompatible with what you're actually returning.
The way to return a closure is by either boxing it or using the impl Trait return type that declares an unnamed type returned by the function:
fn f1(v: &mut i32) -> impl FnMut() {
let fret = || {
let i = v;
println!("inside");
// lots of complicated logic on captured var
*i = 5;
};
fret
}
The above still doesn't compile with the compiler complaining that the closure is FnOnce rather than FnMut. This is because &mut i32 is not Copy, so let i = v actually moves the reference out of v which makes the closure callable only once. That can be fixed with a reborrow, i.e. changing let i = v to let i = &mut *v.
After that the compiler will complain that the lifetime of reference is not captured by the closure, and that you can fix that by adding + '_. (This is equivalent to declaring a lifetime 'a and declaring v: &'a mut i32 and impl FnMut() + 'a.) With this change the code compiles:
fn f1(v: &mut i32) -> impl FnMut() + '_ {
let fret = || {
let i = &mut *v;
println!("inside");
// lots of complicated logic on captured var
*i = 5;
};
fret
}
Playground
There is a code that produces the following error:
fn foo<'a, F1, F2>(f: F1)
where
F1: FnOnce(&'a mut i32) -> F2,
F2: FnOnce() + 'a,
{
let mut a = 0;
f(&mut a)();
}
fn main() {
foo(|a| {
|| {
*a += 1;
()
}
});
}
error[E0597]: `a` does not live long enough
--> src/main.rs:7:7
|
1 | fn foo<'a, F1, F2>(f: F1)
| -- lifetime `'a` defined here
...
7 | f(&mut a)();
| --^^^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `a` is borrowed for `'a`
8 | }
| - `a` dropped here while still borrowed
I'm very confused because error message says that a is still borrowed but foo returns nothing!
Your code has two issues:
a Lifetime issue
it tries to return a closure
1. The lifetime issue
You can reduce the example in your question to
fn foo<'a, F1>(f: F1)
where
F1: FnOnce(&'a mut i32),
{
let mut a = 0;
f(&mut a);
}
fn main() {
foo(|a| {
*a += 1;
});
}
and here you can probably see more clearly, that
fn foo<'a, F1>(f: F1)
where
F1: FnOnce(&'a mut i32)
means, that the caller of foo can decide, what 'a
will be.
But in
let mut a = 0;
f(&mut a);
you create a, which has its concrete lifetime (essentially
the body of foo) and try to call f with it. But 'a which
the caller has chosen, doesn't need to have to do anything
with the lifetime of a, it can be wildly larger for example.
Worded differently
fn foo<'a, F1>(f: F1)
where
F1: FnOnce(&'a mut i32)
means: If you call foo you can hand it something function
like f (function pointer, closure, ...), which needs its parameter
to live at least as long as 'a. 'a could be 'static
for example.
So inside foo you can't just create a thing with its concrete
lifetime and hand if to f, as you don't know, what 'a the
caller of foo will choose.
You might now be surprised, that
fn foo<F1>(f: F1)
where
F1: FnOnce(&mut i32),
{
let mut a = 0;
f(&mut a);
}
fn main() {
foo(|a| {
*a += 1;
});
}
compiles just fine, but that is simply, because foo in this example desugars to
fn foo<F1>(f: F1)
where
F1: for<'a> FnOnce(&'a mut i32),
{
let mut a = 0;
f(&mut a);
}
which means, that not the caller of foo decides, what the lifetime 'a will be, but f has to be something, that can work with a parameter of any lifetime. So foo can choose 'a in this case.
But to be able to return anything, which has the lifetime 'a chosen by foo, you would need to be able to write it into foos signature, which you can't, to the best of my knowledge.
2. it tries to return a closure
The Rust Book, "Returning Closures" tells us, you can't
return closures.
But
fn returns_closure(a: &mut i32) -> impl FnMut() + '_ {
|| {
*a += 1;
println!("{}", *a);
}
}
fn main() {
let mut sum: i32 = 5;
let mut f = returns_closure(&mut sum);
f();
f();
let mut sum2: i32 = 10;
let mut g = returns_closure(&mut sum2);
g();
g();
}
just works fine for me on rustc version 1.58.1.
The Rust Book, "Returning Types that Implement Traits" tells us that you can use the
impl SomeTrait syntax only in return types of functions and methods, which
always return the same concrete type on all function exits.
So the above syntax doesn't apply to closures.
But with generic types we can still do something similar.
fn foo<F1, F2>(f: &mut F1)
where
F1: FnMut(i32) -> F2,
F2: FnMut(),
{
let mut g = f(1);
g();
g();
let mut h = f(2);
h();
h();
}
fn main() {
foo(&mut (|_a: i32| || println!("wtf")));
foo(&mut (|_a: i32| {
let mut a = 5;
move || {
a += 1;
println!("wtf: {}", a);
}
}));
}
still works fine for me. So it seems as long as the compiler can
figure out the concrete types and the closure returning a closures
also always returns the same concrete type, it works for
rustc 1.58.1.
So my guess is, the syntax simply lacks. There needs to be
some syntax to write something like the following pseudo code:
fn foo<F1, F2>(f: &mut F1)
where
F2: FnMut(),
F1: for<'a> FnMut(&'a mut i32) -> F2 + 'a,
{
...
}
I guess the lifetime bound is part of the type at the moment and so
the caller, which chooses F2 would need to know the required lifetime,
that gets choosen in the call ... but thats just guessing.
As the required lifetime bound shouldn't influence the size
needed for the closure returned by f,
something like that might be possible?
In
fn foo2<F1>(f: &mut F1)
where
F1: for<'a> FnMut(&'a mut i32) -> Box<dyn FnMut() + 'a>,
{
...
}
we can neatly express what we want at the unfortunate cost of allocating ...
But now f can of course return different closures with different types
wrapped up in the Box, so it is also more flexible.
The following error goes away if I do as rustc tells me to, and change the bound to
where F: Fn() -> () + 'static
pub struct Struct(Box<dyn Fn() -> ()>);
pub fn example<F>(f: F)
where
F: Fn() -> ()
{
Struct(Box::new(|| ())); // ok
Struct(Box::new(f)); // error: The parameter type `F` may not live long eneough
// help: consider adding an explicit lifetime bound: `F: 'static`
}
However, I just don't understand what 'static means here. It doesn't seem to mean that the closure itself lives forever.
At least, it doesn't seem like the closure lives forever. If I have the closure own some data, that data gets dropped at some point, which makes me suspect the closure is dropped:
pub struct Struct(Box<dyn Fn() -> ()>);
#[derive(Debug)]
struct DropMe;
impl Drop for DropMe {
fn drop(&mut self) {
println!("dropped");
}
}
/// prints:
/// "got DropMe"
/// "got DropMe"
/// "dropped"
/// "end of program"
pub fn main() {
let d = DropMe;
example(move || {
println!("got {:?}", d);
});
println!("end of program")
}
pub fn example<F>(f: F)
where
F: Fn() -> () + 'static
{
let s = Struct(Box::new(f));
s.0();
s.0();
}
Is 'static an upper bound on the lifetime of the closure rather than a lower bound? 'static makes sense as an upper bound, then example would be saying "give me a function that I can hold on to as long as I want", which in practice can be less than the 'static lifetime.
How can I tell when + 'lifetime adds 'lifetime as an upper bound vs. a lower bound?
The Rustonomicon chapter on subtyping+variance doesn't seem to cover this, and I couldn't find the information I was looking for in the Rust Book or Rust Reference.
T: 'a isn't a constraint on the lifetime of instances of T, but of things that T borrows (if any): that is, all borrows in T must outlive 'a.
Thus F: Trait + 'static requires that any borrows in F be for 'static lifetime (or longer, which doesn't exist), regardless that Trait in this case is Fn() -> ().
In your case, the closure takes ownership of d (and borrows the &'static str literal); hence it satisfies F: 'static. But if instead of move || ... the closure merely borrowed d with || ..., then it would not be able to satisfy 'static (as the lifetime of the borrow of d cannot exceed the scope of the call to main).
Yes, 'static is an upper bound.
In fact, all lifetimes constraints are upper bounds. For the callee (that is, whom uses the lifetime).
For the caller (i.e. the provider of the lifetime), on the other hand, they're usually lower bounds: give me something that lives at least as 'static (of course, nothing lives more than 'static, so it actually means "give me something 'static". But it does matter when talking about other lifetimes).
Variance is about changing the caller's respect regarding the lifetime: whether it can pass a longer lifetime (covariance, i.e. a lower bound), a shorter lifetime (contravariance, i.e. an upper bound), or only exactly this lifetime (invariance).
The 'static variant is not obligatory. It seems like it's just what compiler proposes by default.
The two edits below should hopefully illustrate the capture of lifetime of variables from environment.
//rustc 1.62.1 (e092d0b6b)
pub struct Struct<'a>(Box<dyn Fn() -> () + 'a>);
#[derive(Debug)]
struct DropMe;
impl Drop for DropMe {
fn drop(&mut self) {
println!("dropped");
}
}
pub fn main() {
let d = DropMe;
let x = 43;
example(move || {
println!("got {:?} {:?}", d, &x);
});
println!("end of program")
}
pub fn example<'env_borrow, F>(f: F)
where
F: Fn() -> () + 'env_borrow
{
let s = Struct(Box::new(f));
s.0();
s.0();
// drop(s);
println!("end of higher bound func");
}
// rustc 1.62.1 (e092d0b6b)
pub struct Struct<'a>(Box<dyn FnMut() -> () + 'a>);
#[derive(Debug)]
struct DropMe;
impl Drop for DropMe {
fn drop(&mut self) {
println!("dropped");
}
}
pub fn main() {
let d = DropMe;
let mut x = 43;
let y = &mut x;
let closure_boxed = example(move || {
println!("got {:?} , *&mut x = {:?}", d, y);
*y += 1;
});
// drop(x);
// ^ ERROR: try uncomment
drop(closure_boxed);
// ^ ERROR: try comment
println!("x is now {}", x);
println!("end of program")
}
pub fn example<'env_borrow, F>(f: F) -> Struct<'env_borrow>
where
F: FnMut() -> () + 'env_borrow
{
let mut s = Struct(Box::new(f));
s.0();
s.0();
s
}
I believe the issue is that Struct has no generic lifetime parameters, which means the lifetime of one of its instances has no name. The function an instance of Struct boxes (obviously) has to live as long as the instance, but the only way to guarantee that to the compiler is to give it a named lifetime. But because Struct has no lifetime parameters, the only named lifetime that is guaranteed to last sufficient long is 'static.
To fix this, you can give Struct a generic lifetime parameter and then constrain the boxed function to also only live that long.
pub struct Struct<'a>(Box<dyn 'a + Fn() -> ()>);
pub fn example<F>(f: F)
where
F: Fn() -> (),
{
Struct(Box::new(|| ())); // ok
Struct(Box::new(f)); // also ok; 'a will be inferred from the lifetime of f
}
I'm having trouble with lifetimes on an impl trait. I'm trying to get the following code to work:
struct Foo<'op, Input> {
op: Box<dyn Fn(Input) -> i32 + 'op>,
}
impl<'op, Input> Foo<'op, Input> {
fn new<Op>(op: Op) -> Foo<'op, Input>
where
Op: Fn(Input) -> i32 + 'op,
{
Foo { op: Box::new(op) }
}
fn apply<'input_iter, InputIter>(
self,
input_iter: InputIter,
) -> impl Iterator<Item = i32> + 'op + 'input_iter
where
InputIter: IntoIterator<Item = Input> + 'input_iter,
{
input_iter.into_iter().map(move |input| (self.op)(input))
}
}
(Playground)
This is giving me the following error:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/lib.rs:20:36
|
20 | input_iter.into_iter().map(move |input| (self.op)(input))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime 'op as defined on the impl at 5:6...
--> src/lib.rs:5:6
|
5 | impl<'op, Input> Foo<'op, Input> {
| ^^^
= note: ...so that the types are compatible:
expected Foo<'_, _>
found Foo<'op, _>
note: but, the lifetime must be valid for the lifetime 'input_iter as defined on the method body at 13:14...
--> src/lib.rs:13:14
|
13 | fn apply<'input_iter, InputIter>(
| ^^^^^^^^^^^
note: ...so that return value is valid for the call
--> src/lib.rs:16:10
|
16 | ) -> impl Iterator<Item = i32> + 'op + 'input_iter
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Here's my understanding of the lifetimes involved. Foo owns an op, which is a closure that may have a reference in it somewhere, so it may have a bound lifetime. This is denoted by 'op, and Foo is constrained such that it can't outlive it. So far, so good.
In apply(), the idea is that we want to consume an input_iter and self and return an iterator of each element in input_iter mapped using self.op. input_iterator may also contain references, so it may have its own lifetime bounds, denoted by 'input_iter.
What I want is to return an iterator that takes ownership of both self and input_iter. In doing so, it would have to take on both of their lifetime parameters to ensure that it didn't outlast either the input_iter references or the op references. I thought impl Iterator<Item = i32> + 'op + 'input_iter would accomplish this, but I seem to have taken a wrong turn somewhere.
It's also weird that it's complaining about the closure. I understand that the closure can't outlive 'op, because it takes ownership of the operator and its references. That makes perfect sense. What I don't understand is why it needs to live as long as 'input_iter. The closure and the input iterator shouldn't care about each other at all; the only thing connecting them is that they both have the same owner (the output iterator).
What am I missing here?
Lifetime parameters don't always represent the exact lifetime of an object (or of a borrow). Let's consider this example:
fn main() {
let a = 3;
let b = 5;
let c = min(&a, &b);
println!("{:?}", c);
}
fn min<'a>(a: &'a i32, b: &'a i32) -> &'a i32 {
if a < b {
a
} else {
b
}
}
Here, min takes two reference arguments with the same lifetime. However, we call it with borrows of different variables, which have different lifetimes. So why does this code compile?
The answer is subtyping and variance. Larger lifetimes are subtypes of shorter lifetimes; conversely, shorter lifetimes are supertypes of larger lifetimes. In the example above, the compiler has to find one lifetime that is compatible with both input lifetimes. The compiler does this by finding the common supertype of both lifetimes (i.e. the shortest lifetime that contains both). This is called unification.
Back to your problem. It seems like the compiler doesn't handle multiple lifetime bounds on impl Trait too well at the moment. However, there's no real need to have two lifetime bounds: one is enough. The compiler will shorten this single lifetime if necessary to satisfy the requirements of both self and input_iter.
struct Foo<'op, Input> {
op: Box<dyn Fn(Input) -> i32 + 'op>,
}
impl<'op, Input> Foo<'op, Input> {
fn new<Op>(op: Op) -> Foo<'op, Input>
where
Op: Fn(Input) -> i32 + 'op,
{
Foo { op: Box::new(op) }
}
fn apply<InputIter>(
self,
input_iter: InputIter,
) -> impl Iterator<Item = i32> + 'op
where
Input: 'op,
InputIter: IntoIterator<Item = Input> + 'op,
{
input_iter.into_iter().map(move |input| (self.op)(input))
}
}
fn main() {
let y = 1;
let foo = Foo::new(|x| x as i32 + y);
let s = "abc".to_string();
let sr: &str = &*s;
let bar = sr.chars();
let baz: Vec<_> = foo.apply(bar).collect();
println!("{:?}", baz);
}
Try moving the lines in main around to convince yourself that the lifetimes are indeed unified.
At this point, calling the lifetime 'op is somewhat misleading; it might as well be called 'a.
My goal is to make a function (specifically, floodfill) that works independent of the underlying data structure. I tried to do this by passing in two closures: one for querying, that borrows some data immutably, and another for mutating, that borrows the same data mutably.
Example (tested on the Rust Playground):
#![feature(nll)]
fn foo<F, G>(n: i32, closure: &F, mut_closure: &mut G)
where
F: Fn(i32) -> bool,
G: FnMut(i32) -> (),
{
if closure(n) {
mut_closure(n);
}
}
fn main() {
let mut data = 0;
let closure = |n| data == n;
let mut mut_closure = |n| {
data += n;
};
foo(0, &closure, &mut mut_closure);
}
Error: (Debug, Nightly)
error[E0502]: cannot borrow `data` as mutable because it is also borrowed as immutable
--> src/main.rs:16:27
|
15 | let closure = |n| data == n;
| --- ---- previous borrow occurs due to use of `data` in closure
| |
| immutable borrow occurs here
16 | let mut mut_closure = |n| {
| ^^^ mutable borrow occurs here
17 | data += n;
| ---- borrow occurs due to use of `data` in closure
18 | };
19 | foo(0, &closure, &mut mut_closure);
| -------- borrow later used here
I did come up with a solution, but it is very ugly. It works if I combine the closures into one and specify which behavior I want with a parameter:
// #![feature(nll)] not required for this solution
fn foo<F>(n: i32, closure: &mut F)
where
F: FnMut(i32, bool) -> Option<bool>,
{
if closure(n, false).unwrap() {
closure(n, true);
}
}
fn main() {
let mut data = 0;
let mut closure = |n, mutate| {
if mutate {
data += n;
None
} else {
Some(data == n)
}
};
foo(0, &mut closure);
}
Is there any way I can appease the borrow checker without this weird way of combining closures?
The problem is rooted in the fact that there's information that you know that the compiler doesn't.
As mentioned in the comments, you cannot mutate a value while there is a immutable reference to it — otherwise it wouldn't be immutable! It happens that your function needs to access the data immutably once and then mutably, but the compiler doesn't know that from the signature of the function. All it can tell is that the function can call the closures in any order and any number of times, which would include using the immutable data after it's been mutated.
I'm guessing that your original code indeed does that — it probably loops and accesses the "immutable" data after mutating it.
Compile-time borrow checking
One solution is to stop capturing the data in the closure. Instead, "promote" the data to an argument of the function and the closures:
fn foo<T, F, G>(n: i32, data: &mut T, closure: F, mut mut_closure: G)
where
F: Fn(&mut T, i32) -> bool,
G: FnMut(&mut T, i32),
{
if closure(data, n) {
mut_closure(data, n);
}
}
fn main() {
let mut data = 0;
foo(
0,
&mut data,
|data, n| *data == n,
|data, n| *data += n,
);
}
However, I believe that two inter-related closures like that will lead to poor maintainability. Instead, give a name to the concept and make a trait:
trait FillTarget {
fn test(&self, _: i32) -> bool;
fn do_it(&mut self, _: i32);
}
fn foo<F>(n: i32, mut target: F)
where
F: FillTarget,
{
if target.test(n) {
target.do_it(n);
}
}
struct Simple(i32);
impl FillTarget for Simple {
fn test(&self, n: i32) -> bool {
self.0 == n
}
fn do_it(&mut self, n: i32) {
self.0 += n
}
}
fn main() {
let data = Simple(0);
foo(0, data);
}
Run-time borrow checking
Because your code has a temporal need for the mutability (you only need it either mutable or immutable at a given time), you could also switch from compile-time borrow checking to run-time borrow checking. As mentioned in the comments, tools like Cell, RefCell, and Mutex can be used for this:
use std::cell::Cell;
fn main() {
let data = Cell::new(0);
foo(
0,
|n| data.get() == n,
|n| data.set(data.get() + n),
);
}
See also:
When I can use either Cell or RefCell, which should I choose?
Situations where Cell or RefCell is the best choice
Need holistic explanation about Rust's cell and reference counted types
Unsafe programmer-brain-time borrow checking
RefCell and Mutex have a (small) amount of runtime overhead. If you've profiled and determined that to be a bottleneck, you can use unsafe code. The usual unsafe caveats apply: it's now up to you, the fallible programmer, to ensure your code doesn't perform any undefined behavior. This means you have to know what is and is not undefined behavior!
use std::cell::UnsafeCell;
fn main() {
let data = UnsafeCell::new(0);
foo(
0,
|n| unsafe { *data.get() == n },
|n| unsafe { *data.get() += n },
);
}
I, another fallible programmer, believe this code to be safe because there will never be temporal mutable aliasing of data. However, that requires knowledge of what foo does. If it called one closure at the same time as the other, this would most likely become undefined behavior.
Additional comments
There's no reason to take references to your generic closure types for the closures
There's no reason to use -> () on the closure type, you can just omit it.