Can somebody help me to rewrite this piece of code with new unboxed closures:
struct Builder;
pub fn build(rules: |params: &mut Builder|) -> Builder {
let mut builder = Builder::new();
rules(&mut builder);
builder
}
I tried to write like this, but I got a lifetime error:
pub fn build<F>(rules: F) -> Builder where F: FnOnce<(&mut Builder,), ()> {
let mut builder = Builder::new();
rules(&mut builder);
builder
}
valico/src/builder.rs:48:59: 48:71 error: missing lifetime specifier [E0106]
valico/src/builder.rs:48 pub fn build<F>(rules: F) -> Builder where F: FnOnce<(&mut Builder,), ()> {
^~~~~~~~~~~~
What lifetime I need to specify? Simplified example in the sandbox.
This requires higher rank trait bounds, specifically, higher rank lifetimes. The full unsugared syntax would be F: for<'a> FnOnce<(&'a mut Builder,), ()>.
Using a lifetime on the function can't work, e.g. if we had
pub fn build<'b, F>(rules: F) -> Builder where F: FnOnce<(&'b mut Builder,), ()>
This says that build works with whatever lifetime the caller wishes (e.g. they could chose 'b == 'static), but this is invalid, because there is a specific concrete lifetime that needs to be the used: the lifetime of the &mut builder inside the function. Using F: for<'a> ... in a bound says that F works with any lifetime 'a, so the compiler sees that it is legal to substitute in the one of &mut builder.
As I hinted above, that's the really ugly unsugared syntax. There's two successive ways this can be made much nicer. Firstly, the canonical way to use the closure traits is the () sugar: for<'a> FnOnce(&'a mut Builder) -> (), or, like with the rest of Rust, the -> () can be dropped: for<'a> FnOnce(&'a mut Builder). (NB. this is just sugar for FnOnce<...>, but only the sugared syntax will be stabilised for interacting with these traits at 1.0.)
Then, the paren syntax has a little extra rule: it automatically inserts lifetimes that act like for<'a> (specifically, it undergoes lifetime elision with any inserted lifetime placed into a for on the trait), so just F: FnOnce(&mut Builder) is equivalent to F: for<'a> FnOnce(&'a mut Builder), and it's the recommended version.
Applying these fixes to your playpen example:
pub fn initialize_with_closure<F>(rules: F) -> uint where F: FnOnce(&mut uint) {
let mut i = 0;
rules(&mut i);
i
}
// equivalently
pub fn initialize_with_closure_explicit<F>(rules: F) -> uint where F: for<'a> FnOnce(&'a mut uint) -> () {
let mut i = 0;
rules(&mut i);
i
}
pub fn main() {
initialize_with_closure(|i: &mut uint| *i = *i + 20);
initialize_with_closure_explicit(|i: &mut uint| *i = *i + 20);
}
playpen
Related
Why does this compile:
fn func<T>(
callback: impl FnOnce(&mut i64) -> T,
) -> T {
let v = 42;
callback(&mut 42)
}
but this not?:
fn func<'a, T>(
callback: impl FnOnce(&'a mut i64) -> T,
) -> T {
let v = 42;
callback(&mut 42)
}
Even this doesn't compile:
fn func<'a, T: 'static>(
callback: impl FnOnce(&'a mut i64) -> T,
) -> T {
let v = 42;
callback(&mut 42)
}
Is there a way to tell Rust that the T returned from the callback doesn't hold any references to 'a? I thought : 'static would forbid references in general but it doesn't seem to work. Unfortunately, I do need to give 'a a name because I'm using the lifetime elsewhere, the actual code is somewhat more complicated than this minimal example.
In your first snippet, the lifetimes are elided via Higher-Ranked Trait Bounds:
fn func<T>(
callback: impl for<'a> FnOnce(&'a mut i64) -> T,
) -> T {
let v = 42;
callback(&mut 42)
}
That means the closure can be called with any lifetime, and that includes the lifetime of v.
In your second snippet, the lifetime is chosen by the caller. This is not related to T: for example, the caller can choose 'static, then store the parameter in a static. There is just no way to satisfy this requirement with a variable defined in the function.
I have a function that expects a short lived object. I would expect that I would be able to always pass it a long lived object. But I am getting a strange error when I try to encode that:
type F<'arg> = Box<dyn FnOnce(&'arg ())>;
fn contravar<'small, 'large: 'small>(f: F<'small>) -> F<'large> {
f
}
playground
Particularly:
error: lifetime may not live long enough
--> src/lib.rs:3:5
|
2 | fn contravar<'small, 'large: 'small>(f: F<'small>) -> F<'large> {
| ------ ------ lifetime `'large` defined here
| |
| lifetime `'small` defined here
3 | f
| ^ function was supposed to return data with lifetime `'large` but it is returning data with lifetime `'small`
|
= help: consider adding the following bound: `'small: 'large`
It seems like F is invariant for its argument but I would have guessed that it's contravariant. Am I missing something? Is there a way to make F<'arg> really contravariant for 'arg?
Edit: it looks like the "problem" is that rust wants to treat all generic traits the same (including Fn/FnMut/FnOnce). My opinion is that those 3 are and should be treated special especially given that they are the only way to refer to closures. For that reason I opened an issue
The Rust Reference's page on Subtyping and Variance documents that, as of Rust 1.63.0, fn(T) -> () is contravariant over T and that dyn Trait<T> + 'a is invariant over T.
FnOnce, FnMut and Fn are traits, so that means dyn FnOnce(&'a ()) is unfortunately invariant over &'a ().
// Compiles
pub fn contravariant<'a, 'b: 'a>(x: fn(&'a ())) -> fn(&'b ()) { x }
// Doesn't compile
pub fn contravariant2<'a, 'b: 'a>(x: Box<dyn FnOnce(&'a ())>) -> Box<dyn FnOnce(&'b ())> { x }
Is there a way to wrap FnOnce somehow to convince the compiler of the correct variance?
Here's what I could come up with using unsafe code. Note that I'm not making any guarantees as to whether this is sound or not. I don't know of any way to do this without unsafe code.
use std::marker::PhantomData;
trait Erased {}
impl<T> Erased for T {}
pub struct VariantBoxedFnOnce<Arg, Output> {
boxed_real_fn: Box<dyn Erased + 'static>,
_phantom_fn: PhantomData<fn(Arg) -> Output>,
}
impl<Arg, Output> VariantBoxedFnOnce<Arg, Output> {
pub fn new(real_fn: Box<dyn FnOnce(Arg) -> Output>) -> Self {
let boxed_real_fn: Box<dyn Erased + '_> = Box::new(real_fn);
let boxed_real_fn: Box<dyn Erased + 'static> = unsafe {
// Step through *const T because *mut T is invariant over T
Box::from_raw(Box::into_raw(boxed_real_fn) as *const (dyn Erased + '_) as *mut (dyn Erased + 'static))
};
Self {
boxed_real_fn,
_phantom_fn: PhantomData,
}
}
pub fn call_once(self, arg: Arg) -> Output {
let boxed_real_fn: Box<Box<dyn FnOnce(Arg) -> Output>> = unsafe {
// Based on Box<dyn Any>::downcast()
Box::from_raw(Box::into_raw(self.boxed_real_fn) as *mut Box<dyn FnOnce(Arg) -> Output>)
};
boxed_real_fn(arg)
}
}
pub fn contravariant<'a, 'b: 'a>(x: VariantBoxedFnOnce<&'a (), ()>) -> VariantBoxedFnOnce<&'b (), ()> { x }
#[cfg(test)]
mod tests {
use super::*;
fn foo(_x: &()) {}
#[test]
pub fn check_fn_does_not_require_static() {
let f = VariantBoxedFnOnce::new(Box::new(foo));
let x = ();
f.call_once(&x);
}
#[test]
pub fn check_fn_arg_is_contravariant() {
let f = VariantBoxedFnOnce::new(Box::new(foo));
let g = contravariant(f);
let x = ();
g.call_once(&x);
}
}
Here, VariantBoxedFnOnce is limited to functions taking one argument.
The trick is to store a type-erased version of the Box<dyn FnOnce(Arg) -> Output> such that Arg disappears, because we don't want the variance of VariantBoxedFnOnce<Arg, Output> to depend on Box<dyn FnOnce(Arg) -> Output> (which is invariant over Arg). However, there's also a PhantomData<fn(Arg) -> Output> field to provide the proper contravariance over Arg (and covariance over Output).
We can't use Any as our erased type, because only 'static types implement Any, and we have a step in VariantBoxedFnOnce::new() where we have a Box<dyn Erased + '_> where '_ is not guaranteed to be 'static. We then immediately "transmute" it into 'static, to avoid having a redundant lifetime parameter on VariantBoxedFnOnce, but that 'static is a lie (hence the unsafe code). call_once "downcasts" the erased type to the "original" Box<dyn FnOnce(Arg) -> Output>, except that Arg and Output may be different from the original due to variance.
Consider next code:
fn get_ref<'a, R>(slice: &'a Vec<i32>, f: fn(&'a Vec<i32>) -> R) -> R
where
R: 'a,
{
f(slice)
}
fn main() {
let v = [1,2,3,4,5,6];
let iter = get_ref(&v, |x| x.iter().skip(1).take(2));
println!("{:?}", iter.collect::<Vec<_>>());
}
I create some static variable, then apply some function to its reference and get a result.
It seems to work totally fine. At least it successfully compiles.
Now I am trying to add next level of abstraction. And things are getting weird...
fn owned<'a, R>(owner: Vec<i32>, f: fn(&'a Vec<i32>) -> R)
where
R: 'a,
{
let _ = get_ref(&owner, f); // error occurs here
// `owner` does not live long enough.
}
// get_ref is the same as in the first example
fn get_ref<'a, R>(slice: &'a Vec<i32>, f: fn(&'a Vec<i32>) -> R) -> R
where
R: 'a,
{
f(slice)
}
fn main() {
let v = [1,2,3,4,5,6];
owned(v, |x| x.iter().skip(1).take(2));
}
For me it looks like pretty the same code. But Rust fails to compile it. I really don't understand why this is happening and how should I rewrite my code to compile.
Imagine if I decide to call the function owned<'static, i32>(Vec<i32>, foo) with foo defined as:
fn foo(vec: &'static Vec<i32>) -> i32 { ... }
This satisfies the bounds for owned, since i32: 'static. However this means that you must have a static reference to call f, but owner does not live forever, since it is destroyed at the end of owned.
One way to fix it is to use the following:
fn owned<R>(owner: Vec<i32>, f: for<'a> fn(&'a Vec<i32>) -> R) {
let _ = get_ref(&owner, f);
}
It says that f must be callable with any lifetime, rather than just some specific lifetime. However, this has the consequence that R cannot borrow from the argument, since R is declared in a larger scope than 'a. There isn't any way to fix that while keeping the generics as is.
This answer was taken from my response to this thread on URLO.
I've stumbled upon an interesting edge case: using higher-ranked lifetime bounds to accept closures that return generic parameters, such as for<'a> FnOnce(&'a T) -> R: MyTrait. There's no way to specify that R lives for at most 'a. Perhaps it's best to explain with an example.
Let's define a simple reference-like type wrapping a value:
struct Source;
struct Ref<'a> {
source: &'a Source,
value: i32,
}
For convenience, let's add a helper constructor. Here I will use explicit lifetimes to make the borrowing self-explanatory:
impl Source {
fn new_ref<'a>(&'a self, value: i32) -> Ref<'a> {
Ref { source: self, value }
}
}
This is an extremely fancy implementation of a integer copying routine using HRTBs with a closure over our Ref:
fn call_1<F>(callback: F) -> i32
where
for<'a> F: FnOnce(&'a Source) -> Ref<'a>,
{
let source = Source;
callback(&source).value
}
fn fancy_copy_1(value: i32) -> i32 {
call_1(|s| s.new_ref(value))
}
This is fine and is working as expected. We know that Ref does not outlive the Source and the compiler is also able to pick it up. Now let's create a simple trait and implement it for our reference:
trait MyTrait {
fn value(&self) -> i32;
}
impl<'a> MyTrait for Ref<'a> {
fn value(&self) -> i32 {
self.value
}
}
And modify our integer copying routine to return a generic type implementing that trait instead of just returning Ref:
fn call_2<R, F>(callback: F) -> i32
where
for<'a> F: FnOnce(&'a Source) -> R,
R: MyTrait,
{
let source = Source;
callback(&source).value()
}
fn fancy_copy_2(value: i32) -> i32 {
call_2(|s| s.new_ref(value))
}
Suddenly I get an error: cannot infer an appropriate lifetime for autoref due to conflicting requirements. Rust playground link for convenience. That actually makes sense from some perspective: unlike with Ref<'a> in the first example I never said that R has to live for at most 'a. could very well live longer and thus have access to freed memory. So I need to annotate it with it's own lifetime. But there's no place to do it! The first instinct is to put the lifetime in the bounds:
where
for<'a> F: FnOnce(&'a Source) -> R,
R: MyTrait + 'a,
which is of course incorrect, as 'a is only defined for the first bound.
This is where I got confused, started searching and never found anything about combining HRTBs and generic types in one bound. Perhaps more experienced people in Rust have any suggestions?
Upd 1.
While I was thinking about the problem some more, I remembered I could use the impl Trait syntax. This looks like a solution to my problem:
fn call_3<F>(callback: F) -> i32
where
F: for<'a> FnOnce(&'a Source) -> (impl MyTrait + 'a),
{
let source = Source;
callback(&source).value()
}
fn fancy_copy_3(value: i32) -> i32 {
call_3(|s| Box::new(s.new_ref(value)))
}
That, however does not work because impl MyTrait is not allowed in this place for some reason (probably temporarily). But that made me think of dyn Trait syntax and that indeed does work!
fn call_4<F>(callback: F) -> i32
where
F: for<'a> FnOnce(&'a Source) -> Box<dyn MyTrait + 'a>,
{
let source = Source;
let value = callback(&source); // note how a temporary is required
value.value()
}
fn fancy_copy_4(value: i32) -> i32 {
call_4(|s| Box::new(s.new_ref(value)))
}
Here's my solution: with dyn Trait syntax there is a place to put the + 'a! Unfortunately this solution still doesn't quite work for me too well, as it requires object-safety on the trait plus adds an overhead of allocating a boxed value. But at least it's something.
I have two types: Lexer and SFunction.
SFunction stands for stateful function and is definined like so:
struct SFunction {
f: Option<Box<FnMut() -> SFunction>>,
}
The important part is that any SFunction references a closure that returns an SFunction.
Now I want to have these functions carry state by each affecting the same Lexer. This means that each of these SFunctions has to have a lifetime that depends on a specific Lexer.
Here's some more code if you want to get more of a sense of what I'm doing with this:
impl Lexer {
fn lex(&mut self) {
self.sfunction(Lexer::lexNormal).call()
}
fn sfunction(&mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(|| f(self)))
// SFunction { f: Some(Box::new(move ||f(self))) }
}
fn lexNormal(&mut self) -> SFunction {
return SFunction::empty()
}
}
(Here’s a full version of the code in the Rust playground.)
How do I specify this lifetime requirement in the code?
The compiler errors I'm getting say "cannot infer an appropriate lifetime for capture of self by closure due to conflicting requirements". I'm pretty sure the "conflicting requirements" here is that a Box type assumes the lifetime to be 'static. I could do something like Box<FnMut() -> SFunction + 'a> where 'a is a lifetime defined by the Lexer it depends upon, but I'm not sure how to define such an 'a.
Thanks for your help!
The problem is in this line:
SFunction::new(Box::new(|| f(self)))
Here, self is a reference to a Lexer, but there's no guarantee that the lexer will live long enough. In fact, it needs to live for the 'static lifetime! Without a lifetime specified, a boxed trait object will use the 'static lifetime. Said in code, these two declarations are equivalent:
<Box<FnMut() -> SFunction>>
<Box<FnMut() -> SFunction> + 'static>
And you can make your code compile (in an unsatisfactory way) by restricting it to accept only references that will live for the 'static lifetime:
fn lex(&'static mut self) {
self.sfunction(Lexer::lex_normal).call()
}
fn sfunction(&'static mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(move || f(self)))
}
Of course, it's very doubtful that you will have a Lexer with the static lifetime, as that would mean that it's lexing static data, which wouldn't be very useful. That means we need to include lifetimes in your trait object... as you suggested.
Ultimately what helped to see the problem was to restructure your closure a bit:
fn sfunction(&mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(move || {
// f(self)
let s2 = self;
let f2 = f;
f2(s2)
}))
}
Compiling this produces an error that points to what seems to be the real problem:
<anon>:31:22: 31:26 error: cannot move out of captured outer variable in an `FnMut` closure [E0507]
<anon>:31 let s2 = self;
^~~~
<anon>:31:17: 31:19 note: attempting to move value to here
<anon>:31 let s2 = self;
^~
<anon>:31:17: 31:19 help: to prevent the move, use `ref s2` or `ref mut s2` to capture value by reference
I believe this is because a FnMut closure may be called multiple times, which would mean that the reference enclosed in the closure would need to be copied around, which would be bad news as &mut references should be unique.
All together, this code works:
struct SFunction<'a> {
f: Option<Box<FnOnce() -> SFunction<'a> + 'a>>,
}
impl<'a> SFunction<'a> {
fn new(f: Box<FnOnce() -> SFunction<'a> + 'a>) -> SFunction<'a> {
SFunction {
f: Some(f),
}
}
fn empty() -> SFunction<'a> {
SFunction {
f: None,
}
}
fn call(self) { }
}
struct Lexer;
impl Lexer {
fn lex(&mut self) {
self.sfunction(Lexer::lex_normal).call()
}
fn sfunction(&mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(move || f(self)))
}
fn lex_normal<'z>(&'z mut self) -> SFunction<'z> {
SFunction::empty()
}
}
fn main() {
let mut l = Lexer;
l.lex()
}
I hope my explanation is right and that the changed code still suits your use case!