This is of course a simplified version of my actual problem.
fn my_func<'a, 'b>(a: &'a i32, b: &'b i32) -> i32 {
a + b
}
fn my_func_generic<'a, 'b, T>(a: &'a T, b: &'b T) -> T
where
&'a T: std::ops::Add<&'b T, Output = T>,
{
a + b
}
fn apply_function(x: &i32, y: &i32, function: fn(&i32, &i32) -> i32) -> i32 {
function(x, y)
}
fn main() {
let x = 10;
let y = 11;
println!("{}", my_func(&x, &y));
println!("{}", my_func_generic(&x, &y));
println!("{}", apply_function(&x, &y, my_func));
println!("{}", apply_function(&x, &y, my_func_generic));
}
error[E0308]: mismatched types
--> src/main.rs:23:43
|
23 | println!("{}", apply_function(&x, &y, my_func_generic));
| ^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected fn pointer `for<'r, 's> fn(&'r i32, &'s i32) -> _`
found fn pointer `fn(&i32, &i32) -> _`
The question I have is: How can I use my_func_generic as a function pointer to pass it into apply_function?
It can definitely be used as such a function, as println!("{}", my_func_generic(&x, &y)) demonstrates.
Trying to write it in a pointer gives the same error:
let function_pointer: fn(&i32, &i32) -> i32 = my_func_generic;
Annotating generic types manually gives the same error:
let function_pointer: fn(&i32, &i32) -> i32 = my_func_generic::<'_, '_, i32>;
Sadly I don't understand the error message, and don't have enough knowledge about the inner type system of Rust generics to understand how I could solve this issue, or if it's simply not possible due to some mechanism yet unknown to me.
TL;DR: In your case, you can do it as follows:
fn apply_function<'a, 'b>(x: &'a i32, y: &'b i32, function: fn(&'a i32, &'b i32) -> i32) -> i32
But in general, this is impossible.
There are two kinds of lifetimes in Rust: late-bound and early-bound. The difference between the two boils to when we bind the generic lifetime to a concrete lifetime.
For early-bound lifetime, we bind them when you're referring to their function. For late-bound lifetimes, referring to their function gives you a function pointer (or item) generic over the lifetime - what we call HRTB (for<'lifetime> fn(...)), and we choose the actual lifetime only when you call them.
The important point is that any constraints on the lifetime turns it into an early bound lifetime. This is because it is generally impossible to express constrained lifetimes as HRTB (more details on the above linked rustc-dev-guide page), although to be honest I still don't fully understand the reason.
So in my_func_generic(), 'a and 'b are early bound. However, apply_function() expects them to be late-bound (since the default desugaring for elided lifetimes in function pointers is HRTB, fn(&i32, &i32) -> i32 is actually for<'a, 'b> fn(&'a i32, &'b i32) -> i32), and so this is a type mismatch.
In this case, you can just change apply_function() to bind the lifetimes as parameters, as I suggested above. This will still accept late-bound lifetimes, as we'll just bind them immediately. However, as far as I know, in general it is impossible to convert an early-bound lifetime to a late-bound one.
It works if you change your trait bounds slightly:
fn my_func_generic<'a, 'b, T>(a: &'a T, b: &'b T) -> T
where
T: Copy + std::ops::Add<Output = T>,
{
*a + *b
}
Related
Question 1
What are the differences between these two functions in rust?
fn foo<'a>(x: &'a i32) -> &'a i32 {
x
}
fn foo<'a: 'b, 'b>(x: &'a i32) -> &'b i32 {
x
}
Question 2
What constraints are provided when the same lifetime parameter 'a is used among multiple function parameters and also in the return value?
fn foo<'a>(x: &'a i32, y: &'a i32) -> &'a i32 {
...
}
Two differences as far as I can tell:
The second has two lifetimes that can be different, the first has only one.
The second's lifetimes are early-bound,the first's is late-bound. There are some differences, the main one being that you can't create a HRTB function pointer with early bound lifetime (let f: fn(&i32) -> &i32 = foo2; does not compile).
You almost always want (1), since it is less verbose and allows HRTB.
2.
The lifetime is the smallest of all. Ignoring the points at question 1, this is the same as:
fn foo<'a, 'b: 'a, 'c: 'a>(x: &'b i32, y: &'c i32) -> &'a i32 {
...
}
I am seeking help to understand why the borrow checker fails for the following minimal non-working example, and I would be very happy to learn how to correctly implement what I was trying to do:
use std::collections::HashSet;
struct Foo {
data: HashSet<usize>
}
impl Foo {
fn test<'a, F, T>(&mut self, _operation: F) -> ()
where F: Fn(&'a HashSet<usize>, &'a HashSet<usize>) -> T,
T: Iterator<Item=&'a usize>
{
let update: HashSet<usize> = vec![4, 2, 9].into_iter().collect();
self.data = _operation(&self.data, &update).copied().collect();
}
fn new() -> Self {
Foo { data: HashSet::new() }
}
}
fn main() {
let mut foo: Foo = Foo::new();
foo.test(HashSet::intersection);
}
My main source of confusion is that, if I replace the call to _operation with HashSet::intersection, the code compiles. I thought that the type of the parameter _operation would allow me to pass both HashSet::intersection and HashSet::union as operations here.
For the record, this is the error I receive:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src\main.rs:13:32
|
13 | self.data = _operation(&self.data, &update).copied().collect();
| ^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime defined on the method body at 8:23...
--> src\main.rs:8:23
|
8 | fn test<'a, F, T>(&mut self, _operation: F) -> ()
| ^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src\main.rs:13:32
|
13 | self.data = _operation(&self.data, &update).copied().collect();
| ^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the method body at 8:13...
--> src\main.rs:8:13
|
8 | fn test<'a, F, T>(&mut self, _operation: F) -> ()
| ^^
note: ...so that reference does not outlive borrowed content
--> src\main.rs:13:32
|
13 | self.data = _operation(&self.data, &update).copied().collect();
| ^^^^^^^^^^
For more information about this error, try `rustc --explain E0495`.
error: could not compile `aoc06` due to previous error
The issue, as the (albeit cryptic) compiler message suggests, is that there is a lifetime mismatch: _operation expects HashSet references that live as long as 'a, but &self.data has a lifetime 'b, the elided lifetime of &mut self, and &update has a different lifetime that lasts the duration of the test function body.
To resolve this issue, we must specify that the function type F takes in HashMap references of arbitrary lifetimes, not just the specific lifetime 'a -- this lets the compiler infer the appropriate lifetime when _operation is invoked. This is why we need Higher-Rank Trait Bounds (HRTBs):
fn test<F, T>(&mut self, _operation: F) -> ()
where F: for<'a> Fn(&'a HashSet<usize>, &'a HashSet<usize>) -> T,
However, this raises another issue. How do we apply the higher-ranked lifetime 'a to the type parameter T? Unfortunately Rust does not support higher-kinded types, but we can get away with "abstracting" out the the function type F and the higher-kinded type T to a trait and an associated type on said trait.
trait Operation<'a, T: 'a> {
type Output: Iterator<Item = &'a T>;
fn operate(self, a: &'a HashSet<T>, b: &'a HashSet<T>) -> Self::Output;
}
The Operation trait represents an operation on two HashSets that returns an iterator over references, equivalent to the functions HashSet::union, HashSet::intersection, and the like. We can achieve this using the following impl, which ensures that HashSet::intersection and the like implement Operation:
impl<'a, T: 'a, I, F> Operation<'a, T> for F
where
I: Iterator<Item = &'a T>,
F: FnOnce(&'a HashSet<T>, &'a HashSet<T>) -> I,
{
type Output = I;
fn operate(self, a: &'a HashSet<T>, b: &'a HashSet<T>) -> Self::Output {
self(a, b)
}
}
We can then use a HRTB on the Operation trait instead, which does not require any nested higher-kinded types:
fn test(&mut self, _operation: impl for<'a> Operation<'a, usize>) -> () {
let update: HashSet<usize> = vec![4, 2, 9].into_iter().collect();
self.data = _operation.operate(&self.data, &update).copied().collect();
println!("{:?}", self.data);
}
Playground
The arguments you are passing do not match the lifetime which you declared the Fn bound with.
fn test<'a, F, T>(&mut self, _operation: F) -> ()
'a is some arbitrary lifetime that may be specified by the caller,
F: Fn(&'a HashSet<usize>, &'a HashSet<usize>) -> T,
which must be adequate for the references given to _operation,
let update: HashSet<usize> = vec![4, 2, 9].into_iter().collect();
self.data = _operation(&self.data, &update).copied().collect();
but here you pass in a borrow from self (whose lifetime is not specified to outlive 'a) and a borrow from update (which is a local variable, which cannot outlive 'a).
In order to correctly write this, you need to specify that _operation may be called with any lifetime (which thus includes the lifetimes of borrows of local variables). That's simple, by itself:
fn test<F, T>(&mut self, _operation: F) -> ()
where
F: for<'a> Fn(&'a HashSet<usize>, &'a HashSet<usize>) -> T,
Note that 'a is no longer a lifetime parameter of test. Instead it's part of the bound on F: you can read this for<'a> notation as “for any lifetime, which we will call 'a, F may be called as a function with references &'a ...”.
However, this isn't actually a solution, because you also have T: Iterator<Item = &'a usize>, which uses 'a again. It isn't currently possible to write a where clause that expresses this relationship, particularly as even without the item being a reference, the iterator will be borrowing the &'a HashSets.
This is an unfortunate limitation of current Rust — it also comes up in trying to write a function that takes an async function which borrows an input (which is structurally the same as your situation, with Future in place of Iterator). However, there is a workaround: you can define a trait for the function, which has a single lifetime parameter that links everything together. (This doesn't impose any extra work on the caller of your function, because the trait is blanket implemented for all suitable functions.)
Here's your code with such a trait added and the needed modifications to fn test():
use std::collections::HashSet;
trait IteratorCallback<'a> {
type Output: Iterator<Item = &'a usize> + 'a;
fn call(self, a: &'a HashSet<usize>, b: &'a HashSet<usize>) -> Self::Output;
}
impl<'a, F, T> IteratorCallback<'a> for F
where
F: FnOnce(&'a HashSet<usize>, &'a HashSet<usize>) -> T,
T: Iterator<Item = &'a usize> + 'a,
{
type Output = T;
fn call(self, a: &'a HashSet<usize>, b: &'a HashSet<usize>) -> T {
// Delegate to FnOnce
self(a, b)
}
}
struct Foo {
data: HashSet<usize>,
}
impl Foo {
fn test<F>(&mut self, _operation: F) -> ()
where
F: for<'a> IteratorCallback<'a>,
{
let update: HashSet<usize> = vec![4, 2, 9].into_iter().collect();
self.data = _operation.call(&self.data, &update).copied().collect();
}
fn new() -> Self {
Foo {
data: HashSet::new(),
}
}
}
fn main() {
let mut foo: Foo = Foo::new();
foo.test(HashSet::intersection);
}
Note: I changed the function bound to FnOnce because that's more permissive than Fn and all you need in this case, but the same technique will work with Fn as long as you change fn call(self, to fn call(&self,.
Credit: I used this Reddit comment by user Lej77 as an example to work from for the trait technique.
Sometimes I struggle with lifetimes. I'm still learning and I don't know what's happening here:
use std::future::Future;
use futures::future::{BoxFuture, FutureExt};
struct M{}
struct Client{}
impl Client {
async fn send_and_expect<'a>(
&'a mut self,
m: &M
) -> std::result::Result<(), ()> {
Ok(())
}
pub fn connection_retrier<'a, T>(
f: fn(&'a mut Self, &M) -> T,
f_self: &'a mut Self,
f_m: &'a M,
)-> BoxFuture<'a, std::result::Result<(),()>>
where
T: Future<Output = std::result::Result<(), ()>> + 'a
{
async move {
Client::send_and_expect(f_self, f_m).await
}.boxed()
}
async fn send_with_retry<'a>(&'a mut self) -> std::result::Result<(), ()> {
let m = M{};
Client::connection_retrier(
Client::send_and_expect,
&mut *self, &m).await
}
}
Error:
Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
--> src/lib.rs:31:21
|
11 | ) -> std::result::Result<(), ()> {
| --------------------------- the `Output` of this `async fn`'s found opaque type
...
31 | Client::send_and_expect,
| ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected fn pointer `for<'r> fn(&mut Client, &'r M) -> impl futures::Future`
found fn pointer `for<'a, '_> fn(&'a mut Client, &M) -> impl futures::Future
Playground
I'm now completely confused about why in for<'r> fn(&mut Client, &'r M) -> impl futures::Future, &mut Client has no lifetime. And what does _ means in for<'a, '_> fn(&'a mut Client, &M) -> impl futures::Future`?
I'm very intersted in learning what's happening here.
Cleaning up the code
In order to see this code from the compiler's perspective, let's change all the lifetime parameters in this example to be explicit and have distinct names, expand the async fn sugar, and then look at how the error changes.
The above example is equivalent to the following after lifetime inference and desugaring.
// name the second lifetime, and expand the async fn sugar.
fn send_and_expect<'a, 'b>(&'a mut self, m: &'b M) -> impl Future<Item=Result<(), ()>> + 'a + 'b
{ ... }
// rename 'a to 'c to avoid ambiguous lifetime names
pub fn connection_retrier<'c, T>(
f: for<'d> fn(&'c mut Self, &'d M) -> T, // name the implicit higher-ranked lifetime here
f_self: &'c mut Self,
f_m: &'c M,
)-> BoxFuture<'c, std::result::Result<(), ()>>
where T: Future<Output = std::result::Result<(), ()>> + 'c
{ ... }
// rename 'a to 'e to avoid ambiguous lifetime names
async fn send_with_retry<'e>(&'e mut self) -> std::result::Result<(), ()> {
After making this change, the error becomes:
Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
--> src/lib.rs:31:21
|
11 | ) -> std::result::Result<(), ()> {
| --------------------------- the `Output` of this `async fn`'s found opaque type
...
31 | Client::send_and_expect,
| ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected fn pointer `for<'d> fn(&mut Client, &'d M) -> impl futures::Future`
found fn pointer `for<'a, 'b> fn(&'a mut Client, &'b M) -> impl futures::Future`
This should clarify your question about '_: it's just the name the compiler gave the inferred second lifetime parameter of send_and_expect. As for the missing lifetime on &mut Client, you can see that it's still missing here. For reasons I do not completely understand, and in ways that depend on the exact error message given, the compiler will sometimes omit concrete lifetimes when printing the type of references, but make no mistake, the lifetime of that reference is 'c.
Solving the error
Onto the actual problem. The signature of f indicates that connection_retrier is expecting a function which (1) takes a reference of lifetime &'c and (2) a reference of any other lifetime, and (3) returns a Future type which will remain valid as long as 'c does, as specified by your where bound.
When we pass send_and_expect to connection_retrier, in order for the signatures to match the compiler is coercing it to the type for<'d> send_and_expect::<'c, 'd>. But that type doesn't meet condition (3) above! Unlike a regular function, the default behavior of an async function is to capture all input lifetimes in its return type, so the return type of for<'d> send_and_expect::<'c, 'd> is in fact impl Future<Item=Result<(), ()>> + 'c + 'd, as you can tell by looking at send_and_expect's expanded signature.
Since this type borrows from the two lifetimes 'c and 'd, and there is no constraint that 'd: 'c (read: 'd outlives 'c), it may not remain valid for the entirety of the lifetime 'c, if 'd ends first. It is this mismatch that results in the rather cryptic lifetime error you received. There are two ways you can solve this problem, depending on your preferred semantics. You can either:
Remove the higher-ranked bound from f entirely, and specify exactly the lifetimes you will be calling it with in connection_retrier (both &'c.)
pub fn connection_retrier<'c, T>(
f: fn(&'c mut Self, &'c M) -> T
f_self: &'c mut Self,
f_m: &'c M,
) -> BoxFuture<'c, std::result::Result<(), ()>>
where T: Future<Output = std::result::Result<(), ()>> + 'c
{ ... }
Keep the signature of connection_retrier the same and specify that the future returned by send_and_expect only borrows from its first argument. To do this, you will need to drop the async fn sugar on the signature and wrap the body in an async move block.
fn send_and_expect<'a, 'b>(&'a mut self, m: &'b M) -> impl Future<Output=Result<(), ()>> + 'a
{ async move { ... } }
Note that you cannot solve this by writing the type of f as for<'d: 'c> fn(&'c mut Self, &'d M) -> T, as bounds are currently not permitted for universally quantified lifetimes.
I've stumbled upon an interesting edge case: using higher-ranked lifetime bounds to accept closures that return generic parameters, such as for<'a> FnOnce(&'a T) -> R: MyTrait. There's no way to specify that R lives for at most 'a. Perhaps it's best to explain with an example.
Let's define a simple reference-like type wrapping a value:
struct Source;
struct Ref<'a> {
source: &'a Source,
value: i32,
}
For convenience, let's add a helper constructor. Here I will use explicit lifetimes to make the borrowing self-explanatory:
impl Source {
fn new_ref<'a>(&'a self, value: i32) -> Ref<'a> {
Ref { source: self, value }
}
}
This is an extremely fancy implementation of a integer copying routine using HRTBs with a closure over our Ref:
fn call_1<F>(callback: F) -> i32
where
for<'a> F: FnOnce(&'a Source) -> Ref<'a>,
{
let source = Source;
callback(&source).value
}
fn fancy_copy_1(value: i32) -> i32 {
call_1(|s| s.new_ref(value))
}
This is fine and is working as expected. We know that Ref does not outlive the Source and the compiler is also able to pick it up. Now let's create a simple trait and implement it for our reference:
trait MyTrait {
fn value(&self) -> i32;
}
impl<'a> MyTrait for Ref<'a> {
fn value(&self) -> i32 {
self.value
}
}
And modify our integer copying routine to return a generic type implementing that trait instead of just returning Ref:
fn call_2<R, F>(callback: F) -> i32
where
for<'a> F: FnOnce(&'a Source) -> R,
R: MyTrait,
{
let source = Source;
callback(&source).value()
}
fn fancy_copy_2(value: i32) -> i32 {
call_2(|s| s.new_ref(value))
}
Suddenly I get an error: cannot infer an appropriate lifetime for autoref due to conflicting requirements. Rust playground link for convenience. That actually makes sense from some perspective: unlike with Ref<'a> in the first example I never said that R has to live for at most 'a. could very well live longer and thus have access to freed memory. So I need to annotate it with it's own lifetime. But there's no place to do it! The first instinct is to put the lifetime in the bounds:
where
for<'a> F: FnOnce(&'a Source) -> R,
R: MyTrait + 'a,
which is of course incorrect, as 'a is only defined for the first bound.
This is where I got confused, started searching and never found anything about combining HRTBs and generic types in one bound. Perhaps more experienced people in Rust have any suggestions?
Upd 1.
While I was thinking about the problem some more, I remembered I could use the impl Trait syntax. This looks like a solution to my problem:
fn call_3<F>(callback: F) -> i32
where
F: for<'a> FnOnce(&'a Source) -> (impl MyTrait + 'a),
{
let source = Source;
callback(&source).value()
}
fn fancy_copy_3(value: i32) -> i32 {
call_3(|s| Box::new(s.new_ref(value)))
}
That, however does not work because impl MyTrait is not allowed in this place for some reason (probably temporarily). But that made me think of dyn Trait syntax and that indeed does work!
fn call_4<F>(callback: F) -> i32
where
F: for<'a> FnOnce(&'a Source) -> Box<dyn MyTrait + 'a>,
{
let source = Source;
let value = callback(&source); // note how a temporary is required
value.value()
}
fn fancy_copy_4(value: i32) -> i32 {
call_4(|s| Box::new(s.new_ref(value)))
}
Here's my solution: with dyn Trait syntax there is a place to put the + 'a! Unfortunately this solution still doesn't quite work for me too well, as it requires object-safety on the trait plus adds an overhead of allocating a boxed value. But at least it's something.
A while ago I looked into writing a generic iterator for the Fibonacci sequence
that could accept both primitive numbers as well as custom types (such as
bignums). After failing to get a version working for both the primitive types
and bignums, I stumbled upon this question:
How to write a trait bound for adding two references of a generic type?
Which used so called Higher Ranked Trait Bounds to solve the problem with this
particular issue.
Now however, I'm trying to use a similar strategy to use the *_assign
operators instead. In particular, I'm trying to get something similar to this
working:
use std::ops::{Add, AddAssign};
fn add_test<'a, T>(x: &'a T, y: &'a T) -> T
where
for<'b> &'b T: Add<Output = T>,
{
x + y
}
fn add_assign_test<'a, T>(x: &'a mut T, y: &'a T) -> T
where
for<'b> &'b mut T: AddAssign<&'b T>,
T: Clone,
{
x += y;
x.clone()
}
fn main() {
println!("add_test()={}", add_test(&1, &2));
println!("add_assign_test()={}", add_assign_test(&mut 2, &2));
}
add_test() works as expected but I'm unable to get add_assign_test() to work in a similar way. The errors I'm getting suggest that there might not actually exist an implementation for this kind of behaviour on the primitive types:
error[E0277]: the trait bound `for<'b> &'b mut _: std::ops::AddAssign<&'b _>` is not satisfied
--> src/main.rs:21:38
|
21 | println!("add_assign_test()={}", add_assign_test(&mut 2, &2));
| ^^^^^^^^^^^^^^^ no implementation for `&'b mut _ += &'b _`
|
= help: the trait `for<'b> std::ops::AddAssign<&'b _>` is not implemented for `&'b mut _`
= note: required by `add_assign_test`
I could create a macro that creates implementations for these operators that actually takes references to the primitive types, but that seems a little wasteful. Is there any other way to achieve the same effect?
Just a tiny oversight in your code. Let's look at the trait:
pub trait AddAssign<Rhs = Self> {
fn add_assign(&mut self, rhs: Rhs);
}
The receiver of the method is already &mut self and not self. The reason that you had to do the extra work with Add is because it accepts self as receiver. For AddAssign this means: if a type T implements AddAssign, you can call the method add_assign() on a &mut T!
Thus, instead of writing:
where for <'b> &'b mut T: AddAssign<&'b T>,
... you would write:
where for <'b> T: AddAssign<&'b T>,
(No other line changed so far)
However, you notice that the code still won't compile:
error[E0277]: the trait bound `for<'b> {integer}: std::ops::AddAssign<&'b {integer}>` is not satisfied
--> src/main.rs:13:38
|
13 | println!("add_assign_test()={}", add_assign_test(&mut 2, &2));
| ^^^^^^^^^^^^^^^ no implementation for `{integer} += &'b {integer}`
|
= help: the trait `for<'b> std::ops::AddAssign<&'b {integer}>` is not implemented for `{integer}`
= note: required by `add_assign_test`
The reason is simple: there is simply no implementation of AddAssign for primitive types which takes an immutable reference as rhs (Docs). I don't know if this is an oversight -- it could be worth opening an issue on the Rust repo.
To verify the above code works, I write my own type and implemented AddAssign appropriately: Playground.
Corrected the code snippet based on Lukas' reply:
use std::ops::{Add, AddAssign};
fn add_test<'a, T>(x: &'a T, y: &'a T) -> T
where
for<'b> &'b T: Add<Output = T>,
{
x + y
}
fn add_assign_test<'a, T>(x: &'a mut T, y: &'a T) -> T
where
for<'b> T: AddAssign<&'b T>,
T: Clone,
{
*x += y;
x.clone()
}
fn main() {
println!("add_test()={}", add_test(&1, &2));
println!("add_assign_test()={}", add_assign_test(&mut 2, &2));
}
It appears that this is an issue in Rust itself. There is currently a tracker for this issue.
Until this is fixed, there's two possible workarounds:
Create named-tuples for each primitives and implement OpAssign for those types instead. This does however force you to 'cast' all primitives to your custom type.
Duplicate the 'generic' code with specializations for the primitives.