I am trying to create a high order function which takes in (a function that takes &str and returns iterator of &str). What I am having difficulty is to relate lifetime variables of for<'a> for the function and the return type of that function. It is hard to describe in words, so let's jump right into the code:
use std::fs::File;
use std::io::{BufRead, BufReader, Result};
fn static_dispatcher<'b, I>(
tokenize: impl for<'a> Fn(&'a str) -> I,
ifs: BufReader<File>,
) -> Result<()>
where
I: Iterator<Item = &'b str>,
{
ifs.lines()
.map(|line| tokenize(&line.unwrap()).count())
.for_each(move |n| {
println!("{n}");
});
Ok(())
}
fn main() -> Result<()> {
let ifs = BufReader::new(File::open("/dev/stdin")?);
static_dispatcher(|line| line.split_whitespace(), ifs)
// static_dispatcher(|line| line.split('\t'), ifs)
}
The compiler complains that the lifetime relation of the tokenize's input 'a and output 'b is not specified.
--> src/main.rs:21:30
|
21 | static_dispatcher(|line| line.split_whitespace(), ifs)
| ----- ^^^^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
| | |
| | return type of closure is SplitWhitespace<'2>
| has type `&'1 str`
|
= note: requirement occurs because of the type `SplitWhitespace<'_>`, which makes the generic argument `'_` invariant
= note: the struct `SplitWhitespace<'a>` is invariant over the parameter `'a`
= help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance
I want to specify 'a = 'b, but I can't because 'a comes from for<'a>, which is not visible for the type I.
I also tried
fn static_dispatcher<'a, I>(
tokenize: impl Fn(&'a str) -> I,
ifs: BufReader<File>,
) -> Result<()>
where
I: Iterator<Item = &'a str>,
but this does not work either b/c the lifetime of tokenize argument must be generic, i.e., must be used with for <'a>.
How can I fix this problem?
Related
The following is the code:
struct A;
fn f<'a, I>(default_args: I)
where
I: IntoIterator<Item = &'a A>,
{
{
let a1 = A;
let more_args = [&a1];
{
let i = default_args.into_iter();
let i = i.chain(more_args);
// I will use `i` here.
}
}
// I will NOT use `*_args` here.
}
The following is error:
error[E0597]: `a1` does not live long enough
--> src\main.rs:27:26
|
21 | fn f<'a, I>(default_args: I)
| -- lifetime `'a` defined here
...
27 | let more_args = [&a1];
| ^^^ borrowed value does not live long enough
...
30 | let i = i.chain(more_args);
| ------------------ argument requires that `a1` is borrowed for `'a`
...
33 | }
| - `a1` dropped here while still borrowed
For more information about this error, try `rustc --explain E0597`.
I want to get a new default_args_2 from default_args, where the Item inside default_args_2 has a shorter lifetime, as long as it valid inside the function.
You can add a little (almost) noop .map(|v| v):
let i = default_args.into_iter().map(|v| v);
The problem was that the iterator item types have to match, and therefore the chained iterator has to also yield &'a T, which it does not (it yields a shorter lifetime).
Since you don't really need to use the items for 'a, the trick of the noop map() is to insert a subtyping point: Now, the closure inside map() (conceptually) converts the &'a T, to a &'shorter_lifetime T, which is fine since 'a: 'shorter_lifetime, but now the chained iterator can produce the desired item type.
Putting it differently, you cannot convert impl Iterator<Item = &'a T> into impl Iterator<Item = &'shorter_lifetime T> (as it may not be covariant over the lifetime), but you can convert &'a T to &'shorter_lifetime T, and we use that fact because from the outside we have impl Iterator<Item = &'a T>, but inside map()'s closure we got &'a T.
The idea is to to have one closure (change_x in this case) that captures state (x in this case) that takes a function as its parameter(alterer) that would dictate how the inner state changes.
pub fn plus17(h: & u64) -> u64 {
*h + 17
}
pub fn main() {
let x = 0; //take x by reference
let mut change_x = move |alterer: &dyn FnOnce(&u64) ->u64 | alterer(&x) ;
change_x(&mut plus17);
println!("{}", x);
}
I can't seem to get the types right however:
error[E0161]: cannot move a value of type dyn for<'r> FnOnce(&'r u64) -> u64: the size of dyn for<'r> FnOnce(&'r u64) -> u64 cannot be statically determined
--> playground/src/main.rs:19:69
|
19 | let mut increment_x = move |alterer: &dyn FnOnce(&u64) ->u64 | alterer(&x) ;
| ^^^^^^^
error[E0507]: cannot move out of `*alterer` which is behind a shared reference
--> playground/src/main.rs:19:69
|
19 | let mut increment_x = move |alterer: &dyn FnOnce(&u64) ->u64 | alterer(&x) ;
| ^^^^^^^ move occurs because `*alterer` has type `dyn for<'r> FnOnce(&'r u64) -> u64`, which does not implement the `Copy` trait
I'm not sure if i'm justified in putting the dyn where i put it, it was a compiler's suggestion and im not really sure why i have to put it there. Is it because the alterer can be of arbitrary size despite the input/return type of &u64->u64?
I have also tried to make alterer a FnMut as opposed to FnOnce, but im also pretty shaky as to their distinction and the fact that a given alterer would run only once (at the moment of invocation by outer closure change_x) seemed reasonable.
FnOnce needs an owned self. Thus alterer cannot be FnOnce, because it is not owned but a reference.
You can either make it &dyn Fn or &mut dyn FnMut (I'd recommend going with FnMut), or take Box<dyn FnOnce>.
Playing around with async/await I stumbled across this lifetime puzzle. I assume that predicate takes a reference/value and produces static future - it has no reference to its arguments and captures no values. Instead I got complain about lifetime being too short.
Why it fails and where's the lifetime '2?
use futures::future;
use std::future::Future;
fn validate<F, Fut>(_: F)
where
F: FnMut(&str) -> Fut,
Fut: Future<Output = bool>,
{
}
fn predicate(_: impl AsRef<str>) -> impl Future<Output = bool> {
future::ready(true)
}
fn main() {
validate(|x| predicate(x));
}
Error message:
error: lifetime may not live long enough
--> src\main.rs:16:18
|
16 | validate(|x| predicate(x));
| -- ^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
| ||
| |return type of closure is impl futures::Future
| has type `&'1 str`
error: aborting due to previous error
The error message here is a little confusing because it's talking about two lifetimes, '1 and '2, but doesn't actually say what '2 refers to.
There are two lifetimes that matter here:
the &str argument, which the message labels with lifetime '1
the future type variable Fut, which is labeled with lifetime '2 in the message, though it doesn't clearly state that.
The inferred lifetimes (using the lifetime elision rules) for predicate are like this:
fn predicate<'x>(_: impl AsRef<str> + 'x) -> impl Future<Output = bool> + 'x {
future::ready(true)
}
Even though the implementation doesn't actually use the string input, it would be weird if it didn't, so this is probably correct for a real implementation. The compiler has inferred that the return value of a function uses the input, so they must have compatible lifetimes.
However, the lifetimes for validate are inferred to be different:
fn validate<'1, '2, F, Fut>(_: F)
where
F: FnMut(&'1 str) -> Fut,
Fut: Future<Output = bool> + '2,
{
}
This function accepts arguments with broader requirements than predicate, which means it can't guarantee to enforce the constraints that predicate needs.
You can fix that by making it explicit in the type of validate:
fn validate<'a, F, Fut>(_: F)
where
F: FnMut(&'a str) -> Fut,
Fut: Future<Output = bool> + 'a,
{
}
Constraining both types with the 'a lifetime communicates that validate expects F to use data from the &str, and therefore will ensure that the string lives long enough. Any implementation of validate that broke this contract would also not compile.
The following code
struct Cat<'a, T> {
coolness: &'a T,
}
complains saying
error[E0309]: the parameter type `T` may not live long enough
--> src/main.rs:2:5
|
1 | struct Cat<'a, T> {
| - help: consider adding an explicit lifetime bound `T: 'a`...
2 | coolness: &'a T,
| ^^^^^^^^^^^^^^^
|
note: ...so that the reference type `&'a T` does not outlive the data it points at
--> src/main.rs:2:5
|
2 | coolness: &'a T,
| ^^^^^^^^^^^^^^^
With an explicit lifetime bound, it compiles. When I instantiate the struct where T is an &i32 and despite each reference having a different lifetime, the code compiles. My understanding is that the compiler sees that the inner & outlives the outer &:
struct Cat<'a, T>
where
T: 'a,
{
coolness: &'a T,
}
fn main() {
let b = 10;
{
let c = &b;
{
let fluffy = Cat { coolness: &c };
}
}
}
Does Cat { coolness: &c } expand to Cat { coolness: &'a &'a i32 }? Does the inner reference also assume the same lifetime and so forth for more nested references?
Does Cat { coolness: &c } expand to Cat { coolness: &'a &'a i32 }?
Yes, the Cat ends up with a reference to a reference. This can be demonstrated by the following code compiling:
let fluffy = Cat { coolness: &c };
fn is_it_a_double_ref(_x: &Cat<&i32>) {}
is_it_a_double_ref(&fluffy);
However, the lifetime on each reference is not necessarily the same.
My understanding is that the compiler sees that the inner & outlives the outer &
That's right. And this is precisely where the T: 'a bound comes into play.
Lifetime bounds are a bit tricky to understand at first. They put restrictions on the references contained in T. For example, given the bound T: 'static, types that don't contain any references, or only contain 'static references, e.g. i32 and &'static str, satisfy the bound, while types that contain non-'static references, e.g. &'a i32, don't, because 'a: 'static is false. More generally, given the bound T: 'a, the type T satisfies the bound if, for every lifetime parameter 'x on T, 'x: 'a is true (types with no lifetime parameters trivially satisfy the bound).
Back to your code now. Let's give some names to these references. We'll say the type of coolness is &'fluffy &'c i32. 'c is the lifetime of the variable c and 'fluffy is the lifetime of the variable fluffy (counterintuitively, lifetimes encode the scope of a borrow, not the lifetime of the referent, although the compiler does check that the borrow doesn't extend beyond the referent's lifetime). That means the type of Fluffy is Cat<'fluffy, &'c i32>. Is it true that &'c i32: 'fluffy?
To check if &'c i32: 'fluffy is true, we need to check if 'c: 'fluffy is true. 'c: 'fluffy is true because the variable c goes out of scope after fluffy.
This is related to my earlier question on making a modular exponentiation method generic. I've now arrived at the following code:
fn powm<T>(fbase: &T, exponent: &T, modulus: &T) -> T
where
T: Mul<T, Output = T>
+ From<u8>
+ PartialEq<T>
+ Rem<T, Output = T>
+ Copy
+ for<'a> Rem<&'a T, Output = T>
+ Clone
+ PartialOrd<T>
+ ShrAssign<T>,
for<'a> &'a T: PartialEq<T> + Rem<&'a T, Output = T>,
{
if modulus == T::from(1) {
T::from(0)
} else {
let mut result = T::from(1);
let mut base = fbase % modulus;
let mut exp = exponent.clone();
while exp > T::from(0) {
if exp % T::from(2) == T::from(1) {
result = (result * base) % modulus;
}
exp >>= T::from(1);
base = (base * base) % modulus;
}
result
}
}
It is my understanding that by defining the trait bound where for<'a> &'a T: Rem<&'a T, Output=T> that it is understood that I can use the modulo operator % on two operands of type &'a T, and the result will be of type T. However, I get the following error:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:20:30
|
20 | let mut base = fbase % modulus;
| ^
|
note: first, the lifetime cannot outlive the anonymous lifetime #3 defined on the function body at 3:1...
--> src/main.rs:3:1
|
3 | / fn powm<T>(fbase: &T, exponent: &T, modulus: &T) -> T
4 | | where
5 | | T: Mul<T, Output = T>
6 | | + From<u8>
... |
30 | | }
31 | | }
| |_^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:20:32
|
20 | let mut base = fbase % modulus;
| ^^^^^^^
note: but, the lifetime must be valid for the anonymous lifetime #1 defined on the function body at 3:1...
--> src/main.rs:3:1
|
3 | / fn powm<T>(fbase: &T, exponent: &T, modulus: &T) -> T
4 | | where
5 | | T: Mul<T, Output = T>
6 | | + From<u8>
... |
30 | | }
31 | | }
| |_^
note: ...so that types are compatible (expected std::ops::Rem, found std::ops::Rem<&T>)
--> src/main.rs:20:30
|
20 | let mut base = fbase % modulus;
| ^
The code does work if I replace the line in question by
let mut base = fbase.clone() % modulus;
I don't see why I would need to clone in the first place if I can use the modulo operator already to return a "fresh" element of type T. Do I need to modify my trait bounds instead? Why does this go wrong?
When programming, it's very useful to learn how to create a Minimal, Complete, and Verifiable example (MCVE). This allows you to ignore irrelevant details and focus on the core of the problem.
As one example, your entire blob of code can be reduced down to:
use std::ops::Rem;
fn powm<T>(fbase: &T, modulus: &T)
where
for<'a> &'a T: Rem<&'a T, Output = T>,
{
fbase % modulus;
}
fn main() {}
Once you have a MCVE, you can make permutations to it to explore. For example, we can remove the lifetime elision:
fn powm<'a, 'b, T>(fbase: &'a T, modulus: &'b T)
where
for<'x> &'x T: Rem<&'x T, Output = T>,
{
fbase % modulus;
}
Now we start to see something: what is the relation between all three lifetimes? Well, there isn't one, really. What happens if we make one?
If we say that the input references can be unified to the same
lifetime, it works:
fn powm<'a, T>(fbase: &'a T, modulus: &'a T)
If we say that 'b outlives 'a, it works:
fn powm<'a, 'b: 'a, T>(fbase: &'a T, modulus: &'b T)
If we say that we can have two different lifetimes in the operator, it works:
for<'x, 'y> &'x T: Rem<&'y T, Output = T>,
What about if we poke at the call site?
If we directly call the Rem::rem method, it works:
Rem::rem(fbase, modulus);
If we dereference and re-reference, it works:
&*fbase % &*modulus;
I don't know exactly why the original doesn't work — conceptually both the input references should be able to be unified to one lifetime. It's possible that there's a piece of inference that either cannot or isn't happening, but I'm not aware of it.
Some further discussion with a Rust compiler developer led to an issue as it doesn't quite seem right. This issue has now been resolved and should theoretically be available in Rust 1.23.