argument validation in clap v4 - rust

I am using crate clap v4。When I try to write something validating arguments against regex, I had some problem with lifetimes.
Document of ValueParser for convenience
My code as following:
pub fn validator_regex(r: &'static str) -> ValueParser {
ValueParser::from(move |s: &str| -> std::result::Result<&str, Error> {
let reg = regex::Regex::new(r).unwrap();
match reg.is_match(s) {
true => Ok(s),
false => Err(Error::from(format!("not matches {}", r))),
}
})
}
pub fn validator_identifier() -> ValueParser {
validator_regex("^[-_0-9a-zA-Z]+$")
}
And compilation error:
error: lifetime may not live long enough
--> main\./src\cmd\cmd_util.rs:440:21
|
437 | ValueParser::from(move |s: &str| -> std::result::Result<&str, Error> {
| - - let's call the lifetime of this reference `'2`
| |
| let's call the lifetime of this reference `'1`
...
440 | true => Ok(s),
| ^^^^^ returning this value requires that `'1` must outlive `'2`
Could any help me on these two questions:
how to validate regex in clap v4
why there' s lifetime may not live long enough error in this code, since the returned &str lives as long as the closure argument, and this can be compiled normally
pub fn validator_regex(r: &'static str) -> impl Fn(&str) -> Result<&str, String> {
move |s: &str| -> Result<&str, String> {
let reg = regex::Regex::new(r).unwrap();
match reg.is_match(s) {
true => Ok(s),
false => Err(format!("not match {}", r)),
}
}
}

Let's take a look at the From impl you're invoking:
impl<P> From<P> for ValueParser
where
P: TypedValueParser + Send + Sync + 'static,
Ok, take a look at TypedValueParser:
impl<F, T, E> TypedValueParser for F
where
F: Fn(&str) -> Result<T, E> + Clone + Send + Sync + 'static,
E: Into<Box<dyn std::error::Error + Send + Sync + 'static>>,
T: Send + Sync + Clone,
So, the signature is not Fn(&str) -> Result<&str, E>, it is <T> Fn(&str) -> Result<T, E>. Is it different? Can't we just put &str in the place of T?
It is very different. Let's annotate the lifetimes:
// Fn(&str) -> Result<&str, E>
for<'a> Fn(&'a str) -> Result<&'a str, E>
// <T> Fn(&str) -> Result<T, E>
<&'b str> for<'a> Fn(&'a str) -> Result<&'b str, E>
Where does 'b come from? It can't be HRTB, as it is declared in the impl header (T) and not the trait bound (Fn). Therefore, it must be a fixed lifetime. See the mismatch? The lifetimes I called 'a and 'b are what the compiler calls '1 and '2, respectively. If 'b is fixed, it cannot be derived from the dynamic HRTB 'a.
The fix is simple: just don't return &str, return String instead:
pub fn validator_regex(r: &'static str) -> ValueParser {
ValueParser::from(move |s: &str| -> std::result::Result<String, Error> {
let reg = regex::Regex::new(r).unwrap();
match reg.is_match(s) {
true => Ok(s.to_owned()),
false => Err(Error::from(format!("not matches {}", r))),
}
})
}

Related

Implementation of `AsRef` is not general enough

I'm new to rust's async programming and encountered something weird.
I want to introduce async in traits without boxing overhead.
I'm using nightly with the #![feature(type_alias_impl_trait)] feature.
Here's minimized code:
#![feature(type_alias_impl_trait)]
use std::path::Path;
use std::future::Future;
type Error = Box<dyn std::error::Error>;
#[tokio::main]
async fn main() -> Result<(), Error> {
tokio::spawn(async move {
"test".to_owned().super_async("x").await
});
Ok(())
}
trait AsyncTrait {
type Fut<'a, P>: Future<Output=()>
where
Self: 'a,
P: AsRef<Path> + 'a;
fn test_async<P: AsRef<Path>>(&self, p: P) -> Self::Fut<'_, P>;
}
impl AsyncTrait for String {
type Fut<'a, P> = impl Future<Output = ()> + 'a
where
Self: 'a,
P: AsRef<Path> + 'a;
fn test_async<P: AsRef<Path>>(&self, p: P) -> Self::Fut<'_, P> {
async move {
let bs = p.as_ref();
()
}
}
}
trait SuperAsync: AsyncTrait {
type SuperFut<'a, P>: Future<Output=()>
where
P: AsRef<Path> + 'a,
Self: 'a;
fn super_async<P: AsRef<Path>>(&self, p: P) -> Self::SuperFut<'_, P>;
}
impl<T: AsyncTrait> SuperAsync for T {
type SuperFut<'a, P> = impl Future<Output = ()> + 'a
where
P: AsRef<Path> + 'a,
Self: 'a;
fn super_async<P: AsRef<Path>>(&self, p: P) -> Self::SuperFut<'_, P> {
async move {
self.test_async(p).await;
()
}
}
}
Then I got the error message:
error: implementation of `AsRef` is not general enough
--> src/main.rs:45:5
|
45 | / tokio::spawn(async move {
46 | | "test".to_owned().super_async("x").await
47 | | });
| |______^ implementation of `AsRef` is not general enough
|
= note: `AsRef<Path>` would have to be implemented for the type `&'0 str`, for any lifetime `'0`...
= note: ...but `AsRef<Path>` is actually implemented for the type `&'1 str`, for some specific lifetime `'1`
I don't understand the error message. If I remove tokio::spawn or impl SuperAsync only for String, then the error disappers. Any help?
I was able to implement this:
#![feature(type_alias_impl_trait)]
use std::path::Path;
use std::future::Future;
type Error = Box<dyn std::error::Error>;
#[tokio::main]
async fn main() -> Result<(), Error> {
tokio::spawn(async move {
"test".to_owned().super_async("x").await
});
Ok(())
}
trait AsyncTrait {
type Fut<'a, P>: Future<Output=()>
where
Self: 'a,
P: AsRef<Path> + 'a + ?Sized;
fn test_async<'a, P: AsRef<Path> + ?Sized>(&self, p: &'a P) -> Self::Fut<'a, P>;
}
impl AsyncTrait for String {
type Fut<'a, P> = impl Future<Output = ()> + 'a
where
Self: 'a,
P: AsRef<Path> + 'a + ?Sized;
fn test_async<'a, P: AsRef<Path> + ?Sized>(&self, p: &'a P) -> Self::Fut<'a, P> {
async move {
let bs = p.as_ref();
()
}
}
}
trait SuperAsync: AsyncTrait {
type SuperFut<'a, P>: Future<Output=()>
where
P: AsRef<Path> + 'a + ?Sized,
Self: 'a;
fn super_async<'a, P: AsRef<Path> + ?Sized>(&'a self, p: &'a P) -> Self::SuperFut<'a, P>;
}
impl<T: AsyncTrait> SuperAsync for T {
type SuperFut<'a, P> = impl Future<Output = ()> + 'a
where
P: AsRef<Path> + 'a + ?Sized,
Self: 'a;
fn super_async<'a, P: AsRef<Path> + ?Sized>(&'a self, p: &'a P) -> Self::SuperFut<'a, P> {
async move {
self.test_async(p).await;
()
}
}
}
I changed the definition a little bit, now lifetime 'a is associated to self and P, also changing P into a reference, so we can also associate his and the AsRef lifetimes to to 'a.
By changing P into a reference, we enforce that we are not consuming it, and the data that this reference points to will live longer then the future returned. Maybe there is a way to receive a owned P value and ensure it's lifetime, but I couldn't figure that out.
The ?Sized was added because P solves into the str type (not &str) that is unsized.

Why does this closure's lifetime change based on seemingly unrelated types?

With the code
fn foo<'a, 'b>(
state: &'b mut i32,
) -> impl FnMut(&'a str) -> &'static str + 'b {
|s| "hi"
}
I get an error
error[E0482]: lifetime of return value does not outlive the function call
112 | ) -> impl FnMut(&'a str) -> &'static str + 'b {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: the return value is only valid for the lifetime `'a` as defined on the function body at 110:9
110 | fn foo1<'a, 'b>(
| ^^
But somehow the code
fn foo2<'a, 'b>(
state: &'b mut Option<&'a i32>,
) -> impl FnMut(&'a str) -> &'static str + 'b {
|s| "hi"
}
compiles without errors. Why does the type of state change the lifetime of the closure?
With type &'b mut Option<&'a i32>, Rust infers the lifetime bound 'a: 'b ('a outlives 'b). This bound is required for the function signature to be well-formed. You can add this bound explicitly to avoid the error:
fn foo<'a, 'b>(
state: &'b mut i32,
) -> impl FnMut(&'a str) -> &'static str + 'b
where
'a: 'b,
{
|s| "hi"
}
However, if no parameter uses 'a, then 'a should be a higher-rank trait bound instead:
fn foo<'b>(
state: &'b mut i32,
) -> impl for<'a> FnMut(&'a str) -> &'static str + 'b {
|s| "hi"
}
The Fn family of traits are special in that they allow you to omit the lifetime entirely (following the same elision rules as fn signatures), so this is equivalent:
fn foo<'b>(
state: &'b mut i32,
) -> impl FnMut(&str) -> &'static str + 'b {
|s| "hi"
}

One type (function pointer) is more general than the other

I reached this error in this example, and I'm trying to understand what's happening.
use futures::future::{BoxFuture, FutureExt};
use std::sync::Arc;
use futures::lock::Mutex;
struct Message {}
struct Client {}
enum Error {
ConnectionClosed,
}
#[derive(Default)]
struct MyType{}
impl Client {
fn send_and_expect<'a, R: 'a + Default + Send>(
client: Arc<Mutex<Self>>,
message: &'a Message,
) -> BoxFuture<'a, Result<R, Error>> {
async move { Ok(R::default()) }.boxed()
}
pub fn connection_retrier<'a, R: 'a + Default + Send>(
client: Arc<Mutex<Self>>,
f: for<'b> fn(Arc<Mutex<Self>>, &'b Message) -> BoxFuture<'b, Result<R, Error>>,
f_m: &'a Message,
) -> BoxFuture<'a, Result<R, Error>>
{
async move {Ok(R::default())}.boxed()
}
async fn send_with_retry<'a>(
client: Arc<Mutex<Self>>,
message: &'a Message,
) -> Result<MyType, Error> {
let client = Arc::new(Mutex::new(Client{}));
Self::connection_retrier::<MyType>(client, Self::send_and_expect, message)
.await
}
}
Error:
error[E0308]: mismatched types
--> src/lib.rs:36:52
|
36 | Self::connection_retrier::<MyType>(client, Self::send_and_expect, message)
| ^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected fn pointer `for<'b> fn(Arc<_>, &'b Message) -> Pin<Box<(dyn futures::Future<Output = std::result::Result<MyType, Error>> + std::marker::Send + 'b)>>`
found fn pointer `fn(Arc<_>, &Message) -> Pin<Box<dyn futures::Future<Output = std::result::Result<MyType, Error>> + std::marker::Send>>`
Playground
Well, connection_retrier requres a function f: for<'b> fn(Arc<Mutex<Self>>, &'b Message) -> BoxFuture<'b, Result<R, Error>>,. The one I passed seems to fit the signature.
The way I view for<'b> fn... is: I accept a function f for any lifetime 'b presented. So, for the particular lifetime 'a, send_and_expect seems to fit everything.
The only suspicion I have is for the lifetime of R being fixed at 'a in connection_retrier so the f with its for<'b> is not free to give any lifetime for its R. I think the lifetime of the R in f should be 'b. Is it possible to force f to have R with lifetime 'b?
Something like this:
f: for<'b> fn<RR: 'b>(Arc<Mutex<Self>>, &'b Message) -> BoxFuture<'b, Result<RR, Error>>,
This error looks to be caused by limitation of the compiler's ability to automatically convert generic functions which include a type bounded by a lifetime into a function pointer that is fully generic over that lifetime. I'm not quite sure why this limitation exists, but it can be worked around by wrapping the generic call in a non-generic closure. All you need to do to make your example compile is to change:
Self::send_and_expect
to
|c, m| Self::send_and_expect(c, m)
Thus hiding the generic from the compiler.

lifetime of function pointer is for<'a, '_> while it should be for<'r>

Sometimes I struggle with lifetimes. I'm still learning and I don't know what's happening here:
use std::future::Future;
use futures::future::{BoxFuture, FutureExt};
struct M{}
struct Client{}
impl Client {
async fn send_and_expect<'a>(
&'a mut self,
m: &M
) -> std::result::Result<(), ()> {
Ok(())
}
pub fn connection_retrier<'a, T>(
f: fn(&'a mut Self, &M) -> T,
f_self: &'a mut Self,
f_m: &'a M,
)-> BoxFuture<'a, std::result::Result<(),()>>
where
T: Future<Output = std::result::Result<(), ()>> + 'a
{
async move {
Client::send_and_expect(f_self, f_m).await
}.boxed()
}
async fn send_with_retry<'a>(&'a mut self) -> std::result::Result<(), ()> {
let m = M{};
Client::connection_retrier(
Client::send_and_expect,
&mut *self, &m).await
}
}
Error:
Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
--> src/lib.rs:31:21
|
11 | ) -> std::result::Result<(), ()> {
| --------------------------- the `Output` of this `async fn`'s found opaque type
...
31 | Client::send_and_expect,
| ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected fn pointer `for<'r> fn(&mut Client, &'r M) -> impl futures::Future`
found fn pointer `for<'a, '_> fn(&'a mut Client, &M) -> impl futures::Future
Playground
I'm now completely confused about why in for<'r> fn(&mut Client, &'r M) -> impl futures::Future, &mut Client has no lifetime. And what does _ means in for<'a, '_> fn(&'a mut Client, &M) -> impl futures::Future`?
I'm very intersted in learning what's happening here.
Cleaning up the code
In order to see this code from the compiler's perspective, let's change all the lifetime parameters in this example to be explicit and have distinct names, expand the async fn sugar, and then look at how the error changes.
The above example is equivalent to the following after lifetime inference and desugaring.
// name the second lifetime, and expand the async fn sugar.
fn send_and_expect<'a, 'b>(&'a mut self, m: &'b M) -> impl Future<Item=Result<(), ()>> + 'a + 'b
{ ... }
// rename 'a to 'c to avoid ambiguous lifetime names
pub fn connection_retrier<'c, T>(
f: for<'d> fn(&'c mut Self, &'d M) -> T, // name the implicit higher-ranked lifetime here
f_self: &'c mut Self,
f_m: &'c M,
)-> BoxFuture<'c, std::result::Result<(), ()>>
where T: Future<Output = std::result::Result<(), ()>> + 'c
{ ... }
// rename 'a to 'e to avoid ambiguous lifetime names
async fn send_with_retry<'e>(&'e mut self) -> std::result::Result<(), ()> {
After making this change, the error becomes:
Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
--> src/lib.rs:31:21
|
11 | ) -> std::result::Result<(), ()> {
| --------------------------- the `Output` of this `async fn`'s found opaque type
...
31 | Client::send_and_expect,
| ^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected fn pointer `for<'d> fn(&mut Client, &'d M) -> impl futures::Future`
found fn pointer `for<'a, 'b> fn(&'a mut Client, &'b M) -> impl futures::Future`
This should clarify your question about '_: it's just the name the compiler gave the inferred second lifetime parameter of send_and_expect. As for the missing lifetime on &mut Client, you can see that it's still missing here. For reasons I do not completely understand, and in ways that depend on the exact error message given, the compiler will sometimes omit concrete lifetimes when printing the type of references, but make no mistake, the lifetime of that reference is 'c.
Solving the error
Onto the actual problem. The signature of f indicates that connection_retrier is expecting a function which (1) takes a reference of lifetime &'c and (2) a reference of any other lifetime, and (3) returns a Future type which will remain valid as long as 'c does, as specified by your where bound.
When we pass send_and_expect to connection_retrier, in order for the signatures to match the compiler is coercing it to the type for<'d> send_and_expect::<'c, 'd>. But that type doesn't meet condition (3) above! Unlike a regular function, the default behavior of an async function is to capture all input lifetimes in its return type, so the return type of for<'d> send_and_expect::<'c, 'd> is in fact impl Future<Item=Result<(), ()>> + 'c + 'd, as you can tell by looking at send_and_expect's expanded signature.
Since this type borrows from the two lifetimes 'c and 'd, and there is no constraint that 'd: 'c (read: 'd outlives 'c), it may not remain valid for the entirety of the lifetime 'c, if 'd ends first. It is this mismatch that results in the rather cryptic lifetime error you received. There are two ways you can solve this problem, depending on your preferred semantics. You can either:
Remove the higher-ranked bound from f entirely, and specify exactly the lifetimes you will be calling it with in connection_retrier (both &'c.)
pub fn connection_retrier<'c, T>(
f: fn(&'c mut Self, &'c M) -> T
f_self: &'c mut Self,
f_m: &'c M,
) -> BoxFuture<'c, std::result::Result<(), ()>>
where T: Future<Output = std::result::Result<(), ()>> + 'c
{ ... }
Keep the signature of connection_retrier the same and specify that the future returned by send_and_expect only borrows from its first argument. To do this, you will need to drop the async fn sugar on the signature and wrap the body in an async move block.
fn send_and_expect<'a, 'b>(&'a mut self, m: &'b M) -> impl Future<Output=Result<(), ()>> + 'a
{ async move { ... } }
Note that you cannot solve this by writing the type of f as for<'d: 'c> fn(&'c mut Self, &'d M) -> T, as bounds are currently not permitted for universally quantified lifetimes.

How to indicate that the lifetime of an async function's return value is the same as a parameter?

We can match a normal function with non-static parameter like this:
fn processor(data: &i32) -> &i32 {
data
}
fn process<'b>(data: &'b i32, processor: impl 'static + for<'a> Fn(&'a i32) -> &'a i32) -> &'b i32 {
processor(data)
}
fn main() {
let data = 1;
println!("data: {}", process(&data, processor));
}
Since async functions return anonymous futures, we cannot indicate that the lifetime of anonymous future is the same as the parameter:
use std::future::Future;
async fn processor(data: &i32) -> &i32 {
data
}
async fn process<'b, F>(data: &'b i32, processor: impl 'static + Fn(&i32) -> F) -> &'b i32
where
F: 'b + Future<Output = &'b i32>,
{
processor(data).await
}
async fn _main() {
let data = 1;
println!("data: {}", process(&data, processor).await);
}
The compiler will complain:
error[E0271]: type mismatch resolving `for<'r> <for<'_> fn(&i32) -> impl std::future::Future {processor} as std::ops::FnOnce<(&'r i32,)>>::Output == _`
--> src/lib.rs:16:26
|
7 | async fn process<'b, F>(data: &'b i32, processor: impl 'static + Fn(&i32) -> F) -> &'b i32
| ------- - required by this bound in `process`
...
16 | println!("data: {}", process(&data, processor).await);
| ^^^^^^^ expected bound lifetime parameter, found concrete lifetime
How can I match it?
You need to state that:
the closure accepts a reference with the same lifetime as the parameter.
the returned future returns a reference with the same lifetime as the parameter.
the returned future captures a reference with the same lifetime as the parameter.
async fn process<'b, F, Fut>(data: &'b i32, processor: F) -> &'b i32
where
F: Fn(&'b i32) -> Fut,
// ^^ [1]
F: 'static,
Fut: Future<Output = &'b i32> + 'b,
// ^^ [2] ^^ [3]
{
processor(data).await
}

Resources