In the following example, I don't understand why the first example allows the closure to be Fn, but the second only FnOnce.
Example 1:
fn with_client(
client: BasicClient,
) -> impl Filter<Extract = (BasicClient,), Error = Infallible> + Clone {
warp::any().map(move || client.clone())
}
Example 2:
fn with_client(
client: BasicClient,
) -> impl Filter<Extract = (BasicClient,), Error = Infallible> + Clone {
let clone = client.clone();
warp::any().map(move || clone)
}
If I try and run the second example, I get a compiler warning about the closure being FnOnce as it needs to move clone into the closure. This is certainly the case, but I can't see how this differs from the first example, where we need to move client into the closure?
Let's desugar the closures manually.
struct Closure1 {
client: BasicClient,
}
impl FnOnce<()> for Closure1 {
type Output = BasicClient;
extern "rust-call" fn call_once(self, (): ()) -> BasicClient {
<Self as Fn<()>>::call(&self, ())
}
}
impl FnMut<()> for Closure1 {
extern "rust-call" fn call_mut(&mut self, (): ()) -> BasicClient {
<Self as Fn<()>>::call(&*self, ())
}
}
impl Fn<()> for Closure1 {
extern "rust-call" fn call(&self, (): ()) -> BasicClient {
<BasicClient as Clone>::clone(&self.client)
}
}
struct Closure2 {
client: BasicClient,
}
impl FnOnce<()> for Closure2 {
type Output = BasicClient;
extern "rust-call" fn call_once(self, (): ()) -> BasicClient {
self.client
}
}
fn with_client(
client: BasicClient,
) -> impl Filter<Extract = (BasicClient,), Error = Infallible> + Clone {
warp::any().map(Closure1 { client })
}
fn with_client(
client: BasicClient,
) -> impl Filter<Extract = (BasicClient,), Error = Infallible> + Clone {
let clone = <BaseClient as Clone>::clone(&clone);
warp::any().map(Closure1 { client: clone })
}
Like you see, the problem is not that we move client into the closure. The problem is that we move it out of the closure. When you move out of something, you must have ownership of the data: thus, FnOnce. The fact that you're creating the closure with a clone does not matter. But when you clone inside the closure, you only use shared references - and thus you don't need ownership, or even exclusive access - thus Fn.
Basically because clone() is fn clone(&self) -> Self;, only a reference of client is moved into the closure. Therefore in example 1 the closure can be called multiple times. In example 2 the object clone itself is moved into the closure. Since the object is only created once, the closure is FnOnce.
If BasicClient has another method, say fn not_clone(self); , and example 1 calls warp::any().map(move || client.not_clone()), the closure will also be a FnOnce, because the method consumes self. client itself, not a reference, is moved into the closure. However if BasicClient implements Copy trait in addition to Clone the compiler will makes a copy and moves the copy into the closure for each invocation of the closure. The not_clone closure, as well as the closure in example 2, will be Fn.
Related
While trying to implement a simple web server application using actix-web, I encountered apparently inconsistent behavior of Rust closures that do not know how to explain.
I had the following code:
use actix_web::{web, App, HttpServer};
#[derive(Clone)]
struct Config {
val1: String,
val2: String,
val3: String,
}
fn main() {
let conf = Config {
val1: "just".to_string(),
val2: "some".to_string(),
val3: "data".to_string(),
};
HttpServer::new(move ||
App::new().configure(create_config(&conf))
)
.bind("127.0.0.1:8088")
.unwrap()
.run()
.unwrap();
}
fn create_config<'a>(conf: &'a Config) -> impl FnOnce(&mut web::ServiceConfig) + 'a {
move |app: &mut web::ServiceConfig| {
// Have to clone config because web::get().to by definition requires
// its argument to have static lifetime, which is longer than 'a
let my_own_conf_clone = conf.clone();
app.service(
web::scope("/user")
.route("", web::get().to(move || get_user(&my_own_conf_clone)))
);
}
}
fn get_user(conf: &Config) -> String {
println!("Config {} is {} here!", conf.val3, conf.val1);
"User McUser".to_string()
}
This code works. Notice the closure I pass to web::get().to. I used it to pass the Config object down to get_user and still present web::get().to with a function that has no arguments, as it requires. At this point I decided to move the closure generation to a separate function:
fn create_config<'a>(conf: &'a Config) -> impl FnOnce(&mut web::ServiceConfig) + 'a {
move |app: &mut web::ServiceConfig| {
app.service(
web::scope("/user")
.route("", web::get().to(gen_get_user(conf)))
);
}
}
fn gen_get_user(conf: &Config) -> impl Fn() -> String {
let my_own_conf_clone = conf.clone();
move || get_user(&my_own_conf_clone)
}
fn get_user(conf: &Config) -> String {
println!("Config {} is {} here!", conf.val3, conf.val1);
"User McUser".to_string()
}
This code fails to compile with the following error:
error[E0277]: the trait bound `impl std::ops::Fn<()>: actix_web::handler::Factory<_, _>` is not satisfied
--> src/main.rs:30:39
|
30 | .route("", web::get().to(gen_get_user(conf)))
| ^^ the trait `actix_web::handler::Factory<_, _>` is not implemented for `impl std::ops::Fn<()>`
Why does it fail in the second case but not in first? Why was the trait Factory satisfied in the first case but isn't in the second? May be it's the factory's (its sources are here) fault? Is there a different way to return a closure, that would work in this situation? Any other approaches you could suggest? (Notice that Factory is not public so I cannot implement it directly myself)
If you wish to fool around with the code, I have it here: https://github.com/yanivmo/rust-closure-experiments
Notice that you can move between the commits to see the code in its working or failing states.
Then using impl Trait as return type all other type information other then that value implements Trait get erased.
In this particular case closure move || get_user(&my_own_conf_clone) implements Fn() -> String and Clone, but after returning Clone get erased.
But since Factory implemented for Fn() -> String + Clone, and not for Fn() -> String return value no longer implement factory.
This can be fixed by changing gen_get_user to
fn gen_get_user(conf: &Config) -> impl Fn() -> String + Clone{
let my_own_conf_clone = conf.clone();
move || get_user(&my_own_conf_clone)
}
How can I reduce the lifetime of a closure?
I was trying to make a method, which returns an iterator related to self. I didn't want to make new struct or something, so I just made it return filters and maps, and confronted some borrow checker errors.
The following code was my first try.
fn f<'b>(&'b self) -> impl Iterator<Item = u8> {
(0..self.some_number())
.filter(|&i| self.some_bool_function(i))
.map(|i| i as u8)
}
The following code replicates my question.
struct A(bool);
impl A {
fn f<'a>(&'a self) -> impl Iterator<Item = u8> + 'a {
(0..1).filter(|&i| self.0)
}
}
or even shorter,
fn g<'a>(t:&'a ()) -> impl 'a + FnMut() {
|| *t
}
This would not compile, because the closure may outlive self. I don't know how to make this work, without moving self.
If you return a closure, you must ensure that the closure has everything it needs - even after returning (i.e. after the (temporary) function parameters are popped from the stack).
Thus, I think you want to move the stuff you return into the closure:
impl A {
fn f<'a>(&'a self) -> impl Iterator<Item = u8> + 'a {
(0..1).filter(move |&i| self.0)
}
}
Resp.
fn g<'a>(t:&'a ()) -> impl 'a + FnMut() {
move || *t
}
Resp (extending your first example):
struct A(bool);
impl A {
fn some_number(&self) -> usize {
6
}
fn some_bool_function(&self, i: usize) -> bool {
i%2==0
}
fn f<'b>(&'b self) -> impl Iterator<Item = u8> + 'b {
(0..self.some_number())
.filter(move |&i| self.some_bool_function(i))
.map(|i| i as u8)
}
}
The tokio::fs::File::open(path: T + 'static) requires a 'static lifetime on its path parameter.
This makes sense because it is handled in runtime threads during the program's execution. I think it would make more sense if you could pass your own lifetimes, because the runtime does not need to run the whole time and so you could throw away some stuff. Do I understand something wrong?
I'd like to stay for 'static at the moment and so my problem is this...
I have a trait TraitN and some struct StructX { path: String, } with a fn new(path: &String) -> Box<TraitN>. The new creates and sets self.path = path.to_string();.
In some impl fn doit(&self) { ... } for StructX, I'd like to call tokio::fs::File::open(&self.path).
How can I pass &self.path with a 'static lifetime?
This is a complete example:
extern crate futures;
extern crate tokio;
#[macro_use]
extern crate error_chain;
use futures::future;
use futures::future::{loop_fn, ok, Future, Loop};
use futures::Stream;
use std::io::BufReader;
use tokio::{fs, io};
mod error {
error_chain!{}
}
use error::*;
type FutureResult<T> = future::FutureResult<T, Error>;
trait HandlerTrait {
fn new(path: &str) -> Box<HandlerTrait>
where
Self: Sized;
fn get_all(&self) -> FutureResult<Vec<String>>;
}
#[derive(Debug)]
pub struct Handler {
path: String,
}
impl HandlerTrait for Handler {
fn new(path: &str) -> Box<HandlerTrait> {
Box::new(Handler {
path: path.to_string(),
})
}
fn get_all(&self) -> FutureResult<Vec<String>> {
let file = fs::File::open(self.path.clone())
.and_then(|file: fs::File| ok(file))
.wait()
.unwrap();
let lines = io::lines(BufReader::new(file));
ok(lines
.filter(|line| line.len() > 80)
.map(|all| all[0..80].to_string())
.collect()
.wait()
.unwrap())
}
}
fn get_handler(path: &str) -> Option<Box<HandlerTrait>> {
Some(Handler::new(path))
}
fn get_path() -> FutureResult<String> {
ok("./somepath/file".to_string())
}
fn start_runtime() -> Result<()> {
let path: &str = get_path().wait().unwrap().as_str();
tokio::run(doit(path.clone()));
Ok(())
}
fn doit(path: &'static str) -> impl Future<Item = (), Error = ()> + 'static {
let n = 0;
loop_fn(n, move |_nr| {
let lh = get_handler(path).unwrap();
lh.get_all()
.or_else(|_| Err(()))
.and_then(|_all| ok(Loop::Break(())))
})
}
#[test]
fn test() {
start_runtime().unwrap();
assert!(true);
}
error[E0597]: borrowed value does not live long enough
--> src/lib.rs:63:22
|
63 | let path: &str = get_path().wait().unwrap().as_str();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ - temporary value only lives until here
| |
| temporary value does not live long enough
|
= note: borrowed value must be valid for the static lifetime...
playground
TL;DR Use a String instead of a &str. This might change when async / await syntax is stabilized.
Here's the MCVE I made of your original question:
extern crate tokio; // 0.1.11
trait TraitN {}
struct StructX {
path: String,
}
impl TraitN for StructX {}
fn new(path: &str) -> Box<TraitN> {
Box::new(StructX {
path: path.to_string(),
})
}
impl StructX {
fn doit(&self) {
tokio::fs::File::open(self.path.clone());
}
}
To solve this, clone the String and give ownership of it to the function:
impl StructX {
fn doit(&self) {
tokio::fs::File::open(self.path.clone());
}
}
With your example code, there are numerous problems:
fn start_runtime() -> Result<()> {
let path: &str = get_path().wait().unwrap().as_str();
tokio::run(doit(path.clone()));
Ok(())
}
You cannot take a reference to the result of unwrap because nothing will own that value. You cannot have a reference to this kind of temporary.
Cloning a &'a str returns a &'a str, not a String.
It doesn't make sense to call wait on the value because that blocks the thread. Run everything in the reactor loop.
This function should look like
fn start_runtime() -> Result<()> {
tokio::run({
get_path()
.map_err(|e| panic!("{}", e))
.and_then(|path| doit(path))
});
Ok(())
}
Then all of your code should switch to impl Into<String> instead of &str of &'static str. doit also needs to be able to create duplicate Strings:
fn doit(path: impl Into<String> + Clone) -> impl Future<Item = (), Error = ()> + 'static {
let n = 0;
let path = path.into();
loop_fn(n, move |_nr| {
let lh = get_handler(path.clone()).unwrap();
lh.get_all()
.or_else(|_| Err(()))
.and_then(|_all| ok(Loop::Break(())))
})
}
this [...] is config which doesn't change [...] read from a configfile during app init.
In that case, create a singleton which will give you an effectively-static value:
extern crate lazy_static; // 1.1.0
use lazy_static::lazy_static;
lazy_static! {
static ref PATH: String = {
// Should be read from a file.
String::from("/the/path/to/the/thing")
};
}
Then change all of the values to &'static str:
#[derive(Debug)]
pub struct Handler {
path: &'static str,
}
impl HandlerTrait for Handler {
fn new(path: &'static str) -> Box<HandlerTrait> {
Box::new(Handler {
path
})
}
}
And take a reference to the singleton:
fn start_runtime() -> Result<()> {
tokio::run(doit(&PATH));
Ok(())
}
You can couple this with phimuemue's answer to get a &'static MyConfigStruct, which could then have a fn foo(&'static self) that is available.
There must be something wrong with a language if this becomes so difficult and needs mem-io multiple times.
You are partially correct. It's difficult to have maximally performant async code with today's Rust (1.30) because Rust wants to ensure memory safety above all else. This doesn't mean that the code is unperformant, just that there's a bit of room to do better.
Honestly, making clones here is unlikely to be a performance bottleneck, but it is annoying. That's where async and await syntax comes in. This will allow futures to more easily make use of references in an idiomatic Rust manner.
because the runtime does not need to run the whole time [...] Do I understand something wrong?
However, async and await still might not help you, as by default Tokio will run your future on a different thread. That's one of the primary reasons it requires a 'static bound. This prevents a Tokio thread from having a reference to a stack local that goes out of scope, introducing memory unsafety. This isn't a unique problem to Tokio, however.
See also:
How can I pass a reference to a stack variable to a thread?
The compiler suggests I add a 'static lifetime because the parameter type may not live long enough, but I don't think that's what I want
Why is the bound `T: 'a` required in order to store a reference `&'a T`?
Why does the Rust compiler request I constrain a generic type parameter's lifetime (error E0309)?
Other bits
It appears that every single call to wait in this code is a misuse of futures. You may wish to re-read the Tokio docs to better understand understand how you are supposed to chain futures. If there's a wait call, it's usually at the end of everything, and even that is rare when using Tokio.
See also:
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
You can restrict the lifetime of &self:
impl StructX {
fn doit(&'static self) {
// here, we know that self and its members are 'static
}
}
If you do this, you may actually be better off to have StructX store the 'static borrow of the path in the first place (instead of a string).
I can answer this now by myself:
fn make_static_str<T>(s: T) -> &'static str where T: Into<String>
A solution with Arc<Mutex<String>> - The test fails because there is no file to read on playground.
There are already a lot of threads on this topic but I fail to see if the discussed problems apply to my specific problem.
I have a structure that stores a name and a callback function. Stripped down to the problem it looks like this:
pub struct Command<'a> {
name: &'a str,
callback: &'a Fn(&[&str]) -> ()
}
impl <'a> Command<'a> {
pub fn new(name: &'a str, callback: &'a Fn(&[&str]) -> ()) -> Command<'a> {
Command {
name: name,
callback: callback
}
}
}
What I want to do is store a callback function associated with a name (and prob. more stuff in the future).
But when I try to use this bit of code like so:
fn main() {
let play_callback = |args| {
println!("Playing something.");
for arg in args {
println!("{}", arg);
}
};
let play_command = Command::new("play", &play_callback);
}
I get the following error message:
src/main.rs:22:42: 22:56 error: type mismatch resolving `for<'r, 'r> <[closure#src/main.rs:16:22: 21:3] as std::ops::FnOnce<(&'r [&'r str],)>>::Output == ()`:
expected bound lifetime parameter ,
found concrete lifetime [E0271]
src/main.rs:22 let play_command = Command::new("play", &play_callback);
^~~~~~~~~~~~~~
I tried to inline the closure like this
fn main() {
let play_command = Command::new("play", &|args| {
println!("Playing something.");
for arg in args {
println!("{}", arg);
}
});
}
but then I get another error
src/main.rs:16:47: 21:7 error: borrowed value does not live long enough
which I believe I understand why I get.
I tried using a generic type parameter for Command before switching to a function reference first to store in my Command structure, but when I wanted to initialize a HashSet of command objects like this:
let mut commands: HashSet<Command> = HashSet::new();
the compiler wanted me to specify the generic parameter which I think I cannot do as that would mean I could only store the same closure in all my Command objects.
So my question would be: How can I achieve what I want and what is the best way to do so (and why)?
The easy fix (by ljedrz) is that args: &[&str] is not inferred in this case. However, it may not be enough to solve your problem.
When you use a reference to some trait as a function argument, it is treated as a trait object. In this case, the &Fn is a trait object that references the closure on the stack.
An easy analogy of trait objects are objects that implement interfaces in other languages.
However, lifetimes work a bit differently with the trait objects. You can think of them as "detached" from the usual ownership flow. If we were to annotate the Fn trait object lifetime 'c in your example, we would get the following code:
pub struct Command<'a> {
name: &'a str,
callback: &'a for<'c> Fn(&'c [&'c str]) -> ()
}
impl <'a> Command<'a> {
pub fn new<'r>(name: &'r str, callback: &'r for<'c> Fn(&'c [&'c str]) -> ()) -> Command<'r> {
Command {
name: name,
callback: callback
}
}
}
fn main() {
let play_callback = |args: &[&str]| {
println!("Playing something.");
for arg in args {
println!("{}", arg);
}
};
let play_command = Command::new("play", &play_callback);
}
In the code above, the lifetime 'c describes a scope in which the callback function will be called.
The above code, however, is not very practical. It couples the command to the scope in which the closure was created (remember, the trait object references the closure in that scope). So you can't exit from the function in which your Command was created, because that would destroy the closure!
The likely solution is to store the closure on the heap, in Box<Fn(&[&str])>. The lifetime of a trait object in the box (heap memory) is controlled by the creation and destruction of the box, so it is the broadest possible ('static).
pub struct Command<'a> {
name: &'a str,
callback: Box<Fn(&[&str]) -> ()>
}
impl <'a> Command<'a> {
pub fn new<'r>(name: &'r str, callback: Box<Fn(&[&str]) -> ()>) -> Command<'r> {
Command {
name: name,
callback: callback
}
}
}
fn main() {
let play_callback = |args: &[&str]| {
println!("Playing something.");
for arg in args {
println!("{}", arg);
}
};
let play_command = Command::new("play", Box::new(play_callback));
}
In the example above, the closure will be moved into the box when the box is created, and will be destroyed together with Command.
Have you tried specifying the type of args? The following compiles for me:
fn main() {
let play_callback = |args: &[&str]| {
println!("Playing something.");
for arg in args {
println!("{}", arg);
}
};
let play_command = Command::new("play", &play_callback);
}
I don't know why it is not inferred, though.
I have two types: Lexer and SFunction.
SFunction stands for stateful function and is definined like so:
struct SFunction {
f: Option<Box<FnMut() -> SFunction>>,
}
The important part is that any SFunction references a closure that returns an SFunction.
Now I want to have these functions carry state by each affecting the same Lexer. This means that each of these SFunctions has to have a lifetime that depends on a specific Lexer.
Here's some more code if you want to get more of a sense of what I'm doing with this:
impl Lexer {
fn lex(&mut self) {
self.sfunction(Lexer::lexNormal).call()
}
fn sfunction(&mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(|| f(self)))
// SFunction { f: Some(Box::new(move ||f(self))) }
}
fn lexNormal(&mut self) -> SFunction {
return SFunction::empty()
}
}
(Here’s a full version of the code in the Rust playground.)
How do I specify this lifetime requirement in the code?
The compiler errors I'm getting say "cannot infer an appropriate lifetime for capture of self by closure due to conflicting requirements". I'm pretty sure the "conflicting requirements" here is that a Box type assumes the lifetime to be 'static. I could do something like Box<FnMut() -> SFunction + 'a> where 'a is a lifetime defined by the Lexer it depends upon, but I'm not sure how to define such an 'a.
Thanks for your help!
The problem is in this line:
SFunction::new(Box::new(|| f(self)))
Here, self is a reference to a Lexer, but there's no guarantee that the lexer will live long enough. In fact, it needs to live for the 'static lifetime! Without a lifetime specified, a boxed trait object will use the 'static lifetime. Said in code, these two declarations are equivalent:
<Box<FnMut() -> SFunction>>
<Box<FnMut() -> SFunction> + 'static>
And you can make your code compile (in an unsatisfactory way) by restricting it to accept only references that will live for the 'static lifetime:
fn lex(&'static mut self) {
self.sfunction(Lexer::lex_normal).call()
}
fn sfunction(&'static mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(move || f(self)))
}
Of course, it's very doubtful that you will have a Lexer with the static lifetime, as that would mean that it's lexing static data, which wouldn't be very useful. That means we need to include lifetimes in your trait object... as you suggested.
Ultimately what helped to see the problem was to restructure your closure a bit:
fn sfunction(&mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(move || {
// f(self)
let s2 = self;
let f2 = f;
f2(s2)
}))
}
Compiling this produces an error that points to what seems to be the real problem:
<anon>:31:22: 31:26 error: cannot move out of captured outer variable in an `FnMut` closure [E0507]
<anon>:31 let s2 = self;
^~~~
<anon>:31:17: 31:19 note: attempting to move value to here
<anon>:31 let s2 = self;
^~
<anon>:31:17: 31:19 help: to prevent the move, use `ref s2` or `ref mut s2` to capture value by reference
I believe this is because a FnMut closure may be called multiple times, which would mean that the reference enclosed in the closure would need to be copied around, which would be bad news as &mut references should be unique.
All together, this code works:
struct SFunction<'a> {
f: Option<Box<FnOnce() -> SFunction<'a> + 'a>>,
}
impl<'a> SFunction<'a> {
fn new(f: Box<FnOnce() -> SFunction<'a> + 'a>) -> SFunction<'a> {
SFunction {
f: Some(f),
}
}
fn empty() -> SFunction<'a> {
SFunction {
f: None,
}
}
fn call(self) { }
}
struct Lexer;
impl Lexer {
fn lex(&mut self) {
self.sfunction(Lexer::lex_normal).call()
}
fn sfunction(&mut self, f: fn(&mut Lexer) -> SFunction) -> SFunction {
SFunction::new(Box::new(move || f(self)))
}
fn lex_normal<'z>(&'z mut self) -> SFunction<'z> {
SFunction::empty()
}
}
fn main() {
let mut l = Lexer;
l.lex()
}
I hope my explanation is right and that the changed code still suits your use case!