Sharing a variable with a closure - rust

I know, the answers for similar questions already exist... kinda... all that I found were very high level instructions rather than code examples and I couldn't really follow them and I've been stuck in place for days now...
I need to share a variable with a closure. So I have my MockSlave struct. I want it to create a vector that can then be shared with the shell closure. The current version looks something like this
impl MockSlave {
pub async fn new(port: i32) -> Result<Self, Box<dyn std::error::Error>> {
let addr = String::from(format!("https://127.0.0.1:{}", port).as_str());
let endpoint = Endpoint::from_str(addr.as_str())?;
let commands = Rc::new(RefCell::new(Vec::new()));
let client = SlaveClient::new(endpoint, CommandProcessors {
shell: Some(|command| {
commands.borrow_mut().push(command);
Ok(())
})
}).await?;
Ok(MockSlave {
client,
commands
})
}
For clarity, here's SlaveClient::new
pub async fn new(endpoint: Endpoint, processors: CommandProcessors) -> Result<Self, tonic::transport::Error> {
let slave = SlaveClient {
client: SlaveManagerClient::connect(endpoint).await?,
processors
};
Ok(slave)
}
And here's CommandProcessors:
pub struct ReadWriteStreamingCommand {
pub command: Command
}
pub type ReadWriteSteamingCommandprocessor = fn(Box<ReadWriteStreamingCommand>) -> Result<(), Box<dyn Error>>;
pub struct CommandProcessors {
pub shell: Option<ReadWriteSteamingCommandprocessor>
}
Now, the fun part is that VSCode and cargo build give me slightly different errors about the same thing.
VSCode says:
closures can only be coerced to `fn` types if they do not capture any variables
Meanwhile, cargo build says
error[E0308]: mismatched types
--> tests/api_tests/src/mock_slave.rs:20:25
|
20 | shell: Some(|command| {
| _________________________^
21 | | commands.borrow_mut().push(command);
22 | | Ok(())
23 | | })
| |_____________^ expected fn pointer, found closure
Please help. This is at least my 10th approach to this problem over the course of a week. I am not going to move forward by myself...

fn(Box<ReadWriteStreamingCommand>) -> Result<(), Box<dyn Error>>;
This is not the type you think it is. This is the type of pointers to functions that exist statically in your code. That is, functions defined at the top-level or closures which don't actually close over any variables. To accept general Rust callables (including traditional functions and closures), you need one of Fn, FnMut, or FnOnce.
I can't tell from the small code you've shown which of the three you need, but the basic idea is this.
Use FnOnce if you are only going to call the function at most once. This is useful if you're writing some kind of "control flow" type of construct or mapping on the inside of an Option (where at most one value exists).
Use FnMut if you are going to call the function several times but never concurrently. This is what's used by most Rust built-in collection functions like map and filter, since it's always called in a single thread. This is the one I find myself using the most often for general-purpose things.
Use Fn if you need concurrent access and are planning to share across threads.
Every Fn is an FnMut (any function that can be called concurrently can be called sequentially), and every FnMut is an FnOnce (any function that can be called can obviously be called once), so you should choose the highest one on the list above that you can handle.
I'll assume you want FnMut. If you decide one of the others is better, the syntax is the same, but you'll just need to change the name. The type you want, then, is
dyn FnMut(Box<ReadWriteStreamingCommand>) -> Result<(), Box<dyn Error>>
But this is a trait object and hence is not Sized, so you can't do much with it. In particular, you definitely can't put it in another Sized data structure. Instead, we'll wrap it in a Box.
pub type ReadWriteSteamingCommandprocessor = Box<dyn FnMut(Box<ReadWriteStreamingCommand>) -> Result<(), Box<dyn Error>>>;
And then
shell: Some(Box::new(|command| {
commands.borrow_mut().push(command);
Ok(())
}))

Related

Rust Generics, "`T` may not live long enough"

I'm trying to imbed Lua into Rust using Mlua. Using its guided tour, I successfully loaded and evaluated a chunk of Lua code. My next goal is to combine this with opening a file and reading its contents. This is where things get problematic:
use anyhow::Result;
use mlua::{Lua, UserData};
use std::fs;
pub struct LuaManager {
lua_state: Lua
}
impl LuaManager {
pub fn new() -> Self {
LuaManager {
lua_state: Lua::new(),
}
}
pub fn eval<T>(&self, chunk: &str) -> Result<T> where T: Clone + UserData {
let v = self.lua_state.load(chunk).eval::<T>()?;
Ok(v)
}
}
Since scripts can return literally anything, I thought a trait-locked generic annotation would solve the issue: The problem is this:
Error generated by rustc: the parameter type `T` may not live long enough... so that the type `T` will meet its required lifetime bounds...
For reference, here is the function signature as defined in lua.rs:
pub fn eval<R: FromLuaMulti<'lua>>(self) -> Result<R>
I don't understand why I'm having this issue. I've looked this up and this question pertained to user-created types, which to my knowledge I'm not doing... so what is the problem here? It's clear that things like <i32> won't die anytime soon. Cargo did suggest adding a static lifetime annotation, but doing some research into it points me to trying to solve the problem without using it first, since in 99% of cases you actually can get away without it.
Unfortunately in this case it seems like 'static is really required.
Lua::load() takes &'lua self and returns Chunk<'lua, '_>. Chunk::<'lua, '_>::eval::<R>() requires R: FromLuaMulti<'lua>. FromLuaMulti<'lua> is implemented for T: FromLua<'lua>. FromLua<'lua> is implemented for T: UserData, however it requires T: 'static (and also T: Clone, but this is irrelevant).

How to store async closure created at runtime in a struct?

I'm learning Rust's async/await feature, and stuck with the following task. I would like to:
Create an async closure (or better to say async block) at runtime;
Pass created closure to constructor of some struct and store it;
Execute created closure later.
Looking through similar questions I wrote the following code:
use tokio;
use std::pin::Pin;
use std::future::Future;
struct Services {
s1: Box<dyn FnOnce(&mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()>>>>,
}
impl Services {
fn new(f: Box<dyn FnOnce(&mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()>>>>) -> Self {
Services { s1: f }
}
}
enum NumberOperation {
AddOne,
MinusOne
}
#[tokio::main]
async fn main() {
let mut input = vec![1,2,3];
let op = NumberOperation::AddOne;
let s = Services::new(Box::new(|numbers: &mut Vec<usize>| Box::pin(async move {
for n in numbers {
match op {
NumberOperation::AddOne => *n = *n + 1,
NumberOperation::MinusOne => *n = *n - 1,
};
}
})));
(s.s1)(&mut input).await;
assert_eq!(input, vec![2,3,4]);
}
But above code won't compile, because of invalid lifetimes.
How to specify lifetimes to make above example compile (so Rust will know that async closure should live as long as input). As I understand in provided example Rust requires closure to have static lifetime?
Also it's not clear why do we have to use Pin<Box> as return type?
Is it possible somehow to refactor code and eliminate: Box::new(|arg: T| Box::pin(async move {}))? Maybe there is some crate?
Thanks
Update
There is similar question How can I store an async function in a struct and call it from a struct instance?
. Although that's a similar question and actually my example is based on one of the answers from that question. Second answer contains information about closures created at runtime, but seems it works only when I pass an owned variable, but in my example I would like to pass to closure created at runtime mutable reference, not owned variable.
How to specify lifetimes to make above example compile (so Rust will know that async closure should live as long as input). As I understand in provided example Rust requires closure to have static lifetime?
Let's take a closer look at what happens when you invoke the closure:
(s.s1)(&mut input).await;
// ^^^^^^^^^^^^^^^^^^
// closure invocation
The closure immediately returns a future. You could assign that future to a variable and hold on to it until later:
let future = (s.s1)(&mut input);
// do some other stuff
future.await;
The problem is, because the future is boxed, it could be held around for the rest of the program's life without ever being driven to completion; that is, it could have 'static lifetime. And input must obviously remain borrowed until the future resolves: else imagine, for example, what would happen if "some other stuff" above involved modifying, moving or even dropping input—consider what would then happen when the future is run?
One solution would be to pass ownership of the Vec into the closure and then return it again from the future:
let s = Services::new(Box::new(move |mut numbers| Box::pin(async move {
for n in &mut numbers {
match op {
NumberOperation::AddOne => *n = *n + 1,
NumberOperation::MinusOne => *n = *n - 1,
};
}
numbers
})));
let output = (s.s1)(input).await;
assert_eq!(output, vec![2,3,4]);
See it on the playground.
#kmdreko's answer shows how you can instead actually tie the lifetime of the borrow to that of the returned future.
Also it's not clear why do we have to use Pin as return type?
Let's look at a stupidly simple async block:
async {
let mut x = 123;
let r = &mut x;
some_async_fn().await;
*r += 1;
x
}
Notice that execution may pause at the await. When that happens, the incumbent values of x and r must be stored temporarily (in the Future object: it's just a struct, in this case with fields for x and r). But r is a reference to another field in the same struct! If the future were then moved from its current location to somewhere else in memory, r would still refer to the old location of x and not the new one. Undefined Behaviour. Bad bad bad.
You may have observed that the future can also hold references to things that are stored elsewhere, such as the &mut input in #kmdreko's answer; because they are borrowed, those also cannot be moved for the duration of the borrow. So why can't the immovability of the future similarly be enforced by r's borrowing of x, without pinning? Well, the future's lifetime would then depend on its content—and such circularities are impossible in Rust.
This, generally, is the problem with self-referential data structures. Rust's solution is to prevent them from being moved: that is, to "pin" them.
Is it possible somehow to refactor code and eliminate: Box::new(|arg: T| Box::pin(async move {}))? Maybe there is some crate?
In your specific example, the closure and future can reside on the stack and you can simply get rid of all the boxing and pinning (the borrow-checker can ensure stack items don’t move without explicit pinning). However, if you want to return the Services from a function, you'll run into difficulties stating its type parameters: impl Trait would normally be your go-to solution for this type of problem, but it's limited and does not (currently) extend to associated types, such as that of the returned future.
There are work-arounds, but using boxed trait objects is often the most practical solution—albeit it introduces heap allocations and an additional layer of indirection with commensurate runtime cost. Such trait objects are however unavoidable where a single instance of your Services structure may hold different closures in s1 over the course of its life, where you're returning them from trait methods (which currently can’t use impl Trait), or where you're interfacing with a library that does not provide any alternative.
If you want your example to work as is, the missing component is communicating to the compiler what lifetime associations are allowed. Trait objects like dyn Future<...> are constrained to be 'static by default, which means it cannot have references to non-static objects. This is a problem because your closure returns a Future that needs to keep a reference to numbers in order to work.
The direct fix is to annotate that the dyn FnOnce can return a Future that can be bound to the life of the first parameter. This requires a higher-ranked trait bound and the syntax looks like for<'a>:
struct Services {
s1: Box<dyn for<'a> FnOnce(&'a mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()> + 'a>>>,
}
impl Services {
fn new(f: Box<dyn for<'a> FnOnce(&'a mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()> + 'a>>>) -> Self {
Services { s1: f }
}
}
The rest of your code now compiles without modification, check it out on the playground.

Is there any way to implement Into<&str> or From<T> for &str?

Was writing some code in Rust trying to define a CLI, using the (otherwise pretty great) crate clap, and run into a rather critical issue. Methods of App accept an Into<&'help str>, and i've been unable to find a way to implement this trait.
Indeed from what i understand, it is absolutely unimplementable:
struct JustWorkDamnIt {
string: String
}
impl From<JustWorkDamnIt> for &str {
fn from(arg: JustWorkDamnIt) -> Self {
return arg.string.as_str()
}
}
...which yields:
error[E0515]: cannot return value referencing local data `arg.string`
--> src/cmd/interactive.rs:25:16
|
25 | return arg.string.as_str()
| ----------^^^^^^^^^
| |
| returns a value referencing data owned by the current function
| `arg.string` is borrowed here
Interestingly enough, however, returning a literal compiles just fine (which i reckon is why clap doesn't mind using this trait). Presumably that's because the literal is compiled to some static region of memory and therefore isn't owned by the function:
impl From<JustWorkDamnIt> for &str {
fn from(arg: JustWorkDamnIt) -> Self {
return "literal"
}
}
But, i mean, surely there's a way to implement this trait and return dynamic strings? Maybe some clever use of Box<> or something, idk. I believe i've tried all the things i could think of.
And if there isn't a way, then this seems like a pretty glaring flaw for Rust - traits which can be declared and used in function headers, but cannot be actually implemented meaningfully (there's not much of a point in returning a literal). If this turns out to be the case i'll create an issue on the rust-lang repository for this flow, but first i wanted to sanity-check my findings here.
UPD: Thanks for the comments, made me think more in-depth about this issue.
The problem has to do with lifetimes, it seems. I apologize for not showing the entire code, i thought the snippets i shared would describe the problem sufficiently, but in hindsight it does make sense that the context would be important with Rust's lifetimes at play.
I did find a "solution" for this particular instance of the problem. Since the code in question will only run once per executable start, i can just Box::leak the &'static str and call it a day. Still, i would like to figure out if there's a more general solution which could be used for often-created dynamic strings.
impl Cmd for InteractiveCmd {
fn build_args_parser<'a, 'b>(&'b self, builder: App<'a>) -> App<'a> {
// leak() works here but i'm curious if there's a better way
let staticStr : &'static str = Box::leak(Box::from(format!("Interactive {} shell", APPNAME).as_str()));
let rv = builder
// pub fn about<S: Into<&'b str>>(mut self, about: S) -> Self
.about(staticStr)
.version("0.1");
return rv;
}
}
fn main() {
let interactive = InteractiveCmd::new();
let mut app = App::new(APPNAME)
.version(APPVER)
.author(AUTHORS)
.subcommand(interactive.build_args_parser(App::new("interactive")));
}
Currently i am faced with 2 points of confusion here:
Why is there no impl From<String> for &str? The compiler claims that one exists for impl From<String> for &mut str, and i'm not seeing the importance of mut here.
...and if there is a good reason to not impl From<String> for &str, would it make sense to somehow discourage the usage of Into<&str> in the public APIs of libraries?
Or maybe the issue is with my build_args_parser function and it's just not an option to offload the work to it as far as Rust is concerned?
Seems like you're trying to convert a String into a &'a str. You can do it like this:
use clap::App;
fn main() {
let s = format!("Interactive {} shell", "Testing");
let app = App::new("Testing")
.about(&*s); // or `.about(s.as_str());` it's the same
}
Here's the signature of the function that we want to call:
pub fn about<S: Into<&'b str>>(self, about: S) -> Self
So the about parameter must implement trait Into<&'b str>. According to std::convert::Into, we know that &'b str does implement trait Into<&'b str>. So now we just need a way to convert String into &'b str. The std::string::String tell us that there are two ways: use either s.as_str() or &*s.

How to create a function returning a borrowed value? [duplicate]

This question already has answers here:
Is there any way to return a reference to a variable created in a function?
(5 answers)
Closed 4 years ago.
I have the following function as part of a Rust WASM application to convert a Boxed closure into the Rust-representation for a JavaScript function.
use js_sys::Function;
type Callback = Rc<RefCell<Option<Closure<FnMut()>>>>;
fn to_function(callback: &Callback) -> &Function {
callback.borrow().as_ref().unwrap().as_ref().unchecked_ref()
}
However, the compiler complains that the return value uses a borrowed value (obtained with callback.borrow()) so cannot be returned.
Hence, I decided to add lifetime annotations to inform the compiler that this new reference should live as long as the input.
use js_sys::Function;
type Callback = Rc<RefCell<Option<Closure<FnMut()>>>>;
fn to_function<'a>(callback: &'a Callback) -> &'a Function {
callback.borrow().as_ref().unwrap().as_ref().unchecked_ref()
}
Unfortunately, this hasn't helped and I get the same error. What am I doing wrong here?
Yeah, this isn't going to work.
callback.borrow().as_ref().unwrap().as_ref().unchecked_ref()
Let's break this down in steps:
You're borrowing &RefCell<Option<Closure<FnMut()>>> - so you now have Ref<Option<...>>, which is step #1 of your issues. When this happens, this intermediary value now has a different lifetime than 'a (inferior, to be precise). Anything stemming from this will inherit this lesser lifetime. Call it 'b for now
You then as_ref this Ref, turning it into Option<&'b Closure<FnMut()>>
Rust then converts &'b Closure<FnMut()> into &'b Function
Step 1 is where the snafu happens. Due to the lifetime clash, you're left with this mess. A semi-decent way to solve it the following construct:
use std::rc::{Rc};
use std::cell::{RefCell, Ref};
use std::ops::Deref;
struct CC<'a, T> {
inner: &'a Rc<RefCell<T>>,
borrow: Ref<'a, T>
}
impl<'a, T> CC<'a, T> {
pub fn from_callback(item:&'a Rc<RefCell<T>>) -> CC<'a, T> {
CC {
inner: item,
borrow: item.borrow()
}
}
pub fn to_function(&'a self) -> &'a T {
self.borrow.deref()
}
}
It's a bit unwieldy, but it's probably the cleanest way to do so.
A new struct CC is defined, containing a 'a ref to Rc<RefCell<T>> (where the T generic in your case would end up being Option<Closure<FnMut()>>) and a Ref to T with lifetime 'a, auto-populated on the from_callback constructor.
The moment you generate this object, you'll have a Ref with the same lifetime as the ref you gave as an argument, making the entire issue go away. From there, you can call to_function to retrieve a &'a reference to your inner type.
There is a gotcha to this: as long as a single of these objects exists, you will (obviously) not be able to borrow_mut() on the RefCell, which may or may not kill your use case (as one doesn't use a RefCell for the fun of it). Nevertheless, these objects are relatively cheap to instantiate, so you can afford to bin them once you're done with them.
An example with Function and Closure types replaced with u8 (because js_sys cannot be imported into the sandbox) is available here.
Although I really like Sébastien's answer and explanation, I ended up going for Ömer's suggestion of using a macro, simply for the sake of conciseness. I'll post the macro in case it's of use to anyone else.
macro_rules! callback_to_function {
($callback:expr) => {
$callback
.borrow()
.as_ref()
.unwrap()
.as_ref()
.unchecked_ref()
};
}
I'll leave Sébastien's answer as the accepted one as I believe it is the more "correct" way to solve this issue and he provides a great explanation.

Use Trait as Vec Type

I'm new to Rust and have seen some examples of people using Box to allow pushing many types that implement a certain Trait onto a Vec. When using a Trait with Generics, I have run into an issue.
error[E0038]: the trait `collision::collision_detection::Collidable` cannot be made into an object
--> src/collision/collision_detection.rs:19:5
|
19 | collidables: Vec<Box<Collidable<P, M>>>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `collision::collision_detection::Collidable` cannot be made into an object
|
= note: method `get_ncollide_shape` has generic type parameters
error: aborting due to previous error
error: Could not compile `game_proto`.
To learn more, run the command again with --verbose.
Here is my code
extern crate ncollide;
extern crate nalgebra as na;
use self::ncollide::shape::Shape;
use self::ncollide::math::Point;
use self::ncollide::math::Isometry;
use self::na::Isometry2;
pub trait Collidable<P: Point, M> {
fn get_ncollide_shape<T: Shape<P, M>>(&self) -> Box<T>;
fn get_isometry(&self) -> Isometry2<f64>;
}
pub struct CollisionRegistry<P, M>
where
P: Point,
M: Isometry<P>,
{
collidables: Vec<Box<Collidable<P, M>>>,
}
impl<P: Point, M: Isometry<P>> CollisionRegistry<P, M> {
pub fn new() -> Self {
let objs: Vec<Box<Collidable<P, M>>> = Vec::new();
CollisionRegistry { collidables: objs }
}
pub fn register<D>(&mut self, obj: Box<D>)
where
D: Collidable<P, M>,
{
self.collidables.push(obj);
}
}
I'm trying to use collidables as a list of heterogenous game objects that will give me ncollide compatible Shapes back to feed into the collision detection engine.
EDIT:
To clear up some confusion. I'm not trying to construct and return an instance of a Trait. I'm just trying to create a Vec that will allow any instance of the Collidable trait to be pushed onto it.
Rust is a compiled language, so when it compiles your code, it needs to know all of the information it might need to generate machine code.
When you say
trait MyTrait {
fn do_thing() -> Box<u32>;
}
struct Foo {
field: Box<MyTrait>
}
you are telling Rust that Foo will contain a box containing anything implementing MyTrait. By boxing the type, the compiler will erase any additional data about the data type that isn't covered by the trait. These trait objects are implemented as a set of data fields and a table of functions (called a vtable) that contains the functions exposed by the trait, so they can be called.
When you change
fn do_thing() -> Box<u32>;
to
fn do_thing<T>() -> Box<T>;
it may look similar, but the behavior is much different. Let's take a normal function example
fn do_thing<T>(val: T) { }
fn main() {
do_thing(true);
do_thing(45 as u32);
}
the compiler performs what is a called monomorphization, which means your code in the compiler becomes essentially
fn do_thing_bool(val: bool) { }
fn do_thing_num(val: u32) { }
fn main() {
do_thing_bool(true);
do_thing_num(45 as u32);
}
The key thing to realize is that you are asking it to do the same thing for your trait. The problem is that the compiler can't do it. The example above relies on knowing ahead of time that do_thing is called with a number in one case and a boolean in another, and it can know with 100% certainty that those are the only two ways the function is used.
With your code
trait MyTrait {
fn do_thing<T>() -> Box<T>;
}
the compiler does not know what types do_thing will be called with, so it has no way to generate functions you'd need to call. To do that, wherever you convert the struct implementing Collidable into a boxed object it would have to know every possible return type get_ncollide_shape could have, and that is not supported.
Other links for this:
Understanding Traits and Object Safety
https://www.reddit.com/r/rust/comments/3an132/how_to_wrap_a_trait_object_that_has_generic/

Resources