on-the-fly substitution of `Option<Arc<Mutex<Box<dyn T>>>>` - rust

Suppose I have an object video_source: Option<Arc<Mutex<Box<dyn GetVideo>>>> and I pass it to a thread:
std::thread::spawn(||{
loop {
if let Some(video_source) = video_source {
let video_frame = video_source.lock().unwrap().get();
}
}
})
where
trait GetVideo {
fn get() -> Vec<u8>
}
What if I want to change the video source on the fly? Well, I'd do this on another thread:
video_frame.unwrap().lock().unwrap() = Box::new(other_source);
I want to make this idea more generic. I want a type that permits such thing. Here's my sketch:
use std::sync::{Arc, Mutex};
pub type OnTheFlyInner<T> = Box<T + Send + Sync>;
pub type OnTheFly<T> = Arc<Mutex<OnTheFlyInner<T>>>;
//I'd like this to be a method of `OnTheFly`
pub fn on_the_fly_substitute(on_the_fly: &mut Option<OnTheFly>, substitute_by: Option<OnTheFlyInner>) {
if let Some(substitute_by) = substitute_by {
if let Some(on_the_fly) = on_the_fly {
*on_the_fly.lock().unwrap() = substitute_by;
}
} else {
on_the_fly.take();
}
}
However, I cannot make something generic over T where T is a trait, it should be a type.
Any ideas?
Bounty
This is solved by #user4815162342. But what if I want to make one OnTheFly object point to the same thing as the other one?

First, you are correct that T cannnot be a trait like GetVideo; traits are not types. However, T can be dyn GetVideo.
Second, your aliases have generic parameters, so they should be reflected as such in the function signature:
pub fn on_the_fly_substitute<T>(on_the_fly: &mut Option<OnTheFly<T>>, substitute_by: Option<OnTheFlyInner<T>>)
^^^ ^^^ ^^^
Third, your alias looks like an attempt to constrain T to be Send + Sync, but aliases cannot define additional bounds. You would instead put them on the function (with ?Sized since you want to allow trait objects):
pub fn on_the_fly_substitute<T: ?Sized>(on_the_fly: &mut Option<OnTheFly<T>>, substitute_by: Option<OnTheFlyInner<T>>)
where
T: ?Sized + Send + Sync
{
...
}
Note: your function body does not require Send and Sync so these bounds should probably not be included.
Fourth, Option<Arc<Mutex<Box<dyn GetVideo>>>> is not thread safe. You'll need to constrain that the trait object is at least Send:
Option<Arc<Mutex<Box<dyn GetVideo + Send>>>>
^^^^^^
Fifth, a complete example is lacking, but you appear to be wanting multiple threads to modify the same video_source. This would likely not compile since you would need multiple threads to keep a &mut _ in order to change it.
If you want shared ownership of a value that might not exist, move the option into the Mutex and adjust your function and aliases accordingly:
video_source: Arc<Mutex<Option<Box<dyn GetVideo>>>>
Sixth, your comment "I'd like this to be a method of OnTheFly" is misguided. Aliases are just aliases, you'd need a method on the aliased Option/Arc type. Keep it as a free function, introduce an extension trait for it, or create it as a wrapper type instead of an alias if you want more fine-grained control.

Related

Is there a way to reference an anonymous type as a struct member?

Consider the following Rust code:
use std::future::Future;
use std::pin::Pin;
fn main() {
let mut v: Vec<_> = Vec::new();
for _ in 1..10 {
v.push(wrap_future(Box::pin(async {})));
}
}
fn wrap_future<T>(a: Pin<Box<dyn Future<Output=T>>>) -> impl Future<Output=T> {
async {
println!("doing stuff before awaiting");
let result=a.await;
println!("doing stuff after awaiting");
result
}
}
As you can see, the futures I'm putting into the Vec don't need to be boxed, since they are all the same type and the compiler can infer what that type is.
I would like to create a struct that has this Vec<...> type as one of its members, so that I could add a line at the end of main():
let thing = MyStruct {myvec: v};
without any additional overhead (i.e. boxing).
Type inference and impl Trait syntax aren't allowed on struct members, and since the future type returned by an async block exists entirely within the compiler and is exclusive to that exact async block, there's no way to reference it by name. It seems to me that what I want to do is impossible. Is it? If so, will it become possible in a future version of Rust?
I am aware that it would be easy to sidestep this problem by simply boxing all the futures in the Vec as I did the argument to wrap_future() but I'd prefer not to do this if I can avoid it.
I am well aware that doing this would mean that there could be only one async block in my entire codebase whose result values could possibly be added to such a Vec, and thus that there could be only one function in my entire codebase that could create values that could possibly be pushed to it. I am okay with this limitation.
Nevermind, I'm stupid. I forgot that structs could have type parameters.
struct MyStruct<F> where F: Future<Output=()> {
myvec: Vec<F>,
}

Is it possible to assign the return value of different functions that return different structs (that implement a common trait) to a single variable?

Let's say I have something like this:
trait SomeTrait {}
struct One;
impl SomeTrait for One {}
struct Two;
impl SomeTrait for Two {}
fn return_one() -> One {
One
}
fn return_two() -> Two {
Two
}
Somewhere else, I want to essentially do:
fn do_stuff(op: Operation) {
let result = match op {
Operation::OpOne => return_one(),
Operation::OpTwo => return_two(),
};
}
That of course doesn't compile, as those two return_*() functions return distinct types. I've tried:
Declaring result as dyn SomeTrait (still error: mismatched types)
Casting the return values, e.g. return_one() as dyn SomeTrait (error: cast to unsized type: One as dyn SomeTrait)
Making Sized a supertrait of SomeTrait (this won't work for me in this particular case as I don't have control over the real-world version of SomeTrait, but it doesn't compile anyway: error: the trait SomeTrait cannot be made into an object
Things I think would work but don't want to or can't do:
Boxing values on return, e.g. Box::new(return_one()) as dyn Box<SomeTrait> (having to move the values into a box, and thus off the stack, seems excessive)
Having return_one() and return_two() instead return impl SomeTrait (this would allow me to accidentally return Two from return_one(), for example, and I want to use the type system to prevent that)
Wrapping with an enum: I don't want the functions to return a wrapper enum, because then we have the same problem as the previous bullet point. I could wrap the return values in an enum at the call site, and that could work, but let's say there's some function on SomeTrait that I want to call at the end; it seems like a lot of extra boilerplate to then unwrap the enum and call that function for each inner type. If I were to do that, I might as well just copy-paste the trait function call to each match arm.
I found a few crates on crates.io that claim to be able to do this, but AFAICT they all require implementing a trait on the types, which are foreign types for me, so I can't do that.
Is it possible to make this work?
A possible option is to do the following
fn do_stuff(op: Operation) {
let (one, two);
let _result: &dyn SomeTrait = match op {
Operation::OpOne => {one = return_one(); &one},
Operation::OpTwo => {two = return_two(); &two},
};
}
You can also use &mut dyn SomeTrait instead if you need to borrow it mutably.
This is somewhat verbose, but if you find yourself doing it a lot, a macro
that declares the anonymous variables, assigns them and returns a reference might help.
Another solution could be to use the auto_enums crate, which automaticaly creates the enum and implements the trait for it, the downside is that it only supports certain traits, (mostly in std, I believe) and that for this specific use case it requires nightly, or putting the match in a separate function.
I'm not sure I can link a specific part of the docs, but if you scroll down to "#Rust Nightly", you'll see your specific use of it, something like as follows
use auto_enums::auto_enum;
fn do_stuff(op: Operation) {
#[auto_enum(SomeTrait)]
let _result = match op {
Operation::OpOne => return_one(),
Operation::OpTwo => return_two(),
};
}
Although keep in mind this only works if auto_enums supports SomeTrait.

How to store async closure created at runtime in a struct?

I'm learning Rust's async/await feature, and stuck with the following task. I would like to:
Create an async closure (or better to say async block) at runtime;
Pass created closure to constructor of some struct and store it;
Execute created closure later.
Looking through similar questions I wrote the following code:
use tokio;
use std::pin::Pin;
use std::future::Future;
struct Services {
s1: Box<dyn FnOnce(&mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()>>>>,
}
impl Services {
fn new(f: Box<dyn FnOnce(&mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()>>>>) -> Self {
Services { s1: f }
}
}
enum NumberOperation {
AddOne,
MinusOne
}
#[tokio::main]
async fn main() {
let mut input = vec![1,2,3];
let op = NumberOperation::AddOne;
let s = Services::new(Box::new(|numbers: &mut Vec<usize>| Box::pin(async move {
for n in numbers {
match op {
NumberOperation::AddOne => *n = *n + 1,
NumberOperation::MinusOne => *n = *n - 1,
};
}
})));
(s.s1)(&mut input).await;
assert_eq!(input, vec![2,3,4]);
}
But above code won't compile, because of invalid lifetimes.
How to specify lifetimes to make above example compile (so Rust will know that async closure should live as long as input). As I understand in provided example Rust requires closure to have static lifetime?
Also it's not clear why do we have to use Pin<Box> as return type?
Is it possible somehow to refactor code and eliminate: Box::new(|arg: T| Box::pin(async move {}))? Maybe there is some crate?
Thanks
Update
There is similar question How can I store an async function in a struct and call it from a struct instance?
. Although that's a similar question and actually my example is based on one of the answers from that question. Second answer contains information about closures created at runtime, but seems it works only when I pass an owned variable, but in my example I would like to pass to closure created at runtime mutable reference, not owned variable.
How to specify lifetimes to make above example compile (so Rust will know that async closure should live as long as input). As I understand in provided example Rust requires closure to have static lifetime?
Let's take a closer look at what happens when you invoke the closure:
(s.s1)(&mut input).await;
// ^^^^^^^^^^^^^^^^^^
// closure invocation
The closure immediately returns a future. You could assign that future to a variable and hold on to it until later:
let future = (s.s1)(&mut input);
// do some other stuff
future.await;
The problem is, because the future is boxed, it could be held around for the rest of the program's life without ever being driven to completion; that is, it could have 'static lifetime. And input must obviously remain borrowed until the future resolves: else imagine, for example, what would happen if "some other stuff" above involved modifying, moving or even dropping input—consider what would then happen when the future is run?
One solution would be to pass ownership of the Vec into the closure and then return it again from the future:
let s = Services::new(Box::new(move |mut numbers| Box::pin(async move {
for n in &mut numbers {
match op {
NumberOperation::AddOne => *n = *n + 1,
NumberOperation::MinusOne => *n = *n - 1,
};
}
numbers
})));
let output = (s.s1)(input).await;
assert_eq!(output, vec![2,3,4]);
See it on the playground.
#kmdreko's answer shows how you can instead actually tie the lifetime of the borrow to that of the returned future.
Also it's not clear why do we have to use Pin as return type?
Let's look at a stupidly simple async block:
async {
let mut x = 123;
let r = &mut x;
some_async_fn().await;
*r += 1;
x
}
Notice that execution may pause at the await. When that happens, the incumbent values of x and r must be stored temporarily (in the Future object: it's just a struct, in this case with fields for x and r). But r is a reference to another field in the same struct! If the future were then moved from its current location to somewhere else in memory, r would still refer to the old location of x and not the new one. Undefined Behaviour. Bad bad bad.
You may have observed that the future can also hold references to things that are stored elsewhere, such as the &mut input in #kmdreko's answer; because they are borrowed, those also cannot be moved for the duration of the borrow. So why can't the immovability of the future similarly be enforced by r's borrowing of x, without pinning? Well, the future's lifetime would then depend on its content—and such circularities are impossible in Rust.
This, generally, is the problem with self-referential data structures. Rust's solution is to prevent them from being moved: that is, to "pin" them.
Is it possible somehow to refactor code and eliminate: Box::new(|arg: T| Box::pin(async move {}))? Maybe there is some crate?
In your specific example, the closure and future can reside on the stack and you can simply get rid of all the boxing and pinning (the borrow-checker can ensure stack items don’t move without explicit pinning). However, if you want to return the Services from a function, you'll run into difficulties stating its type parameters: impl Trait would normally be your go-to solution for this type of problem, but it's limited and does not (currently) extend to associated types, such as that of the returned future.
There are work-arounds, but using boxed trait objects is often the most practical solution—albeit it introduces heap allocations and an additional layer of indirection with commensurate runtime cost. Such trait objects are however unavoidable where a single instance of your Services structure may hold different closures in s1 over the course of its life, where you're returning them from trait methods (which currently can’t use impl Trait), or where you're interfacing with a library that does not provide any alternative.
If you want your example to work as is, the missing component is communicating to the compiler what lifetime associations are allowed. Trait objects like dyn Future<...> are constrained to be 'static by default, which means it cannot have references to non-static objects. This is a problem because your closure returns a Future that needs to keep a reference to numbers in order to work.
The direct fix is to annotate that the dyn FnOnce can return a Future that can be bound to the life of the first parameter. This requires a higher-ranked trait bound and the syntax looks like for<'a>:
struct Services {
s1: Box<dyn for<'a> FnOnce(&'a mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()> + 'a>>>,
}
impl Services {
fn new(f: Box<dyn for<'a> FnOnce(&'a mut Vec<usize>) -> Pin<Box<dyn Future<Output = ()> + 'a>>>) -> Self {
Services { s1: f }
}
}
The rest of your code now compiles without modification, check it out on the playground.

Is there a more idiomatic way to use traits like `io::Read` with lifetimes?

I have a public trait, Parser, that defines an external interface. I then have a private ParserImpl struct that implements the methods (actually, I have several implementations, which is the idea behind using the trait to abstract away).
use std::io;
pub trait Parser {
// ...omitted
}
struct ParserImpl<R: io::Read> {
// ...omitted
stream: R,
}
impl<R: io::Read> ParserImpl<R> {
// ...methods
fn new(stream: R) -> ParserImpl<R> {
ParserImpl {
// ...omitted
stream: stream,
}
}
}
impl<R: io::Read> Parser for ParserImpl<R> {
// ...methods
}
To create a parser instance, I use a function to hide ParserImpl.
pub fn make_parser<'a, R>(stream: R) -> Box<Parser + 'a>
where
R: io::Read + 'a,
{
Box::new(ParserImpl::new(stream))
}
This is all well and good... and it works... but the make_parser function troubles me. I feel that there must be a simpler way to approach this and like I'm missing something important, as this seems like a potential pitfall whenever using a trait like io::Read to abstract away the source of data.
I understand the need to specify lifetimes (Parameter type may not live long enough?) but I am a bit stumped on whether I can have both a clean and simple interface, and also use a trait like io::Read.
Is there a "cleaner," or perhaps more idiomatic way, to use traits like io::Read that I am missing? If not, that's okay, but I'm pretty new to Rust and when I wrote the above function I kept thinking "this can't be right..."
To make this sample runnable, here's a main:
fn main() {
use std::fs;
let file: fs::File = fs::File::open("blabby.txt").unwrap();
let parser = make_parser(file);
}
That is the idiomatic way of writing the code that has that meaning, but you may not want that meaning.
For example, if you don't need to create a boxed trait object, you can just return the parameterized value directly, or in this case just use the result of ParserImpl::new. This is my default form until I know I need dynamic dispatch provided by some trait object.
You could also require the 'static lifetime instead of introducing a new lifetime 'a, but this reduces the range of allowed types that you can pass into make_parser:
pub fn make_parser<R>(stream: R) -> Box<Parser>
where
R: io::Read + 'static,
{
Box::new(ParserImpl::new(stream))
}

How do I access the target's associated functions when using `Deref`?

I need to implement the fmt::Display method for an object coming from an external crate, so I created a wrapper for this object. I'd like to be able to use all the methods from the original object, without having to redefine all of them. I tried to implement Deref as advised on the awesome IRC channel #rust-beginners:
struct CustomMap(ObjectComingFromAnExternalCrate<char, char>);
impl std::ops::Deref for CustomMap {
type Target = ObjectComingFromAnExternalCrate<char, char>;
fn deref(&self) -> &Self::Target {
&self.0
}
}
fn main() {
let cm = CustomMap::with_capacity(10);
println!("Hello, world!");
}
However, I'm getting this error :
error: no associated item named `with_capacity` found for type `CustomMap` in the current scope
--> <anon>:16:13
|
16 | let a = CustomMap::with_capacity(10);
| ^^^^^^^^^^^^^^^^^^^^^^^^
I assume it's because deref() doesn't work with associated functions.
How can I work around this? Reimplementing every associated function I use, just to be able to implement one method I need seems like overkill.
Newtypes are specifically designed to provide encapsulation, so they do not necessarily lend them well to just "adding new stuff".
That being said, a combination of:
Deref and DerefMut to get access to the methods
From and Into to easily convert from one to the other
OR making the inner type pub
should be able to tackle this.
The From/Into recommendation comes from the fact that most associated functions are generally constructors1.
impl From<ObjectComingFromAnExternalCrate<char, char>> for CustomMap { ... }
and then you can do:
let cm: CustomMap = ObjectComingFromAnExternalCrate<char, char>::with_capacity(10).into();
The other solution is to define CustomMap as:
struct CustomMap(pub ObjectComingFromAnExternalCrate<char, char>);
and then:
let cm = CustomMap(ObjectComingFromAnExternalCrate<char, char>::with_capacity(10));
If you do not wish to enforce any other invariant, and do not care about encapsulation, either should get you going.
1 Pointer types, such as Rc, use them heavily to avoid hiding methods of the Deref'ed to type.

Resources