This question already has answers here:
Why are trait methods with generic type parameters object-unsafe?
(2 answers)
Closed 2 months ago.
I have this signature that I wanna use in a Trait to create many adapters (file system, aws, google cloud etc.).
I'm trying to create the below Trait but this error occurs:
use tokio::io::AsyncRead;
#[async_trait::async_trait]
pub trait Trait: Send + Sync {
async fn put_file<S>(&self, filename: &str, stream: S) -> Result<()>
where
S: AsyncRead + Send;
}
pub struct Client {
location: String,
}
#[async_trait::async_trait]
impl Trait for Client {
async fn put_file<S>(&self, filename: &str, stream: S) -> Result<()>
where
S: AsyncRead + Send,
{
futures::pin_mut!(stream);
let path = Path::new(&self.location).join(filename);
let mut file = BufWriter::new(File::create(path).await?);
tokio::io::copy(&mut stream, &mut file).await?;
Ok(())
}
}
error[E0038]: the trait `Trait` cannot be made into an object
|
23 | pub client: Arc<dyn Trait>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `Trait` cannot be made into an object
|
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
|
5 | pub trait Trait: Send + Sync {
| ----- this trait cannot be made into an object...
6 | async fn put_file<S>(&self, filename: &str, stream: S) -> Result<()>
| ^^^^^^^^ ...because method `put_file` has generic type parameters
= help: consider moving `put_file` to another trait
Why?
If you read your error message it will both mention that it's
...because method put_file has generic type parameters
as well as where you can find further info on object safety
You can just not create a trait object instead:
struct YourStructThatContainsTrait<T: Trait> {
pub client: Arc<T>,
}
Or go with dynamic dispatch all the way:
async fn put_file(&self, filename: &str, stream: Box<dyn AsyncRead + Unpin + Send>) -> Result<()>;
Related
I'm trying to use the #[async_recursion] macro on a constructor that takes an impl trait as an argument. The impl trait is just a shim around reqwest so I can insert a mock for testing:
#[async_trait]
pub trait NetFuncs {
async fn get(&self, url: &str) -> Result<String, Error>;
}
It was working fine until I made my constructor recursive:
#[derive(Debug)]
pub struct Foo {
map: serde_yaml::mapping::Mapping,
filename: String,
parent: Option<Box<Foo>>,
receipt: Option<Receipt>,
}
impl Foo {
#[async_recursion]
pub async fn from_str(s: &str, filename: &str, net: &impl NetFuncs) -> Result<Foo, Error> {
throws the error:
error: future cannot be sent between threads safely
--> src/main.rs:97:5
|
97 | #[async_recursion]
| ^^^^^^^^^^^^^^^^^^ future created by async block is not `Send`
|
note: captured value is not `Send` because `&` references cannot be sent unless their referent is `Sync`
--> src/main.rs:125:17
|
125 | net,
| ^^^ has type `&impl NetFuncs` which is not `Send`, because `impl NetFuncs` is not `Sync`
= note: required for the cast to the object type `dyn Future<Output = Result<Foo, Error>> + Send`
= note: this error originates in the attribute macro `async_recursion` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider further restricting this bound
|
98 | pub async fn from_str(s: &str, filename: &str, net: &impl NetFuncs + std::marker::Sync) -> Result<Foo, Error> {
| +++++++++++++++++++
There are other ways to mock a network for testing then the way I did it, but I liked my solution, at least until I hit this error. How do I fix this error without removing the net: &impl NetFuncs argument?
MRE
[package]
name = "mre2"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-recursion = "1.0"
async-trait = "0.1"
use async_trait::async_trait;
use async_recursion::async_recursion;
#[derive(Debug)]
pub struct Foo {
s: String,
filename: String,
foo: String,
parent: Option<Box<Foo>>,
}
#[async_trait]
pub trait NetFuncs {
async fn get(&self, url: &str) -> String;
}
#[derive(Debug)]
pub struct FakeNet {}
#[async_trait]
impl NetFuncs for FakeNet {
async fn get(&self, url: &str) -> String {
"".to_string()
}
}
impl Foo {
#[async_recursion]
pub async fn from_str(s: &str, filename: &str, net: &impl NetFuncs) -> Foo {
Foo { s: s.to_string(), filename: filename.to_string(), parent: None, foo: net.get("").await.to_string() }
}
}
The problem is, like explained by the compiler, that &impl NetFuncs may not necessarily impl Send but the async_recursion macro by default requires it, so, you have two options:
Require impl NetFuncs to be Sync, so that &impl NetFuncs is Send. This can be done either with &(impl NetFuncs + Sync) or by requiring every implementor to implement Send: trait NetFuncs: Sync.
Not requiring the resulting future to be Send. As documented in the async_recursion documentation, this can be done by changing #[async_recursion] to #[async_recursion(?Send)].
Without the macro it works since the compiler make the resulting future Send depend on whether all types kept across .await points are Send: if they are, the future is also Send. If they are not, it is not too. The macro changes the async fn to fn ...() -> Pin<Box<dyn Future>>, and unfortunately, it is not possible to have the same behavior as with async fn - this is something only the compiler can implement. Thus, the macro allows you to choose whether you want the resulting future to be Send - meaning all types should be too, or not.
In the example code below, a non-send value, Vec<T>, is moved into a function that returns something else. At this point, I no longer care about that vector. The returned object stores no reference to it, it no longer exists.
However, when I .await on the next line I get the error "captured value is not Send". Which it isn't, but since it should have been destroyed when vector_as_string exited, it doesn't need to send it across threads when the future restarts, because that variable is never used again.
use async_trait::async_trait;
async fn write_value(value: Vec<u8>) {
println!("something")
}
fn vector_as_string<T>(vec: Vec<T>) -> Vec<u8> {
Vec::new()
}
#[async_trait]
trait Writer {
async fn write_array<T>(&mut self, value: Vec<T>);
}
pub struct WriterImplementation {}
#[async_trait]
impl Writer for WriterImplementation {
async fn write_array<T>(&mut self, value: Vec<T>) {
let string = vector_as_string(value);
write_value(string).await
}
}
#[tokio::main]
async fn main() {
println!("Hi");
}
Dependencies:
[dependencies]
tokio = { version = "1.9.0", features = ["full"]}
async-trait = "0.1.51"
Error:
error: future cannot be sent between threads safely
--> src/main.rs:20:55
|
20 | async fn write_array<T>(&mut self, value: Vec<T>) {
| _______________________________________________________^
21 | | let string = vector_as_string(value);
22 | |
23 | | write_value(string).await
24 | | }
| |_____^ future created by async block is not `Send`
|
note: captured value is not `Send`
--> src/main.rs:20:40
|
20 | async fn write_array<T>(&mut self, value: Vec<T>) {
| ^^^^^ has type `Vec<T>` which is not `Send`
= note: required for the cast to the object type `dyn Future<Output = ()> + Send`
help: consider further restricting this bound
|
20 | async fn write_array<T + std::marker::Send>(&mut self, value: Vec<T>) {
| ^^^^^^^^^^^^^^^^^^^
Adding T: Send as it suggests allows it to compile, but why does T need to be Send if we aren't holding any T's across the await, as it has already been moved?
From the async_trait documentation:
Async fns get transformed into methods that return Pin<Box<dyn Future + Send + 'async>> and delegate to a private async freestanding function.
Not all async traits need futures that are dyn Future + Send. To avoid having Send and Sync bounds placed on the async trait methods, invoke the async trait macro as #[async_trait(?Send)] on both the trait and the impl blocks.
Applied to your case:
#[async_trait(?Send)]
trait Writer {
async fn write_array<T>(&mut self, value: Vec<T>);
}
#[async_trait(?Send)]
impl Writer for WriterImplementation {
async fn write_array<T>(&mut self, value: Vec<T>) {
let string = vector_as_string(value);
write_value(string).await
}
}
This question already has answers here:
How to clone a struct storing a boxed trait object?
(3 answers)
Closed 2 years ago.
If I want a non-copyable type-erased (dynamically-typed) callable, that's
Box<dyn Fn(i32) -> ()>
If I want a ref-counted type-erased callable, that's (depending on whether I need thread-safety or not)
Rc<dyn Fn(i32) -> ()>
Arc<dyn Fn(i32) -> ()>
But here, the copies all refer to the same underlying memory - they're not distinct.
If I want distinct callable objects, how can I do that? Box<T> already implements Clone when T implements Clone, but Fn does not implement Clone so that doesn't apply here. Doing something like:
Box<dyn Fn(i32) -> () + Clone>
fails with:
error[E0225]: only auto traits can be used as additional traits in a trait object
--> src/main.rs:7:35
|
7 | fn foo(f: Box<dyn Fn(i32) -> () + Clone>) {
| ------------- ^^^^^ additional non-auto trait
| |
| first non-auto trait
|
= help: consider creating a new trait with all of these as super-traits and using that trait here instead: `trait NewTrait: Fn<(i32,)> + Clone {}`
= note: auto-traits like `Send` and `Sync` are traits that have special properties; for more information on them, visit <https://doc.rust-lang.org/reference/special-types-and-traits.html#auto-traits>
The suggestion in the error doesn't work because of the spelling of Fn, but this:
trait CopyableFn: Fn(i32) -> () + Clone {}
Box<dyn CopyableFn>
By itself also doesn't work because:
error[E0038]: the trait `CopyableFn` cannot be made into an object
--> src/main.rs:7:11
|
5 | trait CopyableFn: Fn(i32) -> () + Clone {}
| ---------- ----- ...because it requires `Self: Sized`
| |
| this trait cannot be made into an object...
6 |
7 | fn foo(f: Box<dyn CopyableFn>) {
| ^^^^^^^^^^^^^^^^^^^ the trait `CopyableFn` cannot be made into an object
Is there a way of creating a type object for Fn that is clonable such that the copies are distinct?
Instead of having CloneableFn be a supertrait of Clone, implement a clone_box method that clones it into a box:
trait CloneableFn: Fn(i32) -> () {
fn clone_box<'a>(&self) -> Box<dyn 'a + CloneableFn>
where
Self: 'a;
}
Since unsized types like dyn CloneableFn cannot be cloned (Clone requires Sized), there's little reason to have Clone as a supertrait here. However, having Fn(i32) -> () as a supertrait allows the functions to be called as normal.
Then the CloneableFn can be implemented for all types that implement both Fn(i32) -> () and Clone:
impl<F> CloneableFn for F
where
F: Fn(i32) -> () + Clone,
{
fn clone_box<'a>(&self) -> Box<dyn 'a + CloneableFn>
where
Self: 'a,
{
Box::new(self.clone())
}
}
Finally, Box<dyn CloneableFn> does not automatically implement Clone since dyn CloneableFn doesn't, so we can implement that ourselves:
impl<'a> Clone for Box<dyn 'a + CloneableFn> {
fn clone(&self) -> Self {
(**self).clone()
}
}
With this, you can now clone Box<dyn CloneableFn> and call it as a regular function:
// let closure borrow some shared state
use std::sync::atomic::{AtomicI32, Ordering};
let x = AtomicI32::new(0);
let f = |n| {
println!("x = {}", x.fetch_add(n, Ordering::Relaxed));
};
let f: Box<dyn CloneableFn> = Box::new(f);
let g = f.clone();
f(3);
g(5);
f(7);
Here's a full playground example
This is related to How to clone a struct storing a boxed trait object?, but in that case the targeted trait (Animal) can be altered to have a supertrait, while in this case that's not possible (since the targeted trait is Fn(i32) -> ()). In a way, it's the opposite approach: adding a trait of which the target is a supertrait instead of adding a supertrait to the target.
I need to implement a trait that is returning the futures::StreamExt trait.
In general this sounds easy and there are several answers to this e.g. this here.
I tried this with StreamExt but this does - for some reason - not work. Here my sample code:
// [dependencies]
// futures = "0.3.6"
// async-std = { version = "1.6.5", features = ["attributes", "unstable"] } // interval function is unstable atm.
use std::time::Duration;
use futures::StreamExt;
trait StreamProvidingTrait {
fn returnastream(self: &Self) -> Box<dyn StreamExt<Item=i32>>;
}
struct StreamProvider {}
impl StreamProvidingTrait for StreamProvider {
fn returnastream(self: &Self) -> Box<dyn StreamExt<Item=i32>> {
return Box::new(async_std::stream::interval(Duration::from_millis(1000)).map(|_| 1));
}
}
#[async_std::main]
async fn main() {
let mut counter = 0;
let object = StreamProvider {};
// creates a stream
let worx = object.returnastream();
// mappes the stream into something....
let mut mapped_stream = worx.map(|_| {
counter = counter + 1;
counter
});
// subscribing to the items
while let item = mapped_stream.next().await {
match item {
Some(value) => println!("{}", value),
_ => {}
}
}
}
And here the error:
Compiling traittest v0.1.0 (/Users/andre/repos/traittest)
warning: the trait `futures_util::stream::stream::StreamExt` cannot be made into an object
--> /Users/andre/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.6/src/stream/stream/mod.rs:1326:8
|
1326 | fn poll_next_unpin(&mut self, cx: &mut Context<'_>) -> Poll<Option<Self::Item>>
| ^^^^^^^^^^^^^^^ the trait cannot be made into an object because method `poll_next_unpin` references the `Self` type in its `where` clause
|
= note: `#[warn(where_clauses_object_safety)]` on by default
= warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
= note: for more information, see issue #51443 <https://github.com/rust-lang/rust/issues/51443>
error[E0038]: the trait `futures_util::stream::stream::StreamExt` cannot be made into an object
--> src/main.rs:6:36
|
6 | fn returnastream(self: &Self) -> Box<dyn StreamExt<Item=i32>>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `futures_util::stream::stream::StreamExt` cannot be made into an object
|
::: /Users/andre/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.6/src/stream/stream/mod.rs:226:8
|
226 | fn next(&mut self) -> Next<'_, Self>
| ---- the trait cannot be made into an object because method `next` references the `Self` type in its return type
...
976 | fn by_ref(&mut self) -> &mut Self {
| ------ the trait cannot be made into an object because method `by_ref` references the `Self` type in its return type
...
1381 | fn select_next_some(&mut self) -> SelectNextSome<'_, Self>
| ---------------- the trait cannot be made into an object because method `select_next_some` references the `Self` type in its return type
error[E0038]: the trait `futures_util::stream::stream::StreamExt` cannot be made into an object
--> src/main.rs:12:36
|
12 | fn returnastream(self: &Self) -> Box<dyn StreamExt<Item=i32>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `futures_util::stream::stream::StreamExt` cannot be made into an object
|
::: /Users/andre/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.6/src/stream/stream/mod.rs:226:8
|
226 | fn next(&mut self) -> Next<'_, Self>
| ---- the trait cannot be made into an object because method `next` references the `Self` type in its return type
...
976 | fn by_ref(&mut self) -> &mut Self {
| ------ the trait cannot be made into an object because method `by_ref` references the `Self` type in its return type
...
1381 | fn select_next_some(&mut self) -> SelectNextSome<'_, Self>
| ---------------- the trait cannot be made into an object because method `select_next_some` references the `Self` type in its return type
error: aborting due to 2 previous errors; 1 warning emitted
For more information about this error, try `rustc --explain E0038`.
error: could not compile `traittest`.
To learn more, run the command again with --verbose.
Process finished with exit code 101
When I exchange the StreamExt trait with my own one, this code works perfectly.
trait SomeTrait {
fn returnatrait(self: &Self) -> Box<dyn SomeOtherTrait>;
}
trait SomeOtherTrait {
fn sayHelloWorld(&self);
}
struct DummyStruct {}
impl SomeOtherTrait for DummyStruct {
fn sayHelloWorld(&self) {
println!("hello world");
}
}
struct Implementation {}
impl SomeTrait for Implementation {
fn returnatrait(self: &Self) -> Box<dyn SomeOtherTrait> {
return Box::new(DummyStruct{});
}
}
fn main() {
let implementation = Implementation{};
let worx = implementation.returnatrait();
worx.sayHelloWorld();
}
What's wrong here? There's obviously something I don't understand. Please help me understand this!
A function returning a trait can use the impl Trait syntax to return an opaque type that implements the trait. A trait method returning a trait currently doesn't support this feature and requires the trait to be returned as a trait object - a dynamically dispatched reference or smart pointer such as Box or Rc. Not all traits are object-safe, though, and the bad news is that StreamExt is among those that aren't, for reasons pointed out by the compiler, such as referencing Self in method return types and where clauses.
The good news, however, is that this is not a problem: StreamExt is an extension trait, one that provides a blanket implementation for all types that implement Stream. So you don't need to bother returning a dyn StreamExt trait object, you can return a dyn Stream one, and you'll still get access to StreamExt methods simply by requesting them with use StreamExt. In other words, you can just replace Box<dyn StreamExt> with Box<dyn Stream> in the return type of your trait.
Another issue you might encounter is that Box<dyn Stream> doesn't work on methods that need to move the stream, which includes many methods provided by StreamExt. Those will require the stream to be pinned, which can be fixed by returning Pin<Box<dyn Stream>> instead of Box<dyn Stream>. There is even a boxed() method on StreamExt that pins and boxes the stream in one operation; code that uses it would look like this (playground):
use futures::stream::{Stream, StreamExt};
use std::pin::Pin;
trait StreamProvidingTrait {
fn returnastream(&self) -> Pin<Box<dyn Stream<Item = i32>>>;
}
struct StreamProvider {}
impl StreamProvidingTrait for StreamProvider {
fn returnastream(&self) -> Pin<Box<dyn Stream<Item = i32>>> {
return tokio::stream::once(0).boxed();
}
}
fn main() {
let provider = StreamProvider {};
let stream = provider.returnastream();
let _fut = stream.into_future();
}
I'm trying to store closures in a Vec that is part of a struct. The closure is a factory function which receives 2 references as arguments and produces a trait object which stores the references the closure receives as arguments.
Because of that, the produced trait object has a lifetime that must not exceed the lifetime of the references. Also component_registrations will be accessed from multiple threads and is therefore wrapped in an Arc<Mutex>.
I tried implementing it but the compiler says that the generic parameter F of the register_component function doesn't satisfy the trait bound used in component_registrations.
This is the relevant part of the code:
use std::sync::Mutex;
use std::sync::Arc;
pub mod gl {
pub struct Gl();
}
pub struct ComponentRegistry<'a> {
gl: &'a gl::Gl
}
pub trait Component<'a> {
}
pub struct Application {
component_registrations: Arc<Mutex<Vec<Box<for<'b> Fn(&'b ComponentRegistry<'b>, &'b gl::Gl) -> Box<Component<'b>> + Send + 'static>>>>
}
impl Application {
pub fn new() -> Application {
Application {
component_registrations: Arc::new(Mutex::new(vec![]))
}
}
pub fn register_component<'a, F>(&mut self, register: F) where F: Fn(&'a ComponentRegistry<'a>, &'a gl::Gl) -> Box<Component<'a>> + Send + 'static {
self.component_registrations.lock().unwrap().push(Box::new(register));
}
}
error[E0277]: the trait bound `for<'b> F: std::ops::Fn<(&'b ComponentRegistry<'b>, &'b gl::Gl)>` is not satisfied
--> src/main.rs:27:59
|
27 | self.component_registrations.lock().unwrap().push(Box::new(register));
| ^^^^^^^^^^^^^^^^^^ the trait `for<'b> std::ops::Fn<(&'b ComponentRegistry<'b>, &'b gl::Gl)>` is not implemented for `F`
|
= help: consider adding a `where for<'b> F: std::ops::Fn<(&'b ComponentRegistry<'b>, &'b gl::Gl)>` bound
= note: required for the cast to the object type `for<'b> std::ops::Fn(&'b ComponentRegistry<'b>, &'b gl::Gl) -> std::boxed::Box<Component<'b>> + std::marker::Send`
error[E0271]: type mismatch resolving `for<'b> <F as std::ops::FnOnce<(&'b ComponentRegistry<'b>, &'b gl::Gl)>>::Output == std::boxed::Box<Component<'b>>`
--> src/main.rs:27:59
|
27 | self.component_registrations.lock().unwrap().push(Box::new(register));
| ^^^^^^^^^^^^^^^^^^ expected bound lifetime parameter 'b, found concrete lifetime
|
note: concrete lifetime that was found is the lifetime 'a as defined on the method body at 26:5
--> src/main.rs:26:5
|
26 | / pub fn register_component<'a, F>(&mut self, register: F) where F: Fn(&'a ComponentRegistry<'a>, &'a gl::Gl) -> Box<Component<'a>> + Send + 'static {
27 | | self.component_registrations.lock().unwrap().push(Box::new(register));
28 | | }
| |_____^
= note: required for the cast to the object type `for<'b> std::ops::Fn(&'b ComponentRegistry<'b>, &'b gl::Gl) -> std::boxed::Box<Component<'b>> + std::marker::Send`
If you use a higher ranked lifetime when you define your component_registrations struct field, you should use a higher ranked lifetime for F as well.
Also, if you say Box<Component<'b>>, it really means Box<Component<'b> + 'static> (so the trait object can contain only owned data). What you really need is Box<Component<'b> + 'b>, which means it is a trait object that implements Component<'b> and it can also contain borrowed data which live at least as long as 'b.
The relevant part is
pub struct Application {
component_registrations: Vec<Box<for<'b> Fn(&'b ComponentRegistry<'b>, &'b gl::Gl) -> Box<Component<'b> + 'b> + Send + 'static>>
}
impl Application {
pub fn register_component<F>(&mut self, register: F) where F: for<'b> Fn(&'b ComponentRegistry<'b>, &'b gl::Gl) -> Box<Component<'b> + 'b> + Send + 'static {
self.component_registrations.push(Box::new(register));
}
}
You can see the full example. Note, that I removed the Arc and Mutex types from your example since they were not relevant.