Avoiding `Sync` for shared static variables - rust

I have the following definitions:
struct MyCustomFactory;
trait Factory {
fn new<'a>(entity: &'a str) -> Arc<dyn Display + 'a>
where
Self: Sized;
}
impl Factory for MyCustomFactory {
fn new<'a>(entity: &'a str) -> Arc<dyn Display + 'a>
where
Self: Sized,
{
Arc::new(entity)
}
}
Higher level problem:
I'm trying to create a global static (and immutable) map of strings to structs which implement the Factory trait. The goal is that for any value member of the map, I can treat it like a Factory, call new against it, and in turn will get something back that (in this case) implements the Display trait.
I have the following:
static ITEMS: Lazy<BTreeMap<&'static str, Arc<dyn Factory>>> =
Lazy::new(|| {
let mut map = BTreeMap::new();
let value = Arc::new(MyCustomFactory) as Arc<dyn Factory>;
map.insert("foo", value);
map
});
The compiler complains with:
77 | / static ITEMS: Lazy<BTreeMap<&'static str, Arc<dyn Factory>>> =
78 | | Lazy::new(|| {
79 | | let map = BTreeMap::new();
80 | | let value = Arc::new(MyCustomFactory) as Arc<dyn Factory>;
... |
88 | | });
| |_______^ `(dyn Factory + 'static)` cannot be shared between threads safely
|
= help: the trait `std::marker::Sync` is not implemented for `(dyn Factory + 'static)`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<(dyn Factory + 'static)>`
= note: required because of the requirements on the impl of `std::marker::Send` for `alloc::collections::btree::node::Root<&'static str, std::sync::Arc<(dyn Factory + 'static)>>`
= note: required because it appears within the type `std::collections::BTreeMap<&'static str, std::sync::Arc<(dyn Factory + 'static)>>`
= note: required because of the requirements on the impl of `std::marker::Sync` for `once_cell::imp::OnceCell<std::collections::BTreeMap<&'static str, std::sync::Arc<(dyn Factory + 'static)>>>`
= note: required because it appears within the type `once_cell::sync::OnceCell<std::collections::BTreeMap<&'static str, std::sync::Arc<(dyn Factory + 'static)>>>`
= note: required because of the requirements on the impl of `std::marker::Sync` for `once_cell::sync::Lazy<std::collections::BTreeMap<&'static str, std::sync::Arc<(dyn Factory + 'static)>>>`
= note: shared static variables must have a type that implements `Sync`
Things I have tried:
Implement Sync against the trait: Declaring the trait to have trait Factory : Sync, (meaning all items which implement the trait, must implement Sync) and/or defining:
|
66 | unsafe impl Sync for dyn Factory {}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't implement cross-crate trait with a default impl for non-struct/enum type
Wrapping Arc<dyn Factory>: I've tried using Mutex / RefCell / OnceCell and others, all resulting in the same error. It seems like there should be a way to treat the underlying MyCustomFactory as a Singleton that can be locked around. It's fine (and expected) that's there's only once instance of this struct globally.

You were close with adding Sync. If you look closely you'll see that the error then recommends adding Send:
= help: the trait `std::marker::Send` is not implemented for `(dyn Factory + 'static)`
Add both Send and Sync and it compiles. You can do:
trait Factory: Send + Sync {
...
}
Or, my preference, require them specifically in the static map, since that's where they're needed:
static ITEMS: Lazy<BTreeMap<&'static str, Arc<dyn Factory + Send + Sync>>> =
Lazy::new(|| {
let mut map = BTreeMap::new();
let value = Arc::new(MyCustomFactory) as Arc<dyn Factory + Send + Sync>;
map.insert("foo", value);
map
});
You could also reduce duplication by letting it infer the second Arc:
let value = Arc::new(MyCustomFactory) as Arc<_>;

Related

Trait Bound is not satisfied for Result<(), Box<(dyn SomeTrait + 'static)>>

use once_cell::sync::OnceCell;
pub trait SomeTrait {}
pub struct Impl1 {}
impl SomeTrait for Impl1 {}
pub static GLOBAL_THING: OnceCell<Box<dyn SomeTrait>> = OnceCell::new();
pub fn main() {
GLOBAL_THING.set(Box::new(Impl1 {})).unwrap();
}
I'm getting this error and not sure how to interpret it's meaning or fix it.
error[E0599]: the method `unwrap` exists for enum `Result<(), Box<(dyn SomeTrait + 'static)>>`, but its trait bounds were not satisfied
--> src/main.rs:11:42
|
11 | GLOBAL_THING.set(Box::new(Impl1 {})).unwrap();
| ^^^^^^ method cannot be called on `Result<(), Box<(dyn SomeTrait + 'static)>>` due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
`Box<dyn SomeTrait>: Debug`
An easy workaround is to just do if let Err() = GLOBAL_THING.set() {panic!(...)} instead of using unwrap(), but I'd like to understand what's going on here and fix if possible.
If you take a look at the unwrap method documentation you'll see that it's not defined for all Results, but only ones where E: Debug:
impl<T, E> Result<T, E>
where
E: Debug,
{
pub fn unwrap(self) -> T;
}
It needs the error type E to implement Debug so that the error can be printed if unwrapping fails.
The error message shows that the result type is Result<(), Box<(dyn SomeTrait + 'static)>>, so E = Box<(dyn SomeTrait + 'static)>. You can make this error type debuggable by having SomeTrait: Debug, which requires that any type that implements SomeTrait must also implement Debug.
pub trait SomeTrait: Debug {}
#[derive(Debug)]
pub struct Impl1 {}
Once you do this you'll hit the next error:
error[E0277]: `dyn SomeTrait` cannot be shared between threads safely
--> src/main.rs:11:1
|
11 | pub static GLOBAL_THING: OnceCell<Box<dyn SomeTrait>> = OnceCell::new();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `dyn SomeTrait` cannot be shared between threads safely
|
= help: the trait `Sync` is not implemented for `dyn SomeTrait`
= note: required because of the requirements on the impl of `Sync` for `Unique<dyn SomeTrait>`
= note: required because it appears within the type `Box<dyn SomeTrait>`
= note: required because of the requirements on the impl of `Sync` for `once_cell::imp::OnceCell<Box<dyn SomeTrait>>`
= note: required because it appears within the type `once_cell::sync::OnceCell<Box<dyn SomeTrait>>`
= note: shared static variables must have a type that implements `Sync`
To fix this you'll also want to make the boxed trait object Send + Sync so that it can be shared across threads.
pub static GLOBAL_THING: OnceCell<Box<dyn SomeTrait + Send + Sync>> = OnceCell::new();
You can see the final result on the Playground.

`FuturesUnordered` does not satisfy `Stream`?

I want to implement a Stream that is based on a FuturesUnordered, which again is supposed to evaluate async functions with a return type of Result<SomeEnum>, though for simplicity of the argument let's assume it's only a Result<f64>. As async fns eventually return Futures, I assumed that the following way would be how I have to define my struct:
use anyhow::Result;
use futures::{Future, Stream, stream::FuturesUnordered};
use std::{pin::Pin, task::Poll};
use pin_project::pin_project;
#[pin_project]
pub struct MyDerivedStream<'a> {
#[pin]
from_futures: FuturesUnordered<&'a (dyn Future<Output = Result<f64>> + Send)>,
}
impl Stream for MyDerivedStream<'_> {
type Item = Result<f64>;
fn poll_next(
self: Pin<&mut Self>,
c: &mut std::task::Context<'_>,
) -> Poll<Option<<Self as Stream>::Item>> {
let this = self.project();
this.from_futures.poll_next(c)
}
}
The problem I'm running into now is that for some reason the poll_next function on the FuturesUnordered fails to compile due to not satisfying Stream trait bounds. (See for yourself on this Playground example):
error[E0599]: the method `poll_next` exists for struct `Pin<&mut FuturesUnordered<&dyn futures::Future<Output = std::result::Result<f64, anyhow::Error>> + std::marker::Send>>`, but its trait bounds were not satisfied
--> src/lib.rs:21:27
|
21 | this.from_futures.poll_next(c)
| ^^^^^^^^^ method cannot be called on `Pin<&mut FuturesUnordered<&dyn futures::Future<Output = std::result::Result<f64, anyhow::Error>> + std::marker::Send>>` due to unsatisfied trait bounds
|
::: /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.14/src/stream/futures_unordered/mod.rs:55:1
|
55 | pub struct FuturesUnordered<Fut> {
| -------------------------------- doesn't satisfy `_: futures::Stream`
|
= note: the following trait bounds were not satisfied:
`&dyn futures::Future<Output = std::result::Result<f64, anyhow::Error>> + std::marker::Send: futures::Future`
which is required by `FuturesUnordered<&dyn futures::Future<Output = std::result::Result<f64, anyhow::Error>> + std::marker::Send>: futures::Stream`
I'm struggling to understand the problem here. For all I can see FuturesUnordered does implement Stream, so what is the actual issue here? Is is the &'a dyn Future - and if so, how else would I need to define the type here to make this work?
&'a dyn Future does not implement Future, which is required by impl Stream for FuturesUnordered. One solution would be replacing &'a dyn Future by Pin<&mut 'a dyn Future>:
use anyhow::Result;
use futures::{Future, Stream, stream::FuturesUnordered};
use std::{pin::Pin, task::Poll};
use pin_project::pin_project;
#[pin_project]
pub struct MyDerivedStream<'a> {
#[pin]
from_futures: FuturesUnordered<Pin<&'a mut(dyn Future<Output = Result<f64>> + Send)>>,
}
impl<'a> Stream for MyDerivedStream<'a> {
type Item = Result<f64>;
fn poll_next(
self: Pin<&mut Self>,
c: &mut std::task::Context<'_>,
) -> Poll<Option<<Self as Stream>::Item>> {
let this = self.project().from_futures;
this.poll_next(c)
}
}
It's necessary to mutably borrow the items in FuturesUnordered, this becomse evident by checking out the Futures::poll function which takes self: Pin<&mut Self>. The implementation of Stream for FuturesUnordered needs to poll the wrapped items in order to determine when a new item can be yielded, which is not possible with shared references.
Without the Pin around the &mut Future, it would be possible to mem::replace the wrapped future and cause that Future to never be actually polled.
This is a great resource to learn more about Pinning: https://fasterthanli.me/articles/pin-and-suffering

How do I resolve the compilation error "Unpin not implemented for GenFuture" resulting from adapting a stream using a combinator?

I'm unable to work out how to resolve the compilation error resulting from adapting a stream using a combinator.
The following Rust Playground demonstrates a fairly minimal example:
use futures::prelude::*;
use futures::StreamExt;
#[derive(Debug)]
pub enum Input {
A,
B(i32),
C(u16),
}
#[derive(Debug)]
enum Output {
Int(i32),
Short(u16),
}
pub struct StreamMaker;
impl StreamMaker {
/// make a stream with a series of inputs
pub fn create(self) -> impl Stream<Item = Input> {
stream::iter(vec![Input::A, Input::C(1u16), Input::B(2)])
}
}
/// consume the creator, and make output messages for a subset
pub fn adapt_stream(creator: StreamMaker) -> impl Stream<Item = Output> {
let mut upstream = creator.create();
upstream.filter_map(|message| async move {
match message {
Input::A => None,
Input::B(v) => Some(Output::Int(v)),
Input::C(v) => Some(Output::Short(v)),
}
})
}
#[tokio::main]
async fn main() -> Result<(), ()> {
let creator = StreamMaker {};
let mut stream = adapt_stream(creator);
while let Some(message) = stream.next().await {
println!("message: {:?}", message)
}
Ok(())
}
Compilation fails with:
error[E0277]: the trait bound `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>: std::marker::Unpin` is not satisfied in `impl core::future::future::Future`
--> src/main.rs:43:38
|
43 | while let Some(message) = stream.next().await {
| ^^^^ within `impl core::future::future::Future`, the trait `std::marker::Unpin` is not implemented for `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>`
|
= help: the following implementations were found:
<std::future::GenFuture<T> as std::marker::Unpin>
= note: required because it appears within the type `impl core::future::future::Future`
= note: required because of the requirements on the impl of `std::marker::Unpin` for `futures_util::stream::stream::filter_map::FilterMap<impl futures_core::stream::Stream, impl core::future::future::Future, [closure#src/main.rs:29:25: 35:6]>`
= note: required because it appears within the type `impl futures_core::stream::Stream`
error[E0277]: the trait bound `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>: std::marker::Unpin` is not satisfied in `impl core::future::future::Future`
--> src/main.rs:43:31
|
43 | while let Some(message) = stream.next().await {
| ^^^^^^^^^^^^^^^^^^^ within `impl core::future::future::Future`, the trait `std::marker::Unpin` is not implemented for `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>`
|
= help: the following implementations were found:
<std::future::GenFuture<T> as std::marker::Unpin>
= note: required because it appears within the type `impl core::future::future::Future`
= note: required because of the requirements on the impl of `std::marker::Unpin` for `futures_util::stream::stream::filter_map::FilterMap<impl futures_core::stream::Stream, impl core::future::future::Future, [closure#src/main.rs:29:25: 35:6]>`
= note: required because it appears within the type `impl futures_core::stream::Stream`
= note: required because of the requirements on the impl of `core::future::future::Future` for `futures_util::stream::stream::next::Next<'_, impl futures_core::stream::Stream>`
I can put a pin_mut!(stream); in the main, but I'd like to be able to push that upstream.
If you do not want the consumer of your stream to have to pin it themselves, you need to return a stream that implements the Unpin trait, meaning that it is safe to move around in memory even after is has been pinned.
pub fn adapt_stream(creator: StreamMaker) -> impl Stream<Item = Output> + Unpin {
// add Unpin trait --^
Adding this, your compiler should complain that the return value doesn't implement Unpin. This is because async move { ... } blocks don't implement Unpin, since they may be self-referential (e.g. contain references to variables they own). The most general way to work around this is to pin the stream to the heap with a Pin<Box<_>>, using the Box::pin constructor:
pub fn adapt_stream(creator: StreamMaker) -> impl Stream<Item = Output> + Unpin {
let mut upstream = creator.create();
Box::pin(upstream.filter_map(|message| async move {
// ^-- pin stream to heap
match message {
Input::A => None,
Input::B(v) => Some(Output::Int(v)),
Input::C(v) => Some(Output::Short(v)),
}
}))
}
Since we're now returning a Pin<Box<_>> pointer to the stream, that pointer can be safely moved around in memory while the inner stream is kept at the same location.
Full playground example

How to pass data struct with trait over multiple threads in rust? [duplicate]

This question already has answers here:
Sharing a struct with trait objects as properties across threads
(1 answer)
Sending trait objects between threads in Rust
(1 answer)
Closed 2 years ago.
I am trying to store the state of my app which includes a list of traits that are initialized in the beginning and pass the state between multiple threads but I am getting an error.
use std::sync::{Arc, RwLock};
use std::thread;
use std::time::Duration;
trait TestTrait {
fn test(&self);
}
struct MyTestStruct {}
impl TestTrait for MyTestStruct {
fn test(&self) {
println!("Test trait called");
}
}
struct DataStore {
t: Box<dyn TestTrait>,
}
fn main() {
let s = Arc::new(RwLock::new(DataStore {
t: Box::new(MyTestStruct {}),
}));
let s_clone = s.clone();
thread::spawn(move || {
for i in 1..10 {
s_clone.read().unwrap().t.test();
println!("hi number {} from the spawned thread!", i);
thread::sleep(Duration::from_millis(1));
}
});
for i in 1..5 {
println!("hi number {} from the main thread!", i);
thread::sleep(Duration::from_millis(1));
}
}
When I run this, I get the following error:
error[E0277]: `(dyn TestTrait + 'static)` cannot be sent between threads safely
--> src/main.rs:26:5
|
26 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `(dyn TestTrait + 'static)` cannot be sent between threads safely
|
= help: the trait `std::marker::Send` is not implemented for `(dyn TestTrait + 'static)`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::ptr::Unique<(dyn TestTrait + 'static)>`
= note: required because it appears within the type `std::boxed::Box<(dyn TestTrait + 'static)>`
= note: required because it appears within the type `DataStore`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::RwLock<DataStore>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::sync::RwLock<DataStore>>`
= note: required because it appears within the type `[closure#src/main.rs:26:19: 32:6 s_clone:std::sync::Arc<std::sync::RwLock<DataStore>>]`
error[E0277]: `(dyn TestTrait + 'static)` cannot be shared between threads safely
--> src/main.rs:26:5
|
26 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `(dyn TestTrait + 'static)` cannot be shared between threads safely
|
= help: the trait `std::marker::Sync` is not implemented for `(dyn TestTrait + 'static)`
= note: required because of the requirements on the impl of `std::marker::Sync` for `std::ptr::Unique<(dyn TestTrait + 'static)>`
= note: required because it appears within the type `std::boxed::Box<(dyn TestTrait + 'static)>`
= note: required because it appears within the type `DataStore`
= note: required because of the requirements on the impl of `std::marker::Sync` for `std::sync::RwLock<DataStore>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::sync::RwLock<DataStore>>`
= note: required because it appears within the type `[closure#src/main.rs:26:19: 32:6 s_clone:std::sync::Arc<std::sync::RwLock<DataStore>>]`
How do I fix this?
thread::spawn requires a closure that implements Send. Arc<T> only implements Send if T implements both Send and Sync.
Send and Sync are auto traits. You can add auto traits to a dyn Trait type to provide more information about the erased type:
struct DataStore {
t: Box<dyn TestTrait + Send + Sync>,
}
If you are going to write dyn TestTrait + Send + Sync everywhere, then another option is to declare Send and Sync as supertraits of TestTrait:
trait TestTrait: Send + Sync {
fn test(&self);
}

How to return an iterator over the keys of a HashMap from a trait implementation?

I'm trying to build a simple graph library in Rust. There is a trait Graph that any graph must implement. This trait has only one function at the moment, nodes, which allows iteration of the graph's nodes using a for-in loop.
An implementation of Graph, MapGraph, is a lightweight wrapper around a HashMap. MapGraph must implement the Graph trait method nodes. I'm having problems getting this to work.
Here's the code for Graph:
pub trait Graph<N> {
fn nodes(&self) -> Box<dyn Iterator<Item = &N>>;
}
And here's the code for MapGraph:
use std::collections::HashMap;
use crate::rep::Graph;
pub struct MapGraph<N> {
map: HashMap<N, HashMap<N, ()>>
}
impl<N> MapGraph<N> {
pub fn new(map: HashMap<N, HashMap<N, ()>>) -> Self {
MapGraph { map }
}
}
impl<N> Graph<N> for MapGraph<N> {
fn nodes(&self) -> Box<dyn Iterator<Item=&N>> {
let keys = self.map.keys();
Box::new(keys)
}
}
The compiler gives this error:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/lib.rs:19:29
|
19 | let keys = self.map.keys();
| ^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 18:5...
--> src/lib.rs:18:5
|
18 | / fn nodes(&self) -> Box<dyn Iterator<Item = &N>> {
19 | | let keys = self.map.keys();
20 | |
21 | | Box::new(keys)
22 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:19:20
|
19 | let keys = self.map.keys();
| ^^^^^^^^
= note: but, the lifetime must be valid for the static lifetime...
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::iter::Iterator<Item = &N> + 'static)>
found std::boxed::Box<dyn std::iter::Iterator<Item = &N>>
I've found other references to this error, but those cases don't seem to look like the one I have here.
I'm using Box because the Graph trait has a function that itself returns a trait. What is the correct way to return an Iterator (or any other trait)? gives this approach as one option, and I haven't been able to implement any of the the others. If there's another way to do this, that would be fine.
What are my options for resolving this specific problem?
It works if you explicitly specify that the trait object (dyn Iterator) that you are returning contains references that are tied to the lifetime of self.
Without adding this bound, the compiler cannot infer from the function signature that the iterator cannot be used after self is moved or destroyed. Because the compiler cannot infer this, it cannot safely use self.map.keys() in the function's output.
Working example with this bound added:
pub trait Graph<N> {
fn nodes<'a>(&'a self) -> Box<dyn Iterator<Item = &N> + 'a>;
}
use std::collections::HashMap;
pub struct MapGraph<N> {
map: HashMap<N, HashMap<N, ()>>,
}
impl<N> MapGraph<N> {
pub fn new(map: HashMap<N, HashMap<N, ()>>) -> Self {
MapGraph { map }
}
}
impl<N> Graph<N> for MapGraph<N> {
fn nodes<'a>(&'a self) -> Box<dyn Iterator<Item = &N> + 'a> {
let keys = self.map.keys();
Box::new(keys)
}
}
Playground
I had thought that a bound of Item = &'a N would also be required, but I guess that's already covered by the "+ 'a"...
N.B. that to make sense of an error like:
expected std::boxed::Box<(dyn std::iter::Iterator<Item = &N> + 'static)>
found std::boxed::Box<dyn std::iter::Iterator<Item = &N>>
you have to understand that the compiler, for ergonomic reasons, automatically adds a + 'static lifetime qualifier to any unqualified trait object. This means that an unqualified Box<dyn MyTrait> is transformed by the compiler into a Box<(dyn MyTrait + 'static)>, which in turn means that the object cannot contain any references except those that last for the lifetime of the entire program.
With this in mind you can see why self.map.keys() does not fit this strict bound, and a more specific explicit bound is required.

Resources