pub struct AsyncTransporter {
stream: Arc<dyn AsyncRW + Send + Sync>,
}
Suppose that a library expects AsyncTransporter to be Send + Sync, and that dyn AsyncRW is a struct that holds a channel tuple:
pub struct SmolSocket {
channel: (Sender<Arc<Packet>>, Receiver<Arc<Packet>>),
}
How can I make it such that SmolSocket implements Send + Sync safely? Even though SmolSocket holds Sender and Receiver, it's going to be inside an Arc so I don't see how it could go wrong.
Unfortunately AsyncTransporter must be Send+Sync.
Context: AsyncTransporter is the connector in https://docs.rs/hyper/0.14.9/hyper/client/struct.Builder.html#method.build so it automatically needs to be Send+Sync. At least this is what I think.
Related
I am making a library in Rust around a TCP protocol. And I am also trying to make it async runtime independent.
I would like to make it accept ownership of an existing connection OR accept a mutable reference to it. I.e. I want to let the caller say who should drop the connection.
I have this
pub struct ReadableStream<T: AsyncRead + AsyncWrite> {
reader: FramedRead<ReadHalf<T>, WordCodec>,
}
pub struct WriteableStream<T: AsyncRead + AsyncWrite> {
writer: FramedWrite<WriteHalf<T>, WordCodec>,
}
pub struct AsyncStream<T: AsyncRead + AsyncWrite> {
read_stream: ReadableStream<T>,
write_stream: WriteableStream<T>,
}
impl<T: AsyncRead + AsyncWrite> From<T> for AsyncStream<T> {
fn from(stream: T) -> Self {
let parts = stream.split();
Self {
read_stream: ReadableStream::new(parts.0),
write_stream: WriteableStream::new(parts.1),
}
}
}
impl<'stream, T: AsyncRead + AsyncWrite + Unpin> From<&'stream mut T>
for AsyncStream<&'stream mut T>
{
fn from(stream: &'stream mut T) -> Self {
let parts = stream.split();
Self {
read_stream: ReadableStream::new(parts.0),
write_stream: WriteableStream::new(parts.1),
}
}
}
But this fails to compile with "conflicting implementation for AsyncStream<&mut _>". If I uncomment the second implementation, the first one compiles. I can also compile only the second one... But I want to have both. I would expect either to be called based on whether from() was called like AsyncStream::from(source) or like AsyncStream::from(&mut source).
So how could I accomplish that?
Note: FramedRead/FramedWrite are from asynchronous-codec. AsyncRead, AsyncWrite, ReadHalf, WriteHalf are from the futures crate. WordCodec is my implementation of the protocol tailored to asynchronous-codec that is irrelevant to the question at hand.
I decided what I thought is biting the bullet by implementing a method in my struct ( AsyncStream ) to do the conversion, and maybe implement From for the specific runtimes I plan to test against, enabling users to use the struct's method when their runtime is not covered.
This approach works... but furthermore than that, it revealed that what I was trying to do is not exactly feasible for a different reason. I want to have the ability to split the connection into reader and writer, and I can't do that since split() takes ownership.
I'll solve the ownership/mutable reference problem by accepting ownership only, and providing a method that consumes self and returns the wrapped stream. Not exactly the ergonomics I was hoping for, but it addresses the scenario about wanting to let the user use my lib from a point, to a point, and switch from/to something else.
I find myself playing around with warp.
I would like to pass a database Trait to a warp::Filter using static dispatch, such that the concrete database may vary.
I read that Send + Sync are unsafe to implement, but I don't fully understand when it is appropriate to implement them.
Is it sound to implement Send + Sync in this context:
use std::sync::{Arc, RwLock};
use warp::Filter;
use warp::reply::{json};
use serde_json::{Value};
pub trait IDatabase {
fn db_read(&self) -> Value;
fn db_write(&self, data: Value);
}
pub fn get_api_v0_test<T: IDatabase + Send + Sync>(
db: Arc<RwLock<T>>
) -> impl Filter<Extract = impl warp::Reply, Error = warp::Rejection> + Clone {
warp::path!("api" / "v0" / "test")
.and(warp::get())
.map(move || {
json(&db.read().unwrap().db_read())
})
}
Edit:
Yes, this example is completely sound!
As #trentcl and #Masklinn pointed out, I am in fact not implementing Send + Sync, I am simply requiring that any type T passed into the filter, implements the Send + Sync Traits, which is completely safe Rust.
I read that Send + Sync are unsafe to implement, but I don't fully understand when it is appropriate to implement them.
Because Send and Sync are automatically derived, it is only appropriate to derive them explicitely when wrapping raw pointers, usually because you're wrapping a native library, and after having made sure that the trait is appropriate for the underlying type.
Is it sound to implement Send + Sync in this context:
I don't understand what you'd want to "implement Send + Sync" on in this context.
pub trait IDatabase {
Rust is not C#, IWhatever is not a thing.
I am pretty new to Rust so maybe I'm missing something obvious.
This code works fine:
pub fn say_hello() {
let fut = tokio::io::write_all(tokio::io::stdout, "Hello, world!").then(|_| {});
tokio::run(fut);
}
Whereas the following code fails to compile:
pub fn say_hello(w: Box<dyn tokio::io::AsyncWrite>) {
let fut = tokio::io::write_all(w, "Hello, world!").then(|_| {});
tokio::run(fut);
}
The compiler error is:
error[E0277]: `dyn tokio_io::async_write::AsyncWrite` cannot be sent between threads safely
Is there any way to accomplish what I want (actually dispatching dynamically, not just making the function generic).
TL;DR tokio::run expects Future that implements Send and also in 'static life time.
If you add required restrictions to your parameter it will work same as Stdout :
pub fn say_hello(w: Box<dyn tokio::io::AsyncWrite + Send + 'static>) {
let fut = tokio::io::write_all(w, "Hello, world!").then(|_| Ok(()));
tokio::run(fut);
}
Note :
Stdout works because Stdout implementations already contain Send, and it is an owned data for the enclosing scope.
But how Rust is able to know the Future that created by write_all is Send or not ?
Calling write_all with an implementation of AsyncWrite is fine, since tokio::io::write_all expects implementation of AsyncWrite. But tokio::run expects an owned or 'static Future which implements Send
You are trying to run the WriteAll future, but please check this Send implementation on WriteAll, it only implements Send when T and A implements Send. In your case T is your buf's type which is &'static str, it implements Send and A is the implementation of AsyncWrite.
In this definition there is no proclamation that states w is a Send (or it has a 'static/owned lifetime) :
pub fn say_hello(w: Box<dyn tokio::io::AsyncWrite>)
that's why tokio run is not accepting WriteAll Future.
I have a struct which holds an Arc<Receiver<f32>> and I'm trying to add a method which takes ownership of self, and moves the ownership into a new thread and starts it. However, I'm getting the error
error[E0277]: the trait bound `std::sync::mpsc::Receiver<f32>: std::marker::Sync` is not satisfied
--> src/main.rs:19:9
|
19 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `std::sync::mpsc::Receiver<f32>` cannot be shared between threads safely
|
= help: the trait `std::marker::Sync` is not implemented for `std::sync::mpsc::Receiver<f32>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::sync::mpsc::Receiver<f32>>`
= note: required because it appears within the type `Foo`
= note: required because it appears within the type `[closure#src/main.rs:19:23: 22:10 self:Foo]`
= note: required by `std::thread::spawn`
If I change the struct to hold an Arc<i32> instead, or just a Receiver<f32>, it compiles, but not with a Arc<Receiver<f32>>. How does this work? The error doesn't make sense to me as I'm not trying to share it between threads (I'm moving it, not cloning it).
Here is the full code:
use std::sync::mpsc::{channel, Receiver, Sender};
use std::sync::Arc;
use std::thread;
pub struct Foo {
receiver: Arc<Receiver<f32>>,
}
impl Foo {
pub fn new() -> (Foo, Sender<f32>) {
let (sender, receiver) = channel::<f32>();
let sink = Foo {
receiver: Arc::new(receiver),
};
(sink, sender)
}
pub fn run_thread(self) -> thread::JoinHandle<()> {
thread::spawn(move || {
println!("Thread spawned by 'run_thread'");
self.run(); // <- This line gives the error
})
}
fn run(mut self) {
println!("Executing 'run'")
}
}
fn main() {
let (example, sender) = Foo::new();
let handle = example.run_thread();
handle.join();
}
How does this work?
Let's check the requirements of thread::spawn again:
pub fn spawn<F, T>(f: F) -> JoinHandle<T>
where
F: FnOnce() -> T,
F: Send + 'static, // <-- this line is important for us
T: Send + 'static,
Since Foo contains an Arc<Receiver<_>>, let's check if and how Arc implements Send:
impl<T> Send for Arc<T>
where
T: Send + Sync + ?Sized,
So Arc<T> implements Send if T implements Send and Sync. And while Receiver implements Send, it does not implement Sync.
So why does Arc have such strong requirements for T? T also has to implement Send because Arc can act like a container; if you could just hide something that doesn't implement Send in an Arc, send it to another thread and unpack it there... bad things would happen. The interesting part is to see why T also has to implement Sync, which is apparently also the part you are struggling with:
The error doesn't make sense to me as I'm not trying to share it between threads (I'm moving it, not cloning it).
The compiler can't know that the Arc in Foo is in fact not shared. Consider if you would add a #[derive(Clone)] to Foo later (which is possible without a problem):
fn main() {
let (example, sender) = Foo::new();
let clone = example.clone();
let handle = example.run_thread();
clone.run();
// oopsie, now the same `Receiver` is used from two threads!
handle.join();
}
In the example above there is only one Receiver which is shared between threads. And this is no good, since Receiver does not implement Sync!
To me this code raises the question: why the Arc in the first place? As you noticed, without the Arc, it works without a problem: you clearly state that Foo is the only owner of the Receiver. And if you are "not trying to share [the Receiver]" anyway, there is no point in having multiple owners.
I have some mutable state I need to share between threads. I followed the concurrency section of the Rust book, which shares a vector between threads and mutates it.
Instead of a vector, I need to share a generic struct that is ultimately monomorphized. Here is a distilled example of what I'm trying:
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
use std::marker::PhantomData;
trait Memory {}
struct SimpleMemory;
impl Memory for SimpleMemory {}
struct SharedData<M: Memory> {
value: usize,
phantom: PhantomData<M>,
}
impl<M: Memory> SharedData<M> {
fn new() -> Self {
SharedData {
value: 0,
phantom: PhantomData,
}
}
}
fn main() {
share(SimpleMemory);
}
fn share<M: Memory>(memory: M) {
let data = Arc::new(Mutex::new(SharedData::<M>::new()));
for i in 0..3 {
let data = data.clone();
thread::spawn(move || {
let mut data = data.lock().unwrap();
data.value += i;
});
}
thread::sleep(Duration::from_millis(50));
}
The compiler complains with the following error:
error[E0277]: the trait bound `M: std::marker::Send` is not satisfied
--> src/main.rs:37:9
|
37 | thread::spawn(move || {
| ^^^^^^^^^^^^^
|
= help: consider adding a `where M: std::marker::Send` bound
= note: required because it appears within the type `std::marker::PhantomData<M>`
= note: required because it appears within the type `SharedData<M>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Mutex<SharedData<M>>`
= note: required because of the requirements on the impl of `std::marker::Send` for `std::sync::Arc<std::sync::Mutex<SharedData<M>>>`
= note: required because it appears within the type `[closure#src/main.rs:37:23: 40:10 data:std::sync::Arc<std::sync::Mutex<SharedData<M>>>, i:usize]`
= note: required by `std::thread::spawn`
I'm trying to understand why M would need to implement Send, and what the appropriate way to accomplish this is.
I'm trying to understand why M would need to implement Send, ...
Because, as stated by the Send documentation:
Types that can be transferred across thread boundaries.
If it's not Send, it is by definition not safe to send to another thread.
Almost all of the information you need is right there in the documentation:
thread::spawn requires the callable you give it to be Send.
You're using a closure, which is only Send if all the values it captures are Send. This is true in general of most types (they are Send if everything they're made of is Send, and similarly for Sync).
You're capturing data, which is an Arc<T>, which is only Send if T is Send.
T is a Mutex<U>, which is only Send if U is Send.
U is M. Thus, M must be Send.
In addition, note that thread::spawn also requires that the callable be 'static, so you need that too. It needs that because if it didn't require that, it'd have no guarantee that the value will continue to exist for the entire lifetime of the thread (which may or may not outlive the thread that spawned it).
..., and what the appropriate way to accomplish this is.
Same way as any other constraints: M: 'static + Send + Memory.