Avoiding closure - encapsulating thread variables in a struct - multithreading

I am writing a simple websocket server named BoltServer based on Rust websocket crate(Code is incomplete, I just started). I am using the example program as the base. However the example programs are not modular (having very long methods). So I am trying to break them up into structs and methods. I want to spawn two threads for each client. One sends messages and the other one receives messages. So here, I want to capture all variables used by the thread in a struct and then call the run method in the impl.
extern crate websocket;
extern crate time;
extern crate rustc_serialize;
pub mod ws {
use std::thread;
use std::sync::{Arc, Mutex};
use std::sync::mpsc;
use std::net::ToSocketAddrs;
use websocket;
use websocket::{Server, Message, Sender, Receiver};
use websocket::server::Connection;
use websocket::stream::WebSocketStream;
use std::str::from_utf8;
struct BoltUser {
user_id: u32,
my_tx: mpsc::Sender<String>,
}
struct Broadcaster {
my_rx: mpsc::Receiver<String>,
}
impl Broadcaster {
fn new(receiver: mpsc::Receiver<String>) -> Broadcaster {
Broadcaster { my_rx: receiver }
}
fn run(self) {
while let Ok(msg) = self.my_rx.recv() {
println!("Broadcaster got message: {}", msg);
}
}
}
struct SocketReader {}
impl SocketReader {
fn run(self) {}
}
struct SocketWriter {
my_rx: mpsc::Receiver<String>,
sender: Sender,
}
impl SocketWriter {
fn run(self) {
while let Ok(message) = self.my_rx.recv() {
}
}
}
pub struct BoltServer {
address: String,
connected_users: Arc<Mutex<Vec<BoltUser>>>,
}
impl BoltServer {
pub fn new(address: &str) -> BoltServer {
BoltServer {
address: address.to_string(),
connected_users: Arc::new(Mutex::new(vec![])),
}
}
fn handshake(&mut self,
connection: Connection<WebSocketStream, WebSocketStream>)
-> (SocketWriter, SocketReader) {
let request = connection.read_request().unwrap();
// println!("thread-> Accepting request...");
let response = request.accept();
let (mut sender, mut receiver) = response.send().unwrap().split();
let (user_tx, user_rx) = mpsc::channel::<String>();//Create a channel for writer
let socket_writer = SocketWriter {
my_rx: user_rx,
sender: sender,
};
let socket_reader = SocketReader {};
(socket_writer, socket_reader)
}
pub fn start(&mut self) {
println!("Starting");
let (broadcaster_tx, broadcaster_rx) = mpsc::channel::<String>();
let broadcaster = Broadcaster::new(broadcaster_rx);
let handle = thread::Builder::new()
.name("Broadcaster".to_string())
.spawn(move || broadcaster.run());
let server = Server::bind(&*self.address).unwrap();
let mut user_id: u32 = 0;
// Block and process connection request from a new client
for connection in server {
user_id = user_id + 1;//Create a new user id
let (socket_writer, socket_reader) = self.handshake(connection);
thread::Builder::new()
.name("Socket writer".to_string())
.spawn(move || socket_writer.run());
thread::Builder::new()
.name("Socket reader".to_string())
.spawn(move || socket_reader.run());
}
handle.unwrap().join();
println!("Finished");
}
}
}
The following code gives an idea of what I want to achieve.
// Block and process connection request from a new client
for connection in server {
user_id = user_id + 1;//Create a new user id
let (socket_writer, socket_reader) = self.handshake(connection);
thread::Builder::new().name("Socket writer".to_string()).spawn(move || {
socket_writer.run()
});
thread::Builder::new().name("Socket reader".to_string()).spawn(move || {
socket_reader.run()
});
}
Here I am stuck in the handshake method. I am not able to initialize the SocketWriter struct with the sender that I am getting by calling the split method in the library. I am getting the following compilation error:
error[E0038]: the trait `websocket::Sender` cannot be made into an object
--> src/lib.rs:46:9
|
46 | sender:Sender,
| ^^^^^^^^^^^^^ the trait `websocket::Sender` cannot be made into an object
|
= note: method `send_dataframe` has generic type parameters
= note: method `send_message` has generic type parameters

The error is telling you the immediate problem:
46 | sender:Sender,
| ^^^^^^^^^^^^^ the trait `websocket::Sender` cannot be made into an object
First of all, a variable/field can't have a plain trait type (but &Trait can be possible), but also the websocket::Sender trait is not object safe; it has generic methods which can't work dynamically (ie vtable methods have to have a fixed type).
Instead, you have to have a concrete type (you could also make it a generic struct).
It's not obvious what the right type is, so I like to get the compiler to tell me. So first try the simplest possible:
sender: (),
The compiler replies with some information:
| ^^^^^^ expected (), found struct `websocket::client::Sender`
Ok, let's plug that in:
sender: websocket::client::Sender,
That gives:
46 | sender: websocket::client::Sender,
| ^^^^^^^^^^^^^^^^^^^^^^^^^ expected 1 type arguments, found 0
Ok, that's a generic type. Next try:
sender: websocket::client::Sender<()>,
Finally it gives us the real type:
74 | sender:sender,
| ^^^^^^ expected (), found enum `websocket::WebSocketStream`
So finally we can finish SocketWriter:
struct SocketWriter {
my_rx: mpsc::Receiver<String>,
sender: websocket::client::Sender<websocket::WebSocketStream>,
}
There's a following compile error since the connection you get is a Result<> so you need to check for errors (it compiles if I change to self.handshake(connection.unwrap()), but that's obviously not the best practice.

Related

Can't get async closure to work with Warp::Filter

I am trying to get an async closure working in the and_then filter from Warp.
This is the smallest example I could come up with where I am reasonably sure I didn't leave any important details out:
use std::{convert::Infallible, sync::Arc, thread, time};
use tokio::sync::RwLock;
use warp::Filter;
fn main() {
let man = Manifest::new();
let check = warp::path("updates").and_then(|| async move { GetAvailableBinaries(&man).await });
}
async fn GetAvailableBinaries(man: &Manifest) -> Result<impl warp::Reply, Infallible> {
Ok(warp::reply::json(&man.GetAvailableBinaries().await))
}
pub struct Manifest {
binaries: Arc<RwLock<Vec<i32>>>,
}
impl Manifest {
pub fn new() -> Manifest {
let bins = Arc::new(RwLock::new(Vec::new()));
thread::spawn(move || async move {
loop {
thread::sleep(time::Duration::from_millis(10000));
}
});
Manifest { binaries: bins }
}
pub async fn GetAvailableBinaries(&self) -> Vec<i32> {
self.binaries.read().await.to_vec()
}
}
I am using:
[dependencies]
tokio = { version = "0.2", features = ["full"] }
warp = { version = "0.2", features = ["tls"] }
The error is:
error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnOnce`
--> src/main.rs:9:48
|
9 | let check = warp::path("updates").and_then(|| async move { GetAvailableBinaries(&man).await });
| -------- ^^^^^^^^^^^^^ ------------------------------------ closure is `FnOnce` because it moves the variable `man` out of its environment
| | |
| | this closure implements `FnOnce`, not `Fn`
| the requirement to implement `Fn` derives from here
After making Manifest implement Clone, you can fix the error by balancing when the manifest object is cloned:
fn main() {
let man = Manifest::new();
let check = warp::path("updates").and_then(move || {
let man = man.clone();
async move { get_available_binaries(&man).await }
});
warp::serve(check);
}
This moves man into the closure passed to and_then, then provides a clone of man to the async block each time the closure is executed. The async block then owns that data and can take a reference to it without worrying about executing the future after the data has been deallocated.
I'm not sure this is what you're going for, but this solution builds for me:
use std::{convert::Infallible, sync::Arc, thread, time};
use tokio::sync::RwLock;
use warp::Filter;
fn main() {
let man = Manifest::new();
let check = warp::path("updates").and_then(|| async { GetAvailableBinaries(&man).await });
}
async fn GetAvailableBinaries(man: &Manifest) -> Result<impl warp::Reply, Infallible> {
Ok(warp::reply::json(&man.GetAvailableBinaries().await))
}
#[derive(Clone)]
pub struct Manifest {
binaries: Arc<RwLock<Vec<i32>>>,
}
impl Manifest {
pub fn new() -> Manifest {
let bins = Arc::new(RwLock::new(Vec::new()));
thread::spawn(move || async {
loop {
thread::sleep(time::Duration::from_millis(10000));
//mutate bins here
}
});
Manifest { binaries: bins }
}
pub async fn GetAvailableBinaries(&self) -> Vec<i32> {
self.binaries.read().await.to_vec()
}
}
The move here is the reason the compiler gave a warning regarding the signature: let check = warp::path("updates").and_then(|| async move { GetAvailableBinaries(&man).await });. This means that everything referenced in this closure will be moved into the context of the closure. In this case, the compiler can't guarantee the closure to be Fn but only FnOnce meaning that the closure can only be guaranteed to execute once.

Calling an FnMut callback from another thread

I am writing a Phoenix client library for Rust, taking advantage of the async websocket client from rust-websockets. Right now I am having trouble figuring out how to pass callback functions into the thread that is handling the websocket traffic. I have a simplified struct:
pub struct Socket {
endpoint: String,
connected: Arc<AtomicBool>,
state_change_close: Option<Box<FnMut(String)>>,
}
This struct has a connect function laid out as follows:
pub fn connect(&mut self) -> Result<(), String> {
if self.connected.load(Ordering::Relaxed) {
return Ok(())
}
// Copy endpoint string, otherwise we get an error on thread::spawn
let connection_string = self.endpoint.clone();
let (usr_msg, stdin_ch) = mpsc::channel(0);
let connection_thread = thread::spawn(move || {
// tokio core for running event loop
let mut core = Core::new().unwrap();
let runner = ClientBuilder::new(&connection_string)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (sink, stream) = duplex.split();
stream.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => {
// This is the line where I am trying to call the callback
if let Some(ref mut func) = self.state_change_close {
(func)(e.unwrap().reason);
}
Some(OwnedMessage::Close(e))
},
_ => None,
}
})
.select(stdin_ch.map_err(|_| WebSocketError::NoDataAvailable))
.forward(sink)
});
// Start the event loop
core.run(runner).unwrap();
});
self.connected.store(true, Ordering::Relaxed);
return Ok(())
}
When I try to compile this code I get the following error:
error[E0277]: the trait bound `std::ops::FnMut(std::string::String) + 'static: std::marker::Send` is not satisfied
--> src\socket.rs:99:29
|
99 | let connection_thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ the trait `std::marker::Send` is not implemented for `std::ops::FnMut(std::string::String) + 'static`
|
I have tried changing the type of state_change_close to a Mutex<Option<...>> to avoid thread safety issues, but that did not help with this problem. Is what I'm trying to do possible?
After doing some more research I realized that I just had to modify Option<Box<FnMut(String)>> to be Option<Box<FnMut(String) + Send>> and copy that around my code to everywhere that the callback might be set. Learning more about trait objects!

How to store hyper::server::Server with a default handler as a struct field in a constructor? [duplicate]

This question already has answers here:
"Expected type parameter" error in the constructor of a generic struct
(1 answer)
Mismatched Type while returning generic Struct [duplicate]
(1 answer)
Returning a closure from a function
(4 answers)
Closed 5 years ago.
I am trying to store a hyper::server::Server as a member of my struct (struct MyApp below). I can do this from, for example, my program's main() function.
How can I do this in my struct's MyApp::new() method? I think I need the concrete type for F in MyApp<F>. However, despite trying, I am unable to (correctly) specify a concrete type for this closure.
I can't figure out how to do this. I thought Boxing the closure would work by allowing me to pass the closure as a concrete type, and indeed it does when I do this in main(), but not MyApp::new(). I am hoping there is a way to do this in stable rust, as I would really like to implement a structure which contains a hyper Server.
Here is my struct:
struct MyApp<F> {
hyper_server: Server<MyBoxedClosure<F>, hyper::Body>,
}
And here is the full code that works -- it sets the MyApp.hyper_server field in main():
extern crate hyper;
extern crate futures;
use hyper::Error;
use hyper::server::{Http, Server, NewService, Service, Request, Response};
use hyper::header::ContentLength;
pub struct HelloWorld;
const PHRASE: &'static str = "Hello, World!";
impl Service for HelloWorld {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = futures::future::FutureResult<Self::Response, Self::Error>;
fn call(&self, _req: Request) -> Self::Future {
futures::future::ok(
Response::new()
.with_header(ContentLength(PHRASE.len() as u64))
.with_body(PHRASE),
)
}
}
pub struct MyBoxedClosure<F> {
value: Box<F>,
}
impl<F> NewService for MyBoxedClosure<F>
where
F: Fn() -> std::result::Result<HelloWorld, std::io::Error>,
{
type Request = Request;
type Response = Response;
type Error = Error;
type Instance = HelloWorld;
fn new_service(&self) -> std::result::Result<Self::Instance, std::io::Error> {
self.value.new_service()
}
}
struct MyApp<F> {
hyper_server: Server<MyBoxedClosure<F>, hyper::Body>,
}
fn main() {
let mbc = MyBoxedClosure { value: Box::new(|| Ok(HelloWorld)) };
let addr = "127.0.0.1:3000".parse().unwrap();
let hyper_server = Http::new().bind(&addr, mbc).unwrap();
let my_app = MyApp { hyper_server: hyper_server };
println!("Hello, world!");
}
If I create a MyApp::new() function and call that from the main() instead, I can't figure out how to avoid a compiler error.
impl<F> MyApp<F>
where
F: Fn() -> std::result::Result<HelloWorld, std::io::Error> + Send + Sync,
{
fn new() -> MyApp<F> {
let mbc = MyBoxedClosure { value: Box::new(|| Ok(HelloWorld)) };
let addr = "127.0.0.1:3000".parse().unwrap();
let hyper_server = Http::new().bind(&addr, mbc).unwrap();
MyApp { hyper_server: hyper_server }
}
}
fn main() {
let _my_app = MyApp::new();
println!("Hello, world!");
}
The compiler error is this:
error[E0308]: mismatched types
--> src/main.rs:56:9
|
56 | MyApp { hyper_server: hyper_server }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected type parameter, found closure
|
= note: expected type `MyApp<F>`
found type `MyApp<[closure#src/main.rs:53:52: 53:69]>`

What argument to pass and how to find out its type?

I started with the example for EventLoop from the mio webpage and added the main function:
extern crate mio;
use std::thread;
use mio::{EventLoop, Handler};
struct MyHandler;
impl Handler for MyHandler {
type Timeout = ();
type Message = u32;
fn notify(&mut self, event_loop: &mut EventLoop<MyHandler>, msg: u32) {
assert_eq!(msg, 123);
event_loop.shutdown();
}
}
fn main() {
let mut event_loop = EventLoop::new().unwrap();
let sender = event_loop.channel();
// Send the notification from another thread
thread::spawn(move || {
let _ = sender.send(123);
});
let _ = event_loop.run(&mut MyHandler);
}
Then I had the idea to move the sending thread to a separate function "foo" and started to wonder what type is passed:
extern crate mio;
use std::thread;
use mio::{EventLoop, Handler};
struct MyHandler;
impl Handler for MyHandler {
type Timeout = ();
type Message = u32;
fn notify(&mut self, event_loop: &mut EventLoop<MyHandler>, msg: u32) {
assert_eq!(msg, 123);
event_loop.shutdown();
}
}
fn foo(s: &?) {
let sender = s.clone();
// Send the notification from another thread
thread::spawn(move || {
let _ = sender.send(123);
});
}
fn main() {
let mut event_loop = EventLoop::new().unwrap();
let sender = event_loop.channel();
foo(&sender);
let _ = event_loop.run(&mut MyHandler);
}
So, I let the compiler tell me the type:
fn foo(s: &String) { ...
raises the error:
error: mismatched types:
expected `&collections::string::String`,
found `&mio::event_loop::Sender<_>`
Ok, nice but replacing &String by &mio::event_loop::Sender<u32> raises the error:
error: struct `Sender` is private
fn foo(s: &mio::event_loop::Sender<u32>) {
^
Hm, looks like a dead end, so I thought passing event_loop instead:
fn foo(s: &mio::event_loop::EventLoop<u32>) {
let sender = s.channel().clone();
...
fn main() { ...
foo(&event_loop); ...
but that raises the error:
error: the trait `mio::handler::Handler` is not implemented for the type `u32` [E0277]
src/main.rs:18 fn foo(s: &mio::event_loop::EventLoop<u32>) {
which confuses me completely.
In e.g. C / C++ I would have just passed a pointer either to EventLop or Sender.
What is Rust trying to tell me here? How to get it working in Rust?
Environment: rustc 1.0.0 (a59de37e9 2015-05-13) (built 2015-05-14), mio 0.3.5
The Sender type is re-exported as mio::Sender. The compiler knows that the actual type is mio::event_loop::Sender and reports that. There's currently no way to automatically figure out what type you need in general, but you can look at the documentation of the EventLoop::channel method and see that it returns a Sender. If you click on the Sender type in the documentation you will end up at the documentation of mio::Sender

Recent Changes in Rust Lifetimes

I've been building a websocket server library with Rust for about 4-6 weeks now. Today, I grabbed the latest rustc using the rustup.sh script, and build is breaking from new lifetime requirements. I'm not sure where to look for these changes to figure out what I need to specify in order to meet the requirements? This Week in Rust doesn't have a listing for the recent changes, and the Lifetimes Guide only goes so far with various examples.
Various Struct Definitions
pub struct Server<'a> {
pub sockets: Vec<Socket<'a>>,
pub events: Vec<Event<'a>>,
pub to_event_loop: Sender<Action<'a>>,
pub socket_id: String
}
impl<'a> Clone for Server<'a> {
fn clone(&self) -> Server<'a> {
Server {
sockets: self.sockets.clone(),
events: self.events.clone(),
to_event_loop: self.to_event_loop.clone(),
socket_id: self.socket_id.clone()
}
}
}
pub struct Action<'a> {
pub event: String,
pub socket: Socket<'a>,
pub message: Message
}
pub struct Event<'a> {
pub name: String,
pub execute: fn(data: json::Json, server: super::Server)
}
pub struct Socket<'a> {
pub id: String,
pub stream: TcpStream
}
Function Producing Error
pub fn start(server: Server, ip: &str, port: u16) {
/*
* Communication channel
* - From HTTP Server to Event Loop (Action Passed)
*/
let (to_event_loop, from_conn_pool): (Sender<Action>, Receiver<Action>) = channel();
// Start up event loop
let server_clone = server.clone();
let to_event_loop_clone = to_event_loop.clone();
spawn(proc() {
event_loop(server_clone, from_conn_pool, to_event_loop_clone)
});
// Start TCP server
let listener = TcpListener::bind(ip, port);
let mut acceptor = listener.listen();
for stream in acceptor.incoming() {
match stream {
Ok(stream) => {
let event_loop_msgr = to_event_loop.clone();
spawn(proc() {
process_new_connection(stream, event_loop_msgr)
})
}
Err(e) => {
println!("Error accepting connection: {}", e)
}
}
}
drop(acceptor);
}
Compiler Output
src/rustic_io.rs:47:24: 47:30 error: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
src/rustic_io.rs:47 let server_clone = server.clone();
^~~~~~
src/rustic_io.rs:50:9: 50:70 note: first, the lifetime cannot outlive the call at 50:8...
src/rustic_io.rs:50 event_loop(server_clone, from_conn_pool, to_event_loop_clone)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/rustic_io.rs:50:20: 50:32 note: ...so that argument is valid for the call
src/rustic_io.rs:50 event_loop(server_clone, from_conn_pool, to_event_loop_clone)
^~~~~~~~~~~~
src/rustic_io.rs:38:51: 70:2 note: but, the lifetime must be valid for the anonymous lifetime #1 defined on the block at 38:50...
src/rustic_io.rs:38 pub fn start(server: Server, ip: &str, port: u16) {
src/rustic_io.rs:39
src/rustic_io.rs:40 /*
src/rustic_io.rs:41 * Communication channel
src/rustic_io.rs:42 * - From HTTP Server to Event Loop (Action Passed)
src/rustic_io.rs:43 */
...
src/rustic_io.rs:47:24: 47:30 note: ...so that types are compatible (expected `&server::Server<'_>`, found `&server::Server<'_>`)
src/rustic_io.rs:47 let server_clone = server.clone();
^~~~~~
The compiler is very specific on what the problem is, I just have no idea what syntax it is expecting so that I can satisfy the requirements?

Resources