Hyper TLS cannot be run in Tokio Thread - Does not implement Send - multithreading

In the server example provided by the HyperTLS library, the server provided cannot be run in a tokio::spawn() thread without throwing a litany of errors all stemming from the fact that:
dyn futures_util::Stream<Item = Result<tokio_rustls::server::TlsStream<tokio::net::TcpStream>, std::io::Error>> cannot be sent between threads safely.
The server definition below is exactly like that found in the example with the exception of the config file used to pass in config information.
#[derive(Debug, Clone)]
pub struct HTTPSServer {}
impl HTTPSServer {
pub async fn start<'a>(config: HTTPSServerConfig<'a>) -> Result<(), HTTPSServerError> {
// Build TLS configuration.
let tls_cfg = {
// Load public certificate.
let certs = load_certs(&config.cert_path)?;
// Load private key.
let key = load_private_key(&config.key_path)?;
// Do not use client certificate authentication.
let mut cfg = rustls::ServerConfig::new(rustls::NoClientAuth::new());
// Select a certificate to use.
cfg.set_single_cert(certs, key)
.map_err(|e| error(format!("failed to set single cert: {}", e)))?;
// Configure ALPN to accept HTTP/2, HTTP/1.1 in that order.
cfg.set_protocols(&[b"h2".to_vec(), b"http/1.1".to_vec()]);
sync::Arc::new(cfg)
};
let tls_acceptor = TlsAcceptor::from(tls_cfg);
// Create a TCP listener via tokio.
let tcp_listener = TcpListener::bind(&config.addr)
.await
.map_err(|e| {
eprintln!("failed to create TCP Listener: {:?}", e);
e
})?;
// Prepare a long-running future stream to accept and serve clients.
let incoming_tls_stream = stream! {
loop {
let (socket, _) = tcp_listener.accept().await?;
let stream = tls_acceptor.accept(socket)
.map_err(|e| {
println!("[!] Voluntary server halt due to client-connection error...");
// Errors could be handled here, instead of server aborting.
// Ok(None)
error(format!("TLS Error: {:?}", e))
});
yield stream.await;
}
};
// Setup router
let service = make_service_fn(|_| async {
Ok::<_, io::Error>(service_fn(echo))
});
// Build server
let server = Server::builder(HyperAcceptor {
acceptor: Box::pin(incoming_tls_stream),
}).serve(service);
// Run the future, keep going until an error occurs.
println!("[https-server]: Starting to serve on https://{}.", &config.fqdn);
server
.await
.map_err(|e| {
eprintln!("[https-server]: error: {:?}", e);
HTTPSServerError::from(e)
})
}
}
The following is the main method to run the server in a separate thread
#[tokio::main]
async fn main() {
// Start the server asynchronously
let fqdn = "https://localhost:8080";
let addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127,0,0,1)), 8080);
let cert_path = "localhost.crt";
let key_path = "localhost.key";
let ca_path = "RootCA.crt";
let client_config = HTTPSClientConfig {
fqdn,
ca_path: Some(ca_path)
};
let server_config = HTTPSServerConfig {
fqdn,
addr,
cert_path,
key_path,
ca_path,
};
// Spawn server in a new thread!
tokio::spawn(start(server_config)).await.unwrap();
// A function to kick off a client call to the server.
client(client_config);
}
async fn start<'a>(server_config: HTTPSServerConfig<'a>) {
HTTPSServer::start(server_config)
.await
.map_err(|e| {
eprintln!("[https-server]: failed to start server: {:?}", e)
});
}
The Error
I am not exactly sure how to solve the following error as it doesn't seem trivial to ensure a dyn type implements Send. Not sure how to solve this.
error: future cannot be sent between threads safely
--> https/src/main.rs:61:5
|
61 | tokio::spawn(start(server_config)).await.unwrap();
| ^^^^^^^^^^^^ future returned by `start` is not `Send`
|
::: /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.2.0/src/task/spawn.rs:129:21
|
129 | T: Future + Send + 'static,
| ---- required by this bound in `tokio::spawn`
|
= help: the trait `Send` is not implemented for `dyn futures_util::Stream<Item = Result<tokio_rustls::server::TlsStream<tokio::net::TcpStream>, std::io::Error>>`
note: future is not `Send` as it awaits another future which is not `Send`
--> https/src/server.rs:89:9
|
89 | server
| ^^^^^^ await occurs here on type `hyper::Server<HyperAcceptor<'_>, hyper::service::make::MakeServiceFn<[closure#https/src/server.rs:77:39: 79:10]>>`, which is not `Send`

Related

Making parallel requests to Redis with Fred

I'm a complete Rust newbie and just tried to convert a simple microservice to Rust. It needs to do many parallel Redis requests for HTTP requests it gets and I'm a bit puzzled with the language syntax. I'm trying to do multiple Redis queries in parallel in an actix-web handler. I have the following type for the Redis GET function in Fred:
fn get<R, K>(&self, key: K) -> AsyncResult<R>
where
R: FromRedis + Unpin + Send,
K: Into<RedisKey>,
docs here: https://docs.rs/fred/5.2.0/fred/interfaces/trait.KeysInterface.html#method.get
In my own code I would then have a for-loop like this:
let resp_futures = vec!{};
for key in keys.iter() {
resp_futures.push(state.redis.get(key));
}
let resps = join_all(resp_futures).await;
Each Redis query should basically return an Option. However, this doesn't work due to some issues with type inference. Any ideas what's the correct way to send parallel Redis requests with the Fred Redis library? The complete server with some unnecessary stuff removed is the following:
use actix_web::{get, post, web, App, HttpResponse, HttpServer, Responder};
use actix_web::http::header::ContentType;
use serde::Deserialize;
use std::env;
use std::sync::Arc;
use fred::prelude::*;
use fred::pool::RedisPool;
use futures::future::join_all;
//
// Server configuration
//
fn get_redis_url() -> String {
match env::var("REDIS_URL") {
Err(_) => panic!("REDIS_URL environment variable not set."),
Ok(s) => s
}
}
struct AppState {
redis: Arc<RedisPool>
}
//
// Request parsing
//
fn get_redis_keys_from_accounts(accounts: Option<&String>) -> Vec<String> {
match accounts {
None => vec!{},
Some(s) => s.split(",").map(|s| "sid:".to_owned() + s).collect()
}
}
fn get_redis_keys_from_app_accounts(accounts: Option<&String>) -> Vec<String> {
match accounts {
None => vec!{},
Some(s) => s.split(",").map(|s| "aid:".to_owned() + s).collect()
}
}
fn get_redis_keys(req: web::Form<RouteRequest>) -> Vec<String> {
let ids = get_redis_keys_from_accounts(req.ids.as_ref());
let accounts = get_redis_keys_from_app_accounts(req.accounts.as_ref());
let aliases = get_redis_keys_from_app_accounts(req.aliases.as_ref());
ids.into_iter().chain(accounts.into_iter()).chain(aliases.into_iter()).collect()
}
//
// Request handling
//
#[derive(Debug, Deserialize)]
struct RouteRequest {
ids: Option<String>,
accounts: Option<String>,
aliases: Option<String>
}
#[post("/v1/route")]
async fn route(state: web::Data<AppState>, req: web::Form<RouteRequest>) -> impl Responder {
let keys = get_redis_keys(req);
// TODO: Fix this!
let resp_futures = vec!{};
for key in keys.iter() {
resp_futures.push(state.redis.get(key));
}
let resps = join_all(resp_futures).await;
HttpResponse::Ok().content_type(ContentType::json()).body(r#"{"status": "ok"}"#)
}
#[actix_web::main]
async fn main() -> std::io::Result<()> {
println!("Connecting to Redis backend");
let url = get_redis_url();
let config = match RedisConfig::from_url(&url) {
Ok(x) => x,
Err(_) => panic!("Invalid redis URL")
};
let policy = ReconnectPolicy::default();
let redis = Arc::new(match RedisPool::new(config, 5) {
Ok(x) => x,
Err(_) => panic!("Unable to create Redis connection pool")
});
let _ = redis.connect(Some(policy));
println!("Starting HTTP server");
HttpServer::new(move || {
App::new()
.app_data(web::Data::new(AppState {redis: redis.clone()}))
.service(route)
})
.bind(("0.0.0.0", 8080))?
.run()
.await
}
Output of cargo check is :
error[E0698]: type inside `async fn` body must be known in this context
--> src/main.rs:76:39
|
76 | resp_futures.push(state.redis.get(key));
| ^^^ cannot infer type for type parameter `R` declared on the associated function `get`
|
note: the type is part of the `async fn` body because of this `await`
--> src/main.rs:78:39
|
78 | let resps = join_all(resp_futures).await;
| ^^^^^^
At line 70 you have to give a hint about the type which should be used. For a String the following line should be used:
resp_futures.push(state.redis.get::<String, _>(key));

P2P communications in Rust

I am trying to write a P2P node. A node contains a list of peers, opens a listening port to let other nodes establish a communication channel and, at the same time, actively tries to establish the connection with some other chosen nodes. This choice is as such that there is a single connection between each pair of nodes (not implemented in the following snippet).
I am trying to achieve this with Tokio using futures combinators.
main.rs
use failure::Error;
pub mod networking {
use failure::Error;
use std::net::SocketAddr;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use crate::Config;
use futures::Future;
pub fn start(cfg: &Config) -> Result<(), Error> {
let myself = cfg.myself.parse::<SocketAddr>()?;
let others = cfg
.others
.iter()
.filter_map(|s| s.parse().ok())
.collect::<Vec<SocketAddr>>();
let server = TcpListener::bind(&myself)?
.incoming()
.for_each(|socket| {
println!("Got a socket: {:?}", socket);
future::ok(())
})
.map_err(|e| eprintln!("Error connecting: {:?}", e));
let client = TcpStream::connect(&others[0])
.map(|socket| {
println!("Got a socket: {:?}", socket);
})
.map_err(|e| eprintln!("Error connecting: {:?}", e));
let future = server.join(client);
tokio::run(future);
Ok(())
}
}
struct Config {
myself: String,
others: Vec<String>,
}
fn main() -> Result<(), Error> {
let config = Config {
myself: "127.0.0.1:2501".to_string(),
others: vec!["127.0.0.1:2502".to_string(), "127.0.0.1:2503".to_string()],
};
networking::start(&config)
}
(playground)
This does not work:
error[E0271]: type mismatch resolving `<futures::future::join::Join<futures::future::map_err::MapErr<futures::stream::for_each::ForEach<tokio_tcp::incoming::Incoming, [closure#src/main.rs:23:23: 26:14], futures::future::result_::FutureResult<(), std::io::Error>>, [closure#src/main.rs:27:22: 27:64]>, futures::future::map_err::MapErr<futures::future::map::Map<tokio_tcp::stream::ConnectFuture, [closure#src/main.rs:30:18: 32:14]>, [closure#src/main.rs:33:22: 33:64]>> as futures::future::Future>::Item == ()`
--> src/main.rs:37:9
|
37 | tokio::run(future);
| ^^^^^^^^^^ expected tuple, found ()
|
= note: expected type `((), ())`
found type `()`
= note: required by `tokio::runtime::threadpool::run`
I understand what the compiler says, but I have no clue what exactly I have to correct.
What should I correct to get the types right?
thanks for the hints,
let future = server.join(client);
has to be corrected to
let future = server.join(client).map(|_| ())

Example usage of hyper with bb8 and postgres

I want to use hyper with bb8 and tokio-postgres. In every request I want to acquire a new connection from the pool. Can anybody provide me some example for this scenario?
Currently I do it like this:
fn main() {
let addr = "127.0.0.1:3000".parse().unwrap();
let pg_mgr =
PostgresConnectionManager::new("postgresql://auth:auth#localhost:5433/auth", NoTls);
rt::run(future::lazy(move || {
Pool::builder()
.build(pg_mgr)
.map_err(|e| eprintln!("Database error: {}", e))
.and_then(move |pool| {
let service = || service_fn(|req| router(req, pool.clone()));
let server = Server::bind(&addr)
.serve(service)
.map_err(|e| eprintln!("Server error: {}", e));
println!("Listening on http://{}", addr);
server
})
}))
}
fn router(
_req: Request<Body>,
_pool: Pool<PostgresConnectionManager<NoTls>>,
) -> Result<Response<Body>, hyper::Error> {
// do some staff with pool
}
But it won't compile:
error[E0597]: `pool` does not live long enough
--> src/main.rs:22:63
|
22 | let service = || service_fn(|req| router(req, pool.clone()));
| -- -----------------------------^^^^----------
| | | |
| | | borrowed value does not live long enough
| | returning this value requires that `pool` is borrowed for `'static`
| value captured here
...
30 | })
| - `pool` dropped here while still borrowed
What am I doing wrong? How to make my case work correctly?
The solution is pretty simple but to understand the problem I want to provide some additional info...
When you call and_then on a future to get the result, it passes the value of the variable to the closure passed to and_then which gives you ownership of that data.
The method serve on hypers builder (returned by Server::bind), expects for the closure to have a static lifetime.
Now to address the problem:
Good: pass the value of the closure into serve, this moves it, transferring the ownership.
Good: service_fn is defined outside of the and_then closure so that function lives long enough
Bad: The closure uses the local variable pool to pass it to the service_fn.
To resolve the problem, just move the local data into your closure like so:
let service = move || service_fn(|req| router(req, pool));
Solution found here
The simplest solution looks like:
fn main() {
let addr = "127.0.0.1:3000".parse().unwrap();
let pg_mgr =
PostgresConnectionManager::new("postgresql://auth:auth#localhost:5433/auth", NoTls);
rt::run(future::lazy(move || {
Pool::builder()
.build(pg_mgr)
.map_err(|_| eprintln!("kek"))
.and_then(move |pool| {
let service = move || {
let pool = pool.clone();
service_fn(move |req| router(req, &pool))
};
let server = Server::bind(&addr)
.serve(service)
.map_err(|e| eprintln!("Server error: {}", e));
println!("Listening on http://{}", addr);
server
})
}))
}
fn router(
_req: Request<Body>,
_pool: &Pool<PostgresConnectionManager<NoTls>>,
) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
// some staff
}
It is also possible to construct service outside of rt::run with Arc and Mutex:
fn main() {
let addr = "127.0.0.1:3000".parse().unwrap();
let pg_mgr =
PostgresConnectionManager::new("postgresql://auth:auth#localhost:5433/auth", NoTls);
let pool: Arc<Mutex<Option<Pool<PostgresConnectionManager<NoTls>>>>> =
Arc::new(Mutex::new(None));
let pool2 = pool.clone();
let service = move || {
let pool = pool.clone();
service_fn(move |req| {
let locked = pool.lock().unwrap();
let pool = locked
.as_ref()
.expect("bb8 should be initialized before hyper");
router(req, pool)
})
};
rt::run(future::lazy(move || {
Pool::builder()
.build(pg_mgr)
.map_err(|_| eprintln!("kek"))
.and_then(move |pool| {
*pool2.lock().unwrap() = Some(pool);
let server = Server::bind(&addr)
.serve(service)
.map_err(|e| eprintln!("Server error: {}", e));
println!("Listening on http://{}", addr);
server
})
}))
}
fn router(
_req: Request<Body>,
_pool: &Pool<PostgresConnectionManager<NoTls>>,
) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
// some staff
}

Calling an FnMut callback from another thread

I am writing a Phoenix client library for Rust, taking advantage of the async websocket client from rust-websockets. Right now I am having trouble figuring out how to pass callback functions into the thread that is handling the websocket traffic. I have a simplified struct:
pub struct Socket {
endpoint: String,
connected: Arc<AtomicBool>,
state_change_close: Option<Box<FnMut(String)>>,
}
This struct has a connect function laid out as follows:
pub fn connect(&mut self) -> Result<(), String> {
if self.connected.load(Ordering::Relaxed) {
return Ok(())
}
// Copy endpoint string, otherwise we get an error on thread::spawn
let connection_string = self.endpoint.clone();
let (usr_msg, stdin_ch) = mpsc::channel(0);
let connection_thread = thread::spawn(move || {
// tokio core for running event loop
let mut core = Core::new().unwrap();
let runner = ClientBuilder::new(&connection_string)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (sink, stream) = duplex.split();
stream.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => {
// This is the line where I am trying to call the callback
if let Some(ref mut func) = self.state_change_close {
(func)(e.unwrap().reason);
}
Some(OwnedMessage::Close(e))
},
_ => None,
}
})
.select(stdin_ch.map_err(|_| WebSocketError::NoDataAvailable))
.forward(sink)
});
// Start the event loop
core.run(runner).unwrap();
});
self.connected.store(true, Ordering::Relaxed);
return Ok(())
}
When I try to compile this code I get the following error:
error[E0277]: the trait bound `std::ops::FnMut(std::string::String) + 'static: std::marker::Send` is not satisfied
--> src\socket.rs:99:29
|
99 | let connection_thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ the trait `std::marker::Send` is not implemented for `std::ops::FnMut(std::string::String) + 'static`
|
I have tried changing the type of state_change_close to a Mutex<Option<...>> to avoid thread safety issues, but that did not help with this problem. Is what I'm trying to do possible?
After doing some more research I realized that I just had to modify Option<Box<FnMut(String)>> to be Option<Box<FnMut(String) + Send>> and copy that around my code to everywhere that the callback might be set. Learning more about trait objects!

Lifetime error when spawning a task

I can not understand this error regarding lifetimes. Proxy is a struct:
impl Proxy {
pub fn new<A: ip::ToSocketAddr>(address: A) -> Proxy {
Proxy {
address: address.to_socket_addr().unwrap()
}
}
pub fn listen_and_proxy(&self) {
println!("listening {}", self.address);
for stream in TcpListener::bind(self.address).listen().incoming() {
let stream = stream.unwrap();
spawn(move || { // <- this causes an error
let mut stream = stream;
let name = stream.peer_name();
println!("client: {} -> {}", name, self.handle(stream));
});
}
}
}
Output error:
cannot infer an appropriate lifetime due to conflicting requirements
What is wrong in the code?
You are trying to call self.handle(stream) which would require moving self into the newly spawned thread. This cannot be done.

Resources