I want to use hyper with bb8 and tokio-postgres. In every request I want to acquire a new connection from the pool. Can anybody provide me some example for this scenario?
Currently I do it like this:
fn main() {
let addr = "127.0.0.1:3000".parse().unwrap();
let pg_mgr =
PostgresConnectionManager::new("postgresql://auth:auth#localhost:5433/auth", NoTls);
rt::run(future::lazy(move || {
Pool::builder()
.build(pg_mgr)
.map_err(|e| eprintln!("Database error: {}", e))
.and_then(move |pool| {
let service = || service_fn(|req| router(req, pool.clone()));
let server = Server::bind(&addr)
.serve(service)
.map_err(|e| eprintln!("Server error: {}", e));
println!("Listening on http://{}", addr);
server
})
}))
}
fn router(
_req: Request<Body>,
_pool: Pool<PostgresConnectionManager<NoTls>>,
) -> Result<Response<Body>, hyper::Error> {
// do some staff with pool
}
But it won't compile:
error[E0597]: `pool` does not live long enough
--> src/main.rs:22:63
|
22 | let service = || service_fn(|req| router(req, pool.clone()));
| -- -----------------------------^^^^----------
| | | |
| | | borrowed value does not live long enough
| | returning this value requires that `pool` is borrowed for `'static`
| value captured here
...
30 | })
| - `pool` dropped here while still borrowed
What am I doing wrong? How to make my case work correctly?
The solution is pretty simple but to understand the problem I want to provide some additional info...
When you call and_then on a future to get the result, it passes the value of the variable to the closure passed to and_then which gives you ownership of that data.
The method serve on hypers builder (returned by Server::bind), expects for the closure to have a static lifetime.
Now to address the problem:
Good: pass the value of the closure into serve, this moves it, transferring the ownership.
Good: service_fn is defined outside of the and_then closure so that function lives long enough
Bad: The closure uses the local variable pool to pass it to the service_fn.
To resolve the problem, just move the local data into your closure like so:
let service = move || service_fn(|req| router(req, pool));
Solution found here
The simplest solution looks like:
fn main() {
let addr = "127.0.0.1:3000".parse().unwrap();
let pg_mgr =
PostgresConnectionManager::new("postgresql://auth:auth#localhost:5433/auth", NoTls);
rt::run(future::lazy(move || {
Pool::builder()
.build(pg_mgr)
.map_err(|_| eprintln!("kek"))
.and_then(move |pool| {
let service = move || {
let pool = pool.clone();
service_fn(move |req| router(req, &pool))
};
let server = Server::bind(&addr)
.serve(service)
.map_err(|e| eprintln!("Server error: {}", e));
println!("Listening on http://{}", addr);
server
})
}))
}
fn router(
_req: Request<Body>,
_pool: &Pool<PostgresConnectionManager<NoTls>>,
) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
// some staff
}
It is also possible to construct service outside of rt::run with Arc and Mutex:
fn main() {
let addr = "127.0.0.1:3000".parse().unwrap();
let pg_mgr =
PostgresConnectionManager::new("postgresql://auth:auth#localhost:5433/auth", NoTls);
let pool: Arc<Mutex<Option<Pool<PostgresConnectionManager<NoTls>>>>> =
Arc::new(Mutex::new(None));
let pool2 = pool.clone();
let service = move || {
let pool = pool.clone();
service_fn(move |req| {
let locked = pool.lock().unwrap();
let pool = locked
.as_ref()
.expect("bb8 should be initialized before hyper");
router(req, pool)
})
};
rt::run(future::lazy(move || {
Pool::builder()
.build(pg_mgr)
.map_err(|_| eprintln!("kek"))
.and_then(move |pool| {
*pool2.lock().unwrap() = Some(pool);
let server = Server::bind(&addr)
.serve(service)
.map_err(|e| eprintln!("Server error: {}", e));
println!("Listening on http://{}", addr);
server
})
}))
}
fn router(
_req: Request<Body>,
_pool: &Pool<PostgresConnectionManager<NoTls>>,
) -> impl Future<Item = Response<Body>, Error = hyper::Error> {
// some staff
}
Related
I'm trying to use a Mutex<Sender> inside an async closure but I'm not entirely sure why I'm getting the error below:
error: lifetime may not live long enough
--> src/main.rs:120:41
|
120 | io.add_method(MARK_ITSELF, move |_| async {
| ________________________________--------_^
| | | |
| | | return type of closure `impl std::future::Future<Output = Result<jsonrpc::serde_json::Value, jsonrpc_http_server::jsonrpc_core::Error>>` contains a lifetime `'2`
| | lifetime `'1` represents this closure's body
121 | | trace!("Mark itself!");
122 | | let tx = sender.lock().map_err(to_internal)?;
123 | | tx.send(Action::MarkItself)
124 | | .map_err(to_internal)
125 | | .map(|_| Value::Bool(true))
126 | | });
| |_____^ returning this value requires that `'1` must outlive `'2`
|
= note: closure implements `Fn`, so references to captured variables can't escape the closure
My main function looks like this:
use jsonrpc::serde_json::Value;
use jsonrpc::Error as ClientError;
use jsonrpc::{
serde_json::value::RawValue,
simple_http::{self, SimpleHttpTransport},
Client,
};
use jsonrpc_http_server::jsonrpc_core::{Error as ServerError, IoHandler};
use jsonrpc_http_server::ServerBuilder;
use log::{debug, error, trace};
use serde::Deserialize;
use std::rc::Rc;
use std::sync::mpsc::Receiver;
use std::sync::Mutex;
use std::thread;
use std::{
env, fmt,
net::SocketAddr,
sync::mpsc::{channel, Sender},
};
const START_ROLL_CALL: &str = "start_roll_call";
const MARK_ITSELF: &str = "mark_itself";
fn create_client(url: &str, user: &str, pass: &str) -> Result<Client, simple_http::Error> {
let t = SimpleHttpTransport::builder()
.url(url)?
.auth(user, Some(pass))
.build();
Ok(Client::with_transport(t))
}
fn spawn_worker() -> Result<Sender<Action>, failure::Error> {
let (tx, rx): (Sender<Action>, Receiver<Action>) = channel();
let next: SocketAddr = env::var("NEXT")?.parse()?;
thread::spawn(move || {
let remote = Remote::new(next).unwrap();
let mut in_roll_call = false;
for action in rx.iter() {
match action {
Action::StartRollCall => {
if !in_roll_call {
if remote.start_roll_call().is_ok() {
debug!("ON");
in_roll_call = true;
}
} else {
if remote.mark_itself().is_ok() {
debug!("OFF");
in_roll_call = false;
}
}
}
Action::MarkItself => {
if in_roll_call {
if remote.mark_itself().is_ok() {
debug!("OFF");
in_roll_call = false;
}
} else {
debug!("SKIP");
}
}
}
}
});
Ok(tx)
}
enum Action {
StartRollCall,
MarkItself,
}
struct Remote {
client: Client,
}
impl Remote {
fn new(addr: SocketAddr) -> Result<Self, simple_http::Error> {
let url = format!("http://{}", addr);
let client = create_client(&url, "", "")?;
Ok(Self { client })
}
fn call_method<T>(&self, method: &str, params: &[Box<RawValue>]) -> Result<T, ClientError>
where
T: for<'de> Deserialize<'de>,
{
let request = self.client.build_request(method, params);
self.client
.send_request(request)
.and_then(|res| res.result::<T>())
}
fn start_roll_call(&self) -> Result<bool, ClientError> {
self.call_method(START_ROLL_CALL, &[])
}
fn mark_itself(&self) -> Result<bool, ClientError> {
self.call_method(MARK_ITSELF, &[])
}
}
fn main() -> Result<(), failure::Error> {
env_logger::init();
let tx = spawn_worker()?;
let addr: SocketAddr = env::var("ADDRESS")?.parse()?;
let mut io = IoHandler::default();
let sender = Mutex::new(tx.clone());
io.add_method(START_ROLL_CALL, move |_| async move {
trace!("Starting roll call!");
let tx = sender.lock().map_err(to_internal)?;
tx.send(Action::StartRollCall)
.map_err(to_internal)
.map(|_| Value::Bool(true))
});
let sender = Mutex::new(tx.clone());
io.add_method(MARK_ITSELF, move |_| async {
trace!("Mark itself!");
let tx = sender.lock().map_err(to_internal)?;
tx.send(Action::MarkItself)
.map_err(to_internal)
.map(|_| Value::Bool(true))
});
let server = ServerBuilder::new(io).start_http(&addr)?;
Ok(server.wait())
}
fn to_internal<E: fmt::Display>(err: E) -> ServerError {
error!("Error: {}", err);
ServerError::internal_error()
}
The main idea is to pass the mspc sender to the closure so that the method can send the Action(An enum). Is there something I'm doing wrong?
The problem is add_method requires that
your closure is static, and
the Future returned from your closure is static.
Try this
let sender = Arc::new(Mutex::new(tx.clone()));
io.add_method(START_ROLL_CALL, move |_| {
let cloned_sender = sender.clone();
async move {
trace!("Starting roll call!");
let tx = cloned_sender.lock().map_err(to_internal)?;
tx.send(Action::StartRollCall)
.map_err(to_internal)
.map(|_| Value::Bool(true))
}
});
Side note: you are using sync Mutex in an async environment. This undermines the benefits of async. Consider using async Mutex such as Tokio Mutex or async-lock Mutex.
I want to write a program that passes messages from a local websocket to an remote and vice versa, but when I add a while to spawn threads I get an error. How can I fix this?
The exact same error also shows up with ws_local.
error[E0382]: use of moved value: `write_remote`
|
42 | let (mut write_remote, mut read_remote) = ws_remote.split();
| ---------------- move occurs because `write_remote` has type `SplitSink<WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>>, Message>`, which does not implement the `Copy` trait
...
70 | let _handle_two = task::spawn(async move {
| ________________________________________________^
71 | | while let Some(msg) = read_local.next().await {
72 | | let msg = msg?;
73 | | if msg.is_text() || msg.is_binary() {
74 | | write_remote.send(msg).await;
| | ------------ use occurs due to use in generator
... |
78 | | Result::<(), tungstenite::Error>::Ok(())
79 | | });
| |_______^ value moved here, in previous iteration of loop
Here is my code:
#![cfg_attr(
all(not(debug_assertions), target_os = "windows"),
windows_subsystem = "windows"
)]
use tokio::net::{TcpListener, TcpStream};
use futures_util::{future, SinkExt, StreamExt, TryStreamExt};
use tokio_tungstenite::{
connect_async,
accept_async,
tungstenite::{Result},
};
use http::Request;
use tokio::sync::oneshot;
use futures::{
future::FutureExt, // for `.fuse()`
pin_mut,
select,
};
use tokio::io::AsyncWriteExt;
use std::io;
use std::net::SocketAddr;
use std::thread;
use tokio::spawn;
use tokio::task;
async fn client() -> Result<()> {
// Client
let request = Request::builder()
.method("GET")
.header("Host", "demo.piesocket.com")
// .header("Origin", "https://example.com/")
.header("Connection", "Upgrade")
.header("Upgrade", "websocket")
.header("Sec-WebSocket-Version", "13")
.header("Sec-WebSocket-Key", tungstenite::handshake::client::generate_key())
.uri("wss://demo.piesocket.com/v3/channel_1?api_key=VCXCEuvhGcBDP7XhiJJUDvR1e1D3eiVjgZ9VRiaV¬ify_self")
.body(())?;
let (mut ws_remote, _) = connect_async(request).await?;
let (mut write_remote, mut read_remote) = ws_remote.split();
let listener = TcpListener::bind("127.0.0.1:4444").await.expect("Can't listen");
while let Ok((stream, _)) = listener.accept().await {
let mut ws_local = accept_async(stream).await.expect("Failed to accept");
let (mut write_local, mut read_local) = ws_local.split();
// read_remote.try_filter(|msg| future::ready(msg.is_text() || msg.is_binary()))
// .forward(write_local)
// .await
// .expect("Failed to forward messages");
// read_local.try_filter(|msg| future::ready(msg.is_text() || msg.is_binary()))
// .forward(write_remote)
// .await
// .expect("Failed to forward messages");
let _handle_one = task::spawn(async move {
while let Some(msg) = read_remote.next().await {
let msg = msg?;
if msg.is_text() || msg.is_binary() {
write_local.send(msg).await;
}
};
Result::<(), tungstenite::Error>::Ok(())
});
let _handle_two = task::spawn(async move {
while let Some(msg) = read_local.next().await {
let msg = msg?;
if msg.is_text() || msg.is_binary() {
write_remote.send(msg).await;
}
};
Result::<(), tungstenite::Error>::Ok(())
});
// handle_one.await.expect("The task being joined has panicked");
// handle_two.await.expect("The task being joined has panicked");
}
Ok(())
}
fn main() {
tauri::async_runtime::spawn(client());
tauri::Builder::default()
// .plugin(PluginBuilder::default().build())
.run(tauri::generate_context!())
.expect("failed to run app");
}
looks like you need to make the split streams able to cross threads. The context within the while loop can't access the write_local and write_remote values again and again without a super-context that can hold them.
Here is a working example:
use std::sync::Arc;
use futures_util::{SinkExt, StreamExt};
use http::Request;
use tokio::task;
use tokio::{net::TcpListener, sync::Mutex};
use tokio_tungstenite::{accept_async, connect_async, tungstenite::Result};
async fn client() -> Result<()> {
// Client
let request = Request::builder()
.method("GET")
.header("Host", "demo.piesocket.com")
// .header("Origin", "https://example.com/")
.header("Connection", "Upgrade")
.header("Upgrade", "websocket")
.header("Sec-WebSocket-Version", "13")
.header("Sec-WebSocket-Key", tungstenite::handshake::client::generate_key())
.uri("wss://demo.piesocket.com/v3/channel_1?api_key=VCXCEuvhGcBDP7XhiJJUDvR1e1D3eiVjgZ9VRiaV¬ify_self")
.body(())?;
let (ws_remote, _) = connect_async(request).await?;
let (write_remote, read_remote) = ws_remote.split();
let read_remote = Arc::new(Mutex::new(read_remote));
let write_remote = Arc::new(Mutex::new(write_remote));
let listener = TcpListener::bind("127.0.0.1:4444")
.await
.expect("Can't listen");
while let Ok((stream, _)) = listener.accept().await {
let ws_local = accept_async(stream).await.expect("Failed to accept");
let (mut write_local, mut read_local) = ws_local.split();
// read_remote.try_filter(|msg| future::ready(msg.is_text() || msg.is_binary()))
// .forward(write_local)
// .await
// .expect("Failed to forward messages");
// read_local.try_filter(|msg| future::ready(msg.is_text() || msg.is_binary()))
// .forward(write_remote)
// .await
// .expect("Failed to forward messages");
let read_remote = read_remote.clone();
let _handle_one = task::spawn(async move {
let mut read_remote = read_remote.lock_owned().await;
while let Some(msg) = read_remote.next().await {
let msg = msg?;
if msg.is_text() || msg.is_binary() {
write_local.send(msg).await;
}
}
Result::<(), tungstenite::Error>::Ok(())
});
let write_remote = write_remote.clone();
let _handle_two = task::spawn(async move {
let mut write_remote = write_remote.lock_owned().await;
while let Some(msg) = read_local.next().await {
let msg = msg?;
if msg.is_text() || msg.is_binary() {
write_remote.send(msg).await;
}
}
Result::<(), tungstenite::Error>::Ok(())
});
// handle_one.await.expect("The task being joined has panicked");
// handle_two.await.expect("The task being joined has panicked");
}
Ok(())
}
fn main() {
println!("Hello!");
}
In the server example provided by the HyperTLS library, the server provided cannot be run in a tokio::spawn() thread without throwing a litany of errors all stemming from the fact that:
dyn futures_util::Stream<Item = Result<tokio_rustls::server::TlsStream<tokio::net::TcpStream>, std::io::Error>> cannot be sent between threads safely.
The server definition below is exactly like that found in the example with the exception of the config file used to pass in config information.
#[derive(Debug, Clone)]
pub struct HTTPSServer {}
impl HTTPSServer {
pub async fn start<'a>(config: HTTPSServerConfig<'a>) -> Result<(), HTTPSServerError> {
// Build TLS configuration.
let tls_cfg = {
// Load public certificate.
let certs = load_certs(&config.cert_path)?;
// Load private key.
let key = load_private_key(&config.key_path)?;
// Do not use client certificate authentication.
let mut cfg = rustls::ServerConfig::new(rustls::NoClientAuth::new());
// Select a certificate to use.
cfg.set_single_cert(certs, key)
.map_err(|e| error(format!("failed to set single cert: {}", e)))?;
// Configure ALPN to accept HTTP/2, HTTP/1.1 in that order.
cfg.set_protocols(&[b"h2".to_vec(), b"http/1.1".to_vec()]);
sync::Arc::new(cfg)
};
let tls_acceptor = TlsAcceptor::from(tls_cfg);
// Create a TCP listener via tokio.
let tcp_listener = TcpListener::bind(&config.addr)
.await
.map_err(|e| {
eprintln!("failed to create TCP Listener: {:?}", e);
e
})?;
// Prepare a long-running future stream to accept and serve clients.
let incoming_tls_stream = stream! {
loop {
let (socket, _) = tcp_listener.accept().await?;
let stream = tls_acceptor.accept(socket)
.map_err(|e| {
println!("[!] Voluntary server halt due to client-connection error...");
// Errors could be handled here, instead of server aborting.
// Ok(None)
error(format!("TLS Error: {:?}", e))
});
yield stream.await;
}
};
// Setup router
let service = make_service_fn(|_| async {
Ok::<_, io::Error>(service_fn(echo))
});
// Build server
let server = Server::builder(HyperAcceptor {
acceptor: Box::pin(incoming_tls_stream),
}).serve(service);
// Run the future, keep going until an error occurs.
println!("[https-server]: Starting to serve on https://{}.", &config.fqdn);
server
.await
.map_err(|e| {
eprintln!("[https-server]: error: {:?}", e);
HTTPSServerError::from(e)
})
}
}
The following is the main method to run the server in a separate thread
#[tokio::main]
async fn main() {
// Start the server asynchronously
let fqdn = "https://localhost:8080";
let addr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127,0,0,1)), 8080);
let cert_path = "localhost.crt";
let key_path = "localhost.key";
let ca_path = "RootCA.crt";
let client_config = HTTPSClientConfig {
fqdn,
ca_path: Some(ca_path)
};
let server_config = HTTPSServerConfig {
fqdn,
addr,
cert_path,
key_path,
ca_path,
};
// Spawn server in a new thread!
tokio::spawn(start(server_config)).await.unwrap();
// A function to kick off a client call to the server.
client(client_config);
}
async fn start<'a>(server_config: HTTPSServerConfig<'a>) {
HTTPSServer::start(server_config)
.await
.map_err(|e| {
eprintln!("[https-server]: failed to start server: {:?}", e)
});
}
The Error
I am not exactly sure how to solve the following error as it doesn't seem trivial to ensure a dyn type implements Send. Not sure how to solve this.
error: future cannot be sent between threads safely
--> https/src/main.rs:61:5
|
61 | tokio::spawn(start(server_config)).await.unwrap();
| ^^^^^^^^^^^^ future returned by `start` is not `Send`
|
::: /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.2.0/src/task/spawn.rs:129:21
|
129 | T: Future + Send + 'static,
| ---- required by this bound in `tokio::spawn`
|
= help: the trait `Send` is not implemented for `dyn futures_util::Stream<Item = Result<tokio_rustls::server::TlsStream<tokio::net::TcpStream>, std::io::Error>>`
note: future is not `Send` as it awaits another future which is not `Send`
--> https/src/server.rs:89:9
|
89 | server
| ^^^^^^ await occurs here on type `hyper::Server<HyperAcceptor<'_>, hyper::service::make::MakeServiceFn<[closure#https/src/server.rs:77:39: 79:10]>>`, which is not `Send`
I have a gRPC server and need to make a HTTP GET. I'm struggling getting the futures right in the calls.
I'm trying this
fn current(
&self,
_: ::grpc::RequestOptions,
p: quote::CurrentRequest,
) -> ::grpc::SingleResponse<quote::CurrentResponse> {
let symbol = p.get_symbol();
let client = Client::new();
let fut: grpc::GrpcFuture<quote::CurrentResponse> = Box::new(
client
.get(Uri::from_static(AlphaFunction::base_url()))
.and_then(|res| res.into_body().concat2())
.and_then(|body| {
info!("body {:?}", body);
let mut r = quote::CurrentResponse::new();
// TODO: Parse body
r.set_symbol(symbol.to_string());
Ok(r)
})
.map_err(|e| e),
);
grpc::SingleResponse::new(fut)
}
But I get a bunch of errors:
expected struct `hyper::error::Error`, found enum `grpc::error::Error`
and
77 | grpc::SingleResponse::new(fut)
| ^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `protos::quote::CurrentResponse`, found tuple
I figured it out:
let fut = client
.get(Uri::from_static(AlphaFunction::base_url()))
.and_then(|res| res.into_body().concat2())
.and_then(move |body| {
info!("body {:?}", body);
let mut r = quote::CurrentResponse::new();
r.set_symbol(symbol.to_string());
Ok(r)
})
.map_err(|e| grpc::Error::Panic(format!("{}", e)));
I am writing a Phoenix client library for Rust, taking advantage of the async websocket client from rust-websockets. Right now I am having trouble figuring out how to pass callback functions into the thread that is handling the websocket traffic. I have a simplified struct:
pub struct Socket {
endpoint: String,
connected: Arc<AtomicBool>,
state_change_close: Option<Box<FnMut(String)>>,
}
This struct has a connect function laid out as follows:
pub fn connect(&mut self) -> Result<(), String> {
if self.connected.load(Ordering::Relaxed) {
return Ok(())
}
// Copy endpoint string, otherwise we get an error on thread::spawn
let connection_string = self.endpoint.clone();
let (usr_msg, stdin_ch) = mpsc::channel(0);
let connection_thread = thread::spawn(move || {
// tokio core for running event loop
let mut core = Core::new().unwrap();
let runner = ClientBuilder::new(&connection_string)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (sink, stream) = duplex.split();
stream.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => {
// This is the line where I am trying to call the callback
if let Some(ref mut func) = self.state_change_close {
(func)(e.unwrap().reason);
}
Some(OwnedMessage::Close(e))
},
_ => None,
}
})
.select(stdin_ch.map_err(|_| WebSocketError::NoDataAvailable))
.forward(sink)
});
// Start the event loop
core.run(runner).unwrap();
});
self.connected.store(true, Ordering::Relaxed);
return Ok(())
}
When I try to compile this code I get the following error:
error[E0277]: the trait bound `std::ops::FnMut(std::string::String) + 'static: std::marker::Send` is not satisfied
--> src\socket.rs:99:29
|
99 | let connection_thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ the trait `std::marker::Send` is not implemented for `std::ops::FnMut(std::string::String) + 'static`
|
I have tried changing the type of state_change_close to a Mutex<Option<...>> to avoid thread safety issues, but that did not help with this problem. Is what I'm trying to do possible?
After doing some more research I realized that I just had to modify Option<Box<FnMut(String)>> to be Option<Box<FnMut(String) + Send>> and copy that around my code to everywhere that the callback might be set. Learning more about trait objects!