extern crate redis;
use redis::aio::ConnectionManager;
use std::env;
pub async fn connect() -> redis::aio::ConnectionManager {
let redis_host_name =
env::var("REDIS_HOSTNAME").expect("Missing environment variable REDIS_HOSTNAME");
let redis_tls = env::var("REDIS_TLS").map(|redis_tls| redis_tls == "1").unwrap_or(false);
let uri_scheme = match redis_tls {
true => "rediss",
false => "redis",
};
let redis_conn_url = format!("{}://{}", uri_scheme, redis_host_name);
let client = redis::Client::open(redis_conn_url)
.expect("Invalid connection URL");
let redis = ConnectionManager::new(client.clone()).await.unwrap();
redis
}
I have a pub/sub system with Redis in Rust. For now it works perfectly creating the connections separately (one to publish and another to subscribe), but if the redis server goes down, it does not try to reconnect. To try to solve it, I have seen the option of the ConnectionManager of the following link:
https://docs.rs/redis/0.21.5/redis/aio/struct.ConnectionManager.html
Which implements the reconnection. The problem is that I don't see documentation to be able to implement it. With the above function I return a ConnectionManager, which I then want to pass through the AppData in the routes with ActixWeb. But I don't understand how I use those connections to publish and subscribe to the channel. Since it tells me that those functions don't exist in the ConnectionManager. Any ideas?
Related
I have a basic Actix Web server set up, and I have successfully been creating websocket connections in my tests using awc::client::Client.
Now I am trying to test that my server is closing all of the websocket connections when I tell it reset the status of the app.
My current planned test for this is:
#[test]
async fn reset_game_works_basic() {
let server: TestServer = test_fixtures::get_test_server();
let (_resp, mut chris_connection) = Client::new()
.ws(server.url("/join-game?username=Chris"))
.connect()
.await
.unwrap();
let _ = server.post("/reset-game").send().await;
let websocket_connected = chris_connection.websocket_connected_status();
// ^^^^ Not a real function
assert_eq!(websocket_connected, false);
}
From the awc websocket docs, I have only been able to find a CloseCode enum, but that seems to be used for the client to tell the server why it is closing the connection.
I also fruitlessly tried to check if the connection was open by using is_write_ready(), but that returned true.
I have done manual testing with Postman, and the clients are being disconnected when you send a post request to /reset-game.
How should I ask the client if it still has an open connection?
I ended up finding a working (albiet fragile) test for this.
When the websocket connection is closed, .next().await will return None, and when it is open, it will return Some, even if there is no next message!
So this test successfully passes with the working code, and fails if I remove the code that actually closes the websocket!
#[test]
async fn reset_game_works_basic() {
let server: TestServer = test_fixtures::get_test_server();
let (_resp, mut chris_connection) = Client::new()
.ws(server.url("/join-game?username=Chris"))
.connect()
.await
.unwrap();
let _ = server.post("/reset-game").send().await;
let _join = chris_connection.next().await;
let no_message = chris_connection.next().await;
let websocket_disconnected = no_message.is_none();
assert_eq!(websocket_disconnected, true);
}
It certainly is fragile to the problem that if I make the websocket send a second message before disconnecting it will break, but that is a separate problem to the one I was trying to solve here.
I'm new to rust and encountered an issue while building an API with warp. I'm trying to pass some requests to another thread with a channel(trying to avoid using arc/mutex). Still, I noticed that when I pass an mpsc::sync::Sender to a warp handler, I get this error.
"std::sync::mpsc::Sender cannot be shared between threads safely"
and
"the trait Sync is not implemented for `std::sync::mpsc::Sender"
Can someone lead me in the right direction?
use std::sync::mpsc::Sender;
pub async fn init_server(run_tx: Sender<Packet>) {
let store = Store::new();
let store_filter = warp::any().map(move || store.clone());
let run_tx_filter = warp::any().map(move || run_tx.clone());
let update_item = warp::get()
.and(warp::path("v1"))
.and(warp::path("auth"))
.and(warp::path::end())
.and(post_json())
.and(store_filter.clone())
.and(run_tx_filter.clone()) //where I'm trying to send "Sender"
.and_then(request_token);
let routes = update_item;
println!("HTTP server started on port 8080");
warp::serve(routes).run(([127, 0, 0, 1], 3030)).await;
}
pub async fn request_token(
req: TokenRequest,
store: Store,
run_tx: Sender<Packet>,
) -> Result<impl warp::Reply, warp::Rejection> {
let (tmp_tx, tmp_rx) = std::sync::mpsc::channel();
run_tx
.send(Packet::IsPlayerLoggedIn(req.address, tmp_tx))
.unwrap();
let logged_in = tmp_rx.recv().unwrap();
if logged_in {
return Ok(warp::reply::with_status(
"Already logged in",
http::StatusCode::BAD_REQUEST,
));
}
Ok(warp::reply::with_status("some token", http::StatusCode::OK))
}
I've looked through some of the examples for warp, and was also wondering what are some good resources to get knowledgable of the crate. Thank you!
This is because you're using std::sync::mpsc::Sender. std::sync implements !Sync, so you won't be able to use that. You don't want to use that anyway since it's blocking.
When you use async functionality in rust, you need to provide a runtime for it. The good news is that warp runs on the tokio runtime, so you should have access to tokio::sync::mpsc If you take a look at that link, the Sender for that mpsc implementation implements Sync and Send so it's safe to share across threads.
TLDR:
Use tokio::sync::mpsc instead of std::sync::mpsc and this won't be an issue.
I need to call the contract's method from my Indexer. Now I use tokio::process and near-cli written on NodeJs. It looks soundless, and I would like to do that from Rust.
Recommended way
NEAR JSON-RPC Client RS is the recommended way to interact with NEAR Protocol from within the Rust code.
Example from the README
use near_jsonrpc_client::{methods, JsonRpcClient};
use near_jsonrpc_primitives::types::transactions::TransactionInfo;
let mainnet_client = JsonRpcClient::connect("https://archival-rpc.mainnet.near.org");
let tx_status_request = methods::tx::RpcTransactionStatusRequest {
transaction_info: TransactionInfo::TransactionId {
hash: "9FtHUFBQsZ2MG77K3x3MJ9wjX3UT8zE1TczCrhZEcG8U".parse()?,
account_id: "miraclx.near".parse()?,
},
};
// call a method on the server via the connected client
let tx_status = mainnet_client.call(tx_status_request).await?;
println!("{:?}", tx_status);
In the examples folder of the repo, you will find different use cases, and hopefully, you'll find yours there as well.
near-jsonrpc-client-rs is the best option.
Alternative way
NB! This way is using non-documented APIs. It is not recommended way, because using these assumes you will dig into the code and find out how to use it by yourself.
If you're using the NEAR Indexer Framework you're literally running a nearcore node which includes:
JSON RPC server
ClientActor and ViewClient
Based on the kind of call you need to perform to your contract: change method or view method, you can use ClientActor or ViewClient.
ViewClient example
Code is for understanding the concept, not a working example.
let indexer = near_indexer::Indexer::new(indexer_config);
let view_client = indexer.client_actors().0;
let block_response = view_client
.send(query)
.await
.context("Failed to deliver response")?
.context("Invalid request")?;
You can find the usage in NEAR Indexer for Explorer starting from here
ClientActor example
ClientActor is used to send a transaction. I guess here's a good starting point to look for ClientActor example.
async fn send_tx_async(
&self,
request_data: near_jsonrpc_primitives::types::transactions::RpcBroadcastTransactionRequest,
) -> CryptoHash {
let tx = request_data.signed_transaction;
let hash = tx.get_hash().clone();
self.client_addr.do_send(NetworkClientMessages::Transaction {
transaction: tx,
is_forwarded: false,
check_only: false, // if we set true here it will not actually send the transaction
});
hash
}
I am using tungstenite to build a chat server, and the way I want to do it relies on having many threads that communicate with each other through mpsc. I want to start up a new thread for each user that connects to the server and connect them to a websocket, and also have that thread be able to read from mpsc so that the server can send messages out through that connection.
The problem is that the mpsc read blocks the thread, but I can't block the thread if I want to be reading from it. The only thing I could think of to work around that is to make two threads, one for inbound and one for outbound messages, but that requires me to share my websocket connection with both workers, which of course I cannot do.
Here's a heavily truncated version of my code where I try to make two workers in the Action::Connect arm of the match statement, which gives error[E0382]: use of moved value: 'websocket' for trying to move it into the second worker's closure:
extern crate tungstenite;
extern crate workerpool;
use std::net::{TcpListener, TcpStream};
use std::sync::mpsc::{self, Sender, Receiver};
use workerpool::Pool;
use workerpool::thunk::{Thunk, ThunkWorker};
use tungstenite::server::accept;
pub enum Action {
Connect(TcpStream),
Send(String),
}
fn main() {
let (main_send, main_receive): (Sender<Action>, Receiver<Action>) = mpsc::channel();
let worker_pool = Pool::<ThunkWorker<()>>::new(8);
{
// spawn thread to listen for users connecting to the server
let main_send = main_send.clone();
worker_pool.execute(Thunk::of(move || {
let listener = TcpListener::bind(format!("127.0.0.1:{}", 8080)).unwrap();
for (_, stream) in listener.incoming().enumerate() {
main_send.send(Action::Connect(stream.unwrap())).unwrap();
}
}));
}
let mut users: Vec<Sender<String>> = Vec::new();
// process actions from children
while let Some(act) = main_receive.recv().ok() {
match act {
Action::Connect(stream) => {
let mut websocket = accept(stream).unwrap();
let (user_send, user_receive): (Sender<String>, Receiver<String>) = mpsc::channel();
let main_send = main_send.clone();
// thread to read user input and propagate it to the server
worker_pool.execute(Thunk::of(move || {
loop {
let message = websocket.read_message().unwrap().to_string();
main_send.send(Action::Send(message)).unwrap();
}
}));
// thread to take server output and propagate it to the server
worker_pool.execute(Thunk::of(move || {
while let Some(message) = user_receive.recv().ok() {
websocket.write_message(tungstenite::Message::Text(message.clone())).unwrap();
}
}));
users.push(user_send);
}
Action::Send(message) => {
// take user message and echo to all users
for user in &users {
user.send(message.clone()).unwrap();
}
}
}
}
}
If I create just one thread for both in and output in that arm, then user_receive.recv() blocks the thread so I can't read any messages with websocket.read_message() until I get an mpsc message from the main thread. How can I solve both problems? I considered cloning the websocket but it doesn't implement Clone and I don't know if just making a new connection with the same stream is a reasonable thing to try to do, it seems hacky.
The problem is that the mpsc read blocks the thread
You can use try_recv to avoid thread blocking. The another implementation of mpsc is crossbeam_channel. That project is a recommended replacement even by the author of mpsc
I want to start up a new thread for each user that connects to the server
I think the asyn/await approach will be much better from most of the prospectives then thread per client one. You can read more about it there
I'm not able to create a client that tries to connect to a server and:
if the server is down it has to try again in an infinite loop
if the server is up and connection is successful, when the connection is lost (i.e. server disconnects the client) the client has to restart the infinite loop to try to connect to the server
Here's the code to connect to a server; currently when the connection is lost the program exits. I'm not sure what the best way to implement it is; maybe I have to create a Future with an infinite loop?
extern crate tokio_line;
use tokio_line::LineCodec;
fn get_connection(handle: &Handle) -> Box<Future<Item = (), Error = io::Error>> {
let remote_addr = "127.0.0.1:9876".parse().unwrap();
let tcp = TcpStream::connect(&remote_addr, handle);
let client = tcp.and_then(|stream| {
let (sink, from_server) = stream.framed(LineCodec).split();
let reader = from_server.for_each(|message| {
println!("{}", message);
Ok(())
});
reader.map(|_| {
println!("CLIENT DISCONNECTED");
()
}).map_err(|err| err)
});
let client = client.map_err(|_| { panic!()});
Box::new(client)
}
fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let client = get_connection(&handle);
let client = client.and_then(|c| {
println!("Try to reconnect");
get_connection(&handle);
Ok(())
});
core.run(client).unwrap();
}
Add the tokio-line crate with:
tokio-line = { git = "https://github.com/tokio-rs/tokio-line" }
The key question seems to be: how do I implement an infinite loop using Tokio? By answering this question, we can tackle the problem of reconnecting infinitely upon disconnection. From my experience writing asynchronous code, recursion seems to be a straightforward solution to this problem.
UPDATE: as pointed out by Shepmaster (and the folks of the Tokio Gitter), my original answer leaks memory since we build a chain of futures that grows on each iteration. Here follows a new one:
Updated answer: use loop_fn
There is a function in the futures crate that does exactly what you need. It is called loop_fn. You can use it by changing your main function to the following:
fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let client = future::loop_fn((), |_| {
// Run the get_connection function and loop again regardless of its result
get_connection(&handle).map(|_| -> Loop<(), ()> {
Loop::Continue(())
})
});
core.run(client).unwrap();
}
The function resembles a for loop, which can continue or break depending on the result of get_connection (see the documentation for the Loop enum). In this case, we choose to always continue, so it will infinitely keep reconnecting.
Note that your version of get_connection will panic if there is an error (e.g. if the client cannot connect to the server). If you also want to retry after an error, you should remove the call to panic!.
Old answer: use recursion
Here follows my old answer, in case anyone finds it interesting.
WARNING: using the code below results in unbounded memory growth.
Making get_connection loop infinitely
We want to call the get_connection function each time the client is disconnected, so that is exactly what we are going to do (look at the comment after reader.and_then):
fn get_connection(handle: &Handle) -> Box<Future<Item = (), Error = io::Error>> {
let remote_addr = "127.0.0.1:9876".parse().unwrap();
let tcp = TcpStream::connect(&remote_addr, handle);
let handle_clone = handle.clone();
let client = tcp.and_then(|stream| {
let (sink, from_server) = stream.framed(LineCodec).split();
let reader = from_server.for_each(|message| {
println!("{}", message);
Ok(())
});
reader.and_then(move |_| {
println!("CLIENT DISCONNECTED");
// Attempt to reconnect in the future
get_connection(&handle_clone)
})
});
let client = client.map_err(|_| { panic!()});
Box::new(client)
}
Remember that get_connection is non-blocking. It just constructs a Box<Future>. This means that when calling it recursively, we still don't block. Instead, we get a new future, which we can link to the previous one by using and_then. As you can see, this is different to normal recursion since the stack doesn't grow on each iteration.
Note that we need to clone the handle (see handle_clone), and move it into the closure passed to reader.and_then. This is necessary because the closure is going to live longer than the function (it will be contained in the future we are returning).
Handling errors
The code you provided doesn't handle the case in which the client is unable to connect to the server (nor any other errors). Following the same principle shown above, we can handle errors by changing the end of get_connection to the following:
let handle_clone = handle.clone();
let client = client.or_else(move |err| {
// Note: this code will infinitely retry, but you could pattern match on the error
// to retry only on certain kinds of error
println!("Error connecting to server: {}", err);
get_connection(&handle_clone)
});
Box::new(client)
Note that or_else is like and_then, but it operates on the error produced by the future.
Removing unnecessary code from main
Finally, it is not necessary to use and_then in the main function. You can replace your main by the following code:
fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let client = get_connection(&handle);
core.run(client).unwrap();
}