Why can't I send multiple requests in parallel using rust tonic? - rust

I implemented the tonic helloworld tutorial. I then tried to change the client code so that I could send multiple requests before awaiting any.
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let num_requests = 10;
let mut client = GreeterClient::connect("http://[::1]:50051").await?;
let mut responses = Vec::with_capacity(num_requests);
for _ in 0..num_requests {
let request = tonic::Request::new(HelloRequest {
name: "Tonic".into(),
});
responses.push(client.say_hello(request));
}
for resp in responses {
assert!(resp.await.is_ok());
}
Ok(())
}
This results in a compilation error:
error[E0499]: cannot borrow `client` as mutable more than once at a time
--> src/client.rs:19:24
|
19 | responses.push(client.say_hello(request));
| ^^^^^^ mutable borrow starts here in previous iteration of loop
Does that mean 'client.say_hello()' returns a type which still references client, and therefore I can't make another call to 'say_hello', which itself requires '&mut self'? Is there a way to continue to make requests before calling to 'await'?

From the Tonic documentation:
Sending a request on a channel requires a &mut self and thus can only send one request in flight. This is intentional and is required to follow the Service contract from the tower library which this channel implementation is built on top of.
...
To work around this and to ease the use of the channel, Channel provides a Clone implementation that is cheap. This is because at the very top level the channel is backed by a tower_buffer::Buffer which runs the connection in a background task and provides a mpsc channel interface. Due to this cloning the Channel type is cheap and encouraged.
Therefore, you can clone the client for each concurrent request you make. This eliminates the possibility of a single client being mutably borrowed more than once at any given time, so the borrow checker is appeased.
let num_requests = 10;
let client = GreeterClient::connect("http://[::1]:50051").await?;
let mut responses = Vec::with_capacity(num_requests);
for _ in 0..num_requests {
let mut client = client.clone();
let request = tonic::Request::new(HelloRequest {
name: "Tonic".into(),
});
responses.push(tokio::spawn(async move {
client.say_hello(request).await
}));
}
for resp in responses {
let resp = resp.await;
assert!(resp.is_ok());
assert!(resp.unwrap().is_ok());
}

Related

Rust: Using Mutexes to Allow Access to Data from multiple threads

I'm trying to create a reference to the current TCP connections. In this code, I'm trying to just print the current TCP connection's peer addresses in a loop and sleep for 10 seconds and the error I'm having is accessing the data between multiple threads.
I want to be able to manipulate a TCPStream from a different thread at any given point in time, to do things like shut down the TCPStream or get the peer address.
Can you please let me know what I'm doing wrong in a way that I can get a better understanding of how Arc and Mutex work?
use std::io::Read;
use std::net::{TcpListener, TcpStream};
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
fn main() {
let server = Server {
connected_clients: Arc::new(Mutex::new(Vec::new()))
};
thread::spawn(move || {
let listener = TcpListener::bind("127.0.0.1:25565").unwrap();
// For each new connection start a new thread
for stream in listener.incoming() {
let stream = Arc::new(Mutex::new(stream.unwrap()));
let client = Client {
stream: stream.clone()
};
let cc = server.connected_clients.clone();
cc.lock().unwrap().push(client);
thread::spawn(move || {
// TODO: Add client to the connected_clients Vec
let mut buffer = [0; 1024];
loop {
stream.lock().unwrap().read(&mut buffer).unwrap();
println!("{}", String::from_utf8(Vec::from(&buffer[..])).unwrap().trim_end_matches(char::from(0)));
}
});
}
});
loop {
thread::sleep(Duration::from_secs(10));
// let vec = server.lock().unwrap().connected_clients.lock().unwrap().iter();
for client in server.connected_clients.lock().unwrap().iter() {
println!("{:?}", client.stream.lock().unwrap().peer_addr().unwrap())
}
}
}
#[derive(Debug)]
struct Server {
connected_clients: Arc<Mutex<Vec<Client>>>,
}
#[derive(Debug)]
struct Client {
stream: Arc<Mutex<TcpStream>>
}
ERROR:
error[E0382]: borrow of moved value: `server.connected_clients`
--> src\main.rs:40:23
|
12 | thread::spawn(move || {
| ------- value moved into closure here
...
22 | let cc = server.connected_clients.clone();
| ------------------------ variable moved due to use in closure
...
40 | for client in server.connected_clients.lock().unwrap().iter() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ value borrowed here after move
|
= note: move occurs because `server.connected_clients` has type `Arc<Mutex<Vec<Client>>>`, which does not implement the `Copy` trait
= note: borrow occurs due to deref coercion to `Mutex<Vec<Client>>`
Just move the line let cc = server.connected_clients.clone(); before the first line thread::spawn(move || {.
The move keyword of the closure will now take ownership of cc, then the original server.connected_clients will stay available for the loop at the end of the program.
The idea behind Rc::clone() or Arc::clone() is exactly for the purpose of move closures: instead of moving the original ref-counted pointer to the resource into the closure, we move its clone and the original ref-counted pointer to the resource is still available in its original context.

the trait `std::marker::Sync` is not implemented for `std::sync::mpsc::Sender<i32>

I'm attempting to build a multithreaded application using MPSC and I'm running into the error in the title. I'm not sure what the proper pattern for this use case is - I'm looking for a pattern that will allow me to clone the producer channel and move it into a new thread to be used.
This new thread will keep an open websocket and send a subset of the websocket message data through the producer whenever a websocket message is received. Data from other threads will be needed in the consumer thread which is why I assume the MPSC pattern is a suitable choice.
In addition to the message in the title it also shows these:
`std::sync::mpsc::Sender<i32>` cannot be shared between threads safely
help: the trait `std::marker::Sync` is not implemented for `std::sync::mpsc::Sender`
Can/should I implement Send for this? Is this an appropriate time to use Rc or Pin? I believe this is happening because I'm attempting to send a type that doesn't implement Send across an .await in an async closure, but I don't what to make of it or what to reach for in this situation.
I've been able to reduce my issue down to this:
use futures::stream::{self, StreamExt};
use std::sync::mpsc::{channel, Receiver, Sender};
#[tokio::main]
async fn main() {
let (tx, rx): (Sender<i32>, Receiver<i32>) = channel();
tokio::spawn(async move {
let a = [1, 2, 3];
let mut s = stream::iter(a.iter())
.cycle()
.for_each(move |int| async {
tx.send(*int);
})
.await;
});
}
There are several issues with your code. The first is that you're missing a move in the innermost async block, so the compiler tries to borrow a reference to tx. That's why you get the error that Sender (type of tx) doesn't implement Sync.
Once you add the missing move you get a different error:
error[E0507]: cannot move out of `tx`, a captured variable in an `FnMut` closure
Now the issue is that for_each() will call the closure multiple times, so you are actually not allowed to move tx into the async block - because there would be nothing to move after the first invocation of the closure.
Since MPSC channels allow multiple producers, Sender implements Clone, so you can simply clone tx before moving it to the async block. This compiles:
let (tx, _rx): (Sender<i32>, Receiver<i32>) = channel();
tokio::spawn(async move {
let a = [1, 2, 3];
let _s = stream::iter(a.iter())
.cycle()
.for_each(move |int| {
let tx = tx.clone();
async move {
tx.send(*int).unwrap();
}
})
.await;
});
Playground
Finally, as pointed out in the comments, you almost certainly want to use async channels here. While the channel you create is unbounded, so senders will never block, the receiver will block when there are no messages and therefore halt a whole executor thread.
As it happens, the sender side of tokio MPSC channels also implements Sync making code close to the one from your question compile:
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();
tokio::spawn(async move {
let a = [1, 2, 3];
let _s = stream::iter(a.iter())
.cycle()
.for_each(|int| async {
tx.send(*int).unwrap();
})
.await;
});
assert_eq!(rx.recv().await, Some(1));
assert_eq!(rx.recv().await, Some(2));
assert_eq!(rx.recv().await, Some(3));
Playground

Rust ownership in loops

I'm trying to implement rabbitmq send/listen functionality in Rust and I have the following code:
struct RabbitMQ {
connection: Connection,
}
impl RabbitMQ {
fn connect() -> Self {
RabbitMQ {
connection: the created connection
}
}
}
impl MessageBroker for RabbitMQ {
async fn publish(&self, topic: &Topic) -> Result<PublisherConfirm, Error> {
let channel = self.connection.create_channel().await.unwrap();
RabbitMQ::create_exchange(&channel, &topic.exchange).await;
let payload = topic.message.as_bytes();
let res = channel.basic_publish(
topic.exchange.name.as_str(),
topic.queue.routing_key.as_str(),
topic.exchange.publish_options,
payload.to_vec(),
BasicProperties::default(),
);
res.await
}
}
So far so good!
Now I want to publish many messages in a for loop without waiting for the confirmation from the server, the problem is that when I spawn tokio async task I need to move my broker value and this makes it invalid for the next iteration of the loop:
let broker = RabbitMQ::connect(&connection_details).await;
for x in 1..10 {
tokio::spawn(async move {
let confirm = broker.publish(&my_topic).await.unwrap();
}).await.unwrap();
}
The above code won't compile with the following error:
error[E0382]: use of moved value: `broker`
--> src/main.rs:47:33
|
21 | let broker = RabbitMQ::connect(&connection_details).await;
| ------ move occurs because `broker` has type `message_broker::RabbitMQ`, which >does not implement the `Copy` trait
...
47 | tokio::spawn(async move {
| _________________________________^
48 | | let confirm = &broker.publish(&enable_cdn).await.unwrap();
| | ------ use occurs due to use in generator
49 | | }).await.unwrap();
| |_________^ value moved here, in previous iteration of loop
I can't implement the Copy trait as Connection isn't primitive and it seems that I can't use reference "&" to the broker.
My question is how can I accomplish this without writing n publish calls?
You're using an async move block, which means any name which is used in the block is moved into the future, regardless of the operations being performed. So writing
&broker.publish
inside the block makes no difference: first broker is moved, and the future (when polled with .await) takes an internal reference to it. So what you need to do is borrow outside the block then move that borrow inside:
let broker = RabbitMQ::connect(&connection_details).await;
for x in 1..10 {
let broker = &broker;
tokio::spawn(async move {
let confirm = broker.publish(&enable_cdn).await.unwrap();
}).await.unwrap();
}
but I think that's not going to work either: tokio::spawn is not scoped, so even though you're await-ing it, the compiler has no idea that it will not outlive broker. As far as it's concerned a tokio task can live as long as it wants. This means you're now probably going to get a lifetime error (the compiler will assume the borrow can outlive the enclosing function, and thus its origin).
An easy solution to that would be to put the Connection behind an Arc or something.
Alternatively, restructure your system to work better with the requirements of rabbitmq: no idea which you're using but amiquip states that connections are thread-safe, and channels while not thread-safe can be sent to other threads.
So rather than publish-ing to an implicit connection, in each iteration of the loop create a channel and move that into the task in order to actually perform the publication.
Also,
Now I want to publish many messages in a for loop without waiting for the confirmation from the server
aren't you still doing that since you're awaiting the result of tokio::spawn?

Using async tokio-postgres concurrently is not `Send`

I'm trying to implement asynchronous db access using tokio-postgres crate. Here is What I tried:
use tokio_postgres::{Client, NoTls, Error};
pub struct Database{
client: Mutex<Client>
}
impl Database {
pub async fn some_db_operation(&self, /* args */) -> Result<(), Error> {
let connection = &mut self.client.lock().expect("Mutex was poisoned");
let transaction = &mut connection.transaction().await?;
//executing some queries
Ok(())
}
}
The problem is that I want to access the database as a part of incoming http-request handling using warp and therefore everything should be Send. I got the following error:
--> src/db.rs:27:32
|
26 | let connection = &mut self.client.lock().expect("Mutex was poisoned");
| ----------------------------------------------- has type `std::sync::MutexGuard<'_, tokio_postgres::client::Client>` which is not `Send`
27 | let transaction = &mut connection.transaction().await?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ await occurs here, with `self.client.lock().expect("Mutex was poisoned")` maybe used later
Is there a workaround to make it Send?
There was no such a problem with a synchronous client.
MutexGuard (the result of calling lock() on a Mutex) is not Send. The relevant part in the docs is the manual implementation of !Send ("not Send").
I would strongly suggest to look into some sort of connection pooling, perhaps with deadpool_postgres which is built around tokio-postgres. Using a single client behind a mutex will likely tank your async performance anyway.

Spawning tasks with non-static lifetimes with tokio 0.1.x

I have a tokio core whose main task is running a websocket (client). When I receive some messages from the server, I want to execute a new task that will update some data. Below is a minimal failing example:
use tokio_core::reactor::{Core, Handle};
use futures::future::Future;
use futures::future;
struct Client {
handle: Handle,
data: usize,
}
impl Client {
fn update_data(&mut self) {
// spawn a new task that updates the data
self.handle.spawn(future::ok(()).and_then(|x| {
self.data += 1; // error here
future::ok(())
}));
}
}
fn main() {
let mut runtime = Core::new().unwrap();
let mut client = Client {
handle: runtime.handle(),
data: 0,
};
let task = future::ok::<(), ()>(()).and_then(|_| {
// under some conditions (omitted), we update the data
client.update_data();
future::ok::<(), ()>(())
});
runtime.run(task).unwrap();
}
Which produces this error:
error[E0477]: the type `futures::future::and_then::AndThen<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:13:51: 16:10 self:&mut &mut Client]>` does not fulfill the required lifetime
--> src/main.rs:13:21
|
13 | self.handle.spawn(future::ok(()).and_then(|x| {
| ^^^^^
|
= note: type must satisfy the static lifetime
The problem is that new tasks that are spawned through a handle need to be static. The same issue is described here. Sadly it is unclear to me how I can fix the issue. Even some attempts with and Arc and a Mutex (which really shouldn't be needed for a single-threaded application), I was unsuccessful.
Since developments occur rather quickly in the tokio landscape, I am wondering what the current best solution is. Do you have any suggestions?
edit
The solution by Peter Hall works for the example above. Sadly when I built the failing example I changed tokio reactor, thinking they would be similar. Using tokio::runtime::current_thread
use futures::future;
use futures::future::Future;
use futures::stream::Stream;
use std::cell::Cell;
use std::rc::Rc;
use tokio::runtime::current_thread::{Builder, Handle};
struct Client {
handle: Handle,
data: Rc<Cell<usize>>,
}
impl Client {
fn update_data(&mut self) {
// spawn a new task that updates the data
let mut data = Rc::clone(&self.data);
self.handle.spawn(future::ok(()).and_then(move |_x| {
data.set(data.get() + 1);
future::ok(())
}));
}
}
fn main() {
// let mut runtime = Core::new().unwrap();
let mut runtime = Builder::new().build().unwrap();
let mut client = Client {
handle: runtime.handle(),
data: Rc::new(Cell::new(1)),
};
let task = future::ok::<(), ()>(()).and_then(|_| {
// under some conditions (omitted), we update the data
client.update_data();
future::ok::<(), ()>(())
});
runtime.block_on(task).unwrap();
}
I obtain:
error[E0277]: `std::rc::Rc<std::cell::Cell<usize>>` cannot be sent between threads safely
--> src/main.rs:17:21
|
17 | self.handle.spawn(future::ok(()).and_then(move |_x| {
| ^^^^^ `std::rc::Rc<std::cell::Cell<usize>>` cannot be sent between threads safely
|
= help: within `futures::future::and_then::AndThen<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]>`, the trait `std::marker::Send` is not implemented for `std::rc::Rc<std::cell::Cell<usize>>`
= note: required because it appears within the type `[closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]`
= note: required because it appears within the type `futures::future::chain::Chain<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]>`
= note: required because it appears within the type `futures::future::and_then::AndThen<futures::future::result_::FutureResult<(), ()>, futures::future::result_::FutureResult<(), ()>, [closure#src/main.rs:17:51: 20:10 data:std::rc::Rc<std::cell::Cell<usize>>]>`
So it does seem like in this case I need an Arc and a Mutex even though the entire code is single-threaded?
In a single-threaded program, you don't need to use Arc; Rc is sufficient:
use std::{rc::Rc, cell::Cell};
struct Client {
handle: Handle,
data: Rc<Cell<usize>>,
}
impl Client {
fn update_data(&mut self) {
let data = Rc::clone(&self.data);
self.handle.spawn(future::ok(()).and_then(move |_x| {
data.set(data.get() + 1);
future::ok(())
}));
}
}
The point is that you no longer have to worry about the lifetime because each clone of the Rc acts as if it owns the data, rather than accessing it via a reference to self. The inner Cell (or RefCell for non-Copy types) is needed because the Rc can't be dereferenced mutably, since it has been cloned.
The spawn method of tokio::runtime::current_thread::Handle requires that the future is Send, which is what is causing the problem in the update to your question. There is an explanation (of sorts) for why this is the case in this Tokio Github issue.
You can use tokio::runtime::current_thread::spawn instead of the method of Handle, which will always run the future in the current thread, and does not require that the future is Send. You can replace self.handle.spawn in the code above and it will work just fine.
If you need to use the method on Handle then you will also need to resort to Arc and Mutex (or RwLock) in order to satisfy the Send requirement:
use std::sync::{Mutex, Arc};
struct Client {
handle: Handle,
data: Arc<Mutex<usize>>,
}
impl Client {
fn update_data(&mut self) {
let data = Arc::clone(&self.data);
self.handle.spawn(future::ok(()).and_then(move |_x| {
*data.lock().unwrap() += 1;
future::ok(())
}));
}
}
If your data is really a usize, you could also use AtomicUsize instead of Mutex<usize>, but I personally find it just as unwieldy to work with.

Resources