Unexpectedly closed channel in sync::mpsc - multithreading

I have a closure, that uses Sender from std::sync::mpsc:
let node = Arc::new(Mutex::new(node_sender));
let switch_callback =
move |p| match Params::parse::<Value>(p) {
Ok(ref v) if v.as_array().is_some() => {
let chain = v.as_array()
.and_then(|arr| arr[0].as_str())
.and_then(|s| Some(s.to_owned()))
.unwrap();
let channel = node.lock().unwrap().clone();
match channel.send(chain.clone()) {
Ok(_) => futures::done(Ok(Value::String(chain))).boxed(),
Err(err) => futures::failed(JsonRpcError::invalid_params(
format!("Node not responding: {}", err.to_string())))
.boxed(),
}
}
Ok(_) | Err(_) => {
futures::failed(JsonRpcError::invalid_params("Invalid chain label for node"))
.boxed()
}
};
This closure is used as a callback from another thread. I used clone() here, to clone Sender so I expect the channel should stay active. But the channel is actually getting closed, why would this happen?

One possibility for this would be that your Receiver has been dropped. The channel will only stay active while both the Sender and Receiver are alive.
One of the examples for Sender.send shows that dropping the Receiver terminates the channel:
use std::sync::mpsc::channel;
let (tx, rx) = channel();
// This send is always successful
tx.send(1).unwrap();
// This send will fail because the receiver is gone
drop(rx);
assert_eq!(tx.send(1).unwrap_err().0, 1);
Make sure your Receiver is alive for as long as your Sender is and you should not see this error.

Related

Rust waiting for Tokio threads to finish

I'm trying to create a little program that will kick off new threads whenever an MPSC channel receives a message. I'm able to get the program to kick off the threads, but can't figure out how to get to the program to stay open.
Here is the app:
#[tokio::main]
async fn main() {
let (sender, mut receiver) = mpsc::channel(100);
let handles = Arc::new(tokio::sync::Mutex::new(vec![]));
let handles_clone = handles.clone();
let consumer_task = tokio::spawn(async move {
while let Some(item) = receiver.recv().await {
println!("Consumer received: {}", item);
let h = tokio::spawn(async move {
println!("Thread {} working", item);
sleep(Duration::from_secs(2)).await;
println!("Thread {} done", item);
});
{
let mut handles = handles_clone.lock().await;
handles.push(h);
}
}
});
let producer_task = tokio::spawn(async move {
for i in 0..5 {
let _ = sender.send(i).await;
}
});
let mut handles = handles.lock().await;
handles.push(consumer_task);
handles.push(producer_task);
let a = handles.deref();
join_all(a);
My thought is that I need to join_all on the handles that I'm kicking off, but am having trouble figuring out how I can push handles onto the vector inside the receiver thread, AND then join_all at the end of the program to block until those threads are done.
I wrapping the handles vec in an Arc<Mutex<_>> since I want to reference the handles vec inside the receiver thread, but I can't seem to join_all at the end, and get this error:
`&tokio::task::JoinHandle<()>` is not a future
the trait `futures::Future` is not implemented for `&tokio::task::JoinHandle<()>`
&tokio::task::JoinHandle<()> must be a future or must implement `IntoFuture` to be awaited
the trait `futures::Future` is implemented for `tokio::task::JoinHandle<T>`
`futures::Future` is implemented for `&mut tokio::task::JoinHandle<()>`, but not for `&tokio::task::JoinHandle<()>`
Any help would be greatly appreciated.
Do you see that & in the error message? The compiler error comes from the fact you don't own the JoinHandles when you call join_all (JoinHandles can only be joined once, and thus they need to be owned to do that.) You can fix that error by replacing the last two lines with:
join_all(handles.drain(..)).await;
but then you race into a deadlock: The consumer task will try to lock handles_clone, but that is locked by the "main" task, waiting in join_all, which will never finish. You could circumvent this problem by locking, popping handles, and awaiting one by one, but you'd essentially have implemented an inefficient (lifo) channel by doing so. It would be better to actually use a channel, then.
Better yet, though, to make your life easier by moving the handles Vec into your consumer task and joining there. That makes the mutex unnecessary.
let (sender, mut receiver) = mpsc::channel(100);
let consumer_task = tokio::spawn(async move {
let mut handles = vec![];
while let Some(item) = receiver.recv().await {
println!("Consumer received: {}", item);
let h = tokio::spawn(async move {
println!("Thread {} working", item);
sleep(Duration::from_secs(2)).await;
println!("Thread {} done", item);
});
handles.push(h);
}
join_all(handles).await;
});
let producer_task = tokio::spawn(async move {
for i in 0..5 {
let _ = sender.send(i).await;
}
});
consumer_task.await.unwrap();
producer_task.await.unwrap();

Non-blocking recv on Tokio mpsc Receiver

I am using Rust and Tokio 1.6 to build an app which can interact with an Elgato StreamDeck via hidapi = "1.2". I want to poll the HID device for events (key down / key up) and send those events on an mpsc channel, while watching a separate mpsc channel for incoming commands to update the device state (reset, change brightness, update image, etc). Since the device handle is not thread safe, I need to do both things from a single thread.
major edits below
This is a rewrite of my original question. I've left my interim answer below, but in the interest of a more self contained example, here is a the basic process using device_query = "0.2":
use device_query::{DeviceState, Keycode};
use std::time::Duration;
use tokio;
use tokio::sync::mpsc::{Receiver, Sender};
use tokio::time::timeout;
#[tokio::main]
async fn main() {
// channel for key press events coming from device loop
let (key_tx, mut key_rx) = tokio::sync::mpsc::channel(32);
// channel for commands sent to device loop
let (dev_tx, mut dev_rx) = tokio::sync::mpsc::channel(32);
start_device_loop(60, key_tx, dev_rx);
println!("Waiting for key presses");
while let Some(k) = key_rx.recv().await {
match k {
Some(ch) => match ch {
Keycode::Q => dev_tx.clone().try_send(String::from("Quit!")).expect("Could not send command"),
ch => println!("{}", ch),
},
_ => (),
}
}
println!("Done.")
}
/// Starts a tokio task, polling the supplied device and sending key events
/// on the supplied mpsc sender
pub fn start_device_loop(hz: u32, tx: Sender<Option<Keycode>>, mut rx: Receiver<String>) {
let poll_wait = 1000 / hz;
let poll_wait = Duration::from_millis(poll_wait as u64);
tokio::task::spawn(async move {
let dev = DeviceState::new();
loop {
let mut keys = dev.query_keymap();
match keys.len() {
0 => (),
1 => tx.clone().try_send(Some(keys.remove(0))).unwrap(),
_ => println!("So many keys..."),
}
match timeout(poll_wait, rx.recv()).await {
Ok(cmd) => println!("Command '{}' received.", cmd.unwrap()),
_ => (),
};
// std::thread::sleep(poll_wait);
}
});
}
Note this does not compile - I get an error future created by async block is not 'Send' and within 'impl Future', the trait 'Send' is not implemented for '*mut x11::xlib::_XDisplay'. My understanding of the error is that because device_query is not thread-safe, and awaiting introduces the possibility of scope moving across threads, nothing may be awaited while a non-thread-safe object is in scope. And indeed, if I comment out the block around match timeout... and uncomment the std::thread::sleep everything compiles and runs.
Which brings me back to the original question; how can I both send and receive messages in a single thread without using await or the apparently forbidden fruit of poll_recv()?
After much hunting I found noop_waker in the futures crate which appears to do what I need in combination with poll_recv:
pub fn start_device_loop(hz: u32, tx: Sender<Option<Keycode>>, mut rx: Receiver<String>) {
let poll_wait = 1000 / hz;
let poll_wait = Duration::from_millis(poll_wait as u64);
tokio::task::spawn_blocking(move || {
let dev = DeviceState::new();
let waker = futures::task::noop_waker();
let mut cx = std::task::Context::from_waker(&waker);
loop {
let mut keys = dev.query_keymap();
match keys.len() {
0 => (),
1 => tx.clone().try_send(Some(keys.remove(0))).unwrap(),
_ => println!("So many keys..."),
}
match rx.poll_recv(&mut cx) {
Poll::Ready(cmd) => println!("Command '{}' received.", cmd.unwrap()),
_ => ()
};
std::thread::sleep(poll_wait);
}
});
}
After digging through docs and tokio source more I can't find anything that suggests poll_recv is supposed to be an internal-only function or that using it here would have any obvious side effects. Letting the process run at 125hz I'm not seeing any excess resource usage either.
I'm leaving the above code for posterity, but since asking this question the try_recv method has been added to Receivers, making this all much cleaner.

Wrapping blocking mpsc in async Rust (Tokio)

I am trying to wrap a synchronous MQTT client library using Tokio. The code needs to continuously receive messages via std::sync::mpsc channel and send them into the async code. I understand how to use spawn_blocking for wrapping a code that returns a single value. But how this can be applied to wrap a loop that is continuously receiving messages from std::sync::mpsc channel?
Here is the code that I use to send messages into the channel.
let (mut tx, mut rx) = std::sync::mpsc::channel();
tokio::spawn(async move {
let mut mqtt_options = MqttOptions::new("bot", settings.mqtt.host, settings.mqtt.port);
let (mut mqtt_client, notifications) = MqttClient::start(mqtt_options).unwrap();
mqtt_client.subscribe(settings.mqtt.topic_name, QoS::AtLeastOnce).unwrap();
tokio::task::spawn_blocking(move || {
println!("Waiting for notifications");
for notification in notifications {
match notification {
rumqtt::Notification::Publish(publish) => {
let payload = Arc::try_unwrap(publish.payload).unwrap();
let text: String = String::from_utf8(payload).expect("Can't decode payload for notification");
println!("Recieved message: {}", text);
let msg: Message = serde_json::from_str(&text).expect("Error while deserializing message");
println!("Deserialized message: {:?}", msg);
println!("{}", msg);
tx.send(msg);
}
_ => println!("{:?}", notification)
}
}
});
});
But I am unsure how should I use tokio API to receive these messages inside another async closure.
tokio::task::spawn(async move || {
// How to revieve messages via `rx` here? I can't use tokio::sync::mpsc channels
// since the code that sends messages is blocking.
});
I've posted a separate thread on a rust-lang community and got an answer there.
std::sync::mpsc::channel can be swapped to tokio::sync::mpsc::unbounded_channel, which has a non-async send method. It solves the issue.

rust - Send data from function argument (from a thread) into mpsc channel

I'm trying to handle TcpStream using threads in a Rust program (a tcp server). I want to keep tracks of currents clients connections in a HashMap using an Arc<Mutex<HashMap>>.
When a thread is finished, to remove the connection from the HashMap, I thought about using an uuid, passed to the function used to handle the stream. Like this it could be send inside a mpsc channel to the main thread and then remove the connection from the HashMap.
The code from the main thread :
let clients_streams = Arc::new(Mutex::new(HashMap::new()));
// Server thread. Create channel
let (server_sender, server_receiver) = mpsc::channel();
loop {
for stream in server_socket.incoming() {
let clients_streams = Arc::clone(&clients_streams);
let server_sender = server_sender.clone();
let mut cs = clients_streams.lock().unwrap();
match stream {
Ok(stream) => {
let mut cs = clients_streams.lock().unwrap();
let uuid = Uuid::new_v4().to_string();
cs.insert(uuid.clone(), stream.try_clone());
thread::spawn(move || {
handle_client(stream, server_sender.clone(), uuid.clone());
})
}
Err(err) => thread::spawn(move || {
println!("Connection failed : {}", err);
}),
};
}
The handle_client function :
fn handle_client<'a>(stream: TcpStream, sender: mpsc::Sender<&'a [u8]>, uuid: String) {
let mut reader = BufReader::new(&stream);
let mut writer = BufWriter::new(&stream);
writer.write(b"Hello\n").unwrap();
writer.flush().unwrap();
// Read from client
loop {
let mut resp = Vec::new();
let read_bytes = reader.read_until(b'\n', &mut resp);
match read_bytes {
Ok(read_bytes) => {
if read_bytes == 0 {
let msg = format!("end of {}", uuid);
println!("connection closed by remote");
sender.send(msg.as_bytes()).unwrap();
break;
};
}
Err(err) => match err.kind() {
io::ErrorKind::Interrupted => continue,
_ => break,
},
}
}
}
This code doesn't compile, it shows this error but I don't understand how to resolve it. The variable msg doesn't live long enough, because of the uuid inside of it, but why ? I had to add the lifetime because I'm sending uuid inside the channel.
fn handle_client<'a>(stream: TcpStream, sender: mpsc::Sender<&'a [u8]>, uuid: String) {
-- lifetime `'a` defined here
sender.send(msg.as_bytes()).unwrap();
------------^^^------------
| |
| borrowed value does not live long enough
argument requires that `msg` is borrowed for `'a`
break;
};
- `msg` dropped here while still borrowed
Uuid does not affect msg because format macro creates a clean new String. You are trying to send a &[u8] (this is what as_bytes() return). You need to find a way to remove this reference. (maybe this something like How to convert from &[u8] to Vec<u8>?) You can also share references with Rc.
PS: this more a comment than an answer but I can't post answers

Tokio future that reads from a channel and uses poll_fn and try_ready never completes

I have a Tokio future which never completes (rx is a Receiver and sock is a tokio UdpSocket). It basically reads packets from a packet queue and transmits them over a socket:
poll_fn(move || {
match try_ready!(rx
.poll()
.map_err(|_e| tokio::io::Error::new(tokio::io::ErrorKind::Other, "Poll error")))
{
Some((packet, to)) => {
println!(
"Rx: Received {} bytes for {}: {:?}",
packet.len(),
to,
packet.as_slice(),
);
try_ready!(sock.poll_send_to(packet.as_slice(), &to));
println!("Sent");
}
None => println!("Rx end"),
}
Ok(futures::Async::Ready(()))
})
.map_err(|e: tokio::io::Error| println!("Error: {:?}", e))
It executes until the poll_send_to line (the println! just before poll_send_to executes, the println! just after doesn't) and then waits forever without sending the packet.
I replaced the above future with the following one to ensure that it wasn't a socket issue(had some issues with what I think were flaky notifications before):
poll_fn(move || {
let packet = vec![0;10];
let to = SocketAddr::from_str("127.0.0.1:8001").expect("Parse error");
try_ready!(sock.poll_send_to(packet.as_slice(), &to));
Ok(futures::Async::Ready(()))
})
.map_err(|e: tokio::io::Error| println!("Error: {:?}", e))
This future worked perfectly - it sent the packet as expected and exited the program.
I don't think the problem is with the message channels given that rx can poll successfully and prints the println message. I don't think the problem is with the socket either given that the second future works. I am observing packets directly through Wireshark, so I don't think it is an issue with my observations either.
I'm pretty new to Rust and Tokio, so it's possible that I am overlooking some basic fact (e.g. can't try_ready twice in the same future, future doesn't resume from where it left off previously, etc).
Can you help me figure out the problem with the first future?
use futures::future::lazy;
use futures::stream::Stream;
use futures::try_ready;
use std::net::SocketAddr;
use std::str::FromStr;
use tokio;
use tokio::net::UdpSocket;
use tokio::prelude::future::poll_fn;
use tokio::prelude::Future;
fn main() {
let mut sock = UdpSocket::bind(&SocketAddr::from_str("127.0.0.1:8000").expect("Parse error"))
.expect("Bind error");
let (mut tx, mut rx) = tokio::sync::mpsc::channel::<(Vec<u8>, SocketAddr)>(2000);
tokio::run(lazy(move || {
//----------------- This future works ----------------//
// tokio::spawn(
// poll_fn(move || {
// let packet = vec![70; 10];
// let to = SocketAddr::from_str("127.0.0.1:8001").expect("Parse error");
// try_ready!(sock.poll_send_to(packet.as_slice(), &to));
// Ok(futures::Async::Ready(()))
// })
// .map_err(|e: tokio::io::Error| println!("Error: {:?}", e)),
// );
//----------------- This future doesn't ----------------//
tokio::spawn(
poll_fn(move || {
match try_ready!(rx
.poll()
.map_err(|_e| tokio::io::Error::new(tokio::io::ErrorKind::Other, "Poll error")))
{
Some((packet, to)) => {
// This is printed
println!(
"Rx: Received {} bytes for {}: {:?}",
packet.len(),
to,
packet.as_slice(),
);
try_ready!(sock.poll_send_to(packet.as_slice(), &to));
// This is never printed
println!("Sent");
}
None => println!("Rx end"),
}
Ok(futures::Async::Ready(()))
})
.map_err(|e: tokio::io::Error| println!("Error: {:?}", e)),
);
//----------------- This future queues a packet ----------------//
tokio::spawn(
poll_fn(move || {
try_ready!(tx.poll_ready());
tx.try_send((
vec![70; 10],
SocketAddr::from_str("127.0.0.1:8001").expect("Parse error"),
))
.expect("Send error");
// Wait permanently so message channel doesn't get disconnected
// Achieved differently in production
Ok(futures::Async::NotReady)
})
.map_err(|e: tokio::sync::mpsc::error::SendError| println!("Error: {:?}", e)),
);
Ok(())
}));
}
Repo
Using this version of your future shows the problem:
tokio::spawn(
future::poll_fn(move || {
eprintln!("Starting poll_fn");
let from_channel = rx
.poll()
.map_err(|_e| tokio::io::Error::new(tokio::io::ErrorKind::Other, "Poll error"));
if let Some((packet, to)) = futures::try_ready!(dbg!(from_channel)) {
futures::try_ready!(dbg!(sock.poll_send_to(packet.as_slice(), &to)));
}
Ok(futures::Async::Ready(()))
})
.map_err(|e: tokio::io::Error| println!("Error: {:?}", e)),
);
Here's the slightly cleaned-up output:
Starting poll_fn
[src/main.rs:21] from_channel = Ok(NotReady)
Starting poll_fn
[src/main.rs:21] from_channel = Ok(Ready(Some(/* ... */)))
[src/main.rs:22] sock.poll_send_to(packet.as_slice(), &to) = Ok(NotReady)
Starting poll_fn
[src/main.rs:21] from_channel = Ok(NotReady)
In words:
The future starts.
There's nothing ready from the channel; the channel registers a notification.
The future returns.
The channel gets a value and notifies the task.
The future starts again.
There's a value ready from the channel.
Sending on the socket is not ready; the socket registers a notification.
The future returns.
The socket is cleared and notifies the task.
The future starts again.
There's nothing ready from the channel; the channel registers a notification.
The future returns.
Nothing else is ever added to the channel.
In short, you aren't correctly maintaining your state machine inside of your future. You need to know how far you got the last time the future ran and start at that point the next time it ran.
There's a reason that the async / await syntax is much-anticipated: it will write these state machines for you.
I don't know why you've chosen to use the lower-level poll-based interface. I'd use the higher-level Future-based one:
tokio::spawn({
rx.fold(sock, |sock, (packet, to)| {
sock.send_dgram(packet, &to)
.inspect(|_| println!("Sent it!"))
.map(|(sock, _)| sock)
.map_err(|e| panic!("Error: {:?}", e))
})
.map(drop)
.map_err(|e| panic!("Error: {:?}", e))
});
the Future-based interface [...] destroys the socket(and buffer) on error
This is a good reason to use the poll-based interface, but I'd still just dip into it long enough to implement your own future. Something like this:
struct X(UdpSocket);
struct XSendGram<D> {
sock: Option<UdpSocket>,
data: D,
addr: SocketAddr,
}
impl X {
fn send_dgram<D>(self, data: D, addr: SocketAddr) -> XSendGram<D> {
XSendGram {
sock: Some(self.0),
data,
addr,
}
}
}
impl<D> Future for XSendGram<D>
where
D: AsRef<[u8]>,
{
type Item = (X, usize);
type Error = (X, std::io::Error);
fn poll(&mut self) -> Result<Async<Self::Item>, Self::Error> {
let mut sock = self.sock.take().expect("Future called after success or failure");
match sock.poll_send_to(self.data.as_ref(), &self.addr) {
Ok(Async::Ready(bytes)) => Ok(Async::Ready((X(sock), bytes))),
Ok(Async::NotReady) => {
self.sock = Some(sock); // Restore it for the next call
Ok(Async::NotReady)
}
Err(e) => Err((X(sock), e)),
}
}
}
tokio::spawn({
rx.fold(X(sock), |sock, (packet, to)| {
sock.send_dgram(packet, to)
.inspect(|(_, n)| println!("Sent {} bytes", n))
.then(|r| match r {
Ok((sock, _)) | Err((sock, _)) => future::ok(sock),
})
})
.map(drop)
.map_err(|e| panic!("Error: {:?}", e))
});

Resources