With Rust's UdpSocket, it appears to only have send(), recv(), and set_nonblocking(bool). This is a rather limited set of functions, e.g. it would be nice to have recv() which always blocks until a packet is received and recv_nonblocking() which never blocks, and maybe even recv_callback() that calls a function when a packet is received.
Is there any way to do recv_nonblocking() other than something insane like this:
fn recv_nonblocking(socket: &UdpSocket, buf: &mut [u8]) -> Result<usize> {
try!(socket.set_nonblocking(true));
// By the way, how do I know how big to make my buffer?
// There doesn't seem to be any way to know the size of a packet
// before it is returned by socket.recv().
let res = socket.recv(&mut buf);
// This isn't really right either because there is no get_nonblocking()
// so we can't reliably restore the previous state.
try!(socket.set_nonblocking(false));
res
}
Would that even work?
recv_nonblocking
Just call set_nonblocking and then recv. If you're really going to use both blocking and nonblocking I/O on the same socket, you can write trivial wrapper functions:
fn recv_nonblocking(this: &UdpSocket, buf: &mut [u8]) -> io::Result<usize> {
this.set_nonblocking(true);
this.recv(buf)
}
recv_callback
Good asynchronous I/O primitives have been left out of the standard library, so that the design work can be iterated on without locking down to the stability guarantees and release cycle of the language.
The crate to look at is mio.
Generally you either use blocking or non-blocking mode through the whole lifecycle of the socket. In blocking mode you can use set_read_timeout get_read_timeout instead.
Related
I have a Rust-based latency-sensitive application that subscribes to a stream of incoming data, deserializes it, processes the deserialized object, and then forwards it elsewhere.
Sometimes, I receive bursts of messages and this causes the latency to degrade a bit as it is "backed up." It would be great if I could parallelize the deserialization.
However, I need to preserve the order of the messages when I forward them along. Forwarding is extremely fast, almost negligible, so the fact that forwarding is serial is okay.
Naively, I could send a tuple of (sequence_number, data) over a channel to a pool of processor threads, and each thread could, upon processing, send a tuple of (sequence_number, processed) over a different channel to a single thread that simply forwards. The forwarding thread would also keep track of the next sequence_number to send. When it receives something over the channel, it saves to a HashMap<u64, MyData>. Then while the map contains the next sequence_number, it could forward.
But it gives me pause that I couldn't find such a library on GitHub; makes me think this could be a bad idea.
So I am wondering, is there a name for this sort of thing? Does it exist in Rust or some other language? Is there a better pattern I can follow?
Not sure of a common term but you could use FuturesOrdered from the futures crate.
Here is an example (playground):
use rand::{thread_rng, Rng};
use futures::stream::FuturesOrdered;
use futures::StreamExt as _;
use std::thread;
use core::time::Duration;
#[tokio::main]
async fn main() {
let mut ord_futures = FuturesOrdered::new();
for i in 0..100 {
// receive
ord_futures.push(async move {
tokio::time::sleep(Duration::from_secs(thread_rng().gen_range(1..5))).await;
println!("processed {i}");
i
});
}
while let Some(i) = ord_futures.next().await {
// forward
println!("received {i}");
}
}
While learning some Rust, I saw a lot of tutorials that used two very simple models. One is on the server side, where all the accepted tcpstreams are moved to a new thread for use, and the other is on the client side, using blocking reads and then output.
But for a real project, this is definitely not enough. For example, on the client side, it is usually not possible to block the main thread to read the data. So either use non-blocking sockets, or use multi-threaded or asynchronous io.
Since I am new to Rust, I don't plan to use async io or tokio libraries for this.
Suppose I use a thread to block reading data, and send data or close the tcp connection in the main thread.
As a general practice, since the tcp connection is used in two threads, then generally we have to use Arc<Mutex<TcpStream>> to use the connection variable.
But when I need to read in the read thread, I will do Mutex::lock() to get the TcpStream, and when I send or close in the main thread, I also need to do Mutex::lock(). Won't this cause a deadlock?
Of course, another way is to poll a message queue in a new thread, and send commands like this one when the socket has a read event, or when the main thread needs to send data or close the connection. This way the access to the TcpStream is done in one thread. However, it seems to add a lot of extra code for maintaining the message queue.
If the TcpStream can generate two ends, just like channel, a read end and a write end. I will use them in different threads conveniently. But it seems no such function provided.
Is there a recommended approach?
Not sure about a "recommended" approach, but you don't need a mutex to read/write from a TcpStream because io traits are implemented for &TcpStream in addition to TcpStream. This allows you to call methods like read() on &stream, which can be easily shared among threads using Arc. For example:
use std::io::{BufRead, BufReader, BufWriter, Write};
use std::net::TcpStream;
use std::sync::Arc;
fn main() -> std::io::Result<()> {
let stream = Arc::new(TcpStream::connect("127.0.0.1:34254")?);
let (r, w) = (Arc::clone(&stream), stream);
let thr1 = std::thread::spawn(move || -> std::io::Result<()> {
let r = BufReader::new(r.as_ref());
for line in r.lines() {
println!("received: {}", line?);
}
Ok(())
});
let thr2 = std::thread::spawn(move || {
let mut w = BufWriter::new(w.as_ref());
w.write_all(b"Hello\n")
});
thr1.join().unwrap()?;
thr2.join().unwrap()?;
Ok(())
}
Suppose I have an userspace TCP/IP stack. It's natural that I wrap it in Arc<Mutex<>> so I can share it with my threads.
It's also natural that I want to implement AsyncRead and AsyncWrite for it, so libraries that expect impl AsyncWrite and impl AsyncRead like hyper can use it.
This is an example:
use core::task::Context;
use std::pin::Pin;
use std::sync::Arc;
use core::task::Poll;
use tokio::io::{AsyncRead, AsyncWrite};
struct IpStack{}
impl IpStack {
pub fn send(self, data: &[u8]) {
}
//TODO: async or not?
pub fn receive<F>(self, f: F)
where F: Fn(Option<&[u8]>){
}
}
pub struct Socket {
stack: Arc<futures::lock::Mutex<IpStack>>,
}
impl AsyncRead for Socket {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>
) -> Poll<std::io::Result<()>> {
//How should I lock and call IpStack::read here?
Poll::Ready(Ok(()))
}
}
impl AsyncWrite for Socket {
fn poll_write(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<Result<usize, std::io::Error>> {
//How should I lock and call IpStack::send here?
Poll::Ready(Ok(buf.len()))
}
//poll_flush and poll_shutdown...
}
Playground
I don't see anything wrong with my assumptions and I don't see another better way to share a stack with multiple threads unless I wrap it in Arc<Mutex<>>
This is similar to try_lock on futures::lock::Mutex outside of async? which caught my interest.
How should I lock the mutex without blocking? Notice that once I got the lock, the IpStack is not async, it has calls that block. I would like to implement async to it too, but I don't know it the problem will get much harder. Or would the problem get simpler if it had async calls?
I found the tokio documentation page on tokio::sync::Mutex pretty helpful: https://docs.rs/tokio/1.6.0/tokio/sync/struct.Mutex.html
From your description it sounds you want:
Non-blocking operations
One big data structure that manages all the IO resources managed by the userspace TCP/IP stack
To share that one big data structure across threads
I would suggest exploring something like an actor and use message passing to communicate with a task spawned to manage the TCP/IP resources. I think you could wrap the API kind of like the mini-redis example cited in tokio's documentation to implement AsyncRead and AsyncWrite. It might be easier to start with an API that returns futures of complete results and then work on streaming. I think this would be easier to make correct. Could be fun to exercise it with loom.
I think if you were intent on synchronizing access to the TCP/IP stack through a mutex you'd probably end up with an Arc<Mutex<...>> but with an API that wraps the mutex locks like mini-redis. The suggestion the tokio documentation makes is that their Mutex implementation is more appropriate for managing IO resources rather than sharing raw data and that does fit your situation I think.
You should not use an asynchronous mutex for this. Use a standard std::sync::Mutex.
Asynchronous mutexes like futures::lock::Mutex and tokio::sync::Mutex allow locking to be awaited instead of blocking so they are safe to use in async contexts. They are designed to be used across awaits. This is precisely what you don't want to happen! Locking across an await means that the mutex is locked for potentially a very long time and would prevent other asynchronous tasks wanting to use the IpStack from making progress.
Implementing AsyncRead/AsyncWrite is straight-forward in theory: either it can be completed immediately, or it coordinates through some mechanism to notify the context's waker when the data is ready and returns immediately. Neither case requires extended use of the underlying IpStack, so its safe to use a non-asynchronous mutex.
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
use tokio::io::{AsyncRead, AsyncWrite};
struct IpStack {}
pub struct Socket {
stack: Arc<Mutex<IpStack>>,
}
impl AsyncRead for Socket {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>,
) -> Poll<std::io::Result<()>> {
let ip_stack = self.stack.lock().unwrap();
// do your stuff
Poll::Ready(Ok(()))
}
}
I don't see another better way to share a stack with multiple threads unless I wrap it in Arc<Mutex<>>.
A Mutex is certainly the most straightforward way to implement something like this, but I would suggest an inversion of control.
In the Mutex-based model, the IpStack is really driven by the Sockets, which consider the IpStack to be a shared resource. This results in a problem:
If a Socket blocks on locking the stack, it violates the contract of AsyncRead by spending an unbounded amount of time executing.
If a Socket doesn't block on locking the stack, choosing instead to use try_lock(), it may be starved because it doesn't remain "in line" for the lock. A fair locking algorithm, such as that provided by parking_lot, can't save you from starvation if you don't wait.
Instead, you might approach the problem the way the system network stack does. Sockets are not actors: the network stack drives the sockets, not the other way around.
In practice, this means that the IpStack should have some means for polling the sockets to determine which one(s) to write to/read from next. OS interfaces for this purpose, though not directly applicable, may provide some inspiration. Classically, BSD provided select(2) and poll(2); these days, APIs like epoll(7)(Linux) and kqueue(2)(FreeBSD) are preferred for large numbers of connections.
A dead simple strategy, loosely modeled on select/poll, is to repeatedly scan a list of Socket connections in round-robin fashion, handling their pending data as soon as it is available.
For a basic implementation, some concrete steps are:
When creating a new Socket, a bidirectional channel (i.e. one bounded channel in each direction) is established between it and the IpStack.
AsyncWrite on a Socket attempts to send data over the outgoing channel to the IpStack. If the channel is full, return Poll::Pending.
AsyncRead on a Socket attempts to receive data over the incoming channel from the IpStack. If the channel is empty, return Poll::Pending.
The IpStack must be driven externally (for instance, in an event loop on another thread) to continually poll the open sockets for available data, and to deliver incoming data to the correct sockets. By allowing the IpStack to control which sockets' data is sent, you can avoid the starvation problem of the Mutex solution.
I have a scenario where thread 1 writes to the socket, thread 2 reads from the socket. I have done this in the past via split() which would consume the TcpStream and return the ReadHalf/WriteHalf, which then could be neatly passed to the threads. I am running into issues on 1.39.0(tokio - 0.2.0-alpha.6).
Now it has changed to pub fn split(&mut self) -> (ReadHalf, WriteHalf). This doesn't allow passing the ReadHalf/WriteHalf(whose lifetime is tied to the stream) to separate threads, without running into messy lifetime issues
The plain read()/write() variants take &mut self, which makes it impossible to do concurrent reads/writes.
Interestingly, UdpSocket still has the old way(pub fn split(self) -> (UdpSocketRecvHalf, UdpSocketSendHalf))
Also found this related(unresolved) thread: https://github.com/tokio-rs/tokio/issues/1108. Not sure it is even possible anymore with TcpStream.
Appreciate any suggestions here.
Thanks.
You can use tokio::io::split
let (read, write) = tokio::io::split(socket);
This ends into ReadHalf and WriteHalf, both of which implement Send and Sync.
It seems to be a common idiom in Rust to spawn off a thread for blocking IO so you can use non-blocking channels:
use std::sync::mpsc::channel;
use std::thread;
use std::net::TcpListener;
fn main() {
let (accept_tx, accept_rx) = channel();
let listener_thread = thread::spawn(move || {
let listener = TcpListener::bind(":::0").unwrap();
for client in listener.incoming() {
if let Err(_) = accept_tx.send(client.unwrap()) {
break;
}
}
});
}
The problem is, rejoining threads like this depends on the spawned thread "realizing" that the receiving end of the channel has been dropped (i.e., calling send(..) returns Err(_)):
drop(accept_rx);
listener_thread.join(); // blocks until listener thread reaches accept_tx.send(..)
You can make dummy connections for TcpListeners, and shutdown TcpStreams via a clone, but these seem like really hacky ways to clean up such threads, and as it stands, I don't even know of a hack to trigger a thread blocking on a read from stdin to join.
How can I clean up threads like these, or is my architecture just wrong?
One simply cannot safely cancel a thread reliably in Windows or Linux/Unix/POSIX, so it isn't available in the Rust standard library.
Here is an internals discussion about it.
There are a lot of unknowns that come from cancelling threads forcibly. It can get really messy. Beyond that, the combination of threads and blocking I/O will always face this issue: you need every blocking I/O call to have timeouts for it to even have a chance of being interruptible reliably. If one can't write async code, one needs to either use processes (which have a defined boundary and can be ended by the OS forcibly, but obviously come with heavier weight and data sharing challenges) or non-blocking I/O which will land your thread back in an event loop that is interruptible.
mio is available for async code. Tokio is a higher level crate based on mio which makes writing non-blocking async code even more straight forward.