Perpetual tokio TCP stream (client) - rust

Question

Foreword, you can skip to the next section
So I decided to try Rust for my new relatively small project, because I like that it produces a single executable which is easy to deploy on my ARM-based target with relatively little resources in terms of RAM and disk space. I have no previous experience with Rust, but a lot of experience with other languages, and so far I am getting somewhat disappointed. It seems that for many Rust libraries and probably Rust itself, the APIs are changing so fast that 90% of sample code found online will not compile with latest versions of libraries like tokio, tokio-util etc. Also, the documentation is often misleading. For example, if you Google for LinesCodec it will show up in tokio_io::codec::LinesCodec, tokio::codec::LinesCodec, tokio_codec::LinesCodec and tokio_util::codec::LinesCodec, which ultimately seems to be one to use as of today. Same confusion goes for other things like FramedRead, which had an and_then and map member functions in some versions, but they doesn't seem to exist in the latest version. Lastly, the amount of questions and answers related to Rust on SO is far less than for other languages I've used, which makes it harder to start using Rust. What I'm trying to do for the past 2 days is solved relatively easily in most programming languages, and I believe, there must be an easy solution in Rust as well, but so far I had no success.
Question itself
I need to connect a TCP client to a remote server and indefinitely read and process data line by line as it comes in. This needs to be done asynchronously because the same process also acts as a HTTP server, so I'm using tokio.
As far as I understand, the somewhat common way is to use TcpStream, slit it to RX/TX parts, then I'm trying to hook up a LinesCodec (with FramedRead) but I'm unable to hook all these together without getting compilation errors.
[dependencies]
futures = "*"
hyper = "*"
tokio = { version = "*", features = ["full"] }
tokio-util = "0.2.0"
tokio-modbus = { version = "*", features = ["tcp", "server", "tcp-server-unstable"], git = "https://github.com/slowtec/tokio-modbus" }
let stream = TcpStream::connect("172.16.100.10:1001").await.unwrap();
let transport = FramedRead::new(stream, LinesCodec::new()); // need to split?
/* ... what to do next to process incoming data line-by-line ...? */
So far I came with this solution, not sure how good it is though
tokio::spawn(async {
let connection = TcpStream::connect("172.16.100.10:1001").await.unwrap();
let mut reader = BufReader::new(connection);
loop {
let mut line = String::new();
reader.read_line(&mut line).await.unwrap();
println!("{}", line);
}
});

Answer 1

Simple app with a Cargo.toml like:
[dependencies]
tokio = { version = "0.3", features = ["full"] }
tokio-util = { version = "0.4", features = ["codec"] }
and a main.rs like:
use tokio::net::{TcpListener, TcpStream };
use tokio_util::codec::{ Framed, LinesCodec };
use tokio::stream::StreamExt;
use std::error::Error;
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let args: Vec<String> = std::env::args().collect();
if args[1] == "server"
{
let local_addr: String = format!("{}{}",":::",args[2]); // app <server | client> <port>
let listener = TcpListener::bind(&local_addr).await?;
while let Ok((socket, peer)) = listener.accept().await {
tokio::spawn(async move {
println!("Client Connected from: {}",peer.to_string());
let mut client = Framed::new(socket, LinesCodec::new_with_max_length(1024));
while let Some(Ok(line)) = client.next().await {
println!("{}", line);
}
});
}
}
else if args[1] == "client"
{
let port = args[2].parse::<u16>().unwrap(); // app client <port>
let saddr = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), port);
let conn = TcpStream::connect(saddr).await?;
let mut server = Framed::new(conn, LinesCodec::new_with_max_length(1024));
while let Some(Ok(line)) = server.next().await {
println!("{}", line);
}
}
Ok({})
}
To run as server:
cargo run server 8080
(in another shell) nc localhost 8080
to run as client:
(in another shell) nc -l -p 8080
cargo run client 8080

Answer 2

There's an example of how to use LineCodec in the chat example program. The relevant section is the "process" function.
A smaller example (server, but the principle is the same for a client) that reverses each line that it receives and echos it back, with a maximum buffer size of 5000:
use tokio::net::TcpListener;
use tokio::stream::StreamExt;
use tokio_util::codec::{Framed, LinesCodec};
extern crate unicode_segmentation;
use unicode_segmentation::UnicodeSegmentation;
use futures::SinkExt;
async fn talk(sock: tokio::net::TcpStream) {
let mut lines = Framed::new(sock, LinesCodec::LinesCodec::new_with_max_length(5000));
while let Some(Ok(line)) = lines.next().await {
let rev = line.graphemes(true).rev().collect::<String>();
if let Err(_e) = lines.send(rev).await {
break;
}
}
}
#[tokio::main]
async fn main() {
let addr = "127.0.0.1:6200";
let mut listener = TcpListener::bind(addr).await.unwrap();
let mut incoming = listener.incoming();
while let Some(conn) = incoming.next().await {
match conn {
Err(e) => eprintln!("accept failed = {:?}", e),
Ok(sock) => {
tokio::spawn(talk(sock));
}
}
}
}
An issue with the read_line workaround is that read_line doesn't give you an option to limit the line length, so if you have untrusted input, it could cause your program to consume arbitrary amounts of memory. LineCodec does give an option to limit the line length.