I am trying to write a P2P node. A node contains a list of peers, opens a listening port to let other nodes establish a communication channel and, at the same time, actively tries to establish the connection with some other chosen nodes. This choice is as such that there is a single connection between each pair of nodes (not implemented in the following snippet).
I am trying to achieve this with Tokio using futures combinators.
main.rs
use failure::Error;
pub mod networking {
use failure::Error;
use std::net::SocketAddr;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use crate::Config;
use futures::Future;
pub fn start(cfg: &Config) -> Result<(), Error> {
let myself = cfg.myself.parse::<SocketAddr>()?;
let others = cfg
.others
.iter()
.filter_map(|s| s.parse().ok())
.collect::<Vec<SocketAddr>>();
let server = TcpListener::bind(&myself)?
.incoming()
.for_each(|socket| {
println!("Got a socket: {:?}", socket);
future::ok(())
})
.map_err(|e| eprintln!("Error connecting: {:?}", e));
let client = TcpStream::connect(&others[0])
.map(|socket| {
println!("Got a socket: {:?}", socket);
})
.map_err(|e| eprintln!("Error connecting: {:?}", e));
let future = server.join(client);
tokio::run(future);
Ok(())
}
}
struct Config {
myself: String,
others: Vec<String>,
}
fn main() -> Result<(), Error> {
let config = Config {
myself: "127.0.0.1:2501".to_string(),
others: vec!["127.0.0.1:2502".to_string(), "127.0.0.1:2503".to_string()],
};
networking::start(&config)
}
(playground)
This does not work:
error[E0271]: type mismatch resolving `<futures::future::join::Join<futures::future::map_err::MapErr<futures::stream::for_each::ForEach<tokio_tcp::incoming::Incoming, [closure#src/main.rs:23:23: 26:14], futures::future::result_::FutureResult<(), std::io::Error>>, [closure#src/main.rs:27:22: 27:64]>, futures::future::map_err::MapErr<futures::future::map::Map<tokio_tcp::stream::ConnectFuture, [closure#src/main.rs:30:18: 32:14]>, [closure#src/main.rs:33:22: 33:64]>> as futures::future::Future>::Item == ()`
--> src/main.rs:37:9
|
37 | tokio::run(future);
| ^^^^^^^^^^ expected tuple, found ()
|
= note: expected type `((), ())`
found type `()`
= note: required by `tokio::runtime::threadpool::run`
I understand what the compiler says, but I have no clue what exactly I have to correct.
What should I correct to get the types right?
thanks for the hints,
let future = server.join(client);
has to be corrected to
let future = server.join(client).map(|_| ())
Related
I'm facing compiler errors with dynamic traits in a scenario where I either have a std::net::TcpStream or an native_tls::TlsStream<TcpStream>. The idea for my algorithm is the following:
create TCP stream in var "tcp"
if scheme is https => replace same var with new TlsStream<TcpStream>.
Write and receive data on stream.
The important traits I need are std::io::Read and std::io::Write. Therefore I thought, I just model my "tcp" var as Box<dyn Read + Write>. But no matter how I tried, I always got compilation errors. Below you can see a minimal example, that doesn't compile.
use std::net::{IpAddr, Ipv4Addr, TcpStream};
use std::io::{Write as IoWrite, Read as IoRead};
use native_tls::TlsConnector;
fn main() {
let use_https = true;
// IP of "example.com".
let ip_addr = std::net::IpAddr::V4(Ipv4Addr::new(85, 13, 155, 159));
let port = if use_https { 443 } else { 80 };
let tcp_stream = TcpStream::connect((ip_addr, port)).unwrap();
let tcp_stream = maybe_connect_to_tls(tcp_stream, use_https);
}
fn maybe_connect_to_tls(tcp: TcpStream, use_https: bool) -> Box<dyn IoRead + IoWrite> {
if use_https {
let tls = TlsConnector::new().unwrap();
let tcp = tls.connect("example.com", tcp).unwrap();
Box::new(tcp)
} else {
Box::new(tcp)
}
}
Playground link
This is the error I get when I try to compile it:
error[E0225]: only auto traits can be used as additional traits in a trait object
--> src/main.rs:14:78
|
14 | fn maybe_connect_to_tls(tcp: TcpStream, use_https: bool) -> Box<dyn IoRead + IoWrite> {
| ------ ^^^^^^^ additional non-auto trait
| |
| first non-auto trait
|
= help: consider creating a new trait with all of these as super-traits and using that trait here instead: `trait NewTrait: std::io::Read + std::io::Write {}`
= note: auto-traits like `Send` and `Sync` are traits that have special properties; for more information on them, visit <https://doc.rust-lang.org/reference/special-types-and-traits.html#auto-traits>
I also tried maybe_connect_to_tls<T>(tcp: TcpStream, use_https: bool) -> Box<T> where T: IoWrite + IoRead + ?Sized but whatever I try leads to other compilation failures..
Any idea how I can solve this? I want a common stream object and it should be invisible to the application what the specific implementation is.
For Rust internal reasons, your trait objects can contain no more than one non-auto trait. However, you can simply create a trait which depends on the Read and Write traits:
use std::net::{IpAddr, Ipv4Addr, TcpStream};
use std::io::{Write as IoWrite, Read as IoRead};
use native_tls::TlsConnector;
trait ReadAndWrite: IoRead + IoWrite {
}
impl <T: IoRead + IoWrite> ReadAndWrite for T {
}
fn main() {
let use_https = true;
// IP of "example.com".
let ip_addr = std::net::IpAddr::V4(Ipv4Addr::new(85, 13, 155, 159));
let port = if use_https { 443 } else { 80 };
let tcp_stream = TcpStream::connect((ip_addr, port)).unwrap();
let tcp_stream = maybe_connect_to_tls(tcp_stream, use_https);
}
fn maybe_connect_to_tls(tcp: TcpStream, use_https: bool) -> Box<dyn ReadAndWrite> {
if use_https {
let tls = TlsConnector::new().unwrap();
let tcp = tls.connect("example.com", tcp).unwrap();
Box::new(tcp)
} else {
Box::new(tcp)
}
}
Playground link
SOLUTION:
I needed to add "use std::io::prelude::*;" to my code. I do not know why.
I am trying to read from an std::net::TcpStream but I recieve this error when calling stream.read(&buf).unwrap;
the method read exists for struct std::net::TcpStream, but its
trait bounds were not satisfied method cannot be called on
std::net::TcpStream due to unsatisfied trait bounds note: the
following trait bounds were not satisfied:
std::net::TcpStream: futures::AsyncRead
which is required by std::net::TcpStream: futures::AsyncReadExt help: items from traits can only be used if the
trait is in scoperustc(E0599) main.rs(31, 16): method cannot be called
on std::net::TcpStream due to unsatisfied trait bounds tcp.rs(49,
1): doesn't satisfy std::net::TcpStream: futures::AsyncReadExt
tcp.rs(49, 1): doesn't satisfy std::net::TcpStream: futures::AsyncRead mod.rs(580, 8): the method is available for
std::boxed::Box<std::net::TcpStream> here
Code:
use irc::client::prelude::*;
use futures::prelude::*;
use std::net::{IpAddr, Ipv4Addr, SocketAddr, TcpStream};
use std::io;
use futures::{AsyncRead, AsyncReadExt};
const NAME: &str = "nickname";
#[derive(Debug)]
struct DCC {
ip: IpAddr,
port: u16,
}
impl DCC {
fn from_msg(msg: &str) -> Result<DCC, std::num::ParseIntError> {
let msg_split: Vec<&str> = msg.split_whitespace().collect();
let ip: u32 = msg_split[3].parse()?;
let ip_addr: IpAddr = IpAddr::V4(Ipv4Addr::from(ip));
let port_num: u16 = msg_split[4].parse()?;
let dcc = DCC{
ip: ip_addr,
port: port_num,
};
return Ok(dcc);
}
async fn connect(&self) -> Result<(), io::Error>{
let socket_addr = SocketAddr::new(self.ip, self.port);
let mut socket = TcpStream::connect(socket_addr)?;
let mut buf = vec![];
socket.read(&buf).unwrap();
return Err(io::Error::new(io::ErrorKind::Other, "oh no!"));
}
}
#[tokio::main]
async fn irc_get(name: &str) -> Result<String, irc::error::Error>{
let config = Config {
nickname: Some(NAME.to_owned()),
server: Some("irc.irchighway.net".to_owned()),
port: Some(6667),
use_tls: Some(false),
channels: vec!["#ebooks".to_owned()],
..Config::default()
};
let mut client = Client::from_config(config).await?;
client.identify()?;
let mut stream = client.stream()?;
//waits for server to log us in and then sends the search request
loop{
let m = match stream.next().await{
Some(v) => v,
None => panic!("at the disco")
};
let message = match &m {
Ok(message) => match &message.command {Command::NOTICE(_s1, s2)=> {print!("{:?} \n", s2); message}, _ => message},
Err(_e) => panic!("at the disco")};
match &message.command{
Command::NOTICE(_s, msg) => { if msg.contains("Welcome to #ebooks"){break}},
_=> ()
}
}
client.send_privmsg("#ebooks", format!["#Search {}", name])?;
loop{
let m = match stream.next().await.transpose()?{
Some(m) => m,
None => panic!("at the disco")
};
match &m.command{
Command::PRIVMSG(nm, msg) => if nm == NAME {println!("{:?}",m); return Ok(String::from(msg))},
_ => ()
}
}
}
fn main() {
let dcc = DCC::from_msg(&irc_get(&"romeo and juliet").unwrap()[..]);
println!("{:?}", dcc);
}
I'm fairly new at rust and based on all of the examples in the documentation I think I'm using .read correctly. My only thought is that maybe it's because I'm trying to write the code in the impl, but I don't know if rust treats that differently. It also fails with "async fn connect..." and with "fn connect...".
The compiler was telling you the solution:
help: items from traits can only be used if the trait is in scope
You need to import the traits in order to use them:
use futures::{AsyncRead, AsyncReadExt};
Also you would probably want to use tokio::TcpStream which is async and not the std one.
I'm trying to figure out build a feature which requires reading the contents of a file into a futures::stream::BoxStream but I'm having a tough time figuring out what I need to do.
I have figured out how to read a file byte by byte via Bytes which implements an iterator.
use std::fs::File;
use std::io::prelude::*;
use std::io::{BufReader, Bytes};
// TODO: Convert this to a async Stream
fn async_read() -> Box<dyn Iterator<Item = Result<u8, std::io::Error>>> {
let f = File::open("/dev/random").expect("Could not open file");
let reader = BufReader::new(f);
let iter = reader.bytes().into_iter();
Box::new(iter)
}
fn main() {
ctrlc::set_handler(move || {
println!("received Ctrl+C!");
std::process::exit(0);
})
.expect("Error setting Ctrl-C handler");
for b in async_read().into_iter() {
println!("{:?}", b);
}
}
However, I've been struggling a bunch trying to figure out how I can turn this Box<dyn Iterator<Item = Result<u8, std::io::Error>>> into an Stream.
I would have thought something like this would work:
use futures::stream;
use std::fs::File;
use std::io::prelude::*;
use std::io::{BufReader, Bytes};
// TODO: Convert this to a async Stream
fn async_read() -> stream::BoxStream<'static, dyn Iterator<Item = Result<u8, std::io::Error>>> {
let f = File::open("/dev/random").expect("Could not open file");
let reader = BufReader::new(f);
let iter = reader.bytes().into_iter();
std::pin::Pin::new(Box::new(stream::iter(iter)))
}
fn main() {
ctrlc::set_handler(move || {
println!("received Ctrl+C!");
std::process::exit(0);
})
.expect("Error setting Ctrl-C handler");
while let Some(b) = async_read().poll() {
println!("{:?}", b);
}
}
But I keep getting a ton of compiler errors, I've tried other permutations but generally getting no where.
One of the compiler errors:
std::pin::Pin::new
``` --> src/main.rs:14:24
|
14 | std::pin::Pin::new(Box::new(stream::iter(iter)))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected trait object `dyn std::iter::Iterator`, found enum `std::result::Result`
Anyone have any advice?
I'm pretty new to Rust, and specifically Streams/lower level stuff so I apologize if I got anything wrong, feel free to correct me.
For some additional background, I'm trying to do this so you can CTRL-C out of a command in nushell
I think you are overcomplicating it a bit, you can just return impl Stream from async_read, there is no need to box or pin (same goes for the original Iterator-based version). Then you need to set up an async runtime in order to poll the stream (in this example I just use the runtime provided by futures::executor::block_on). Then you can call futures::stream::StreamExt::next() on the stream to get a future representing the next item.
Here is one way to do this:
use futures::prelude::*;
use std::{
fs::File,
io::{prelude::*, BufReader},
};
fn async_read() -> impl Stream<Item = Result<u8, std::io::Error>> {
let f = File::open("/dev/random").expect("Could not open file");
let reader = BufReader::new(f);
stream::iter(reader.bytes())
}
async fn async_main() {
while let Some(b) = async_read().next().await {
println!("{:?}", b);
}
}
fn main() {
ctrlc::set_handler(move || {
println!("received Ctrl+C!");
std::process::exit(0);
})
.expect("Error setting Ctrl-C handler");
futures::executor::block_on(async_main());
}
I'm experimenting with the futures API using the websocket library. I have this code:
use futures::future::Future;
use futures::future;
use futures::sink::Sink;
use futures::stream::Stream;
use futures::sync::mpsc::channel;
use futures::sync::mpsc::{Sender, Receiver};
use tokio_core::reactor::Core;
use websocket::{ClientBuilder, OwnedMessage};
pub fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let handle_clone = handle.clone();
let (send, recv): (Sender<String>, Receiver<String>) = channel(100);
let f = ClientBuilder::new("wss://...")
.unwrap()
.async_connect(None, &handle_clone)
.map_err(|e| println!("error: {:?}", e))
.map(|(duplex, _)| duplex.split())
.and_then(move |(sink, stream)| {
// this task consumes the channel, writes messages to the websocket
handle_clone.spawn(future::loop_fn(recv, |recv: Receiver<String>| {
sink.send(OwnedMessage::Close(None))
.and_then(|_| future::ok(future::Loop::Break(())))
.map_err(|_| ())
}));
// the main tasks listens the socket
future::loop_fn(stream, |stream| {
stream
.into_future()
.and_then(|_| future::ok(future::Loop::Break(())))
.map_err(|_| ())
})
});
loop {
core.turn(None)
}
}
After connecting to the server, I want to run "listener" and "sender" tasks without one blocking the other one. The problem is I can't use sink in the new task, it fails with:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/slack_conn.rs:29:17
|
25 | .and_then(move |(sink, stream)| {
| ---- captured outer variable
...
29 | sink.send(OwnedMessage::Close(None))
| ^^^^ cannot move out of captured outer variable in an `FnMut` closure
I could directly use duplex to send and receive, but that leads to worse errors.
Any ideas on how to make this work? Indeed, I'd be happy with any futures code that allows me to non-blockingly connect to a server and spawn two async tasks:
one that reads from the connection and takes some action (prints to screen etc.)
one that reads from a mpsc channel and writes to the connection
It's fine if I have to write it in a different style.
SplitSink implements Sink which defines send to take ownership:
fn send(self, item: Self::SinkItem) -> Send<Self>
where
Self: Sized,
On the other hand, loop_fn requires that the closure be able to be called multiple times. These two things are fundamentally incompatible — how can you call something multiple times which requires consuming a value?
Here's a completely untested piece of code that compiles — I don't have rogue WebSocket servers lying about.
#[macro_use]
extern crate quick_error;
extern crate futures;
extern crate tokio_core;
extern crate websocket;
use futures::{Future, Stream, Sink};
use futures::sync::mpsc::channel;
use tokio_core::reactor::Core;
use websocket::ClientBuilder;
pub fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let (send, recv) = channel(100);
let f = ClientBuilder::new("wss://...")
.unwrap()
.async_connect(None, &handle)
.from_err::<Error>()
.map(|(duplex, _)| duplex.split())
.and_then(|(sink, stream)| {
let reader = stream
.for_each(|i| {
println!("Read a {:?}", i);
Ok(())
})
.from_err();
let writer = sink
.sink_from_err()
.send_all(recv.map_err(Error::Receiver))
.map(|_| ());
reader.join(writer)
});
drop(send); // Close the sending channel manually
core.run(f).expect("Unable to run");
}
quick_error! {
#[derive(Debug)]
pub enum Error {
WebSocket(err: websocket::WebSocketError) {
from()
description("websocket error")
display("WebSocket error: {}", err)
cause(err)
}
Receiver(err: ()) {
description("receiver error")
display("Receiver error")
}
}
}
The points that stuck out during implementation were:
everything has to become a Future eventually
it's way easier to define an error type and convert to it
Knowing if the Item and Error associated types were "right" was tricky. I ended up doing a lot of "type assertions" ({ let x: &Future<Item = (), Error = ()> = &reader; }).
I am writing a Phoenix client library for Rust, taking advantage of the async websocket client from rust-websockets. Right now I am having trouble figuring out how to pass callback functions into the thread that is handling the websocket traffic. I have a simplified struct:
pub struct Socket {
endpoint: String,
connected: Arc<AtomicBool>,
state_change_close: Option<Box<FnMut(String)>>,
}
This struct has a connect function laid out as follows:
pub fn connect(&mut self) -> Result<(), String> {
if self.connected.load(Ordering::Relaxed) {
return Ok(())
}
// Copy endpoint string, otherwise we get an error on thread::spawn
let connection_string = self.endpoint.clone();
let (usr_msg, stdin_ch) = mpsc::channel(0);
let connection_thread = thread::spawn(move || {
// tokio core for running event loop
let mut core = Core::new().unwrap();
let runner = ClientBuilder::new(&connection_string)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (sink, stream) = duplex.split();
stream.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => {
// This is the line where I am trying to call the callback
if let Some(ref mut func) = self.state_change_close {
(func)(e.unwrap().reason);
}
Some(OwnedMessage::Close(e))
},
_ => None,
}
})
.select(stdin_ch.map_err(|_| WebSocketError::NoDataAvailable))
.forward(sink)
});
// Start the event loop
core.run(runner).unwrap();
});
self.connected.store(true, Ordering::Relaxed);
return Ok(())
}
When I try to compile this code I get the following error:
error[E0277]: the trait bound `std::ops::FnMut(std::string::String) + 'static: std::marker::Send` is not satisfied
--> src\socket.rs:99:29
|
99 | let connection_thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ the trait `std::marker::Send` is not implemented for `std::ops::FnMut(std::string::String) + 'static`
|
I have tried changing the type of state_change_close to a Mutex<Option<...>> to avoid thread safety issues, but that did not help with this problem. Is what I'm trying to do possible?
After doing some more research I realized that I just had to modify Option<Box<FnMut(String)>> to be Option<Box<FnMut(String) + Send>> and copy that around my code to everywhere that the callback might be set. Learning more about trait objects!