Here is an example program:
extern crate futures;
extern crate tokio_core;
use futures::{Async, Future, Stream};
use tokio_core::reactor::Core;
use tokio_core::net::TcpListener;
fn main() {
let mut core = Core::new().unwrap();
futures::sync::oneshot::spawn(
TcpListener::bind(&"127.0.0.1:5000".parse().unwrap(), &core.handle())
.unwrap()
.incoming()
.for_each(|_| {
println!("connection received");
Ok(())
}),
&core,
);
let ft = futures::future::poll_fn::<(), (), _>(|| {
std::thread::sleep_ms(50);
Ok(Async::NotReady)
});
core.run(ft);
}
As you can see, I call oneshot::spawn and then immediately drop its return value, which should theoretically cancel the future contained inside. However, when I run this program and then make a connection to 127.0.0.1:5000, it still prints "connection received." Why does it do this? I expected it to not print anything and drop the TcpListener, unbinding from the port.
This is a (now fixed) bug in the futures crate; version 0.1.18 should include the fix.
It used inverted values for keep_running: bool in SpawnHandle/Executor.
Related
New to Rust, really feel like I'm missing something here.
In the below code, there are two nested closures which are important to the question: app.connect_activate and draw_area.connect_draw. I am trying to use a channel to send values into the connect_draw closure, so that what is drawn will reflect the state of a simulation that changes over time (currently represented by random colors).
What I've found is very weird to me and I do not understand at all:
if I create the channel within connect_activate, I can call rx.recv within connect_draw (this is the first example below)
if I create the channel outside of connect_activate, I can call rx.recv within connect_activate (example omitted for brevity)
if I create the channel outside of connect_activate, I cannot call rx.recv within connect_draw; it fails with cannot move out of rx, a captured variable in an Fn closure (this is the second example below)
In other words: I can move the created channel across either closure boundary, but not both. It can move from the scope of main into the scope of connect_activate, or from the scope of connect_activate into the scope of connect_draw, but it cannot do both in sequence. Why would this be the case? If I can move it into connect_activate, shouldn't it then be just as "owned" as if it were created there? Both cases do seem to be truly "moved", insofar as they fail if I omit the "move" keyword. What's going on here?
Bonus question: There's a hideous placeholder where I'm re-drawing every time I get any event, because I still don't know how to run events off a timer. Halp?
Example 1: (WORKS)
extern crate cairo;
extern crate rand;
extern crate gtk;
extern crate gdk;
extern crate glib;
use std::{thread, time};
use gtk::prelude::*;
use gtk::{Application, ApplicationWindow, DrawingArea};
use std::sync::mpsc;
use std::sync::mpsc::{Receiver, Sender};
fn main(){
let app = Application::builder()
.application_id("org.example.HelloWorld")
.build();
app.connect_activate(move |app| {
let (tx, rx ) : (Sender<f64>, Receiver<f64>)= mpsc::channel();
let draw_area = DrawingArea::new();
let _id = draw_area.connect_draw(move |_unused, f| {
let red = rx.recv().unwrap();
let green = rx.recv().unwrap();
let blue = rx.recv().unwrap();
f.set_source_rgb(red,green, blue);
f.paint().expect("Painting failed");
Inhibit(false)
});
let win = ApplicationWindow::builder()
.application(app)
.default_width(320)
.default_height(200)
.title("Hello, World!")
.build();
win.add(&draw_area);
win.show_all();
win.connect_event(|w, _g|{ //Placeholder until I learn to make queue_draw fire on a timer
w.queue_draw();
Inhibit(false)
});
thread::spawn(move || {
loop {
thread::sleep(time::Duration::from_millis(100));
tx.send(rand::random::<f64>()).unwrap();
}
});
});
app.run();
}
Example 2 (DOES NOT WORK)
extern crate cairo;
extern crate rand;
extern crate gtk;
extern crate gdk;
extern crate glib;
use std::{thread, time};
use gtk::prelude::*;
use gtk::{Application, ApplicationWindow, DrawingArea};
use std::sync::mpsc;
use std::sync::mpsc::{Receiver, Sender};
fn main(){
let app = Application::builder()
.application_id("org.example.HelloWorld")
.build();
let (tx, rx ) : (Sender<f64>, Receiver<f64>)= mpsc::channel();
app.connect_activate(move |app| {
let draw_area = DrawingArea::new();
let _id = draw_area.connect_draw(move |_unused, f| {
let red = rx.recv().unwrap();
let green = rx.recv().unwrap();
let blue = rx.recv().unwrap();
f.set_source_rgb(red,green, blue);
f.paint().expect("Painting failed");
Inhibit(false)
});
let win = ApplicationWindow::builder()
.application(app)
.default_width(320)
.default_height(200)
.title("Hello, World!")
.build();
win.add(&draw_area);
win.show_all();
win.connect_event(|w, _g|{ //Placeholder until I learn to make queue_draw fire on a timer
w.queue_draw();
Inhibit(false)
});
});
thread::spawn(move || {
loop {
thread::sleep(time::Duration::from_millis(100));
tx.send(rand::random::<f64>()).unwrap();
}
});
app.run();
}
A gtk application's connect_activate() method takes in a Fn parameter, meaning it can be called multiple times. You cannot move a captured variable out of a Fn closure since the variable will have been moved next time you tried to call it. The first example works because it only moves a local variable that would be created each time.
I'm trying to benchmark the crate multiqueue with tokio to implement something along the lines of publisher/subscriber by making Streams that can be iterated. I'm not convinced on the efficiency (I may need dozens or hundreds of listeners which filter on the items and the single publisher will be publishing somewhere around 10 messages per millisecond), so I'd like to benchmark the approach before I commit to it. However, right now, I'm encountering a strange bug where sometimes the tokio::timer::Interval just doesn't seem to fire at all.
The full code is below:
#![feature(test)]
extern crate futures;
extern crate multiqueue;
extern crate test;
extern crate tokio;
#[cfg(test)]
mod tests {
use super::*;
use futures::future::lazy;
use futures::sync::mpsc::{channel, Receiver, Sender};
use futures::{Async, Poll, Stream};
use futures::{Future, Sink};
use test::Bencher;
use tokio::timer::Interval;
#[bench]
fn bench_many(b: &mut Bencher) {
tokio::run(lazy(|| {
let (tx, rx) = multiqueue::mpmc_fut_queue(1000);
tokio::spawn(
Interval::new_interval(std::time::Duration::from_micros(100))
.take(100)
.map(|_| 100)
.map_err(|e| {
eprintln!("Got interval error = {:?}", e);
})
.fold(tx, |tx, num| {
println!("Sending {}", num);
tx.send(num).map_err(|e| println!("send err = {:?}", e))
})
.map(|_| ()),
);
for i in 0..3 {
println!("Starting");
let rx = rx.clone();
tokio::spawn(rx.for_each(move |num| {
println!("{} Got a num! {}", i, num);
Ok(())
}));
}
Ok(())
}));
}
}
and I'm running it with cargo bench. futures is on version "0.1", tokio is on version "0.1", and multiqueue is on version "0.3".
Sometimes, the whole test completes with many messages of "[0-2] Got a num! 100" and "Sending 100", but sometimes it hangs either in the middle (after several "Sending" and "Got a" messages) or hangs just with 3 "Starting" messages.
I suspect this may be an issue with the number of tasks I can run at the same time with tokio, but I don't really understand why this would be a limitation I'd be running into as both types of tasks I'm spawning yield time to the executor frequently.
How can I make this more reliable?
I have a future which wraps a TCP stream in a Framed using the LinesCodec.
When I try to wrap this in a test, I get the future blocking around 20% of the time, but because I have nothing listening on the socket I'm trying to connect to, I expect to always get the error:
thread 'tokio-runtime-worker-0' panicked at 'error: Os { code: 111, kind: ConnectionRefused, message: "Connection refused" }', src/lib.rs:35:24 note: Run with 'RUST_BACKTRACE=1' for a backtrace.
This is the test code I have used:
#[macro_use(try_ready)]
extern crate futures; // 0.1.24
extern crate tokio; // 0.1.8
use std::io;
use std::net::SocketAddr;
use tokio::codec::{Framed, LinesCodec};
use tokio::net::TcpStream;
use tokio::prelude::*;
struct MyFuture {
addr: SocketAddr,
}
impl Future for MyFuture {
type Item = Framed<TcpStream, LinesCodec>;
type Error = io::Error;
fn poll(&mut self) -> Result<Async<Framed<TcpStream, LinesCodec>>, io::Error> {
let strm = try_ready!(TcpStream::connect(&self.addr).poll());
Ok(Async::Ready(Framed::new(strm, LinesCodec::new())))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::net::Shutdown;
#[test]
fn connect() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
let fut = MyFuture { addr: addr }
.and_then(|f| {
println!("connected");
let cn = f.get_ref();
cn.shutdown(Shutdown::Both)
}).map_err(|e| panic!("error: {:?}", e));
tokio::run(fut)
}
}
playground
I have seen patterns in other languages where the test binary itself offers a mechanism to return results asynchronously, but haven't found a good way of using a similar mechanism in Rust.
A simple way to test async code may be to use a dedicated runtime for each test: start it, wait for future completion and shutdown the runtime at the end of the test.
#[test]
fn my_case() {
// setup future f
// ...
tokio::run(f);
}
I don't know if there are consolidated patterns already in the Rust ecosystem; see this discussion about the evolution of testing support for future based code.
Why your code does not work as expected
When you invoke poll(), the future is queried to check if a value is available.
If a value is not available, an interest is registered so that poll() will be invoked again when something happens that can resolve the future.
When your MyFuture::poll() is invoked:
TcpStream::connect creates a new future TcpStreamNew
TcpStreamNew::poll is invoked immediately only once on the future's creation at step 1.
The future goes out of scope, so the next time you invoke MyFuture::poll you never resolve the previously created futures.
You have registered an interest for a future that, if not resolved the first time you poll it, you never ask back again (poll) for a resolved value or for an error.
The reason of the "nondeterministic" behavior is because the first poll sometimes resolve immediately with a ConnectionRefused error and sometimes it waits forever for a future connection event or a failure that it is never retrieved.
Look at mio::sys::unix::tcp::TcpStream used by Tokio:
impl TcpStream {
pub fn connect(stream: net::TcpStream, addr: &SocketAddr) -> io::Result<TcpStream> {
set_nonblock(stream.as_raw_fd())?;
match stream.connect(addr) {
Ok(..) => {}
Err(ref e) if e.raw_os_error() == Some(libc::EINPROGRESS) => {}
Err(e) => return Err(e),
}
Ok(TcpStream {
inner: stream,
})
}
When you connect on a non-blocking socket, the system call may connect/fail immediately or return EINPROGRESS, in this last case a poll must be triggered for retrieving the value of the error.
The issue is not with the test but with the implementation.
This working test case based on yours has no custom future implementation and only calls TcpStream::connect(). It works as you expect it to.
extern crate futures;
extern crate tokio;
#[cfg(test)]
mod tests {
use super::*;
use std::net::Shutdown;
use std::net::SocketAddr;
use tokio::net::TcpStream;
use tokio::prelude::*;
#[test]
fn connect() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
let fut = TcpStream::connect(&addr)
.and_then(|f| {
println!("connected");
f.shutdown(Shutdown::Both)
}).map_err(|e| panic!("error: {:?}", e));
tokio::run(fut)
}
}
playground
You are connecting to the same endpoint over and over again in your poll() method. That's not how a future works. The poll() method will be called repeatedly, with the expectation that at some point it will return either Ok(Async::Ready(..)) or Err(..).
If you initiate a new TCP connection every time poll() is called, it will be unlikely to complete in time.
Here is a modified example that does what you expect:
#[macro_use(try_ready)]
extern crate futures;
extern crate tokio;
use std::io;
use std::net::SocketAddr;
use tokio::codec::{Framed, LinesCodec};
use tokio::net::{ConnectFuture, TcpStream};
use tokio::prelude::*;
struct MyFuture {
tcp: ConnectFuture,
}
impl MyFuture {
fn new(addr: SocketAddr) -> MyFuture {
MyFuture {
tcp: TcpStream::connect(&addr),
}
}
}
impl Future for MyFuture {
type Item = Framed<TcpStream, LinesCodec>;
type Error = io::Error;
fn poll(&mut self) -> Result<Async<Framed<TcpStream, LinesCodec>>, io::Error> {
let strm = try_ready!(self.tcp.poll());
Ok(Async::Ready(Framed::new(strm, LinesCodec::new())))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::net::Shutdown;
#[test]
fn connect() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
let fut = MyFuture::new(addr)
.and_then(|f| {
println!("connected");
let cn = f.get_ref();
cn.shutdown(Shutdown::Both)
}).map_err(|e| panic!("error: {:?}", e));
tokio::run(fut)
}
}
I'm not certain what you intend your future to do, though; I can't comment if it's the right approach.
to some degree, you can drop in tokio's test library to make this easier; it supports async/await in unit tests.
#[tokio::test]
async fn my_future_test() {
let addr: SocketAddr = "127.0.0.1:4222".parse().unwrap();
match MyFuture { addr }.poll().await {
Ok(f) => assert!("something good")
Err(e) => assert!("something bad")
}
}
https://docs.rs/tokio/0.3.3/tokio/attr.test.html
I've read the documentation for std::net and mio, and I've found some methods like set_nodelay and set_keepalive, but I haven't found a way to set other socket options like SO_REUSEPORT and SO_REUSEADDR on a given socket. How can I do this?
Because SO_REUSEPORT isn't cross-platform, you will need to dip into platform-specific code. In this case, you can get the raw file descriptor from the socket and then use functions, types, and values from the libc crate to set the options you want:
extern crate libc; // 0.2.43
use std::{io, mem, net::TcpListener, os::unix::io::AsRawFd};
fn main() -> Result<(), io::Error> {
let listener = TcpListener::bind("0.0.0.0:8888")?;
unsafe {
let optval: libc::c_int = 1;
let ret = libc::setsockopt(
listener.as_raw_fd(),
libc::SOL_SOCKET,
libc::SO_REUSEPORT,
&optval as *const _ as *const libc::c_void,
mem::size_of_val(&optval) as libc::socklen_t,
);
if ret != 0 {
return Err(io::Error::last_os_error());
}
}
Ok(())
}
I make no guarantee that this is the right place to set this option, or that I haven't screwed up something in the unsafe block, but it does compile and run on macOS 10.12.
A better solution may be to check out the nix crate, which provides nicer wrappers for most *nix-specific code:
extern crate nix; // 0.11.0
use nix::sys::socket::{self, sockopt::ReusePort};
use std::{error::Error, net::TcpListener, os::unix::io::AsRawFd};
fn main() -> Result<(), Box<Error>> {
let listener = TcpListener::bind("0.0.0.0:8888")?;
socket::setsockopt(listener.as_raw_fd(), ReusePort, &true)?;
Ok(())
}
An even better solution may be to check out the net2 crate, which provides higher-level methods aimed specifically at networking-related code:
extern crate net2; // 0.2.33
use net2::{unix::UnixTcpBuilderExt, TcpBuilder};
fn main() -> Result<(), std::io::Error> {
let listener = TcpBuilder::new_v4()?
.reuse_address(true)?
.reuse_port(true)?
.bind("0.0.0.0:8888")?
.listen(42)?;
Ok(())
}
One uses this to send output to stdout:
println!("some output")
I think there is no corresponding macro to do the same for stderr.
After Rust 1.19
As of Rust 1.19, you can use the eprint and eprintln macros:
fn main() {
eprintln!("This is going to standard error!, {}", "awesome");
}
This was originally proposed in RFC 1896.
Before Rust 1.19
You can see the implementation of println! to dive into exactly how it works, but it was a bit overwhelming when I first read it.
You can format stuff to stderr using similar macros though:
use std::io::Write;
let name = "world";
writeln!(&mut std::io::stderr(), "Hello {}!", name);
This will give you a unused result which must be used warning though, as printing to IO can fail (this is not something we usually think about when printing!). We can see that the existing methods simply panic in this case, so we can update our code to do the same:
use std::io::Write;
let name = "world";
let r = writeln!(&mut std::io::stderr(), "Hello {}!", name);
r.expect("failed printing to stderr");
This is a bit much, so let's wrap it back in a macro:
use std::io::Write;
macro_rules! println_stderr(
($($arg:tt)*) => { {
let r = writeln!(&mut ::std::io::stderr(), $($arg)*);
r.expect("failed printing to stderr");
} }
);
fn main() {
let name = "world";
println_stderr!("Hello {}!", name)
}
print! and println! are convenience methods for writing to standard output. There are other macros with the same formatting features available for different tasks:
write! and writeln! to write a formatted string to a &mut Writer
format! to just generate a formatted String
To write to the standard error stream, you can use e.g. writeln! like this:
use std::io::Write;
fn main() {
let mut stderr = std::io::stderr();
writeln!(&mut stderr, "Error!").unwrap();
}
It's done so:
use std::io::Write;
fn main() {
std::io::stderr().write(b"some output\n");
}
You can test it by sending the program output to /dev/null to ensure it works (I ignore the warning):
$ rustc foo.rs && ./foo > /dev/null
foo.rs:4:5: 4:42 warning: unused result which must be used, #[warn(unused_must_use)] on by default
foo.rs:4 io::stderr().write(b"some output\n");
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
some output
Similarly, one can do the following for stdout:
use std::io::Write;
fn main() {
std::io::stdout().write(b"some output\n");
}
I think this means println! is just a convenience: it's shorter and it also allows some formatting. As an example of the latter, the following displays 0x400:
println!("0x{:x}", 1024u)
While not answering the precise question, maybe it’s of interest that there’s a log crate which specifies an interface for leveled logging that other crates (e.g. env_logger) can fulfill.
The output of such logging will be sent to stderr, and there are additional benefits for users, such as specifying the log level.
This is how using such a logger could look like:
#[macro_use]
extern crate log;
extern crate env_logger;
fn main() {
env_logger::init().unwrap();
error!("this is printed by default");
}
(Example adapted from http://burntsushi.net/rustdoc/env_logger/index.html#example)
Goal
stderr!("Code {}: Danger, Will Robinson! Danger!", 42);
Notes
The other answers generate an unused import warning with the latest nightly, so here's a modern macro that Just Works TM.
Code
macro_rules! stderr {
($($arg:tt)*) => (
use std::io::Write;
match writeln!(&mut ::std::io::stderr(), $($arg)* ) {
Ok(_) => {},
Err(x) => panic!("Unable to write to stderr (file handle closed?): {}", x),
}
)
}