How do you test your futures which are meant to be run in the Tokio runtime?
fn fut_dns() -> impl Future<Item = (), Error = ()> {
let f = dns::lookup("www.google.de", "127.0.0.1:53");
f.then(|result| match result {
Ok(smtptls) => {
println!("{:?}", smtptls);
assert_eq!(smtptls.version, "TLSRPTv1");
assert!(smtptls.rua.len() > 0);
assert_eq!(smtptls.rua[0], "mailto://...");
ok(())
}
Err(e) => {
println!("error: {:?}", e);
err(())
}
})
}
#[test]
fn smtp_log_test() {
tokio::run(fut_dns());
assert!(true);
}
The future runs and the thread of the future panics on an assert. You can read the panic in the console, but the test doesn't recognize the threads of tokio::run.
The How can I test a future that is bound to a tokio TcpStream? doesn't answer this, because it simply says: A simple way to test async code may be to use a dedicated runtime for each test
I do this!
My question is related to how the test can detect if the future works. The future needs a started runtime environment.
The test is successful although the future asserts or calls err().
So what can I do?
Do not write your assertions inside the future.
As described in How can I test a future that is bound to a tokio TcpStream?, create a Runtime to execute your future. As described in How do I synchronously return a value calculated in an asynchronous Future in stable Rust?, compute your value and then exit the async world:
fn run_one<F>(f: F) -> Result<F::Item, F::Error>
where
F: IntoFuture,
F::Future: Send + 'static,
F::Item: Send + 'static,
F::Error: Send + 'static,
{
let mut runtime = tokio::runtime::Runtime::new().expect("Unable to create a runtime");
runtime.block_on(f.into_future())
}
#[test]
fn smtp_log_test() {
let smtptls = run_one(dns::lookup("www.google.de", "127.0.0.1:53")).unwrap();
assert_eq!(smtptls.version, "TLSRPTv1");
assert!(smtptls.rua.len() > 0);
assert_eq!(smtptls.rua[0], "mailto://...");
}
Related
In Rust, I would like to do multiple tasks in parallel and when each task finishes, I would like to do another task handled by the main process.
I know that tasks will finish at different timings, and I don't want to wait for all the tasks to do the next task.
I've tried doing multiple threads handled by the main process but I have to wait for all the threads to finish before doing another action or maybe I did not understand.
for handle in handles {
handle.join().unwrap();
}
How can I manage to do a task handled by the main process after each end of threads without blocking the whole main thread?
Here is a diagram to explain what I want to do :
If i'm not clear or if you have a better idea to handle my problem, don't mind to tell me!
Here's an example how to implement this using FuturesUnordered and Tokio:
use futures::{stream::FuturesUnordered, StreamExt};
use tokio::time::sleep;
use std::{time::Duration, future::ready};
#[tokio::main]
async fn main() {
let tasks = FuturesUnordered::new();
tasks.push(some_task(1000));
tasks.push(some_task(2000));
tasks.push(some_task(500));
tasks.push(some_task(1500));
tasks.for_each(|result| {
println!("Task finished after {} ms.", result);
ready(())
}).await;
}
async fn some_task(delay_ms: u64) -> u64 {
sleep(Duration::from_millis(delay_ms)).await;
delay_ms
}
If you run this code, you can see that the closure passed to for_each() is executed immediately whenever a task finishes, even though they don't finish in the order they were created.
Note that Tokio takes care of scheduling the tasks to different threads for you. By default, there will be one thread per CPU core.
To compile this, you need to add this to your Cargo.toml file:
[dependencies]
futures = "0.3"
tokio = { version = "1", features = ["full"] }
If you want to add some proper error propagation, the code becomes only slightly more complex – most of the added code is for the custom error type:
use futures::{stream::FuturesUnordered, TryStreamExt};
use tokio::time::sleep;
use std::{time::Duration, future::ready};
#[tokio::main]
async fn main() -> Result<(), MyError> {
let tasks = FuturesUnordered::new();
tasks.push(some_task(1000));
tasks.push(some_task(2000));
tasks.push(some_task(500));
tasks.push(some_task(1500));
tasks.try_for_each(|result| {
println!("Task finished after {} ms.", result);
ready(Ok(()))
}).await
}
async fn some_task(delay_ms: u64) -> Result<u64, MyError> {
sleep(Duration::from_millis(delay_ms)).await;
Ok(delay_ms)
}
#[derive(Debug)]
struct MyError {}
impl std::fmt::Display for MyError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "MyError occurred")
}
}
impl std::error::Error for MyError {}
I'm trying to learn async programming, but this very basic example doesn't work:
use std::future::Future;
fn main() {
let t = async {
println!("Hello, world!");
};
t.poll();
}
Everything I've read from the specs says this should work, but cargo complains that method "poll" can't be found in "impl std::future::Future". What am I doing wrong?
poll has this signature:
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
There are two problems with calling this in the way you do:
poll is not implement on a future Fut, but on Pin<&mut Fut>, so you need to get a pinned reference first. pin_mut! is often useful, and if the future implements Unpin, you can use Pin::new as well.
The bigger problem however is that poll takes a &mut Context<'_> argument. The context is created by the asynchronous runtime and passed in to the poll function of the outermost future. This means that you can't just poll a future like that, you need to be in an asynchronous runtime to do it.
Instead, you can use a crate like tokio or async-std to run a future in a synchronous context:
// tokio
use tokio::runtime::Runtime;
let runtime = Runtime::new().unwrap();
let result = runtime.block_on(async {
// ...
});
// async-std
let result = async_std::task::block_on(async {
// ...
})
Or even better, you can use #[tokio::main] or #[async_std::main] to convert your main function into an asynchronous function:
// tokio
#[tokio::main]
async fn main() {
// ...
}
// async-std
#[async_std::main]
async fn main() {
// ...
}
In the Tokio documentation we have this snippet:
extern crate tokio;
extern crate futures;
use futures::future::lazy;
tokio::run(lazy(|| {
for i in 0..4 {
tokio::spawn(lazy(move || {
println!("Hello from task {}", i);
Ok(())
}));
}
Ok(())
}));
The explanation for this is:
The lazy function runs the closure the first time the future is polled. It is used here to ensure that tokio::spawn is called from a task. Without lazy, tokio::spawn would be called from outside the context of a task, which results in an error.
I'm not sure I understand this precisely, despite having some familiarity with Tokio. It seems that these two lazy have slightly different roles, and that this explanation only applies to the first one. Isn't the second call to lazy (inside the for loop) just here to convert the closure into a future?
The purpose of lazy is covered by the documentation for lazy:
Creates a new future which will eventually be the same as the one created by the closure provided.
The provided closure is only run once the future has a callback scheduled on it, otherwise the callback never runs. Once run, however, this future is the same as the one the closure creates.
Like a plain closure, it's used to prevent code from being eagerly evaluated. In synchronous terms, it's the difference between calling a function and calling the closure that the function returned:
fn sync() -> impl FnOnce() {
println!("This is run when the function is called");
|| println!("This is run when the return value is called")
}
fn main() {
let a = sync();
println!("Called the function");
a();
}
And the parallel for futures 0.1:
use futures::{future, Future}; // 0.1.27
fn not_sync() -> impl Future<Item = (), Error = ()> {
println!("This is run when the function is called");
future::lazy(|| {
println!("This is run when the return value is called");
Ok(())
})
}
fn main() {
let a = not_sync();
println!("Called the function");
a.wait().unwrap();
}
With async/await syntax, this function should not be needed anymore:
#![feature(async_await)] // 1.37.0-nightly (2019-06-05)
use futures::executor; // 0.3.0-alpha.16
use std::future::Future;
fn not_sync() -> impl Future<Output = ()> {
println!("This is run when the function is called");
async {
println!("This is run when the return value is called");
}
}
fn main() {
let a = not_sync();
println!("Called the function");
executor::block_on(a);
}
As you've identified, Tokio's examples use lazy to:
ensure that the code in the closure is only run from inside the executor.
ensure that a closure is run as a future
I view these two aspects of lazy as effectively the same.
See also:
Why does calling tokio::spawn result in the panic "SpawnError { is_shutdown: true }"?
What is the purpose of async/await in Rust?
I am learning to use Rust futures and I am finding it extremely confusing. I feel like I am being stupid but when would then, and_then and or_else be used? What return types are expected?
Please provide some examples of the different situations you would expect to see them.
TL;DR: then is used when you want to do something regardless of if the future was successful or not, and_then runs the closure only when the future succeeded, and or_else runs the closure only when the future failed.
and_then and or_else are direct analogs to the methods of the same name on Result .
Your first step should be to read the docs. The documentation contains the exact method signatures (which explain what types it expects and what the return types are), prose describing each method, and example usage as well.
I've extracted small snippets of the docs and emphasized the relevant parts.
Future::then:
This function can be used to ensure a computation runs regardless of
the conclusion of the future. The closure provided will be yielded a
Result once the future is complete.
The returned value of the closure must implement the IntoFuture trait
and can represent some more work to be done before the composed future
is finished.
Future::and_then:
This function can be used to chain two futures together and ensure
that the final future isn't resolved until both have finished. The
closure provided is yielded the successful result of this future and
returns another value which can be converted into a future.
Future::or_else
Return a future that passes along this future's value if it succeeds, and otherwise passes the error to the closure f and waits for the future it returns.
The return type for all three methods is any type that can be converted into another future.
then: no additional restrictions on the return type.
and_then requires that the error type of the returned future match the starting future's error type.
or_else requires that the success type of the returned future match the starting future's success type.
use futures::{future, Future}; // 0.1.25
struct Error;
fn download_from_server(server: u8) -> impl Future<Item = Vec<u8>, Error = Error> {
/* ... */
}
fn upload_to_server(data: Vec<u8>) -> impl Future<Item = usize, Error = Error> {
/* ... */
}
// Uses `or_else` to do work on failure
fn download() -> impl Future<Item = Vec<u8>, Error = Error> {
download_from_server(0)
.or_else(|_| download_from_server(1))
.or_else(|_| download_from_server(2))
}
// Uses `and_then` to do work on success
fn reupload() -> impl Future<Item = usize, Error = Error> {
download().and_then(|data| upload_to_server(data))
}
// Uses `then` to always do work
fn do_things() -> impl Future<Item = (), Error = ()> {
reupload().then(|r| {
match r {
Ok(size) => println!("Uploaded {} bytes", size),
Err(_) => println!("Got an error"),
};
Ok(())
})
}
Some cases are simplified by the async / await syntax stabilized in Rust 1.39:
// Equivalent to `or_else`
async fn download() -> Result<Vec<u8>, Error> {
match download_from_server(0).await {
Ok(v) => Ok(v),
Err(_) => match download_from_server(1).await {
Ok(v) => Ok(v),
Err(_) => download_from_server(2).await,
},
}
}
// Equivalent to `and_then`
async fn reupload() -> Result<usize, Error> {
let data = download().await?;
upload_to_server(data).await
}
// Equivalent to `then`
async fn do_things() -> Result<(), ()> {
match reupload().await {
Ok(size) => println!("Uploaded {} bytes", size),
Err(_) => println!("Got an error"),
}
Ok(())
}
I want to use Delay to do some work later. If I use tokio::run, it just works fine, but it panics when using tokio::spawn:
use std::sync::mpsc;
use std::time::*;
use tokio::prelude::*; // 0.1.14
fn main() {
let (tx, rx) = mpsc::channel();
let task = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(1))
.map(move |_| {
tx.send(String::from("hello")).unwrap();
()
})
.map_err(|e| {
panic!("{:?}", e);
});
tokio::spawn(task);
let msg = rx.recv().unwrap();
println!("{}", msg);
}
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: SpawnError { is_shutdown: true }', src/libcore/result.rs:1009:5
I need to use spawn not run if I want various tasks to work concurrently. How to change the code to make it work?
The documentation for tokio::spawn states:
This function will panic if the default executor is not set or if spawning onto the default executor returns an error.
Effectively, this means that tokio::spawn should only be called from inside a call to tokio::run.
Since you have only a single future to execute, you might as well just pass it directly to tokio::run. If you had multiple futures, then you can make make use of future::lazy to construct a lazily-evaluated future that will call spawn when it eventually runs:
use std::time::*;
use tokio::prelude::*; // 0.1.14
fn main() {
tokio::run(futures::lazy(|| {
tokio::spawn(wait_one_sec().map(|_| println!("One")));
tokio::spawn(wait_one_sec().map(|_| println!("Two")));
Ok(())
}));
}
fn wait_one_sec() -> impl Future<Item = (), Error = ()> {
tokio::timer::Delay::new(Instant::now() + Duration::from_secs(1))
.map(drop)
.map_err(|e| panic!("{:?}", e))
}
Note that if you forget the futures::lazy then you will get the same error. This is because the arguments to functions are evaluated eagerly, which means that the call to tokio::spawn happens first, causing the same sequence of events.
use std::sync::mpsc;
I think it's highly doubtful that you want to use the standard libraries channels, as they are not async-aware and thus will block — a very bad thing in async code.
Instead, you probably want futures::sync::mpsc.