I am trying to send off parallel asynchronous Rusoto SQS requests using FuturesOrdered:
use futures::prelude::*; // 0.1.26
use futures::stream::futures_unordered::FuturesUnordered;
use rusoto_core::{Region, HttpClient}; // 0.38.0
use rusoto_credential::EnvironmentProvider; // 0.17.0
use rusoto_sqs::{SendMessageBatchRequest, SendMessageBatchRequestEntry, Sqs, SqsClient}; // 0.38.0
fn main() {
let client = SqsClient::new_with(
HttpClient::new().unwrap(),
EnvironmentProvider::default(),
Region::UsWest2,
);
let messages: Vec<u32> = (1..12).map(|n| n).collect();
let chunks: Vec<_> = messages.chunks(10).collect();
let tasks: FuturesUnordered<_> = chunks.into_iter().map(|c| {
let batch = create_batch(c);
client.send_message_batch(batch)
}).collect();
let tasks = tasks
.for_each(|t| {
println!("{:?}", t);
Ok(())
})
.map_err(|e| println!("{}", e));
tokio::run(tasks);
}
fn create_batch(ids: &[u32]) -> SendMessageBatchRequest {
let queue_url = "https://sqs.us-west-2.amazonaws.com/xxx/xxx".to_string();
let entries = ids
.iter()
.map(|id| SendMessageBatchRequestEntry {
id: id.to_string(),
message_body: id.to_string(),
..Default::default()
})
.collect();
SendMessageBatchRequest {
entries,
queue_url,
}
}
The tasks complete correctly but tokio::run(tasks) doesn't stop. I assume that is because of tasks.for_each() will force it to continue to run and look for more futures?
Why doesn't tokio::run(tasks) stop? Am I using FuturesOrdered correctly?
I am also a little worried about memory usage when creating up to 60,000 futures to complete and pushing them into the FuturesUnordered combinator.
I discovered that it was the SqsClient in the main function that was causing it to block as it is still doing some house work even though the tasks are finished.
A solution provided by one of the Rusoto people was to add this just above tokio::run
std::mem::drop(client);
Related
I am writing an echo server that is able to listen on multiple ports. My working server below relies on select to accept connections from 2 different listeners
However, instead of defining the listeners as individual variables, is it possible to define select branches based on a Vec<TcpListener> ?
use tokio::{io, net, select, spawn};
#[tokio::main]
async fn main() {
let listener1 = net::TcpListener::bind("127.0.0.1:8001").await.unwrap();
let listener2 = net::TcpListener::bind("127.0.0.1:8002").await.unwrap();
loop {
let (conn, _) = select! {
v = listener1.accept() => v.unwrap(),
v = listener2.accept() => v.unwrap(),
};
spawn(handle(conn));
}
}
async fn handle(mut conn: net::TcpStream) {
let (mut read, mut write) = conn.split();
io::copy(&mut read, &mut write).await.unwrap();
}
While futures::future::select_all() works, it is not very elegant (IMHO) and it creates an allocation for each round. A better solution is to use streams (note this also allocates on each round, but this allocates much less):
use tokio::{io, net, spawn};
use tokio_stream::wrappers::TcpListenerStream;
use futures::stream::{StreamExt, SelectAll};
#[tokio::main]
async fn main() {
let mut listeners = SelectAll::new();
listeners.push(TcpListenerStream::new(net::TcpListener::bind("127.0.0.1:8001").await.unwrap()));
listeners.push(TcpListenerStream::new(net::TcpListener::bind("127.0.0.1:8002").await.unwrap()));
while let Some(conn) = listeners.next().await {
let conn = conn.unwrap();
spawn(handle(conn));
}
}
You can use the select_all function from the futures crate, which takes an iterator of futures and awaits any of them (instead of all of them, like join_all does):
use futures::{future::select_all, FutureExt};
use tokio::{io, net, select, spawn};
#[tokio::main]
async fn main() {
let mut listeners = [
net::TcpListener::bind("127.0.0.1:8001").await.unwrap(),
net::TcpListener::bind("127.0.0.1:8002").await.unwrap(),
];
loop {
let (result, index, _) = select_all(
listeners
.iter_mut()
// note: `FutureExt::boxed` is called here because `select_all`
// requires the futures to be pinned
.map(|listener| listener.accept().boxed()),
)
.await;
let (conn, _) = result.unwrap();
spawn(handle(conn));
}
}
Following code is working, it can be tested in Playground
use std::{thread, time::Duration};
use rand::Rng;
fn main() {
let mut hiv = Vec::new();
let (sender, receiver) = crossbeam_channel::unbounded();
// make workers
for t in 0..5 {
println!("Make worker {}", t);
let receiver = receiver.clone(); // clone for this thread
let handler = thread::spawn(move || {
let mut rng = rand::thread_rng(); // each thread have one
loop {
let r = receiver.recv();
match r {
Ok(x) => {
let s = rng.gen_range(100..1000);
thread::sleep(Duration::from_millis(s));
println!("w={} r={} working={}", t, x, s);
},
_ => { println!("No more work for {} --- {:?}.", t, r); break},
}
}
});
hiv.push(handler);
}
// Generate jobs
for x in 0..10 {
sender.send(x).expect("all threads hung up :(");
}
drop(sender);
// wait for jobs to finish.
println!("Wait for all threads to finish.\n");
for h in hiv {
h.join().unwrap();
}
println!("join() done. Work Finish.");
}
My question is following :
Can I remove boilerplate code by using threadpool, rayon or some other Rust crate ?
I know that I could do my own implementation, but would like to know is there some crate with same functionality ?
From my research threadpool/rayon are useful when you "send" code and it is executed, but I have not found way to make N threads that will have some code/logic that they need to remember ?
Basic idea is in let mut rng = rand::thread_rng(); this is instance that each thread need to have on it own.
Also is there are some other problems with code, please point it out.
Yes, you can use Rayon to eliminate a lot of that code and make the remaining code much more readable, as illustrated in this gist:
https://gist.github.com/BillBarnhill/db07af903cb3c3edb6e715d9cedae028
The worker pool model is not great in Rust, due to the ownership rules. As a result parallel iterators are often a better choice.
I forgot to address your main concern, per thread context, originally. You can see how to store per thread context using a ThreadLocal! in this answer:
https://stackoverflow.com/a/42656422/204343
I will try to come back and edit the code to reflect ThreadLocal! use as soon as I have more time.
The gist requires nightly because of thread_id_value, but that is all but stable and can be removed if needed.
The real catch is that the gist has timing, and compares main_new with main_original, with surprising results. Perhaps not so surprising, Rayon has good debug support.
On Debug build the timing output is:
main_new duration: 1.525667954s
main_original duration: 1.031234059s
You can see main_new takes almost 50% longer to run.
On release however main_new is a little faster:
main_new duration: 1.584190936s
main_original duration: 1.5851124s
A slimmed version of the gist is below, with only the new code.
#![feature(thread_id_value)]
use std::{thread, time::Duration, time::Instant};
use rand::Rng;
#[allow(unused_imports)]
use rayon::prelude::*;
fn do_work(x : u32) -> String {
let mut rng = rand::thread_rng(); // each thread have one
let s = rng.gen_range(100..1000);
let thread_id = thread::current().id();
let t = thread_id.as_u64();
thread::sleep(Duration::from_millis(s));
format!("w={} r={} working={}", t, x, s)
}
fn process_work_product(output : String) {
println!("{}", output);
}
fn main() {
// bit hacky, but lets set number of threads to 5
rayon::ThreadPoolBuilder::new()
.num_threads(4)
.build_global()
.unwrap();
let x = 0..10;
x.into_par_iter()
.map(do_work)
.for_each(process_work_product);
}
I'm trying to run multiple invocation of the same script on a single deno MainWorker concurrently, and waiting for their
results (since the scripts can be async). Conceptually, I want something like the loop in run_worker below.
type Tx = Sender<(String, Sender<String>)>;
type Rx = Receiver<(String, Sender<String>)>;
struct Runner {
worker: MainWorker,
futures: FuturesUnordered<Pin<Box<dyn Future<Output=(String, Result<Global<Value>, Error>)>>>>,
response_futures: FuturesUnordered<Pin<Box<dyn Future<Output=(String, Result<(), SendError<String>>)>>>>,
result_senders: HashMap<String, Sender<String>>,
}
impl Runner {
fn new() ...
async fn run_worker(&mut self, rx: &mut Rx, main_module: ModuleSpecifier, user_module: ModuleSpecifier) {
self.worker.execute_main_module(&main_module).await.unwrap();
self.worker.preload_side_module(&user_module).await.unwrap();
loop {
tokio::select! {
msg = rx.recv() => {
if let Some((id, sender)) = msg {
let global = self.worker.js_runtime.execute_script("test", "mod.entry()").unwrap();
self.result_senders.insert(id, sender);
self.futures.push(Box::pin(async {
let resolved = self.worker.js_runtime.resolve_value(global).await;
return (id, resolved);
}));
}
},
script_result = self.futures.next() => {
if let Some((id, out)) = script_result {
self.response_futures.push(Box::pin(async {
let value = deserialize_value(out.unwrap(), &mut self.worker);
let res = self.result_senders.remove(&id).unwrap().send(value).await;
return (id.clone(), res);
}));
}
},
// also handle response_futures here
else => break,
}
}
}
}
The worker can't be borrowed as mutable multiple times, so this won't work. So the worker has to be a RefCell, and
I've created a BorrowingFuture:
struct BorrowingFuture {
worker: RefCell<MainWorker>,
global: Global<Value>,
id: String
}
And its poll implementation:
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match Pin::new(&mut Box::pin(self.worker.borrow_mut().js_runtime.resolve_value(self.global.clone()))).poll(cx) {
Poll::Ready(result) => Poll::Ready((self.id.clone(), result)),
Poll::Pending => {
cx.waker().clone().wake_by_ref();
Poll::Pending
}
}
}
So the above
self.futures.push(Box::pin(async {
let resolved = self.worker.js_runtime.resolve_value(global).await;
return (id, resolved);
}));
would become
self.futures.push(Box::pin(BorrowingFuture{worker: self.worker, global: global.clone(), id: id.clone()}));
and this would have to be done for the response_futures above as well.
But I see a few issues with this.
Creating a new future on every poll and then polling that seems wrong, but it does work.
It probably has a performance impact because new objects are created constantly.
The same issue would happen for the response futures, which would call send on each poll, which seems completely wrong.
The waker.wake_by_ref is called on every poll, because there is no way to know when a script result
will resolve. This results in the future being polled thousands (and more) times per second (always creating a new object),
which could be the same as checking it in a loop, I guess.
Note My current setup doesn't use select!, but an enum as Output from multiple Future implementations, pushed into
a single FuturesUnordered, and then matched to handle the correct type (script, send, receive). I used select here because
it's far less verbose, and gets the point across.
Is there a way to do this better/more efficiently? Or is it just not the way MainWorker was meant to be used?
main for completeness:
#[tokio::main]
async fn main() {
let main_module = deno_runtime::deno_core::resolve_url(MAIN_MODULE_SPECIFIER).unwrap();
let user_module = deno_runtime::deno_core::resolve_url(USER_MODULE_SPECIFIER).unwrap();
let (tx, mut rx) = channel(1);
let (result_tx, mut result_rx) = channel(1);
let handle = thread::spawn(move || {
let runtime = tokio::runtime::Builder::new_multi_thread().enable_all().build().unwrap();
let mut runner = Runner::new();
runtime.block_on(runner.run_worker(&mut rx, main_module, user_module));
});
tx.send(("test input".to_string(), result_tx)).await.unwrap();
let result = result_rx.recv().await.unwrap();
println!("result from worker {}", result);
handle.join().unwrap();
}
The async example is useful, but being new to Rust and Tokio, I am struggling to work out how to do N requests at once, using URLs from a vector, and creating an iterator of the response HTML for each URL as a string.
How could this be done?
Concurrent requests
As of reqwest 0.10:
use futures::{stream, StreamExt}; // 0.3.5
use reqwest::Client; // 0.10.6
use tokio; // 0.2.21, features = ["macros"]
const CONCURRENT_REQUESTS: usize = 2;
#[tokio::main]
async fn main() {
let client = Client::new();
let urls = vec!["https://api.ipify.org"; 2];
let bodies = stream::iter(urls)
.map(|url| {
let client = &client;
async move {
let resp = client.get(url).send().await?;
resp.bytes().await
}
})
.buffer_unordered(CONCURRENT_REQUESTS);
bodies
.for_each(|b| async {
match b {
Ok(b) => println!("Got {} bytes", b.len()),
Err(e) => eprintln!("Got an error: {}", e),
}
})
.await;
}
stream::iter(urls)
stream::iter
Take a collection of strings and convert it into a Stream.
.map(|url| {
StreamExt::map
Run an asynchronous function on every element in the stream and transform the element to a new type.
let client = &client;
async move {
Take an explicit reference to the Client and move the reference (not the original Client) into an anonymous asynchronous block.
let resp = client.get(url).send().await?;
Start an asynchronous GET request using the Client's connection pool and wait for the request.
resp.bytes().await
Request and wait for the bytes of the response.
.buffer_unordered(N);
StreamExt::buffer_unordered
Convert a stream of futures into a stream of those future's values, executing the futures concurrently.
bodies
.for_each(|b| {
async {
match b {
Ok(b) => println!("Got {} bytes", b.len()),
Err(e) => eprintln!("Got an error: {}", e),
}
}
})
.await;
StreamExt::for_each
Convert the stream back into a single future, printing out the amount of data received along the way, then wait for the future to complete.
See also:
Join futures with limited concurrency
How to merge iterator of streams?
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
What is the difference between `then`, `and_then` and `or_else` in Rust futures?
Without bounded execution
If you wanted to, you could also convert an iterator into an iterator of futures and use future::join_all:
use futures::future; // 0.3.4
use reqwest::Client; // 0.10.1
use tokio; // 0.2.11
#[tokio::main]
async fn main() {
let client = Client::new();
let urls = vec!["https://api.ipify.org"; 2];
let bodies = future::join_all(urls.into_iter().map(|url| {
let client = &client;
async move {
let resp = client.get(url).send().await?;
resp.bytes().await
}
}))
.await;
for b in bodies {
match b {
Ok(b) => println!("Got {} bytes", b.len()),
Err(e) => eprintln!("Got an error: {}", e),
}
}
}
I'd encourage using the first example as you usually want to limit the concurrency, which buffer and buffer_unordered help with.
Parallel requests
Concurrent requests are generally good enough, but there are times where you need parallel requests. In that case, you need to spawn a task.
use futures::{stream, StreamExt}; // 0.3.8
use reqwest::Client; // 0.10.9
use tokio; // 0.2.24, features = ["macros"]
const PARALLEL_REQUESTS: usize = 2;
#[tokio::main]
async fn main() {
let urls = vec!["https://api.ipify.org"; 2];
let client = Client::new();
let bodies = stream::iter(urls)
.map(|url| {
let client = client.clone();
tokio::spawn(async move {
let resp = client.get(url).send().await?;
resp.bytes().await
})
})
.buffer_unordered(PARALLEL_REQUESTS);
bodies
.for_each(|b| async {
match b {
Ok(Ok(b)) => println!("Got {} bytes", b.len()),
Ok(Err(e)) => eprintln!("Got a reqwest::Error: {}", e),
Err(e) => eprintln!("Got a tokio::JoinError: {}", e),
}
})
.await;
}
The primary differences are:
We use tokio::spawn to perform work in separate tasks.
We have to give each task its own reqwest::Client. As recommended, we clone a shared client to make use of the connection pool.
There's an additional error case when the task cannot be joined.
See also:
What is the difference between concurrent programming and parallel programming?
What is the difference between concurrency and parallelism?
What is the difference between concurrency, parallelism and asynchronous methods?
If possible for your problem I recommend using std async and rayon. They are both mature now and really easy to get started with given the async{/* code here */} scope bounds in std. You can also work into/alongside tokio with feature integration https://docs.rs/async-std/1.10.0/async_std/#features
The async example is useful, but being new to Rust and Tokio, I am struggling to work out how to do N requests at once, using URLs from a vector, and creating an iterator of the response HTML for each URL as a string.
How could this be done?
Concurrent requests
As of reqwest 0.10:
use futures::{stream, StreamExt}; // 0.3.5
use reqwest::Client; // 0.10.6
use tokio; // 0.2.21, features = ["macros"]
const CONCURRENT_REQUESTS: usize = 2;
#[tokio::main]
async fn main() {
let client = Client::new();
let urls = vec!["https://api.ipify.org"; 2];
let bodies = stream::iter(urls)
.map(|url| {
let client = &client;
async move {
let resp = client.get(url).send().await?;
resp.bytes().await
}
})
.buffer_unordered(CONCURRENT_REQUESTS);
bodies
.for_each(|b| async {
match b {
Ok(b) => println!("Got {} bytes", b.len()),
Err(e) => eprintln!("Got an error: {}", e),
}
})
.await;
}
stream::iter(urls)
stream::iter
Take a collection of strings and convert it into a Stream.
.map(|url| {
StreamExt::map
Run an asynchronous function on every element in the stream and transform the element to a new type.
let client = &client;
async move {
Take an explicit reference to the Client and move the reference (not the original Client) into an anonymous asynchronous block.
let resp = client.get(url).send().await?;
Start an asynchronous GET request using the Client's connection pool and wait for the request.
resp.bytes().await
Request and wait for the bytes of the response.
.buffer_unordered(N);
StreamExt::buffer_unordered
Convert a stream of futures into a stream of those future's values, executing the futures concurrently.
bodies
.for_each(|b| {
async {
match b {
Ok(b) => println!("Got {} bytes", b.len()),
Err(e) => eprintln!("Got an error: {}", e),
}
}
})
.await;
StreamExt::for_each
Convert the stream back into a single future, printing out the amount of data received along the way, then wait for the future to complete.
See also:
Join futures with limited concurrency
How to merge iterator of streams?
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
What is the difference between `then`, `and_then` and `or_else` in Rust futures?
Without bounded execution
If you wanted to, you could also convert an iterator into an iterator of futures and use future::join_all:
use futures::future; // 0.3.4
use reqwest::Client; // 0.10.1
use tokio; // 0.2.11
#[tokio::main]
async fn main() {
let client = Client::new();
let urls = vec!["https://api.ipify.org"; 2];
let bodies = future::join_all(urls.into_iter().map(|url| {
let client = &client;
async move {
let resp = client.get(url).send().await?;
resp.bytes().await
}
}))
.await;
for b in bodies {
match b {
Ok(b) => println!("Got {} bytes", b.len()),
Err(e) => eprintln!("Got an error: {}", e),
}
}
}
I'd encourage using the first example as you usually want to limit the concurrency, which buffer and buffer_unordered help with.
Parallel requests
Concurrent requests are generally good enough, but there are times where you need parallel requests. In that case, you need to spawn a task.
use futures::{stream, StreamExt}; // 0.3.8
use reqwest::Client; // 0.10.9
use tokio; // 0.2.24, features = ["macros"]
const PARALLEL_REQUESTS: usize = 2;
#[tokio::main]
async fn main() {
let urls = vec!["https://api.ipify.org"; 2];
let client = Client::new();
let bodies = stream::iter(urls)
.map(|url| {
let client = client.clone();
tokio::spawn(async move {
let resp = client.get(url).send().await?;
resp.bytes().await
})
})
.buffer_unordered(PARALLEL_REQUESTS);
bodies
.for_each(|b| async {
match b {
Ok(Ok(b)) => println!("Got {} bytes", b.len()),
Ok(Err(e)) => eprintln!("Got a reqwest::Error: {}", e),
Err(e) => eprintln!("Got a tokio::JoinError: {}", e),
}
})
.await;
}
The primary differences are:
We use tokio::spawn to perform work in separate tasks.
We have to give each task its own reqwest::Client. As recommended, we clone a shared client to make use of the connection pool.
There's an additional error case when the task cannot be joined.
See also:
What is the difference between concurrent programming and parallel programming?
What is the difference between concurrency and parallelism?
What is the difference between concurrency, parallelism and asynchronous methods?
If possible for your problem I recommend using std async and rayon. They are both mature now and really easy to get started with given the async{/* code here */} scope bounds in std. You can also work into/alongside tokio with feature integration https://docs.rs/async-std/1.10.0/async_std/#features