"future cannot be sent between threads safely" when pass Arc<Mutex> into tokio::spawn - rust

I implemented TCP client using tokio. However, my code not compile because I got an error:
error: future cannot be sent between threads safely
--> src/main.rs:81:9
|
81 | tokio::spawn(async move {
| ^^^^^^^^^^^^ future created by async block is not `Send`
|
= help: within `impl Future<Output = ()>`, the trait `Send` is not implemented for `std::sync::MutexGuard<'_, Option<tokio::net::TcpStream>>`
note: future is not `Send` as this value is used across an await
--> src/main.rs:90:42
|
82 | match stream.lock().unwrap().as_mut() {
| ---------------------- has type `std::sync::MutexGuard<'_, Option<tokio::net::TcpStream>>` which is not `Send`
...
90 | stream.write(&packet).await.unwrap();
| ^^^^^^ await occurs here, with `stream.lock().unwrap()` maybe used later
...
94 | };
| - `stream.lock().unwrap()` is later dropped here
help: consider moving this into a `let` binding to create a shorter lived borrow
--> src/main.rs:82:19
|
82 | match stream.lock().unwrap().as_mut() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: required by a bound in `tokio::spawn`
--> /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/task/spawn.rs:127:21
|
127 | T: Future + Send + 'static,
| ^^^^ required by this bound in `tokio::spawn`
This is my code where issue occurs:
async fn handle_write(&mut self) -> JoinHandle<()> {
let stream = Arc::clone(&self.stream);
let session = Arc::clone(&self.session);
let queue = Arc::clone(&self.queue);
tokio::spawn(async move {
match stream.lock().unwrap().as_mut() {
Some(stream) => {
let packet: Vec<u8> = queue.lock().unwrap().pop_front().unwrap();
let packet = match session.lock().unwrap().header_crypt.as_mut() {
Some(header_crypt) => header_crypt.encrypt(&packet),
_ => packet,
};
stream.write(&packet).await.unwrap();
stream.flush().await.unwrap();
},
_ => {},
};
})
}
and same issue here:
async fn handle_read(&mut self) -> JoinHandle<()> {
let queue = Arc::clone(&self.queue);
let stream = Arc::clone(&self.stream);
let session = Arc::clone(&self.session);
tokio::spawn(async move {
match stream.lock().unwrap().as_mut() {
Some(stream) => {
let mut buffer = [0u8; 4096];
match stream.read(&mut buffer).await {
Ok(bytes_count) => {
let raw_data = match session.lock().unwrap().header_crypt.as_mut() {
Some(header_crypt) => header_crypt.decrypt(&buffer[..bytes_count]),
_ => buffer[..bytes_count].to_vec(),
};
queue.lock().unwrap().push_back(raw_data);
},
_ => {},
};
},
_ => {},
};
})
}
Playground.
Could somebody explain, what am I doing wrong ?
P.S. just in case: I am using std::sync::{Arc, Mutex};

Finally, as decided in comments to my question, I used tokio::sync::Mutex instead of std::sync::Mutex. So, now code compiles correctly.
Playground.

In my case the problem was using ThreadRng with thread_rng() which is NOT thread safe. Just as a heads-up for anyone else banging their head against this error message. I refactored to using let mut rng = ::rand::rngs::StdRng::from_seed(OsRng.gen());

Related

Updating UI elements from a async loop

What is the standard method for updating UI elements from a loop calling a web request in Rust using gtk-rs? I am having trouble updating a label in a UI element to display some data from a web request.
Is there a standard way of doing this? The issues I have been having at the moment have all been related to passing data between threads and as such I am curious to know what is the common way of doing this?
I have a code example here where I get an error stating futures are not Send. I am using fragile in order to pass a box into the thread.
At the moment I am trying to solve the issue of making my futures Send however I am not sure that will solve the issue or just leave me with another similar problem.
use gtk::prelude::*;
use gtk::Orientation;
use adw::Application;
use std::thread;
use std::time::Duration;
use fragile;
const APP_ID: &str = "org.currency_trades";
fn main() {
let app = Application::builder().application_id(APP_ID).build();
app.connect_activate(build_ui);
app.run();
}
pub fn build_ui(app: &Application) -> gtk::Box {
let home_box = fragile::Fragile::new(gtk::Box::new(Orientation::Vertical, 15));
thread::spawn(move || {
let box_gbp = gtk::Box::new(Orientation::Horizontal, 250);
let gbp_label = gtk::Label::builder()
.label("GBP")
.margin_top(12)
.margin_start(50)
.build();
let gbp_price_label = gtk::Label::builder()
.label("Uninitialized")
.margin_top(12)
.margin_end(50)
.build();
box_gbp.append(&gbp_label);
box_gbp.append(&gbp_price_label);
home_box.get().append(&box_gbp);
loop {
let runtime = tokio::runtime::Runtime::new().unwrap();
std::thread::sleep(Duration::from_millis(1000));
let _ = runtime.block_on(runtime.spawn(async move {
let gbp_label_request = match reqwest::get("https://www.boredapi.com/api/activity").await {
Ok(label) => label,
Err(_) => panic!("Panic!")
};
let gbp_label = match gbp_label_request.text().await {
Ok(r) => r,
Err(_) => String::from("Unknown")
};
gbp_price_label.set_label(&gbp_label);
}));
}
});
return *home_box.get();
}
The associated error:
error: future cannot be sent between threads safely
--> src/main.rs:89:52
|
89 | let _ = runtime.block_on(runtime.spawn(async move {
| ____________________________________________________^
90 | | let gbp_label_request = match reqwest::get("https://www.boredapi.com/api/activity").await {
91 | | Ok(label) => label,
92 | | Err(_) => panic!("Panic!")
... |
100 | | gbp_price_label.set_label(&gbp_label);
101 | | }));
| |_____________^ future created by async block is not `Send`
|
= help: the trait `Sync` is not implemented for `*mut c_void`
note: captured value is not `Send`
--> src/main.rs:100:17
|
100 | gbp_price_label.set_label(&gbp_label);
| ^^^^^^^^^^^^^^^ has type `gtk4::Label` which is not `Send`
note: required by a bound in `Runtime::spawn`
--> /Users/andy/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.24.1/src/runtime/runtime.rs:192:21
|
192 | F: Future + Send + 'static,
| ^^^^ required by this bound in `Runtime::spawn`
For more information about this error, try `rustc --explain E0271`.
There is an example in the gtk-rs event loop documentation that explains how to do this using MainContext::channel.
let (sender, receiver) = MainContext::channel(PRIORITY_DEFAULT);
// Connect to "clicked" signal of `button`
button.connect_clicked(move |_| {
let sender = sender.clone();
// The long running operation runs now in a separate thread
thread::spawn(move || {
// Deactivate the button until the operation is done
sender.send(false).expect("Could not send through channel");
let ten_seconds = Duration::from_secs(10);
thread::sleep(ten_seconds);
// Activate the button again
sender.send(true).expect("Could not send through channel");
});
});
// The main loop executes the closure as soon as it receives the message
receiver.attach(
None,
clone!(#weak button => #default-return Continue(false),
move |enable_button| {
button.set_sensitive(enable_button);
Continue(true)
}
),
);

Problem regarding borrowing references in Rust

I am trying to write a program in which one thread writes to a queue and another thread
reads from the queue
But I am facing an issue regarding accessing the 'queue' in the thread that reads the queue
Below is the code which is not compiling
use ::std::collections::VecDeque;
use notify::{Config, RecommendedWatcher, RecursiveMode, Watcher};
use std::cell::RefCell;
use std::path::{Path, PathBuf};
use std::thread;
use std::time::Duration;
fn main() {
//let path = std::env::args()
// .nth(1)
// .expect("Argument 1 needs to be a path");
//println!("watching {}", path);
let path = "c:\\testfolder";
if let Err(e) = watch(path) {
println!("error: {:?}", e)
}
}
fn process_queue(queue: &VecDeque<String>) -> () {}
fn watch<P: AsRef<Path>>(path: P) -> notify::Result<()> {
let (tx, rx) = std::sync::mpsc::channel();
// Automatically select the best implementation for your platform.
// You can also access each implementation directly e.g. INotifyWatcher.
let mut watcher = RecommendedWatcher::new(tx, Config::default())?;
// Add a path to be watched. All files and directories at that path and
// below will be monitored for changes.
let mut queue: VecDeque<String> = VecDeque::new();
thread::spawn(|| {
// everything in here runs
process_queue(&queue)
});
watcher.watch(path.as_ref(), RecursiveMode::Recursive)?;
for res in rx {
match res {
Ok(event) => {
println!("changed: {:?}", event.paths);
let os_str: String = String::from(event.paths[0].to_str().unwrap());
//let my_str: String = os_str.unwrap().to_str().unwrap();
//let s =os_str.into_os_string();
queue.push_back(os_str);
}
Err(e) => println!("watch error: {:?}", e),
}
}
Ok(())
}
The output from the Rust compiler
error[E0373]: closure may outlive the current function, but it borrows `queue`, which is owned by the current function
--> src\main.rs:43:19
|
43 | thread::spawn(|| {
| ^^ may outlive borrowed value `queue`
...
47 | process_queue(&queue)
| ----- `queue` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src\main.rs:43:5
|
43 | / thread::spawn(|| {
44 | |
45 | | // everything in here runs
46 | |
47 | | process_queue(&queue)
48 | |
49 | | });
| |______^
help: to force the closure to take ownership of `queue` (and any other referenced variables), use the `move` keyword
|
43 | thread::spawn(move || {
| ++++
error[E0502]: cannot borrow `queue` as mutable because it is also borrowed as immutable
--> src\main.rs:63:17
|
43 | thread::spawn(|| {
| - -- immutable borrow occurs here
| _____|
| |
44 | |
45 | | // everything in here runs
46 | |
47 | | process_queue(&queue)
| | ----- first borrow occurs due to use of `queue` in closure
48 | |
49 | | });
| |______- argument requires that `queue` is borrowed for `'static`
...
63 | queue.push_back(os_str);
| ^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
From the errors I understand that the compiler does not allow both mutable and immutable references at the same time.
But I don't know how to implement what I am trying to do with these restrictions.
One way to solve this is by Box-ing the VecDeque so that you can pass a cloned reference to your process_queue function.
Using a Box allows you to allocate the VecDeque on the heap so that you can give your spawned thread a reference to the Vec and also still mutate the queue in the current thread.
This would look like:
let mut queue = Box::new(VecDeque::new());
let queue_clone = queue.clone();
thread::spawn(|| {
// queue_clone is now moved into the fn closure and is
// not accessible to the code below
process_queue(queue_clone)
});
and you can update process_queue to accept the correct type:
fn process_queue(queue: Box<VecDeque<String>>) -> () { }
Note that with this implementation, process_queue only runs once when the thread is spawned, and if you want to have process_queue do something every time the queue is changed, following the advice of others to use something like Channels makes the most sense.
Thanks for all your responses
From all the responses I understand that using channels and moving the receiver loop to the other thread as suggested bu user4815162342
will be the best solution
I successfully implemented what I was trying to do using channels based on your suggestions.
The final working code is pasted below
use std::thread;
use std::time::Duration;
use notify::{RecommendedWatcher, RecursiveMode, Watcher, Config};
use std::path::Path;
use std::path::PathBuf;
//
fn main() {
//let path = std::env::args()
// .nth(1)
// .expect("Argument 1 needs to be a path");
//println!("watching {}", path);
let path="c:\\testfolder";
if let Err(e) = watch(path) {
println!("error: {:?}", e)
}
}
fn watch<P: AsRef<Path>>(path: P) -> notify::Result<()> {
let (tx, rx) = std::sync::mpsc::channel();
// Automatically select the best implementation for your platform.
// You can also access each implementation directly e.g. INotifyWatcher.
let mut watcher = RecommendedWatcher::new(tx, Config::default())?;
// Add a path to be watched. All files and directories at that path and
// below will be monitored for changes.
let handle=thread::spawn(move || {
// everything in here runs
for res in rx {
match res {
Ok(event) =>{
// println!("changed: {:?}", event.paths);
let os_str:String = String::from(event.paths[0].to_str().unwrap());
println!("file name: {}", os_str);
},
Err(e) => println!("watch error: {:?}", e),
}
}
});
watcher.watch(path.as_ref(), RecursiveMode::Recursive)?;
handle.join();
Ok(())
}
In your situation, using Rust's MPSC (multi-producer single-consumer, ie essentially a queue) would probably be the best. You could also create a variable that is shared between multiple thread using Arc and Mutex structs, but that would be way overkilled and can have a performance impact (only one can access the variable at any time).
Here is an example of a multi-threaded MPSC, I will let you adapt it to your infrastructure :
use std::{sync::mpsc, thread};
fn main() {
let (sender, receiver) = mpsc::channel();
let handle_1 = thread::spawn(|| {
thread_1(sender);
});
let handle_2 = thread::spawn(|| {
thread_2(receiver);
});
handle_1.join().unwrap();
handle_2.join().unwrap();
}
// the enum must have the Send trait (automatically implemented)
enum Instruction {
Print(String),
Exit
}
fn thread_1(sender: mpsc::Sender<Instruction>) {
sender.send(Instruction::Print("I".to_owned())).unwrap();
sender.send(Instruction::Print("like".to_owned())).unwrap();
sender.send(Instruction::Print("Rust".to_owned())).unwrap();
sender.send(Instruction::Print(".".to_owned())).unwrap();
sender.send(Instruction::Exit).unwrap();
}
fn thread_2(receiver: mpsc::Receiver<Instruction>) {
'global_loop: loop {
for received in receiver.recv() {
match received {
Instruction::Print(string) => print!("{} ", string),
Instruction::Exit => {
println!("");
break 'global_loop;
}
}
}
}
}

How to share tokio::net::TcpStream act on it concurrently?

I have a requirement to send and receive normal data on the same TcpStream, while sending heartbeat data at regular intervals. In the current implementation, I used Arc<Mutex<TcpStream>>, but it compiled with errors:
use anyhow::Result;
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<()> {
let stream = TcpStream::connect("127.0.0.1:8888").await.unwrap();
let stream = Arc::new(Mutex::new(stream));
let common_stream = stream.clone();
let handler1 = tokio::spawn(async {
loop {
let mut stream = common_stream.lock().unwrap();
let mut buf = [0u8; 10];
stream.read_exact(&mut buf).await.unwrap();
buf.reverse();
stream.write(&buf).await.unwrap();
}
});
let heartbeat_stream = stream.clone();
let handler2 = tokio::spawn(async {
loop {
let mut stream = heartbeat_stream.lock().unwrap();
stream.write_u8(1).await.unwrap();
thread::sleep(Duration::from_millis(200));
}
});
handler1.await?;
handler2.await?;
Ok(())
}
error: future cannot be sent between threads safely
--> src\main.rs:14:20
|
14 | let handler1 = tokio::spawn(async {
| ^^^^^^^^^^^^ future created by async block is not `Send`
|
= help: within `impl Future<Output = [async output]>`, the trait `Send` is not implemented for `std::sync::MutexGuard<'_, tokio::net::TcpStream>`
note: future is not `Send` as this value is used across an await
--> src\main.rs:20:31
|
16 | let mut stream = common_stream.lock().unwrap();
| ---------- has type `std::sync::MutexGuard<'_, tokio::net::TcpStream>` which is not `Send`
...
20 | stream.write(&buf).await.unwrap();
| ^^^^^^ await occurs here, with `mut stream` maybe used later
21 | }
| - `mut stream` is later dropped here
note: required by a bound in `tokio::spawn`
--> .cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\tokio-1.17.0\src\task\spawn.rs:127:21
|
127 | T: Future + Send + 'static,
| ^^^^ required by this bound in `tokio::spawn`
error: future cannot be sent between threads safely
--> src\main.rs:25:20
|
25 | let handler2 = tokio::spawn(async {
| ^^^^^^^^^^^^ future created by async block is not `Send`
|
= help: within `impl Future<Output = [async output]>`, the trait `Send` is not implemented for `std::sync::MutexGuard<'_, tokio::net::TcpStream>`
note: future is not `Send` as this value is used across an await
--> src\main.rs:28:31
|
27 | let mut stream = heartbeat_stream.lock().unwrap();
| ---------- has type `std::sync::MutexGuard<'_, tokio::net::TcpStream>` which is not `Send`
28 | stream.write_u8(1).await.unwrap();
| ^^^^^^ await occurs here, with `mut stream` maybe used later
...
31 | }
| - `mut stream` is later dropped here
note: required by a bound in `tokio::spawn`
--> .cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\tokio-1.17.0\src\task\spawn.rs:127:21
|
127 | T: Future + Send + 'static,
| ^^^^ required by this bound in `tokio::spawn`
How can these errors be fixed or is there another way to achieve the same goal?
Here is a solution that splits the stream to two parts for reading and writing plus does in a loop:
waiting for heartbeat events and sends a byte to write half of stream when this happens
waits data from read half (10 bytes), reverses it and writes again to write half
Also this does not spawn threads and does everything nicely in current one without locks.
use anyhow::Result;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<()> {
let mut stream = TcpStream::connect("127.0.0.1:8888").await?;
let (mut read, mut write) = stream.split();
let mut heartbeat_interval = tokio::time::interval(Duration::from_millis(200));
let mut buf = [0u8; 10];
loop {
tokio::select! {
_ = heartbeat_interval.tick() => {
write.write(&[1]).await?;
}
result = read.read_exact(&mut buf) => {
let _bytes_read = result?;
buf.reverse();
write.write(&buf).await?;
}
}
}
}
Several errors in your code, although the idea behind it is almost good. You should use any available tool in async as possible. Some of the needed/desired changes:
Use tokio::time::sleep because it is async, otherwise the call is blocking
Use an async version of mutex (the one from futures crate for example)
Use some kind of generic error handling (anyhow would help)
use futures::lock::Mutex;
use anyhow::Error;
use tokio::time::sleep;
use std::sync::Arc;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<(), Error> {
let stream = TcpStream::connect("127.0.0.1:8888").await.unwrap();
let stream = Arc::new(Mutex::new(stream));
let common_stream = stream.clone();
let handler1 = tokio::spawn(async move {
loop {
let mut stream = common_stream.lock().await;
let mut buf = [0u8; 10];
stream.read_exact(&mut buf).await.unwrap();
buf.reverse();
stream.write(&buf).await.unwrap();
}
});
let heartbeat_stream = stream.clone();
let handler2 = tokio::spawn(async move {
loop {
let mut stream = heartbeat_stream.lock().await;
stream.write_u8(1).await.unwrap();
sleep(Duration::from_millis(200)).await;
}
});
handler1.await?;
handler2.await?;
Ok(())
}
Playground

Rust declare first assign later pattern

I have a bi-directional grpc stream that acts as bridge to a kafka cluster. When the stream is first initialised, I was to create the kafka consumer and start using it.
To do so, I thought of initialising an empty consumer, waiting for the first input, then assigning a created consumer to an empty one. I tried to do so by following the pattern here.
https://doc.rust-lang.org/rust-by-example/variable_bindings/declare.html
Rust is throwing a possibly-unitialized variable error, is this because it is being initialised in an asynchronous stream?
use std::pin::Pin;
use futures::{Stream, StreamExt};
use kafka::consumer::{Consumer, FetchOffset, GroupOffsetStorage};
use tonic::transport::Server;
use tonic::{Request, Response, Status};
use bridge::kafka_stream_server::{KafkaStream, KafkaStreamServer};
use bridge::{KafkaResponse, PublishRequest};
pub mod bridge {
tonic::include_proto!("bridge"); // The string specified here must match the proto package name
}
#[derive(Default)]
pub struct KafkaStreamService {}
pub fn create_kafka_consumer(topic: String) -> Consumer {
Consumer::from_hosts(vec!["localhost:9092".to_owned()])
.with_topic(topic.to_owned())
.with_fallback_offset(FetchOffset::Latest)
.with_group("".to_owned())
.with_offset_storage(GroupOffsetStorage::Kafka)
.create()
.unwrap()
}
#[tonic::async_trait]
impl KafkaStream for KafkaStreamService {
type SubscribeStream =
Pin<Box<dyn Stream<Item = Result<KafkaResponse, Status>> + Send + Sync + 'static>>;
async fn subscribe(
&self,
request: Request<tonic::Streaming<PublishRequest>>,
) -> Result<Response<Self::SubscribeStream>, Status> {
println!("Initiated stream!");
let mut stream = request.into_inner();
let mut consumer_created_flag: bool = false;
let consumer: Consumer; //declared here
let output = async_stream::try_stream! {
while let Some(publication) = stream.next().await {
let message = publication?;
let topic = message.topic.clone();
if consumer_created_flag == false {
consumer = create_kafka_consumer(topic); //error occurs here
consumer_created_flag = true;
}
let reply = bridge::KafkaResponse {
content: format!("Hello {}!", "world"),
};
yield reply.clone();
}
};
Ok(Response::new(Box::pin(output) as Self::SubscribeStream))
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "[::1]:50051".parse().unwrap();
println!("KafkaService listening on: {}", addr);
let svc = KafkaStreamServer::new(KafkaStreamService::default());
Server::builder().add_service(svc).serve(addr).await?;
Ok(())
}
EDIT: verbose error as requested:
error[E0381]: use of possibly-uninitialized variable: `consumer`
--> src/server.rs:42:22
|
42 | let output = async_stream::try_stream! {
| ______________________^
43 | | while let Some(publication) = stream.next().await {
44 | | let message = publication?;
45 | | let topic = message.topic.clone();
46 | | if consumer_created_flag == false {
47 | | consumer = create_kafka_consumer(topic);
| | -------- use occurs due to use in generator
... |
54 | | }
55 | | };
| |_________^ use of possibly-uninitialized `consumer`
|
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
The declare first pattern only works with basic control flow (if, match, {}, etc). It falls apart when referenced or moved into another object, like an async block or a closure:
fn main() {
let val: i32;
let func = move || {
val = 5;
};
}
error[E0594]: cannot assign to `val`, as it is not declared as mutable
--> src/main.rs:4:9
|
2 | let val: i32;
| --- help: consider changing this to be mutable: `mut val`
3 | let func = move || {
4 | val = 5;
| ^^^^^^^ cannot assign
error[E0381]: use of possibly-uninitialized variable: `val`
--> src/main.rs:3:16
|
3 | let func = move || {
| ^^^^^^^ use of possibly-uninitialized `val`
4 | val = 5;
| --- use occurs due to use in closure
A potential fix is to move its declaration into the try_stream! macro:
let output = async_stream::try_stream! {
let mut consumer_created_flag: bool = false;
let consumer: Consumer;
while let Some(publication) = stream.next().await {
let message = publication?;
let topic = message.topic.clone();
if consumer_created_flag == false {
consumer = create_kafka_consumer(topic);
consumer_created_flag = true;
}
let reply = KafkaResponse {
content: format!("Hello {}!", "world"),
};
yield reply.clone();
}
};
However, this causes a new error because you're potentially assigning to it twice (the compiler doesn't know that consumer_created_flag is guarding it):
error[E0384]: cannot assign twice to immutable variable `consumer`
--> src\lib.rs:1348:21
|
44 | let consumer: Consumer; //declared here
| -------- help: make this binding mutable: `mut consumer`
...
49 | consumer = create_kafka_consumer(topic); //error occurs here
| ^^^^^^^^ cannot assign twice to immutable variable
Fortunately a quick fix is to simply make consumer mutable. And then the only thing the compiler complains about is that it is unused, but I figure there's a reason you've put it there.

Error on Future generator closure: Captured variable cannot escape `FnMut` closure body

I want to create a simple websocket server. I want to process the incoming messages and send a response, but I get an error:
error: captured variable cannot escape `FnMut` closure body
--> src\main.rs:32:27
|
32 | incoming.for_each(|m| async {
| _________________________-_^
| | |
| | inferred to be a `FnMut` closure
33 | | match m {
34 | | // Error here...
35 | | Ok(message) => do_something(message, db, &mut outgoing).await,
36 | | Err(e) => panic!(e)
37 | | }
38 | | }).await;
| |_____^ returns a reference to a captured variable which escapes the closure body
|
= note: `FnMut` closures only have access to their captured variables while they are executing...
= note: ...therefore, they cannot allow references to captured variables to escape
This gives a few hits on Stack Overflow but I don't see anywhere in my code where a variable is escaping. The async block won't run concurrently, so I don't see any problem. Furthermore, I feel like I am doing something very simple: I get a type which allows me to send data back to the client, but when using a reference to it in the async block, it gives a compile error. The error only occurs when I use the outgoing or db variable in the async code.
This is my code (error is in the handle_connection function):
main.rs
use tokio::net::{TcpListener, TcpStream};
use std::net::SocketAddr;
use std::sync::Arc;
use futures::{StreamExt, SinkExt};
use tungstenite::Message;
use tokio_tungstenite::WebSocketStream;
struct DatabaseConnection;
#[tokio::main]
async fn main() -> Result<(), ()> {
listen("127.0.0.1:3012", Arc::new(DatabaseConnection)).await
}
async fn listen(address: &str, db: Arc<DatabaseConnection>) -> Result<(), ()> {
let try_socket = TcpListener::bind(address).await;
let mut listener = try_socket.expect("Failed to bind on address");
while let Ok((stream, addr)) = listener.accept().await {
tokio::spawn(handle_connection(stream, addr, db.clone()));
}
Ok(())
}
async fn handle_connection(raw_stream: TcpStream, addr: SocketAddr, db: Arc<DatabaseConnection>) {
let db = &*db;
let ws_stream = tokio_tungstenite::accept_async(raw_stream).await.unwrap();
let (mut outgoing, incoming) = ws_stream.split();
// Adding 'move' does also not work
incoming.for_each(|m| async {
match m {
// Error here...
Ok(message) => do_something(message, db, &mut outgoing).await,
Err(e) => panic!(e)
}
}).await;
}
async fn do_something(message: Message, db: &DatabaseConnection, outgoing: &mut futures_util::stream::SplitSink<WebSocketStream<TcpStream>, Message>) {
// Do something...
// Send some message
let _ = outgoing.send(Message::Text("yay".to_string())).await;
}
Cargo.toml
[dependencies]
futures = "0.3.*"
futures-channel = "0.3.*"
futures-util = "0.3.*"
tokio = { version = "0.2.*", features = [ "full" ] }
tokio-tungstenite = "0.10.*"
tungstenite = "0.10.*"
When using async move, I get the following error:
code
incoming.for_each(|m| async move {
let x = &mut outgoing;
let b = db;
}).await;
error
error[E0507]: cannot move out of `outgoing`, a captured variable in an `FnMut` closure
--> src\main.rs:33:38
|
31 | let (mut outgoing, incoming) = ws_stream.split();
| ------------ captured outer variable
32 |
33 | incoming.for_each(|m| async move {
| ______________________________________^
34 | | let x = &mut outgoing;
| | --------
| | |
| | move occurs because `outgoing` has type `futures_util::stream::stream::split::SplitSink<tokio_tungstenite::WebSocketStream<tokio::net::tcp::stream::TcpStream>, tungstenite::protocol::message::Message>`, which does not implement the `Copy` trait
| | move occurs due to use in generator
35 | | let b = db;
36 | | }).await;
| |_____^ move out of `outgoing` occurs here
FnMut is an anonymous struct, since FnMutcaptured the &mut outgoing, it becomes a field inside of this anonymous struct and this field will be used on each call of FnMut , it can be called multiple times. If you lose it somehow (by returning or moving into another scope etc...) your program will not able to use that field for further calls, due to safety Rust Compiler doesn't let you do this(for your both case).
In your case instead of capturing the &mut outgoing we can use it as argument for each call, with this we'll keep the ownership of outgoing. You can do this by using fold from futures-rs:
incoming
.fold(outgoing, |mut outgoing, m| async move {
match m {
// Error here...
Ok(message) => do_something(message, db, &mut outgoing).await,
Err(e) => panic!(e),
}
outgoing
})
.await;
This may seem a bit tricky but it does the job, we are using constant accumulator(outgoing) which will be used as an argument for our FnMut.
Playground (Thanks #Solomon Ucko for creating reproducible example)
See also :
How to return the captured variable from `FnMut` closure, which is a captor at the same time
How can I move a captured variable into a closure within a closure?

Resources