I'm trying to use a closure in a thread but after 2 hours of trying I can't seem to find how. Here's the file discord_async.rs:
use discord::*;
use discord::model::Event;
use std::sync::Arc;
use shared_mutex::SharedMutex;
use std::thread;
pub struct DiscordAsync {
client: Arc<SharedMutex<Discord>>
}
impl DiscordAsync {
pub fn new(bot_token: &str) -> DiscordAsync {
let client = Arc::new(SharedMutex::new(Discord::from_bot_token(bot_token).unwrap()));
DiscordAsync {
client: client
}
}
pub fn start<F>(&self, mut event_handle: F) -> () where F: FnMut(Arc<Event>, Arc<SharedMutex<Discord>>) + Send + 'static {
let (mut connection, _) = self.client.read().unwrap().connect().unwrap();
let event_handle = Arc::new(SharedMutex::new(event_handle));
loop {
let event = Arc::new(connection.recv_event().unwrap());
let event_handle = event_handle.read().unwrap();
// Start a thread so we don't block shit
thread::spawn(move || {
// Match event to actions we want to handle
event_handle(event.clone(), self.client);
});
}
}
}
I use it like this in main.rs:
extern crate colored;
extern crate discord;
extern crate shared_mutex;
mod discord_async;
use std::thread;
use colored::*;
use discord::*;
use discord::model::{Event, Channel, ServerId};
use discord_async::DiscordAsync;
fn main() {
// Server info
let bot_token = "...";
let server_id = ServerId(12345);
let dis_async = DiscordAsync::new(bot_token);
dis_async.start(|event, _| {
println!("{:?}", event);
});
}
Compiler message:
Compiling bottest1 v0.1.0 (file:///home/kindlyfire/Documents/dev/rust/bottest1)
error[E0477]: the type `[closure#src/discord_async.rs:29:18: 33:5 event_handle:shared_mutex::SharedMutexReadGuard<'_, F>, event:std::sync::Arc<discord::model::Event>, self:&discord_async::DiscordAsync]` does not fulfill the required lifetime
--> src/discord_async.rs:29:4
|
29 | thread::spawn(move || {
| ^^^^^^^^^^^^^
|
= note: type must outlive the static lifetime
And my Cargo.toml:
[package]
name = "bottest1"
version = "0.1.0"
authors = ["kindlyfire"]
[dependencies]
discord = "0.7.0"
colored = "1.4"
shared-mutex = "0.2"
I've looked at a lot of different ways to do this, including on SO, but I can't find any that work.
You lock the mutex and then try to move the locked object into the thread. That's the wrong way around. You need to clone the Arc and move that into the thread.
Edit: I haven't tested this, but something like this should work:
pub fn start<F>(&self, mut event_handle: F) -> ()
where F: FnMut(Arc<Event>, Arc<SharedMutex<Discord>>) + Send + 'static
{
let (mut connection, _) = self.client.read().unwrap().connect().unwrap();
let event_handle = Arc::new(SharedMutex::new(event_handle));
loop {
let event = Arc::new(connection.recv_event().unwrap());
let event_handle = event_handle.clone();
let client = self.client.clone();
// Start a thread so we don't block shit
thread::spawn(move || {
// Match event to actions we want to handle
event_handle.read().unwrap()(event, client);
});
}
}
Note that we create clones of the Arcs outside the lambda, and then use them inside. Except for event, which we don't clone, because we're fine with moving the one pointer we have. This moves the clones into the lambda. You could probably get rid of the Arc around Event, though. You don't have any other pointers to it, don't need to keep it alive, and it has to be Send anyway (I think &T is only Sync if T is Send).
If this doesn't work, please update with the new compiler error.
As a side note on terminology, a "handle" is an object that refers to some resource. A function that deals with events is a "handler".
Related
I have a config Struct that I'm sharing across my actix app like so:
pub fn run(addr: &str, pg_pool: PgPool, config: Settings) -> Result<Server, std::io::Error> {
let pool = web::Data::new(pg_pool);
let arc_config = web::Data::new(Arc::new(config)); // <---
let server = HttpServer::new(move || {
App::new()
.service(exhaust)
.app_data(pool.clone())
.app_data(arc_config.clone()) // <---
})
.bind(addr)?
.run();
I then have a handler that is trying to spawn multiple threads and pass that config struct into each:
#[get("/exhaust")]
pub async fn exhaust(pool: web::Data<PgPool>, config: web::Data<Arc<Settings>>) -> impl Responder {
for _ in 1..16 {
let handle = thread::spawn(move || {
let inner_config = Arc::clone(&config);
get_single_tweet(inner_config.as_ref().deref(), "1401287393228038149");
});
}
HttpResponse::Ok()
}
My thinking was that because config is already wrapped in an Arc() I should be able to just Arc::clone() it inside of each thread and then deref into the underlying variable.
But I'm getting this error:
error[E0382]: use of moved value: `config`
--> src/twitter/routes/pull.rs:63:36
|
58 | pub async fn exhaust(pool: web::Data<PgPool>, config: web::Data<Arc<Settings>>) -> impl Responder {
| ------ move occurs because `config` has type `actix_web::web::Data<Arc<Settings>>`, which does not implement the `Copy` trait
...
63 | let handle = thread::spawn(move || {
| ^^^^^^^ value moved into closure here, in previous iteration of loop
64 | let inner_config = Arc::clone(&config);
| ------ use occurs due to use in closure
I'm struggling to understand why this fails. If the config is inside an Arc, then why does the compiler think I'm trying to move it instead of incrementing the reference count?
I've also tried a number of other approaches, all unsuccessful:
Removing the move in front of the closure - compiler complains the borrowed value doesn't live long enough
Dereferencing config and wrapping it in a fresh Arc() - similar error as original
What's the right way to do this?
You need to clone it before it's moved. Otherwise your first iteration will necessarily need to take it (since there is no guarantee config will still exist to be cloned when the task runs). Then you get the error you see for the second iteration, since it too necessarily needs to move config; that's what it means by "value moved into closure here, in previous iteration of loop".
#[get("/exhaust")]
pub async fn exhaust(pool: web::Data<PgPool>, config: web::Data<Arc<Settings>>) -> impl Responder {
for _ in 1..16 {
let handle = thread::spawn({
// Clone this here, so the closure gets its very own to move-capture.
let inner_config = config.get_ref().clone();
move || {
get_single_tweet(inner_config.as_ref().deref(), "1401287393228038149");
});
}
}
HttpResponse::Ok()
}
Note that web::Data itself is already just a wrapper around Arc, so you have some redundancy with the nested Arc. You may just want:
#[get("/exhaust")]
pub async fn exhaust(pool: web::Data<PgPool>, config: web::Data<Settings>) -> impl Responder {
for _ in 1..16 {
let handle = thread::spawn({
let inner_config = config.clone();
move || {
get_single_tweet(inner_config.as_ref().deref(), "1401287393228038149");
});
}
}
HttpResponse::Ok()
}
I want to wait for a backend thread (Like this but in my case the backend manages a database which I want to close properly before the application actually exits) to finish (e.g. join it) after application.run() has finished.
My actual non working main.rs (the closure needs to be non-mut)
the thread to wait for
use gio::prelude::*;
use gtk::prelude::*;
use gtk::{ApplicationWindow, Label};
use std::env::args;
use std::thread;
fn main() {
let application = gtk::Application::new(
Some("com.github.gtk-rs.examples.communication_thread"),
Default::default(),
)
.expect("Initialization failed...");
let (thr, mut receiver) = start_communication_thread();
application.connect_activate(move |application| {
build_ui(application, receiver.take().unwrap())
});
application.run(&args().collect::<Vec<_>>());
thr.join();
}
fn build_ui(application: >k::Application, receiver: glib::Receiver<String>) {
let window = ApplicationWindow::new(application);
let label = Label::new(None);
window.add(&label);
spawn_local_handler(label, receiver);
window.show_all();
}
/// Spawn channel receive task on the main event loop.
fn spawn_local_handler(label: gtk::Label, receiver: glib::Receiver<String>) {
receiver.attach(None, move |item| {
label.set_text(&item);
glib::Continue(true)
});
}
/// Spawn separate thread to handle communication.
fn start_communication_thread() -> (thread::JoinHandle<()>, Option<glib::Receiver<String>>) {
let (sender, receiver) = glib::MainContext::channel(glib::PRIORITY_DEFAULT);
let thr = thread::spawn(move || {
let mut counter = 0;
loop {
let data = format!("Counter = {}!", counter);
println!("Thread received data: {}", data);
if sender.send(data).is_err() {
break
}
counter += 1;
thread::sleep(std::time::Duration::from_millis(100));
}
});
(thr, Some(receiver))
}
As mentioned above, the only error remaining is that application.connect_activate() takes an Fn closure, the current implementation is FnMut.
The error message is:
error[E0596]: cannot borrow `receiver` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:17:31
|
17 | build_ui(application, receiver.take().unwrap())
| ^^^^^^^^ cannot borrow as mutable
So you cannot use "receiver" mutably, which is necessary for you to take() its contents.
But if you wrap the receiver inside a Cell, then you can access the immutable Cell's contents mutably. So add this line directly after the line with start_communication_thread():
let receiver = Cell::new(receiver);
There might be some more correct answer as I am only a beginner at Rust, but at least it seems to work.
Please note that this changes the take() call to be called against the Cell instead of Option, whose implementation has the same effect, replacing the Cell's contents with None.
I'm experimenting with the futures API using the websocket library. I have this code:
use futures::future::Future;
use futures::future;
use futures::sink::Sink;
use futures::stream::Stream;
use futures::sync::mpsc::channel;
use futures::sync::mpsc::{Sender, Receiver};
use tokio_core::reactor::Core;
use websocket::{ClientBuilder, OwnedMessage};
pub fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let handle_clone = handle.clone();
let (send, recv): (Sender<String>, Receiver<String>) = channel(100);
let f = ClientBuilder::new("wss://...")
.unwrap()
.async_connect(None, &handle_clone)
.map_err(|e| println!("error: {:?}", e))
.map(|(duplex, _)| duplex.split())
.and_then(move |(sink, stream)| {
// this task consumes the channel, writes messages to the websocket
handle_clone.spawn(future::loop_fn(recv, |recv: Receiver<String>| {
sink.send(OwnedMessage::Close(None))
.and_then(|_| future::ok(future::Loop::Break(())))
.map_err(|_| ())
}));
// the main tasks listens the socket
future::loop_fn(stream, |stream| {
stream
.into_future()
.and_then(|_| future::ok(future::Loop::Break(())))
.map_err(|_| ())
})
});
loop {
core.turn(None)
}
}
After connecting to the server, I want to run "listener" and "sender" tasks without one blocking the other one. The problem is I can't use sink in the new task, it fails with:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/slack_conn.rs:29:17
|
25 | .and_then(move |(sink, stream)| {
| ---- captured outer variable
...
29 | sink.send(OwnedMessage::Close(None))
| ^^^^ cannot move out of captured outer variable in an `FnMut` closure
I could directly use duplex to send and receive, but that leads to worse errors.
Any ideas on how to make this work? Indeed, I'd be happy with any futures code that allows me to non-blockingly connect to a server and spawn two async tasks:
one that reads from the connection and takes some action (prints to screen etc.)
one that reads from a mpsc channel and writes to the connection
It's fine if I have to write it in a different style.
SplitSink implements Sink which defines send to take ownership:
fn send(self, item: Self::SinkItem) -> Send<Self>
where
Self: Sized,
On the other hand, loop_fn requires that the closure be able to be called multiple times. These two things are fundamentally incompatible — how can you call something multiple times which requires consuming a value?
Here's a completely untested piece of code that compiles — I don't have rogue WebSocket servers lying about.
#[macro_use]
extern crate quick_error;
extern crate futures;
extern crate tokio_core;
extern crate websocket;
use futures::{Future, Stream, Sink};
use futures::sync::mpsc::channel;
use tokio_core::reactor::Core;
use websocket::ClientBuilder;
pub fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let (send, recv) = channel(100);
let f = ClientBuilder::new("wss://...")
.unwrap()
.async_connect(None, &handle)
.from_err::<Error>()
.map(|(duplex, _)| duplex.split())
.and_then(|(sink, stream)| {
let reader = stream
.for_each(|i| {
println!("Read a {:?}", i);
Ok(())
})
.from_err();
let writer = sink
.sink_from_err()
.send_all(recv.map_err(Error::Receiver))
.map(|_| ());
reader.join(writer)
});
drop(send); // Close the sending channel manually
core.run(f).expect("Unable to run");
}
quick_error! {
#[derive(Debug)]
pub enum Error {
WebSocket(err: websocket::WebSocketError) {
from()
description("websocket error")
display("WebSocket error: {}", err)
cause(err)
}
Receiver(err: ()) {
description("receiver error")
display("Receiver error")
}
}
}
The points that stuck out during implementation were:
everything has to become a Future eventually
it's way easier to define an error type and convert to it
Knowing if the Item and Error associated types were "right" was tricky. I ended up doing a lot of "type assertions" ({ let x: &Future<Item = (), Error = ()> = &reader; }).
I'm using an example provided by the Tokio library and attempting to have a vector of all the currently active TCP connections. Ultimately, I would like to be able to broadcast a message to each of the active connections, by looping through them and writing a message to the socket.
To start with, I am trying to print out the current number of connections in one thread whilst accepting connections in another.
To do this, I'm trying to use a shared vector. I've not yet implemented the removal of connections from the vector as and when they disconnect.
// A tiny async echo server with tokio-core
extern crate futures;
extern crate tokio_core;
extern crate tokio_io;
use futures::{Future, Stream};
use tokio_io::{io, AsyncRead};
use tokio_core::net::TcpListener;
use tokio_core::reactor::Core;
use std::thread;
use std::sync::{Arc, Mutex};
use std::io::stdout;
use std::io::Write;
fn main() {
// Create the event loop that will drive this server
let mut core = Core::new().unwrap();
let handle = core.handle();
// Bind the server's socket
let addr = "127.0.0.1:12345".parse().unwrap();
let tcp = TcpListener::bind(&addr, &handle).unwrap();
let mut connections = Arc::new((Mutex::new(Vec::new())));
thread::spawn(move || {
//Every 10 seconds print out the current number of connections
let mut i;
loop {
i = connections.lock().unwrap().len();
println!("There are {} connections", i);
stdout().flush();
thread::sleep_ms(10000);
}
});
// Iterate incoming connections
let server = tcp.incoming().for_each(|(tcp, _)| {
connections.lock().unwrap().push(tcp);
// Split up the read and write halves
let (reader, writer) = tcp.split();
// Future of the copy
let bytes_copied = io::copy(reader, writer);
// ... after which we'll print what happened
let handle_conn = bytes_copied.map(|(n, _, _)| {
println!("wrote {} bytes", n)
}).map_err(|err| {
println!("IO error {:?}", err)
});
// Spawn the future as a concurrent task
handle.spawn(handle_conn);
Ok(())
});
// Spin up the server on the event loop
core.run(server).unwrap();
}
At the moment this is failing to build with the following errors:
error[E0382]: capture of moved value: `connections`
--> src/main.rs:36:42
|
26 | thread::spawn(move || {
| ------- value moved (into closure) here
...
36 | let server = tcp.incoming().for_each(|(tcp, _)| {
| ^^^^^^^^^^ value captured here after move
|
= note: move occurs because `connections` has type `std::sync::Arc<std::sync::Mutex<std::vec::Vec<tokio_core::net::TcpStream>>>`, which does not implement the `Copy` trait
error[E0382]: use of moved value: `tcp`
--> src/main.rs:40:32
|
38 | connections.lock().unwrap().push(tcp);
| --- value moved here
39 | // Split up the read and write halves
40 | let (reader, writer) = tcp.split();
| ^^^ value used here after move
|
= note: move occurs because `tcp` has type `tokio_core::net::TcpStream`, which does not implement the `Copy` trait
Is it possible to achieve this without writing any unsafe code?
You get the first error because of the move closure:
let mut connections = Arc::new((Mutex::new(Vec::new())));
thread::spawn(move || {
let mut i = connections.lock().unwrap().len();
....
}
This actually moves the whole Arc, while you only want to move "a part" of it (that is, move it in such a way that the reference count is incremented, and that both threads can use it).
To do this, we can use Arc::clone:
let mut connections = Arc::new((Mutex::new(Vec::new())));
let conn = connections.clone();
thread::spawn(move || {
let mut i = conn.lock().unwrap().len();
....
}
This way, the cloned Arc, conn, is moved into the closure, and the original Arc, connections, is not, and hence still usable.
I'm not sure exactly what you are doing with your second error, but for the sake of simply counting the connections you do not need to push the entire thing.
I'm trying to share a RwLock amongst several threads without using scoped threads but I can't figure out how to get the lifetimes correct. I assume that this is possible (what's the point of RwLocks otherwise?) but I can't find any examples of it.
Here is a toy example of what I'm trying to accomplish. Any advice would be appreciated.
rust playpen for this code
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(&mut stuff);
println!("done");
}
fn helper(stuff: &mut Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}
&mut references are not safe to send to a non-scoped thread, because the thread may still run after the referenced data has been deallocated. Furthermore, after helper returns, the main thread would still be able to mutate stuff, and the spawned thread would also be able to mutate stuff indirectly, which is not allowed in Rust (there can only be one mutable alias for a variable).
Instead, the RwLock should own the data, rather than borrow it. This means helper should receive a Stuff rather than a &mut Stuff.
use std::sync::{Arc, RwLock};
use std::thread;
struct Stuff {
x: i32
}
fn main() {
let mut stuff = Stuff{x: 5};
helper(stuff);
println!("done");
}
fn helper(stuff: Stuff){
let rwlock = RwLock::new(stuff);
let arc = Arc::new(rwlock);
let local_arc = arc.clone();
for _ in 0..10{
let my_rwlock = arc.clone();
thread::spawn(move || {
let reader = my_rwlock.read().unwrap();
// do some stuff
});
}
let mut writer = local_arc.write().unwrap();
writer.x += 1;
}