Stop actix server if scheduled task dies - rust

I'm playing with an actix server where I have a scheduled task.
I want the entire server to stop in case the thread running the task dies. As I was expecting, adding a panic in the task code will only kill the thread.
This is what I have right now:
#[actix_web::main]
async fn main() -> std::io::Result<()> {
env_logger::init();
actix_rt::spawn(async move {
let mut interval = actix_rt::time::interval(Duration::from_secs(10));
loop {
interval.tick().await;
schedule_task().await.unwrap();
// panic!("Killing thread");
}
});
HttpServer::new(move || {
App::new()
.wrap(Logger::default())
.service(ping)
.route("*", web::post().to(echo))
})
.bind(("0.0.0.0", 9124))?
.run()
.await
I know spawn returns a task handler, but I can't await on that to check if it's an error, since I can't start my server.
Not sure if this can be solved around the current implementation. Should I start my scheduled task differently / is there any way to bind the server to the thread lifetime?
Thanks!

Related

Graceful handling of SIGTERM (Ctrl-c) and shutdown a Threadpool

Currently, I am at the last chapter of the rust book, implementing the graceful shutdown of the HTTP server.
Now I want to extend the logic a bit and trigger the graceful shutdown after pressing Ctrl-c. Therefore I use the ctrlc crate.
But I cannot make it work due to various borrowing checker errors:
fn main() {
let listener = TcpListener::bind("127.0.0.1:7878").unwrap();
let pool = ThreadPool::new(4);
ctrlc::set_handler(|| {
// dropping will trigger ThreadPool::drop and gracefully shutdown the running workers
drop(pool); // compile error, variable is moved here
})
.unwrap();
for stream in listener.incoming() {
let stream = stream.unwrap();
pool.execute(|| {
handle_connection(stream);
});
}
}
I tried multiple approaches with Arc<> and additional mpsc channels, but without success.
What is the best practice here in order to make it work?
I ended up with:
fn main() {
let listener = TcpListener::bind("127.0.0.1:7878").unwrap();
let pool = ThreadPool::new(4);
let (tx, rx) = channel();
ctrlc::set_handler(move || tx.send(()).expect("Could not send signal on channel."))
.expect("Error setting Ctrl-C handler");
for stream in listener.incoming() {
let stream = stream.unwrap();
pool.execute(|| {
handle_connection(stream);
});
match rx.try_recv() {
Ok(_) => break,
Err(_) => continue,
}
}
}
It has a flaw though. Namely after pressing Ctrl-c the shutdown process is not triggered immediately, but only after receiving another request. Then the loop will break and the ThreadPool goes out of scope and gets dropped which triggers the graceful shutdown logic.
The solution is adequate for my learning purposes. In a production environment, one would rely on the graceful shutdown from a web-framework like actix

How to create individual thread for sending pings on client webscoket

I wrote this code, but when sending pings, the program cannot do anything else. How can I spawn another thread to do this work while I do something else in my program?
pub fn sending_ping(addr: Addr<MicroscopeClient>) -> Result<(), ()> {
info!("Pings started");
spawn(async move {
loop {
info!("Ping");
match addr.send(Ping {}).await {
Ok(_) => {
info!("Ping sended")
}
Err(e) => {
warn!("Ping error");
return;
}
}
std::thread::sleep(Duration::from_millis((4000) as u64));
}
});
return Ok(());
}
Assuming you are using tokio, calling tokio::spawn does not necessarily guarantee that the task will be executed on a separate thread (though it may be).
The problem here is that std::thread::sleep() completely blocks the thread that the task is running on, meaning your pings will run asynchronously (not blocking other tasks) but when you move on to the sleep nothing else on the current thread will execute for the next 4 seconds.
You can resolve this by using a non-blocking version of sleep, such as the one provided by tokio https://docs.rs/tokio/latest/tokio/time/fn.sleep.html
Thus your code would look like
pub fn sending_ping(addr: Addr<MicroscopeClient>) -> Result<(), ()> {
info!("Pings started");
spawn(async move {
loop {
info!("Ping");
match addr.send(Ping {}).await {
Ok(_) => {
info!("Ping sended")
}
Err(e) => {
warn!("Ping error");
return;
}
}
tokio::time::sleep(Duration::from_millis(4000)).await;
}
});
return Ok(());
}
If you truly want to ensure that the task spawns on a different thread you would have to use std::thread::spawn but you would then have to set up another async runtime. Instead you could use spawn_blocking which at least guarantees the task is run on a thread where tasks are expected to block
I solved this particular problem in the following way
fn send_heartbeat(&self, ctx: &mut <Self as Actor>::Context) {
ctx.run_interval(HEARTBEAT_INTERVAL, |act, ctx| {
if Instant::now().duration_since(act.hb) > CLIENT_TIMEOUT {
info!("Websocket Client heartbeat failed, disconnecting!");
act.server_addr.do_send(Disconnect {
id: act.user_id.clone(),
});
ctx.stop();
return;
}
});
}
But I still don't know how to start some long process in a websocket in a separate thread, so that it does not block the websocket, with tokio="0.2.0" and actix="0.3.0".

Messaging between two tokio runtimes inside separate threads

I ran into the kind of a problem described in this question: How can I create a Tokio runtime inside another Tokio runtime without getting the error "Cannot start a runtime from within a runtime"? .
Some good rust crates doesn't have asynchronous executor. I decided to put all such libraries calls in one thread which is tolerant of such operations. Another thread should be able to send non-blicking messages using tokio::channel.
I have programmed a demo stand to test implementation options. Call tokio::spawn inside of each runtime is made in order to understand a little more detail in tokio runtimes and handlers - it is a part of a question.
The question.
Please correct me if I misunderstand something further.
There are two tokio runtimes. Each is launched in its own thread. Call tokio::spawn inside first_runtime() spawns task on first runtime. Call tokio::spawn inside second_runtime() spawns task on second runtime. There is a tokio::channel between these two tasks. Call tx.send(...).await does not block sending thread if channel buffer is not full, even if receiving thread is blocked by thread::sleep() call.
Am I getting everything right? The output of this code tells me that I'm right, but I need confirmation of my reasoning.
use std::thread;
use std::time::Duration;
use tokio::sync::mpsc::{Sender, Receiver, channel}; // 1.12.0
#[tokio::main(worker_threads = 1)]
#[allow(unused_must_use)]
async fn first_runtime(tx: Sender<String>) {
thread::sleep(Duration::from_secs(1));
println!("first thread woke up");
tokio::spawn(async move {
for msg_id in 0..10 {
if let Err(e) = tx.send(format!("message {}", msg_id)).await {
eprintln!("[ERR]: {}", e);
} else {
println!("message {} send", msg_id);
}
}
}).await;
println!("first thread finished");
}
#[tokio::main(worker_threads = 1)]
#[allow(unused_must_use)]
async fn second_runtime(mut rx: Receiver<String>) {
thread::sleep(Duration::from_secs(3));
println!("second thread woke up");
tokio::spawn(async move {
while let Some(msg) = rx.recv().await {
println!("{} received", msg);
}
}).await;
println!("second thread finished");
}
fn main() {
let (tx, rx) = channel::<String>(5);
thread::spawn(move || { first_runtime(tx); });
second_runtime(rx);
}

gtk-rs: how to update view from another thread

I am creating a UI application with gtk-rs. In that application, I have to spawn a thread to continuously communicate with another process. Sometimes, I have to update the UI based on what happens in that thread. But, I'm not sure how to do this because I am not able to hold a reference to any part of the UI across threads.
Here is the code I tried:
use gtk;
fn main() {
let application =
gtk::Application::new(Some("com.github.gtk-rs.examples.basic"), Default::default()).unwrap()
application.connect_activate(|app| {
let ui_model = build_ui(app);
setup(ui_model);
});
application.run(&[]);
}
struct UiModel { main_buffer: gtk::TextBuffer }
fn build_ui(application: &gtk::Application) -> UiModel {
let glade_src = include_str!("test.glade");
let builder = gtk::Builder::new();
builder
.add_from_string(glade_src)
.expect("Couldn't add from string");
let window: gtk::ApplicationWindow = builder.get_object("window").unwrap();
window.set_application(Some(application));
window.show_all();
let main_text_view: gtk::TextView = builder.get_object("main_text_view")
return UiModel {
main_buffer: main_text_view.get_buffer().unwrap(),
};
}
fn setup(ui: UiModel) {
let child_process = Command::new("sh")
.args(&["-c", "while true; do date; sleep 2; done"])
.stdout(Stdio::piped())
.spawn()
.unwrap();
let incoming = child_process.stdout.unwrap();
std::thread::spawn(move || { // <- This is the part to pay
&BufReader::new(incoming).lines().for_each(|line| { // attention to.
ui.main_buffer.set_text(&line.unwrap()); // I am trying to update the
}); // UI text from another thread.
});
}
But, I get the error:
| std::thread::spawn(move || {
| _____^^^^^^^^^^^^^^^^^^_-
| | |
| | `*mut *mut gtk_sys::_GtkTextBufferPrivate` cannot be sent between threads safely
This makes sense. I can understand that the Gtk widgets aren't thread safe. But then how do I update them? Is there a way to send signals to the UI thread safely? or is there a way to run the .lines().for_each( loop in the same thread in a way that does not block the UI?
Whatever solution I go with will have to be very high performance. I will be sending much more data than in the example and I want a very low latency screen refresh.
Thanks for your help!
Ok, I solved the problem. For anyone in the future, here is the solution.
glib::idle_add(|| {}) lets you run a closure from another thread on the UI thread (thansk #Zan Lynx). This would be enough to solve the thread safety issue, but it's not enough to get around the borrow checker. No GTKObject is safe to send between threads, so another thread can never even hold a reference to it, even if it will never use it. So you need to store the UI references globally on the UI thread and set up a communication channel between threads. Here is what I did step by step:
Create a way to send data between threads that does not involve passing closures. I used std::sync::mpsc for now but another option might be better long-term.
Create some thread-local global storage. Before you ever start the second thread, store your UI references and the receiving end of that communication pipeline globally on the main thread.
Pass the sending end of the channel to the second thread via a closure. Pass the data you want through that sender.
After passing the data through, use glib::idle_add() -- not with a closure but with a static function -- to tell the UI thread to check for a new message in the channel.
In that static function on the UI thread, access your global UI and receiver variables and update the UI.
Thanks to this thread for helping me figure that out. Here is my code:
extern crate gio;
extern crate gtk;
extern crate pango;
use gio::prelude::*;
use gtk::prelude::*;
use std::cell::RefCell;
use std::io::{BufRead, BufReader};
use std::process::{Command, Stdio};
use std::sync::mpsc;
fn main() {
let application =
gtk::Application::new(Some("com.github.gtk-rs.examples.basic"), Default::default())
.unwrap();
application.connect_activate(|app| {
let ui_model = build_ui(app);
setup(ui_model);
});
application.run(&[]);
}
struct UiModel {
main_buffer: gtk::TextBuffer,
}
fn build_ui(application: &gtk::Application) -> UiModel {
let glade_src = include_str!("test.glade");
let builder = gtk::Builder::new();
builder
.add_from_string(glade_src)
.expect("Couldn't add from string");
let window: gtk::ApplicationWindow = builder.get_object("window").unwrap();
window.set_application(Some(application));
window.show_all();
let main_text_view: gtk::TextView = builder.get_object("main_text_view").unwrap();
return UiModel {
main_buffer: main_text_view.get_buffer().unwrap(),
};
}
fn setup(ui: UiModel) {
let (tx, rx) = mpsc::channel();
GLOBAL.with(|global| {
*global.borrow_mut() = Some((ui, rx));
});
let child_process = Command::new("sh")
.args(&["-c", "while true; do date; sleep 2; done"])
.stdout(Stdio::piped())
.spawn()
.unwrap();
let incoming = child_process.stdout.unwrap();
std::thread::spawn(move || {
&BufReader::new(incoming).lines().for_each(|line| {
let data = line.unwrap();
// send data through channel
tx.send(data).unwrap();
// then tell the UI thread to read from that channel
glib::source::idle_add(|| {
check_for_new_message();
return glib::source::Continue(false);
});
});
});
}
// global variable to store the ui and an input channel
// on the main thread only
thread_local!(
static GLOBAL: RefCell<Option<(UiModel, mpsc::Receiver<String>)>> = RefCell::new(None);
);
// function to check if a new message has been passed through the
// global receiver and, if so, add it to the UI.
fn check_for_new_message() {
GLOBAL.with(|global| {
if let Some((ui, rx)) = &*global.borrow() {
let received: String = rx.recv().unwrap();
ui.main_buffer.set_text(&received);
}
});
}

How do I ensure that a spawned Child process is killed if my app panics?

I'm writing a small test that starts a daemon process and tests it e.g:
let server = Command::new("target/debug/server").spawn();
// do some tests
server.kill();
The typical way to fail a test is to panic. Unfortunately this means that kill() never gets invoked and repeated runs of the test suite fail, because the port is taken by the old process that is still running.
Is there something like a TRAP function that I can use to ensure the Child gets killed?
You can use standard RAII patterns to ensure the child thread is killed if you leave a given scope. If you want to kill your child only if you are panicking, you can insert a check to std::thread::panicking.
use std::process::{Command,Child};
struct ChildGuard(Child);
impl Drop for ChildGuard {
fn drop(&mut self) {
// You can check std::thread::panicking() here
match self.0.kill() {
Err(e) => println!("Could not kill child process: {}", e),
Ok(_) => println!("Successfully killed child process"),
}
}
}
fn main() {
let child = Command::new("/bin/cat").spawn().unwrap();
let _guard = ChildGuard(child);
panic!("Main thread panicking");
}
You can put the possibly-panicking code into a closure and give that closure to catch_panic. catch_panic acts the same way a scoped or spawned thread does on joining. It returns a Result with either Ok(ClosureRetVal) or an Err(Box<Any>) if the closure panicked.
let res = std::thread::catch_panic(|| {
panic!("blub: {}", 35);
});
if let Err(err) = res {
let msg: String = *err.downcast().unwrap();
println!("{}", msg);
}
PlayPen

Resources