I'm writing a service with warp in Rust. When the service receives a SIGTERM signal, I'd like to have it shutdown gracefully and possibly do some logging or other work.
I have tried a number of examples and nothing works. The most promising seems to be from this issue but I cannot seem to get this to work or even compile. I suspect things have changed since this was answered.
# Cargo.toml
[dependencies]
tokio = {version = "1", features = ["full"]}
warp = "0.3"
futures = "0.3"
//! main.rs
use warp::Filter;
use futures;
fn main() {
let (tx, rx) = tokio::sync::oneshot::channel();
tokio::run(futures::future::lazy(move || {
let routes = warp::any().map(|| "Hello, World!");
let (_, server) = warp::serve(routes)
.bind_with_graceful_shutdown(([127, 0, 0, 1], 3030), rx);
warp::spawn(server);
}));
println!("Exiting!");
}
error[E0425]: cannot find function `run` in crate `tokio`
--> src/main.rs:6:12
|
6 | tokio::run(futures::future::lazy(move || {
| ^^^ not found in `tokio`
error[E0425]: cannot find function `spawn` in crate `warp`
--> src/main.rs:10:15
|
10 | warp::spawn(server);
| ^^^^^ not found in `warp`
|
help: consider importing one of these items
|
1 | use std::thread::spawn;
|
1 | use tokio::spawn;
|
error[E0593]: closure is expected to take 1 argument, but it takes 0 arguments
--> src/main.rs:6:16
|
6 | tokio::run(futures::future::lazy(move || {
| ^^^^^^^^^^^^^^^^^^^^^ ------- takes 0 arguments
| |
| expected closure that takes 1 argument
|
help: consider changing the closure to take and ignore the expected argument
|
6 | tokio::run(futures::future::lazy(move |_| {
| ~~~
error[E0271]: type mismatch resolving `<tokio::sync::oneshot::Receiver<_> as warp::Future>::Output == ()`
--> src/main.rs:9:14
|
9 | .bind_with_graceful_shutdown(([127, 0, 0, 1], 3030), rx);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `()`, found enum `Result`
|
= note: expected unit type `()`
found enum `Result<_, tokio::sync::oneshot::error::RecvError>`
note: required by a bound in `warp::Server::<F>::bind_with_graceful_shutdown`
--> /Users/stephen.gibson/.cargo/registry/src/github.com-1ecc6299db9ec823/warp-0.3.2/src/server.rs:281:29
|
281 | signal: impl Future<Output = ()> + Send + 'static,
| ^^^^^^^^^^^ required by this bound in `warp::Server::<F>::bind_with_graceful_shutdown`
Any advice or better yet, updated code would be appreciated.
Thanks to everyone for your thoughts. This is the code that ended up working the way I wanted:
use warp::Filter;
use tokio::signal::unix::{signal, SignalKind};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let routes = warp::any().map(|| "Hello, World!");
let mut stream = signal(SignalKind::terminate())?;
let (_, server) = warp::serve(routes)
.bind_with_graceful_shutdown(([127, 0, 0, 1], 3030), async move {
println!("waiting for signal");
stream.recv().await;
println!("done waiting for signal");
});
match tokio::join!(tokio::task::spawn(server)).0 {
Ok(()) => println!("serving"),
Err(e) => println!("ERROR: Thread join error {}", e)
};
println!("terminating");
Ok(())
}
A simpler solution, using tokio::signal::ctrl_c, a function specifically designed to sleep until a shutdown signal is received.
It works on both Unix and Windows.
use warp::Filter;
#[tokio::main]
async fn main() {
let routes = warp::any().map(|| "Hello, World!");
let (_addr, fut) = warp::serve(routes)
.bind_with_graceful_shutdown(([127, 0, 0, 1], 3030), async move {
tokio::signal::ctrl_c()
.await
.expect("failed to listen to shutdown signal");
});
fut.await;
println!("shutting down");
}
Here is example code that works. I was inspirited by wrap documentation of bind_with_graceful_shutdown
use tokio::sync::oneshot;
use warp::Filter;
#[tokio::main]
async fn main() {
let routes = warp::any().map(|| "Hello, World!");
let (tx, rx) = oneshot::channel();
let (_addr, server) =
warp::serve(routes).bind_with_graceful_shutdown(([127, 0, 0, 1], 3030), async {
rx.await.ok();
});
// Spawn the server into a runtime
tokio::task::spawn(server);
// Later, start the shutdown...
let _ = tx.send(());
}
Related
I am trying to write a program in which one thread writes to a queue and another thread
reads from the queue
But I am facing an issue regarding accessing the 'queue' in the thread that reads the queue
Below is the code which is not compiling
use ::std::collections::VecDeque;
use notify::{Config, RecommendedWatcher, RecursiveMode, Watcher};
use std::cell::RefCell;
use std::path::{Path, PathBuf};
use std::thread;
use std::time::Duration;
fn main() {
//let path = std::env::args()
// .nth(1)
// .expect("Argument 1 needs to be a path");
//println!("watching {}", path);
let path = "c:\\testfolder";
if let Err(e) = watch(path) {
println!("error: {:?}", e)
}
}
fn process_queue(queue: &VecDeque<String>) -> () {}
fn watch<P: AsRef<Path>>(path: P) -> notify::Result<()> {
let (tx, rx) = std::sync::mpsc::channel();
// Automatically select the best implementation for your platform.
// You can also access each implementation directly e.g. INotifyWatcher.
let mut watcher = RecommendedWatcher::new(tx, Config::default())?;
// Add a path to be watched. All files and directories at that path and
// below will be monitored for changes.
let mut queue: VecDeque<String> = VecDeque::new();
thread::spawn(|| {
// everything in here runs
process_queue(&queue)
});
watcher.watch(path.as_ref(), RecursiveMode::Recursive)?;
for res in rx {
match res {
Ok(event) => {
println!("changed: {:?}", event.paths);
let os_str: String = String::from(event.paths[0].to_str().unwrap());
//let my_str: String = os_str.unwrap().to_str().unwrap();
//let s =os_str.into_os_string();
queue.push_back(os_str);
}
Err(e) => println!("watch error: {:?}", e),
}
}
Ok(())
}
The output from the Rust compiler
error[E0373]: closure may outlive the current function, but it borrows `queue`, which is owned by the current function
--> src\main.rs:43:19
|
43 | thread::spawn(|| {
| ^^ may outlive borrowed value `queue`
...
47 | process_queue(&queue)
| ----- `queue` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src\main.rs:43:5
|
43 | / thread::spawn(|| {
44 | |
45 | | // everything in here runs
46 | |
47 | | process_queue(&queue)
48 | |
49 | | });
| |______^
help: to force the closure to take ownership of `queue` (and any other referenced variables), use the `move` keyword
|
43 | thread::spawn(move || {
| ++++
error[E0502]: cannot borrow `queue` as mutable because it is also borrowed as immutable
--> src\main.rs:63:17
|
43 | thread::spawn(|| {
| - -- immutable borrow occurs here
| _____|
| |
44 | |
45 | | // everything in here runs
46 | |
47 | | process_queue(&queue)
| | ----- first borrow occurs due to use of `queue` in closure
48 | |
49 | | });
| |______- argument requires that `queue` is borrowed for `'static`
...
63 | queue.push_back(os_str);
| ^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
From the errors I understand that the compiler does not allow both mutable and immutable references at the same time.
But I don't know how to implement what I am trying to do with these restrictions.
One way to solve this is by Box-ing the VecDeque so that you can pass a cloned reference to your process_queue function.
Using a Box allows you to allocate the VecDeque on the heap so that you can give your spawned thread a reference to the Vec and also still mutate the queue in the current thread.
This would look like:
let mut queue = Box::new(VecDeque::new());
let queue_clone = queue.clone();
thread::spawn(|| {
// queue_clone is now moved into the fn closure and is
// not accessible to the code below
process_queue(queue_clone)
});
and you can update process_queue to accept the correct type:
fn process_queue(queue: Box<VecDeque<String>>) -> () { }
Note that with this implementation, process_queue only runs once when the thread is spawned, and if you want to have process_queue do something every time the queue is changed, following the advice of others to use something like Channels makes the most sense.
Thanks for all your responses
From all the responses I understand that using channels and moving the receiver loop to the other thread as suggested bu user4815162342
will be the best solution
I successfully implemented what I was trying to do using channels based on your suggestions.
The final working code is pasted below
use std::thread;
use std::time::Duration;
use notify::{RecommendedWatcher, RecursiveMode, Watcher, Config};
use std::path::Path;
use std::path::PathBuf;
//
fn main() {
//let path = std::env::args()
// .nth(1)
// .expect("Argument 1 needs to be a path");
//println!("watching {}", path);
let path="c:\\testfolder";
if let Err(e) = watch(path) {
println!("error: {:?}", e)
}
}
fn watch<P: AsRef<Path>>(path: P) -> notify::Result<()> {
let (tx, rx) = std::sync::mpsc::channel();
// Automatically select the best implementation for your platform.
// You can also access each implementation directly e.g. INotifyWatcher.
let mut watcher = RecommendedWatcher::new(tx, Config::default())?;
// Add a path to be watched. All files and directories at that path and
// below will be monitored for changes.
let handle=thread::spawn(move || {
// everything in here runs
for res in rx {
match res {
Ok(event) =>{
// println!("changed: {:?}", event.paths);
let os_str:String = String::from(event.paths[0].to_str().unwrap());
println!("file name: {}", os_str);
},
Err(e) => println!("watch error: {:?}", e),
}
}
});
watcher.watch(path.as_ref(), RecursiveMode::Recursive)?;
handle.join();
Ok(())
}
In your situation, using Rust's MPSC (multi-producer single-consumer, ie essentially a queue) would probably be the best. You could also create a variable that is shared between multiple thread using Arc and Mutex structs, but that would be way overkilled and can have a performance impact (only one can access the variable at any time).
Here is an example of a multi-threaded MPSC, I will let you adapt it to your infrastructure :
use std::{sync::mpsc, thread};
fn main() {
let (sender, receiver) = mpsc::channel();
let handle_1 = thread::spawn(|| {
thread_1(sender);
});
let handle_2 = thread::spawn(|| {
thread_2(receiver);
});
handle_1.join().unwrap();
handle_2.join().unwrap();
}
// the enum must have the Send trait (automatically implemented)
enum Instruction {
Print(String),
Exit
}
fn thread_1(sender: mpsc::Sender<Instruction>) {
sender.send(Instruction::Print("I".to_owned())).unwrap();
sender.send(Instruction::Print("like".to_owned())).unwrap();
sender.send(Instruction::Print("Rust".to_owned())).unwrap();
sender.send(Instruction::Print(".".to_owned())).unwrap();
sender.send(Instruction::Exit).unwrap();
}
fn thread_2(receiver: mpsc::Receiver<Instruction>) {
'global_loop: loop {
for received in receiver.recv() {
match received {
Instruction::Print(string) => print!("{} ", string),
Instruction::Exit => {
println!("");
break 'global_loop;
}
}
}
}
}
I want to periodically fetch data (using asynchronous reqwest), which is then served at an http endpoint using actix-web as a server.
(I have a data source that has a fixed format, that I want to have read by a service that require a different format, so I need to transform the data.)
I've tried to combine actix concepts with the thread sharing state example from the Rust book, but I don't understand the error or how to solve it.
This is the code minified as much as I was able:
use actix_web::{get, http, web, App, HttpResponse, HttpServer, Responder};
use std::sync::{Arc, Mutex};
use tokio::time::{sleep, Duration};
struct AppState {
status: String,
}
#[get("/")]
async fn index(data: web::Data<Mutex<AppState>>) -> impl Responder {
let state = data.lock().unwrap();
HttpResponse::Ok()
.insert_header(http::header::ContentType::plaintext())
.body(state.status.to_owned())
}
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let status_string = get_state().await.unwrap();
let app_data = Arc::new(Mutex::new(web::Data::new(AppState {
status: status_string,
})));
let app_data1 = Arc::clone(&app_data);
actix_web::rt::spawn(async move {
loop {
println!("I get executed every 2-ish seconds!");
sleep(Duration::from_millis(2000)).await;
let res = get_state().await;
let mut app_data = app_data1.lock().unwrap();
// Edit 2: this line is not accepted by the compiler
// Edit 2: *app_data.status = res.unwrap();
// Edit 2: but this line is accepted
*app_data = web::Data::new(AppState { status: res });
}
});
let app_data2 = Arc::clone(&app_data);
// Edit 2: but I get an error here now
HttpServer::new(move || App::new().app_data(app_data2).service(index))
.bind(("127.0.0.1", 9090))?
.run()
.await
}
async fn get_state() -> Result<String, Box<dyn std::error::Error>> {
let client = reqwest::Client::new().get("http://ipecho.net/plain".to_string());
let status = client.send().await?.text().await?;
println!("got status: {status}");
Ok(status)
}
But I get the following error:
error[E0308]: mismatched types
--> src/main.rs:33:32
|
33 | *app_data.status = res.unwrap();
| ---------------- ^^^^^^^^^^^^ expected `str`, found struct `String`
| |
| expected due to the type of this binding
error[E0277]: the size for values of type `str` cannot be known at compilation time
--> src/main.rs:33:13
|
33 | *app_data.status = res.unwrap();
| ^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
|
= help: the trait `std::marker::Sized` is not implemented for `str`
= note: the left-hand-side of an assignment must have a statically known size
Some errors have detailed explanations: E0277, E0308.
For more information about an error, try `rustc --explain E0277`.
Why do I suddenly get a str? Is there an easy fix or is my approach to solving this wrong?
Edit: Maybe removing the * is the right way to go, as Peter Hall suggests, but that gives me the following error instead:
error[E0594]: cannot assign to data in an `Arc`
--> src/main.rs:33:13
|
33 | app_data.status = res.unwrap();
| ^^^^^^^^^^^^^^^ cannot assign
|
= help: trait `DerefMut` is required to modify through a dereference, but it is not implemented for `Arc<AppState>`
error[E0507]: cannot move out of `app_data2`, a captured variable in an `Fn` closure
--> src/main.rs:38:49
|
37 | let app_data2 = Arc::clone(&app_data);
| --------- captured outer variable
38 | HttpServer::new(move || App::new().app_data(app_data2).service(index))
| ------- ^^^^^^^^^ move occurs because `app_data2` has type `Arc<std::sync::Mutex<Data<AppState>>>`, which does not implement the `Copy` trait
| |
| captured by this `Fn` closure
Some errors have detailed explanations: E0507, E0594.
For more information about an error, try `rustc --explain E0507`.
Edit 2: I now get the following error (code changes commented with 'Edit 2' above):
error[E0507]: cannot move out of `app_data2`, a captured variable in an `Fn` closure
--> src/main.rs:46:49
|
45 | let app_data2 = app_data.clone();
| --------- captured outer variable
46 | HttpServer::new(move || App::new().app_data(app_data2).service(index))
| ------- ^^^^^^^^^ move occurs because `app_data2` has type `Arc<Mutex<Data<AppState>>>`, which does not implement the `Copy` trait
| |
| captured by this `Fn` closure
For more information about this error, try `rustc --explain E0507`.
My Cargo.toml dependencies:
[dependencies]
actix-web = "4.2.1"
reqwest = "0.11.12"
tokio = "1.21.2"
async solution
I had my types mixed up a bit, having the app state as Arc<Mutex<T>> seemed to be the way to go, maybe it would be better with Arc<RwLock<T>>.
use actix_web::{get, http, web, App, HttpResponse, HttpServer, Responder};
use std::sync::{Arc, Mutex};
use tokio::time::{sleep, Duration};
struct AppState {
status: String,
}
#[get("/")]
async fn index(data: web::Data<Arc<Mutex<AppState>>>) -> impl Responder {
let state = data.lock().unwrap();
HttpResponse::Ok()
.insert_header(http::header::ContentType::plaintext())
.body(state.status.to_owned())
}
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let status_string = get_state().await.unwrap();
let app_data = Arc::new(Mutex::new(AppState {
status: status_string,
}));
let app_data1 = app_data.clone();
actix_web::rt::spawn(async move {
loop {
println!("I get executed every 2-ish seconds!");
sleep(Duration::from_millis(2000)).await;
let res = get_state().await.unwrap();
let mut app_data = app_data1.lock().unwrap();
*app_data = AppState { status: res };
}
});
HttpServer::new(move || {
App::new()
.app_data(web::Data::new(app_data.clone()))
.service(index)
})
.bind(("127.0.0.1", 9090))?
.run()
.await
}
async fn get_state() -> Result<String, Box<dyn std::error::Error>> {
let client = reqwest::Client::new().get("http://ipecho.net/plain".to_string());
let status = client.send().await?.text().await?;
println!("got status: {status}");
Ok(status)
}
async/sync solution
Instead of doing the async get with reqwest I have a solution with the synchronous crate minreq (that I found after a lot of searching). I also chose to not use the #[actix_web::main] macro, and instead start the runtime explicitly at the end of my main function.
use actix_web::{get, http, rt, web, App, HttpResponse, HttpServer, Responder};
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
struct AppState {
status: String,
}
#[get("/")]
async fn index(data: web::Data<Arc<Mutex<AppState>>>) -> impl Responder {
let state = &data.lock().unwrap();
HttpResponse::Ok()
.insert_header(http::header::ContentType::plaintext())
.body(state.status.clone())
}
fn main() -> std::io::Result<()> {
let status_string = get_state().unwrap();
let app_data = Arc::new(Mutex::new(AppState {
status: status_string,
}));
let app_data1 = Arc::clone(&app_data);
thread::spawn(move || loop {
thread::sleep(Duration::from_millis(2000));
let res = get_state().unwrap();
let mut app_data = app_data1.lock().unwrap();
*app_data = AppState { status: res };
});
rt::System::new().block_on(
HttpServer::new(move || {
App::new()
.app_data(web::Data::new(app_data.clone()))
.service(index)
})
.bind(("127.0.0.1", 9090))?
.run(),
)
}
fn get_state() -> Result<String, Box<dyn std::error::Error>> {
let resp = minreq::get("http://ipecho.net/plain").send().unwrap();
let state = resp.as_str().unwrap();
Ok(state.to_string())
}
I have a requirement to send and receive normal data on the same TcpStream, while sending heartbeat data at regular intervals. In the current implementation, I used Arc<Mutex<TcpStream>>, but it compiled with errors:
use anyhow::Result;
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<()> {
let stream = TcpStream::connect("127.0.0.1:8888").await.unwrap();
let stream = Arc::new(Mutex::new(stream));
let common_stream = stream.clone();
let handler1 = tokio::spawn(async {
loop {
let mut stream = common_stream.lock().unwrap();
let mut buf = [0u8; 10];
stream.read_exact(&mut buf).await.unwrap();
buf.reverse();
stream.write(&buf).await.unwrap();
}
});
let heartbeat_stream = stream.clone();
let handler2 = tokio::spawn(async {
loop {
let mut stream = heartbeat_stream.lock().unwrap();
stream.write_u8(1).await.unwrap();
thread::sleep(Duration::from_millis(200));
}
});
handler1.await?;
handler2.await?;
Ok(())
}
error: future cannot be sent between threads safely
--> src\main.rs:14:20
|
14 | let handler1 = tokio::spawn(async {
| ^^^^^^^^^^^^ future created by async block is not `Send`
|
= help: within `impl Future<Output = [async output]>`, the trait `Send` is not implemented for `std::sync::MutexGuard<'_, tokio::net::TcpStream>`
note: future is not `Send` as this value is used across an await
--> src\main.rs:20:31
|
16 | let mut stream = common_stream.lock().unwrap();
| ---------- has type `std::sync::MutexGuard<'_, tokio::net::TcpStream>` which is not `Send`
...
20 | stream.write(&buf).await.unwrap();
| ^^^^^^ await occurs here, with `mut stream` maybe used later
21 | }
| - `mut stream` is later dropped here
note: required by a bound in `tokio::spawn`
--> .cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\tokio-1.17.0\src\task\spawn.rs:127:21
|
127 | T: Future + Send + 'static,
| ^^^^ required by this bound in `tokio::spawn`
error: future cannot be sent between threads safely
--> src\main.rs:25:20
|
25 | let handler2 = tokio::spawn(async {
| ^^^^^^^^^^^^ future created by async block is not `Send`
|
= help: within `impl Future<Output = [async output]>`, the trait `Send` is not implemented for `std::sync::MutexGuard<'_, tokio::net::TcpStream>`
note: future is not `Send` as this value is used across an await
--> src\main.rs:28:31
|
27 | let mut stream = heartbeat_stream.lock().unwrap();
| ---------- has type `std::sync::MutexGuard<'_, tokio::net::TcpStream>` which is not `Send`
28 | stream.write_u8(1).await.unwrap();
| ^^^^^^ await occurs here, with `mut stream` maybe used later
...
31 | }
| - `mut stream` is later dropped here
note: required by a bound in `tokio::spawn`
--> .cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\tokio-1.17.0\src\task\spawn.rs:127:21
|
127 | T: Future + Send + 'static,
| ^^^^ required by this bound in `tokio::spawn`
How can these errors be fixed or is there another way to achieve the same goal?
Here is a solution that splits the stream to two parts for reading and writing plus does in a loop:
waiting for heartbeat events and sends a byte to write half of stream when this happens
waits data from read half (10 bytes), reverses it and writes again to write half
Also this does not spawn threads and does everything nicely in current one without locks.
use anyhow::Result;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<()> {
let mut stream = TcpStream::connect("127.0.0.1:8888").await?;
let (mut read, mut write) = stream.split();
let mut heartbeat_interval = tokio::time::interval(Duration::from_millis(200));
let mut buf = [0u8; 10];
loop {
tokio::select! {
_ = heartbeat_interval.tick() => {
write.write(&[1]).await?;
}
result = read.read_exact(&mut buf) => {
let _bytes_read = result?;
buf.reverse();
write.write(&buf).await?;
}
}
}
}
Several errors in your code, although the idea behind it is almost good. You should use any available tool in async as possible. Some of the needed/desired changes:
Use tokio::time::sleep because it is async, otherwise the call is blocking
Use an async version of mutex (the one from futures crate for example)
Use some kind of generic error handling (anyhow would help)
use futures::lock::Mutex;
use anyhow::Error;
use tokio::time::sleep;
use std::sync::Arc;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<(), Error> {
let stream = TcpStream::connect("127.0.0.1:8888").await.unwrap();
let stream = Arc::new(Mutex::new(stream));
let common_stream = stream.clone();
let handler1 = tokio::spawn(async move {
loop {
let mut stream = common_stream.lock().await;
let mut buf = [0u8; 10];
stream.read_exact(&mut buf).await.unwrap();
buf.reverse();
stream.write(&buf).await.unwrap();
}
});
let heartbeat_stream = stream.clone();
let handler2 = tokio::spawn(async move {
loop {
let mut stream = heartbeat_stream.lock().await;
stream.write_u8(1).await.unwrap();
sleep(Duration::from_millis(200)).await;
}
});
handler1.await?;
handler2.await?;
Ok(())
}
Playground
Hoping someone can help me understand why running warp with a single route like this compiles fine:
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// GET /stats
let stats = warp::get()
.and(warp::path("stats"))
.map(|| {
let mut sys = System::new_all();
sys.refresh_all();
let local = LocalSystem::from_sys(&sys);
warp::reply::json(&local);
});
// GET /
let index = warp::get()
.and(warp::path::end())
.map(|| warp::reply::json(&last_ten_logs()));
warp::serve(index).run(([127, 0, 0, 1], 4000)).await;
Ok(())
}
But changing the warp::serve() line to serve two routes like in the examples in the repo causes a compilation error:
error[E0277]: the trait bound `(): Reply` is not satisfied
--> src/main.rs:140:17
|
140 | warp::serve(stats.or(index)).run(([127, 0, 0, 1], 4000)).await;
| ----------- ^^^^^^^^^^^^^^^ the trait `Reply` is not implemented for `()`
| |
| required by a bound introduced by this call
|
= note: required because of the requirements on the impl of `Reply` for `((),)`
= note: 2 redundant requirements hidden
= note: required because of the requirements on the impl of `Reply` for `(warp::generic::Either<((),), (Json,)>,)`
I don't understand what the compiler is asking me to change.
The error is explicit:
the trait Reply is not implemented for ()
The problem is that your stats endpoint do not return anything, just remove the last ; so it gets returned as the last expresion in the closure:
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// GET /stats
let stats = warp::get()
.and(warp::path("stats"))
.map(|| {
let mut sys = System::new_all();
sys.refresh_all();
let local = LocalSystem::from_sys(&sys);
warp::reply::json(&local)
});
...
}
I have a bi-directional grpc stream that acts as bridge to a kafka cluster. When the stream is first initialised, I was to create the kafka consumer and start using it.
To do so, I thought of initialising an empty consumer, waiting for the first input, then assigning a created consumer to an empty one. I tried to do so by following the pattern here.
https://doc.rust-lang.org/rust-by-example/variable_bindings/declare.html
Rust is throwing a possibly-unitialized variable error, is this because it is being initialised in an asynchronous stream?
use std::pin::Pin;
use futures::{Stream, StreamExt};
use kafka::consumer::{Consumer, FetchOffset, GroupOffsetStorage};
use tonic::transport::Server;
use tonic::{Request, Response, Status};
use bridge::kafka_stream_server::{KafkaStream, KafkaStreamServer};
use bridge::{KafkaResponse, PublishRequest};
pub mod bridge {
tonic::include_proto!("bridge"); // The string specified here must match the proto package name
}
#[derive(Default)]
pub struct KafkaStreamService {}
pub fn create_kafka_consumer(topic: String) -> Consumer {
Consumer::from_hosts(vec!["localhost:9092".to_owned()])
.with_topic(topic.to_owned())
.with_fallback_offset(FetchOffset::Latest)
.with_group("".to_owned())
.with_offset_storage(GroupOffsetStorage::Kafka)
.create()
.unwrap()
}
#[tonic::async_trait]
impl KafkaStream for KafkaStreamService {
type SubscribeStream =
Pin<Box<dyn Stream<Item = Result<KafkaResponse, Status>> + Send + Sync + 'static>>;
async fn subscribe(
&self,
request: Request<tonic::Streaming<PublishRequest>>,
) -> Result<Response<Self::SubscribeStream>, Status> {
println!("Initiated stream!");
let mut stream = request.into_inner();
let mut consumer_created_flag: bool = false;
let consumer: Consumer; //declared here
let output = async_stream::try_stream! {
while let Some(publication) = stream.next().await {
let message = publication?;
let topic = message.topic.clone();
if consumer_created_flag == false {
consumer = create_kafka_consumer(topic); //error occurs here
consumer_created_flag = true;
}
let reply = bridge::KafkaResponse {
content: format!("Hello {}!", "world"),
};
yield reply.clone();
}
};
Ok(Response::new(Box::pin(output) as Self::SubscribeStream))
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "[::1]:50051".parse().unwrap();
println!("KafkaService listening on: {}", addr);
let svc = KafkaStreamServer::new(KafkaStreamService::default());
Server::builder().add_service(svc).serve(addr).await?;
Ok(())
}
EDIT: verbose error as requested:
error[E0381]: use of possibly-uninitialized variable: `consumer`
--> src/server.rs:42:22
|
42 | let output = async_stream::try_stream! {
| ______________________^
43 | | while let Some(publication) = stream.next().await {
44 | | let message = publication?;
45 | | let topic = message.topic.clone();
46 | | if consumer_created_flag == false {
47 | | consumer = create_kafka_consumer(topic);
| | -------- use occurs due to use in generator
... |
54 | | }
55 | | };
| |_________^ use of possibly-uninitialized `consumer`
|
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
The declare first pattern only works with basic control flow (if, match, {}, etc). It falls apart when referenced or moved into another object, like an async block or a closure:
fn main() {
let val: i32;
let func = move || {
val = 5;
};
}
error[E0594]: cannot assign to `val`, as it is not declared as mutable
--> src/main.rs:4:9
|
2 | let val: i32;
| --- help: consider changing this to be mutable: `mut val`
3 | let func = move || {
4 | val = 5;
| ^^^^^^^ cannot assign
error[E0381]: use of possibly-uninitialized variable: `val`
--> src/main.rs:3:16
|
3 | let func = move || {
| ^^^^^^^ use of possibly-uninitialized `val`
4 | val = 5;
| --- use occurs due to use in closure
A potential fix is to move its declaration into the try_stream! macro:
let output = async_stream::try_stream! {
let mut consumer_created_flag: bool = false;
let consumer: Consumer;
while let Some(publication) = stream.next().await {
let message = publication?;
let topic = message.topic.clone();
if consumer_created_flag == false {
consumer = create_kafka_consumer(topic);
consumer_created_flag = true;
}
let reply = KafkaResponse {
content: format!("Hello {}!", "world"),
};
yield reply.clone();
}
};
However, this causes a new error because you're potentially assigning to it twice (the compiler doesn't know that consumer_created_flag is guarding it):
error[E0384]: cannot assign twice to immutable variable `consumer`
--> src\lib.rs:1348:21
|
44 | let consumer: Consumer; //declared here
| -------- help: make this binding mutable: `mut consumer`
...
49 | consumer = create_kafka_consumer(topic); //error occurs here
| ^^^^^^^^ cannot assign twice to immutable variable
Fortunately a quick fix is to simply make consumer mutable. And then the only thing the compiler complains about is that it is unused, but I figure there's a reason you've put it there.