I'm using this code and I don't understand how to fix this error:
use custom::config;
use actix_web::{web, App, HttpResponse, HttpServer};
pub async fn start(config: &'static config::Config) -> std::io::Result<()> {
return HttpServer::new(move || {
let mut app = App::new();
if config.custom_enabled {
app = app.route("/custom", web::get().to(HttpResponse::Ok))
}
app = app.route("/", web::get().to(HttpResponse::Ok));
return app;
})
.bind(("127.0.0.1", 8080))?
.run()
.await;
}
#[tokio::main]
async fn main() {
let config = config::new();
server::start(&config).await; // throws here
}
throws with:
`config` does not live long enough
borrowed value does not live long enough rustc E0597
Why?
Yes as compiler says: config does not live enough. When you pass config to async function start in this case, it cannot be guranteed weather the parent thread long enough as long the async part does.
In the code you given, main exits just right after triggering the start, Until main exits, start may or may not have been completed so it cannot be guaranteed.
But if main is closed everything is closed! Nope?
Well no guaranteed. Async runtime may manage itself within one single thread or multiple thread as per required or per configuration. And as this is not guaranteed by the async runtime, you async code might be closed with main or it may not. Talking about if it swapns another thread, it can be externally configured as per os to kill or not kill the child thread if parent is killed. So these inconsistent behavior will have to be respected by compiler and it cannot make any assumption about such.
Related
I'm new to actix, and I'm trying to understand how I can run a server on one thread and send requests from another.
This is the code I have so far
use actix_web::{web, App, HttpResponse, HttpServer};
use std::{sync::mpsc::channel, thread};
#[actix_web::main]
async fn main() {
let (tx, rx) = channel();
thread::spawn(move || {
let srv =
HttpServer::new(|| App::new().default_service(web::to(|| HttpResponse::NotFound())))
.bind("localhost:12347")
.unwrap()
.run();
let _ = tx.send(srv);
});
reqwest::get("http://localhost:12347").await.unwrap();
let srv = rx.recv().unwrap();
srv.handle().stop(false).await;
}
It compiles just fine, but it gets stuck on on sending the request. It seems like the server is running, soI can't figure out why I am not getting a response.
EDIT: As suggested by #Finomnis and #cafce25,I changed the code to use tasks instead of threads, and awaited te result of .run()
use actix_web::{web, App, HttpResponse, HttpServer};
use std::{sync::mpsc::channel, thread};
#[actix_web::main]
async fn main() {
let (tx, rx) = channel();
tokio::spawn(async move {
let srv =
HttpServer::new(|| App::new().default_service(web::to(|| HttpResponse::NotFound())))
.bind("localhost:12347")
.unwrap()
.run();
let _ = tx.send(srv.handle());
srv.await.unwrap();
});
reqwest::get("http://localhost:12347").await.unwrap();
let handle = rx.recv().unwrap();
handle.stop(false).await;
}
which solves the problem. I'm still curious if it is possible to do it on different threads since I can't use await inside a synchronous function.
There are a couple of things wrong with your code; the biggest one being that you never .await the run() method.
For that fact alone you cannot run it in a normal thread, it has to exist in an async task.
So what happens is:
you create the server
the server never runs because it doesn't get awaited
you query the server for a response
the response never comes because the server doesn't run, so you get stuck in reqwest::get
What you should do instead:
start the server.
Also:
You don't need to propagate the server object out to stop it. You can create a .handle() first before you move it into the task. The server handle does not contain a reference to the server, it's based on smart pointers instead.
NEVER use synchronous channels with async tasks. It will block the runtime, dead-locking everything. (The only reason it worked in your second example is because it is most likely a multi-threaded runtime and you only dead-locked one of the runtime cores. Still bad.)
(Maybe) don't tokio::spawn if you use #[actix_web::main]. actix-web has its own runtime, you need to actix_web::rt::spawn with it. If you want to use tokio based tasks, you need to do #[tokio::main]. actix-web is compatible with the tokio runtime. (EDIT: actix-web might be compatible with tokio::spawn(), I just didn't find documentation anywhere that says it is)
With all that fixed, here is a working version:
use actix_web::{rt, web, App, HttpResponse, HttpServer};
#[actix_web::main]
async fn main() {
let srv = HttpServer::new(|| App::new().default_service(web::to(|| HttpResponse::NotFound())))
.bind("localhost:12347")
.unwrap()
.run();
let srv_handle = srv.handle();
rt::spawn(srv);
let response = reqwest::get("http://localhost:12347").await.unwrap();
println!("Response code: {:?}", response.status());
srv_handle.stop(false).await;
}
Response code: 404
I know this question has been asked many times, but I still can't figure out what to do (more below).
I'm trying to spawn a new thread using std::thread::spawn and then run an async loop inside of it.
The async function I want to run:
#[tokio::main]
pub async fn pull_tweets(pg_pool2: Arc<PgPool>, config2: Arc<Settings>) {
let mut scheduler = AsyncScheduler::new();
scheduler.every(10.seconds()).run(move || {
let arc_pool = pg_pool2.clone();
let arc_config = config2.clone();
async {
pull_from_main(arc_pool, arc_config).await;
}
});
tokio::spawn(async move {
loop {
scheduler.run_pending().await;
tokio::time::sleep(Duration::from_millis(100)).await;
}
});
}
Spawning a thread to run in:
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let handle = thread::spawn(move || async {
pull_tweets(pg_pool2, config2).await;
});
}
The error:
error[E0277]: `()` is not a future
--> src/main.rs:89:9
|
89 | pull_tweets(pg_pool2, config2).await;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `()` is not a future
|
= help: the trait `std::future::Future` is not implemented for `()`
= note: required by `poll`
The last comment here does an amazing job at generalizing the problem. It seems at some point a return value is expected that implements IntoFuture and I don't have that. I tried adding Ok(()) to both the closure and the function, but no luck.
Adding into closure does literally nothing
Adding into async function gives me a new, but similar-sounding error:
`Result<(), ()>` is not a future
Then I noticed that the answer specifically talks about extension functions, which I'm not using. This also talks about extension functions.
Some other SO answers:
This is caused by a missing async.
This and this are reqwest library specific.
So none seem to work. Could someone help me understand 1)why the error exists here and 2)how to fix it?
Note 1: All of this can be easily solved by replacing std::thread::spawn with tokio::task::spawn_blocking. But I'm purposefully experimenting with thread spawn as per this article.
Note 2: Broader context on what I want to achieve: I'm pulling 150,000 tweets from twitter in an async loop. I want to compare 2 implementations: running on the main runtime vs running on separate thread. The latter is where I struggle.
Note 3: in my mind threads and async tasks are two different primitives that don't mix. Ie spawning a new thread doesn't affect how tasks behave and spawning a new task only adds work on existing thread. Please let me know if this worldview is mistaken (and what I can read).
#[tokio::main] converts your function into the following:
#[tokio::main]
pub fn pull_tweets(pg_pool2: Arc<PgPool>, config2: Arc<Settings>) {
let rt = tokio::Runtime::new();
rt.block_on(async {
let mut scheduler = AsyncScheduler::new();
// ...
});
}
Notice that it is a synchronous function, that spawns a new runtime and runs the inner future to completion. You do not await it, it is a separate runtime with it's own dedicate thread pool and scheduler:
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let handle = thread::spawn(move || {
pull_tweets(pg_pool2, config2);
});
}
Note that your original example was wrong in another way:
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let handle = thread::spawn(move || async {
pull_tweets(pg_pool2, config2).await;
});
}
Even if pull_tweets was an async function, the thread would not do anything, as all you do is create another future in the async block. The created future is not executed, because futures are lazy (and there is no executor context in that thread anyways).
I would structure the code to spawn the runtime directly in the new thread, and call any async functions you want from there:
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let handle = thread::spawn(move || {
let rt = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap();
rt.block_on(async {
pull_tweets(pg_pool2, config2).await;
});
});
}
pub async fn pull_tweets(pg_pool2: Arc<PgPool>, config2: Arc<Settings>) {
// ...
}
I'm trying to get a parent process and a child process to communicate with each other using a tokio::net::UnixStream. For some reason the child is unable to read whatever the parent writes to the socket, and presumably the other way around.
The function I have is similar to the following:
pub async fn run() -> Result<(), Error> {
let mut socks = UnixStream::pair()?;
match fork() {
Ok(ForkResult::Parent { .. }) => {
socks.0.write_u32(31337).await?;
Ok(())
}
Ok(ForkResult::Child) => {
eprintln!("Reading from master");
let msg = socks.1.read_u32().await?;
eprintln!("Read from master {}", msg);
Ok(())
}
Err(_) => Err(Error),
}
}
The socket doesn't get closed, otherwise I'd get an immediate error trying to read from socks.1. If I move the read into the parent process it works as expected. The first line "Reading from master" gets printed, but the second line never gets called.
I cannot change the communication paradigm, since I'll be using execve to start another binary that expects to be talking to a socketpair.
Any idea what I'm doing wrong here? Is it something to do with the async/await?
When you call the fork() system call:
The child process is created with a single thread—the one that called fork().
The default executor in tokio is a thread pool executor. The child process will only get one of the threads in the pool, so it won't work properly.
I found I was able to make your program work by setting the thread pool to contain only a single thread, like this:
use tokio::prelude::*;
use tokio::net::UnixStream;
use nix::unistd::{fork, ForkResult};
use nix::sys::wait;
use std::io::Error;
use std::io::ErrorKind;
use wait::wait;
// Limit to 1 thread
#[tokio::main(core_threads = 1)]
async fn main() -> Result<(), Error> {
let mut socks = UnixStream::pair()?;
match fork() {
Ok(ForkResult::Parent { .. }) => {
eprintln!("Writing!");
socks.0.write_u32(31337).await?;
eprintln!("Written!");
wait().unwrap();
Ok(())
}
Ok(ForkResult::Child) => {
eprintln!("Reading from master");
let msg = socks.1.read_u32().await?;
eprintln!("Read from master {}", msg);
Ok(())
}
Err(_) => Err(Error::new(ErrorKind::Other, "oh no!")),
}
}
Another change I had to make was to force the parent to wait for the child to complete, by calling wait() - also something you probably do not want to be doing in a real async program.
Most of the advice I have read that if you need to fork from a threaded program, either do it before creating any threads, or call exec_ve() in the child immediately after forking (which is what you plan to do anyway).
I'm trying to write a simple Rust program that reads Docker stats using shiplift and exposes them as Prometheus metrics using rust-prometheus.
The shiplift stats example runs correctly on its own, and I'm trying to integrate it in the server as
fn handle(_req: Request<Body>) -> Response<Body> {
let docker = Docker::new();
let containers = docker.containers();
let id = "my-id";
let stats = containers
.get(&id)
.stats().take(1).wait();
for s in stats {
println!("{:?}", s);
}
// ...
}
// in main
let make_service = || {
service_fn_ok(handle)
};
let server = Server::bind(&addr)
.serve(make_service);
but it appears that the stream hangs forever (I cannot produce any error message).
I've also tried the same refactor (using take and wait instead of tokio::run) in the shiplift example, but in that case I get the error executor failed to spawn task: tokio::spawn failed (is a tokio runtime running this future?). Is tokio somehow required by shiplift?
EDIT:
If I've understood correctly, my attempt does not work because wait will block tokio executor and stats will never produce results.
shiplift's API is asynchronous, meaning wait() and other functions return a Future, instead of blocking the main thread until a result is ready. A Future won't actually do any I/O until it is passed to an executor. You need to pass the Future to tokio::run as in the example you linked to. You should read the tokio docs to get a better understanding of how to write asynchronous code in rust.
There were quite a few mistakes in my understanding of how hyper works. Basically:
if a service should handle futures, do not use service_fn_ok to create it (it is meant for synchronous services): use service_fn;
do not use wait: all futures use the same executor, the execution will just hang forever (there is a warning in the docs but oh well...);
as ecstaticm0rse notices, hyper::rt::spawn could be used to read stats asynchronously, instead of doing it in the service
Is tokio somehow required by shiplift?
Yes. It uses hyper, which throws executor failed to spawn task if the default tokio executor is not available (working with futures nearly always requires an executor anyway).
Here is a minimal version of what I ended up with (tokio 0.1.20 and hyper 0.12):
use std::net::SocketAddr;
use std::time::{Duration, Instant};
use tokio::prelude::*;
use tokio::timer::Interval;
use hyper::{
Body, Response, service::service_fn_ok,
Server, rt::{spawn, run}
};
fn init_background_task(swarm_name: String) -> impl Future<Item = (), Error = ()> {
Interval::new(Instant::now(), Duration::from_secs(1))
.map_err(|e| panic!(e))
.for_each(move |_instant| {
futures::future::ok(()) // unimplemented: call shiplift here
})
}
fn init_server(address: SocketAddr) -> impl Future<Item = (), Error = ()> {
let service = move || {
service_fn_ok(|_request| Response::new(Body::from("unimplemented")))
};
Server::bind(&address)
.serve(service)
.map_err(|e| panic!("Server error: {}", e))
}
fn main() {
let background_task = init_background_task("swarm_name".to_string());
let server = init_server(([127, 0, 0, 1], 9898).into());
run(hyper::rt::lazy(move || {
spawn(background_task);
spawn(server);
Ok(())
}));
}
I have the following code:
extern crate futures;
extern crate futures_cpupool;
extern crate tokio_timer;
use std::time::Duration;
use futures::Future;
use futures_cpupool::CpuPool;
use tokio_timer::Timer;
fn work(foo: Foo) {
std::thread::sleep(std::time::Duration::from_secs(10));
}
#[derive(Debug)]
struct Foo { }
impl Drop for Foo {
fn drop(&mut self) {
println!("Dropping Foo");
}
}
fn main() {
let pool = CpuPool::new_num_cpus();
let foo = Foo { };
let work_future = pool.spawn_fn(|| {
let work = work(foo);
let res: Result<(), ()> = Ok(work);
res
});
println!("Created the future");
let timer = Timer::default();
let timeout = timer.sleep(Duration::from_millis(750))
.then(|_| Err(()));
let select = timeout.select(work_future).map(|(win, _)| win);
match select.wait() {
Ok(()) => { },
Err(_) => { },
}
}
It seems this code doesn't execute Foo::drop - no message is printed.
I expected foo to be dropped as soon as timeout future resolves in select, as it's a part of environment of a closure, passed to dropped future.
How to make it execute Foo::drop?
The documentation for CpuPool states:
The worker threads associated with a thread pool are kept alive so long as there is an open handle to the CpuPool or there is work running on them. Once all work has been drained and all references have gone away the worker threads will be shut down.
Additionally, you transfer ownership of foo from main to the closure, which then transfers it to work. work will drop foo at the end of the block. However, work is also performing a blocking sleep operation. This sleep counts as work running on the thread.
The sleep is still going when the main thread exits, which immediately tears down the program, and all the threads, without any time to clean up.
As pointed out in How to terminate or suspend a Rust thread from another thread? (and other questions in other languages), there's no safe way to terminate a thread.
I expected foo to be dropped as soon as timeout future resolves in select, as it's a part of environment of a closure, passed to dropped future.
The future doesn't actually "have" the closure or foo. All it has is a handle to the thread:
pub struct CpuFuture<T, E> {
inner: Receiver<thread::Result<Result<T, E>>>,
keep_running_flag: Arc<AtomicBool>,
}
Strangely, the docs say:
If the returned future is dropped then this CpuPool will attempt to cancel the computation, if possible. That is, if the computation is in the middle of working, it will be interrupted when possible.
However, I don't see any implementation for Drop for CpuFuture, so I don't see how it could be possible (or safe). Instead of Drop, the threadpool itself runs a Future. When that future is polled, it checks to see if the receiver has been dropped. This behavior is provided by the oneshot::Receiver. However, this has nothing to do with threads, which are outside the view of the future.