I'm trying to create a counter in a Hyper web server that counts the number of requests it has received. I'm using a Arc<Mutex<u64>> to hold onto count. However, I haven't been able to figure out the right combination of move and .clone() to satisfy the types of the closures. Here's some code that compiles, but resets the counter on each request:
extern crate hyper;
use hyper::rt::Future;
use hyper::service::service_fn_ok;
use hyper::{Body, Response, Server};
use std::sync::{Arc, Mutex};
fn main() {
let addr = "0.0.0.0:3000".parse().unwrap();
// FIXME want to create the counter here, not below
let server = Server::bind(&addr)
.serve(|| {
service_fn_ok(|_req| {
let counter = Arc::new(Mutex::new(0));
use_counter(counter)
})
})
.map_err(|e| eprintln!("Error: {}", e));
hyper::rt::run(server)
}
fn use_counter(counter: Arc<Mutex<u64>>) -> Response<Body> {
let mut data = counter.lock().unwrap();
*data += 1;
Response::new(Body::from(format!("Counter: {}\n", data)))
}
It turns out I was pretty close, and looking at a few other examples helped me realize the problem. Since there are two layers of closures at play here, I need to move the counter into the outer closure, clone it, and then move that clone into the inner closure and clone there again. To wit:
extern crate hyper; // 0.12.10
use hyper::rt::Future;
use hyper::service::service_fn_ok;
use hyper::{Body, Response, Server};
use std::sync::{Arc, Mutex};
fn main() {
let addr = "0.0.0.0:3000".parse().unwrap();
let counter = Arc::new(Mutex::new(0));
let server = Server::bind(&addr)
.serve(move || {
let counter = counter.clone();
service_fn_ok(move |_req| use_counter(counter.clone()))
})
.map_err(|e| eprintln!("Error: {}", e));
hyper::rt::run(server)
}
fn use_counter(counter: Arc<Mutex<u64>>) -> Response<Body> {
let mut data = counter.lock().unwrap();
*data += 1;
Response::new(Body::from(format!("Counter: {}\n", data)))
}
Update February 2020 Here's a version using hyper 0.13:
use hyper::{Body, Response, Server, Request};
use std::sync::{Arc, Mutex};
use hyper::service::{make_service_fn, service_fn};
use std::convert::Infallible;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "0.0.0.0:3000".parse()?;
let counter = Arc::new(Mutex::new(0));
let make_service = make_service_fn(move |_conn| {
let counter = counter.clone();
async move {
Ok::<_, Infallible>(service_fn(move |_req: Request<Body>| {
let counter = counter.clone();
async move {
Ok::<_, Infallible>(use_counter(counter))
}
}))
}
});
Server::bind(&addr).serve(make_service).await?;
Ok(())
}
fn use_counter(counter: Arc<Mutex<u64>>) -> Response<Body> {
let mut data = counter.lock().unwrap();
*data += 1;
Response::new(Body::from(format!("Counter: {}\n", data)))
}
Related
I have created a very simplified example of the code I am having an issue with:
use core::time;
use std::thread;
use tokio::sync::{mpsc::Receiver, RwLock};
struct MyStruct {
counter: Arc<RwLock<i32>>,
rx: RwLock<Receiver<i32>>,
}
impl MyStruct {
async fn start_here(&self) { // <--------- Lifetime error here on self
while let Some(message) = self.rx.write().await.recv().await {
tokio::spawn(self.do_some_work_then_update_counter());
}
}
async fn do_some_work_then_update_counter(&self) {
let dur = time::Duration::from_millis(10000);
thread::sleep(dur);
let mut counter = self.counter.write().await;
*counter += 1;
}
}
There is a receiver that is receiving messages from another part of the program, and I want to be able to process each message in its own task to prevent blocking the next message from being processed.
As you can imagine it's a lifetime error since the task could outlast self in this case.
One solution I have done is this:
impl MyStruct {
async fn start_here(&self) {
while let Some(message) = self.rx.write().await.recv().await {
let counter = self.counter.clone();
tokio::spawn(do_some_work_then_update_counter(counter));
}
}
}
async fn do_some_work_then_update_counter(counter: Arc<RwLock<i32>>) {
let dur = time::Duration::from_millis(10000);
thread::sleep(dur);
let mut counter = counter.write().await;
*counter += 1;
}
This just doesn't seem like a good option, I want to keep do_some_work_then_update_counter as an impl of MyStruct instead of a free function since it is modifying data on MyStruct.
I am wondering if there is a better solution to this?
You can if you'll return impl Future directly instead of being async fn:
fn do_some_work_then_update_counter(&self) -> impl Future<Output = ()> {
let counter = Arc::clone(&self.counter);
async move {
let dur = time::Duration::from_millis(10000);
thread::sleep(dur);
let mut counter = counter.write().await;
*counter += 1;
}
}
I am writing a server using rustls and hyper and wish to peek and then parse the TcpStream to then accept the corresponding tokio_rustls::TlsAcceptor I want. However, this leads me to use both async and non async functions (tokio::net::TcpStream::peek and tokio_rustls::TlsAcceptor::accept) on the stream, which as been causing me trouble. Simply adding an async block for the peek function gives me an "unused implementer of `core::future::future::Future` that must be used" error and changing move to async move does not work.
I'm wondering if there is some way to get around this, perhaps by not using and_then()?
// Dependencies: futures-util = "0.3.1", rustls = "0.18"
// tokio = {version = "0.2", features = ["full"]}, tokio-rustls = "0.14.0"
use tokio::net::{TcpListener, TcpStream};
use tokio_rustls::server::TlsStream;
use tokio_rustls::TlsAcceptor;
use std::{sync, io};
use futures_util::{
future::TryFutureExt,
stream::{StreamExt, TryStreamExt},
};
#[tokio::main]
async fn run_server() -> Result<(), Box<dyn std::error::Error + Send + Sync>>{
let addr = format!("127.0.0.1:{}", 8000);
let mut tcp = TcpListener::bind(&addr).await?;
let tls_config = sync::Arc::new(rustls::ServerConfig::new(rustls::NoClientAuth::new()));
let tls_acceptor = TlsAcceptor::from(tls_config);
let mut v = vec![0u8; 16 * 1024];
// main focus of question
let incoming_tls_stream = tcp
.incoming()
.map_err(|e| error(format!("Incoming failed: {:?}", e)))
.and_then(move |mut s: TcpStream| {
let n: usize = s.peek(&mut v).await.unwrap();
println!("{:}", n);
// parse something from stream
let parsed = do_something(&v[..n]);
println!("{:}", parsed);
tls_acceptor.accept(s).map_err(|e| {
println!("Client-connection error...");
error(format!("TLS Error: {:?}", e))
})
})
.boxed();
// ...
return Ok(());
}
fn main() {
if let Err(e) = run_server() {
eprintln!("FAILED: {}", e);
std::process::exit(1);
}
}
fn error(err: String) -> io::Error {
io::Error::new(io::ErrorKind::Other, err)
}
fn do_something(bytes: &[u8]) -> &str {
return "test";
}
I am writing the code to bootstrap and connect to a 2G/3G network using a SIM800L modem. This modem is interfaced with a single serial channel, which I've muxed outside of this project into 4 channels (data, text interface, control interface, status messages).
In order to bootstrap this, I need to run a series of sequential commands. This sequence changes based on the output of the modem (is the SIM locked? What kind of info does the SIM need to be unlocked? What kind of APN are we getting on? What kind of network selection do we want?). I initially thought that this would be a perfect application for futures as each individual operation can be very costly in terms of time spent idling (AT+COPS, one of the command, takes up to 10s to return).
I'm on to something like this, which, while it compiles and seems to execute commands sequentially, the third operation comes out empty. My question is twofold: why do the commands run not pop up in the result of the last future, and is there a more robust way of doing something like this?
#![feature(conservative_impl_trait)]
extern crate futures;
extern crate tokio_core;
use std::sync::{Arc, Mutex};
use futures::{future, Future};
use tokio_core::reactor::Core;
use futures::sync::oneshot;
use std::thread;
use std::io;
use std::time::Duration;
pub struct Channel {
operations: Arc<Mutex<Vec<String>>>,
}
impl Channel {
pub fn ops(&mut self) -> Box<Future<Item = Vec<String>, Error = io::Error>> {
println!("{:?}", self.operations);
let ops = Arc::clone(&self.operations);
let ops = ops.lock().unwrap();
future::ok::<Vec<String>, io::Error>(ops.to_vec()).boxed()
}
pub fn run(&mut self, command: &str) -> Box<Future<Item = Vec<String>, Error = io::Error>> {
let (tx, rx) = oneshot::channel::<Vec<String>>();
let ops = Arc::clone(&self.operations);
let str_cmd = String::from(command);
thread::spawn(move || {
thread::sleep(Duration::new(0, 10000));
let mut ops = ops.lock().unwrap();
ops.push(str_cmd.clone());
println!("Pushing op: {}", str_cmd.clone());
tx.send(vec!["OK".to_string()])
});
rx.map_err(|_| io::Error::new(io::ErrorKind::NotFound, "Test"))
.boxed()
}
}
pub struct Channels {
inner_object: Arc<Mutex<Channel>>,
}
impl Channels {
pub fn one(&self, cmd: &str) -> Box<Future<Item = Vec<String>, Error = io::Error>> {
let v = Arc::clone(&self.inner_object);
let mut v = v.lock().unwrap();
v.run(&cmd)
}
pub fn ops(&self) -> Box<Future<Item = Vec<String>, Error = io::Error>> {
let v = Arc::clone(&self.inner_object);
let mut v = v.lock().unwrap();
v.ops()
}
pub fn run_command(&self) -> Box<Future<Item = (), Error = io::Error>> {
let a = self.one("AT+CMEE=2");
let b = self.one("AT+CREG=0");
let c = self.ops();
Box::new(a.and_then(|result_1| {
assert_eq!(result_1, vec![String::from("OK")]);
b.and_then(|result_2| {
assert_eq!(result_2, vec![String::from("OK")]);
c.map(move |ops| {
assert_eq!(
ops.as_slice(),
["AT+CMEE=2".to_string(), "AT+CREG=0".to_string()]
);
})
})
}))
}
}
fn main() {
let mut core = Core::new().expect("Core should be created");
let channels = Channels {
inner_object: Arc::new(Mutex::new(Channel {
operations: Arc::new(Mutex::new(vec![])),
})),
};
let result = core.run(channels.run_command()).expect("Should've worked");
println!("{:?}", result);
}
playground
why do the commands run not pop up in the result of the last future
Because you haven't sequenced the operations to occur in that way:
let a = self.one("AT+CMEE=2");
let b = self.one("AT+CREG=0");
let c = self.ops();
This immediately builds:
a, b — promises that sleep a while before they respond
c — a promise that gets the ops in the vector
At the point in time that c is created, the sleeps have yet to terminate, so there have been no operations performed, so the vector will be empty.
Future::and_then is intended to be used to define sequential operations. This is complicated in your case as you want to use self in the body of the and_then closure. You can clone the Arc<Channel> and use that instead.
You'll note that I've made a number of simplifications:
Returning a String instead of Vec<String>
Removing unused mut qualifiers and a Mutex
Returning the operations Vec directly.
extern crate futures;
extern crate tokio_core;
use std::sync::{Arc, Mutex};
use futures::Future;
use tokio_core::reactor::Core;
use futures::sync::oneshot;
use std::thread;
use std::io;
use std::time::Duration;
pub struct Channel {
operations: Arc<Mutex<Vec<String>>>,
}
impl Channel {
fn ops(&self) -> Vec<String> {
self.operations.lock().unwrap().clone()
}
fn command(&self, command: &str) -> Box<Future<Item = String, Error = io::Error>> {
let (tx, rx) = oneshot::channel();
let ops = Arc::clone(&self.operations);
let str_cmd = String::from(command);
thread::spawn(move || {
thread::sleep(Duration::new(0, 10000));
println!("Pushing op: {}", str_cmd);
ops.lock().unwrap().push(str_cmd);
tx.send("OK".to_string())
});
Box::new(rx.map_err(|_| io::Error::new(io::ErrorKind::NotFound, "Test")))
}
}
struct Channels {
data: Arc<Channel>,
}
impl Channels {
fn run_command(&self) -> Box<Future<Item = (), Error = io::Error>> {
let d2 = Arc::clone(&self.data);
let d3 = Arc::clone(&self.data);
Box::new(
self.data
.command("AT+CMEE=2")
.and_then(move |cmee_answer| {
assert_eq!(cmee_answer, "OK"); // This should be checked in `command` and be a specific Error
d2.command("AT+CREG=0")
})
.map(move |creg_answer| {
assert_eq!(creg_answer, "OK"); // This should be checked in `command` and be a specific Error
let ops = d3.ops();
assert_eq!(ops, ["AT+CMEE=2", "AT+CREG=0"])
}),
)
}
}
fn main() {
let mut core = Core::new().expect("Core should be created");
let channels = Channels {
data: Arc::new(Channel {
operations: Arc::new(Mutex::new(vec![])),
}),
};
let result = core.run(channels.run_command()).expect("Should've worked");
println!("{:?}", result);
}
However, this isn't the type of code I usually see with futures. Instead of taking &self, many futures take self. Let's see how that would look:
extern crate futures;
extern crate tokio_core;
use std::sync::{Arc, Mutex};
use futures::Future;
use tokio_core::reactor::Core;
use futures::sync::oneshot;
use std::thread;
use std::io;
use std::time::Duration;
#[derive(Clone)]
pub struct Channel {
operations: Arc<Mutex<Vec<String>>>,
}
impl Channel {
fn ops(&self) -> Arc<Mutex<Vec<String>>> {
Arc::clone(&self.operations)
}
fn command(self, command: &str) -> Box<Future<Item = (Self, String), Error = io::Error>> {
let (tx, rx) = oneshot::channel();
let str_cmd = String::from(command);
thread::spawn(move || {
thread::sleep(Duration::new(0, 10000));
println!("Pushing op: {}", str_cmd);
self.operations.lock().unwrap().push(str_cmd);
tx.send((self, "OK".to_string()))
});
Box::new(rx.map_err(|_| io::Error::new(io::ErrorKind::NotFound, "Test")))
}
}
struct Channels {
data: Channel,
}
impl Channels {
fn run_command(self) -> Box<Future<Item = (), Error = io::Error>> {
Box::new(
self.data
.clone()
.command("AT+CMEE=2")
.and_then(|(channel, cmee_answer)| {
assert_eq!(cmee_answer, "OK");
channel.command("AT+CREG=0")
})
.map(|(channel, creg_answer)| {
assert_eq!(creg_answer, "OK");
let ops = channel.ops();
let ops = ops.lock().unwrap();
assert_eq!(*ops, ["AT+CMEE=2", "AT+CREG=0"]);
}),
)
}
}
fn main() {
let mut core = Core::new().expect("Core should be created");
let channels = Channels {
data: Channel {
operations: Arc::new(Mutex::new(vec![])),
},
};
let result = core.run(channels.run_command()).expect("Should've worked");
println!("{:?}", result);
}
I'm trying to implement a retry in a client built with Hyper v0.11, but I can't find a way to reuse a request for different attempts:
#[macro_use]
extern crate hyper;
extern crate futures;
extern crate tokio_core;
use futures::Future;
use hyper::{Client, Body, Uri, StatusCode};
use hyper::server::{Request, Response};
use hyper::client::HttpConnector;
use hyper::Get;
use tokio_core::reactor::Core;
fn main() {
let mut core = Core::new().expect("Event Loop");
let handle = core.handle();
let client = Client::new(&handle.clone());
// Request
let json = r#"{"user":"Peter"}"#;
let mut req: Request<Body> = Request::new(Post, "http://localhost:8080/create/user".parse().unwrap());
req.headers_mut().set(ContentType::json());
req.headers_mut().set(ContentLength(json.len() as u64));
req.set_body(json);
dispatch_request(&client, req, 2);
}
fn clone_req(req: &Request) -> Request {
let mut new_req = Request::new(req.method().clone(), req.uri().clone());
new_req.headers_mut().extend(req.headers().iter());
new_req.set_body(req.body()); // <------- here the error occur!
new_req
}
fn dispatch_request(
client: &Client<HttpConnector, Body>,
req: Request<Body>,
n_retry: u32,
) -> Box<Future<Error = hyper::Error, Item = Response>> {
println!("Attemp {}", n_retry);
let max_retry = 3;
let client_clone = client.clone();
let clone_req = clone_req(&req);
let resp = client.request(req).then(move |result| match result {
Ok(client_resp) => {
if client_resp.status() == hyper::StatusCode::Ok {
Box::new(futures::future::ok(client_resp))
} else if n_retry < max_retry {
dispatch_request(&client_clone, clone_req, max_retry + 1)
} else {
Box::new(futures::future::ok(
Response::new().with_status(StatusCode::ServiceUnavailable),
))
}
}
Err(e) => {
println!("Connection error: {:?}", &e);
Box::new(futures::future::ok(
Response::new().with_status(StatusCode::ServiceUnavailable),
))
}
});
Box::new(resp)
}
This is the compilation error:
error[E0507]: cannot move out of borrowed content
--> src/main.rs:28:22
|
28 | new_req.set_body(req.body());
| ^^^ cannot move out of borrowed content
The error is clear, but I don't know how to fix it.
An option is to use the tokio-retry crate. I only tried with hyper v0.12 though.
Why not put retry in main loop? Note you also need to do core.run somewhere.
loop {
let req = Request::new(Get, "http://www.google.com".parse().unwrap());
let resp = dispatch_request(&client, req, );
if let Ok(_) = resp.wait() {
break
}
}
I'm going to have multiple functions that all need access to one main socket.
Would it better to:
Pass this socket to each function that needs access to it
Have a globally accessible socket
Can someone provide an example of the best way to do this?
I come from a Python/Nim background where things like this are easily done.
Edit:
How can I pass a socket as an arg to a function being called within a thread.
Ex.
fn main() {
let mut s = BufferedStream::new((TcpStream::connect(server).unwrap()));
let thread = Thread::spawn(move || {
func1(s, arg1, arg2);
});
while true {
func2(s, arg1);
}
}
Answer for updated question
We can use TcpStream::try_clone:
use std::io::Read;
use std::net::{TcpStream, Shutdown};
use std::thread;
fn main() {
let mut stream = TcpStream::connect("127.0.0.1:34254").unwrap();
let stream2 = stream.try_clone().unwrap();
let _t = thread::spawn(move || {
// close this stream after one second
thread::sleep_ms(1000);
stream2.shutdown(Shutdown::Read).unwrap();
});
// wait for some data, will get canceled after one second
let mut buf = [0];
stream.read(&mut buf).unwrap();
}
Original answer
It's usually (let's say 99.9% of the time) a bad idea to have any global mutable state, if you can help it. Just do as you said: pass the socket to the functions that need it.
use std::io::{self, Write};
use std::net::TcpStream;
fn send_name(stream: &mut TcpStream) -> io::Result<()> {
stream.write(&[42])?;
Ok(())
}
fn send_number(stream: &mut TcpStream) -> io::Result<()> {
stream.write(&[1, 2, 3])?;
Ok(())
}
fn main() {
let mut stream = TcpStream::connect("127.0.0.1:31337").unwrap();
let r = send_name(&mut stream).and_then(|_| send_number(&mut stream));
match r {
Ok(..) => println!("Yay, sent!"),
Err(e) => println!("Boom! {}", e),
}
}
You could also pass the TcpStream to a struct that manages it, and thus gives you a place to put similar methods.
use std::io::{self, Write};
use std::net::TcpStream;
struct GameService {
stream: TcpStream,
}
impl GameService {
fn send_name(&mut self) -> io::Result<()> {
self.stream.write(&[42])?;
Ok(())
}
fn send_number(&mut self) -> io::Result<()> {
self.stream.write(&[1, 2, 3])?;
Ok(())
}
}
fn main() {
let stream = TcpStream::connect("127.0.0.1:31337").unwrap();
let mut service = GameService { stream: stream };
let r = service.send_name().and_then(|_| service.send_number());
match r {
Ok(..) => println!("Yay, sent!"),
Err(e) => println!("Boom! {}", e),
}
}
None of this is really Rust-specific, these are generally-applicable programming practices.