Making parallel requests to Redis with Fred - rust

I'm a complete Rust newbie and just tried to convert a simple microservice to Rust. It needs to do many parallel Redis requests for HTTP requests it gets and I'm a bit puzzled with the language syntax. I'm trying to do multiple Redis queries in parallel in an actix-web handler. I have the following type for the Redis GET function in Fred:
fn get<R, K>(&self, key: K) -> AsyncResult<R>
where
R: FromRedis + Unpin + Send,
K: Into<RedisKey>,
docs here: https://docs.rs/fred/5.2.0/fred/interfaces/trait.KeysInterface.html#method.get
In my own code I would then have a for-loop like this:
let resp_futures = vec!{};
for key in keys.iter() {
resp_futures.push(state.redis.get(key));
}
let resps = join_all(resp_futures).await;
Each Redis query should basically return an Option. However, this doesn't work due to some issues with type inference. Any ideas what's the correct way to send parallel Redis requests with the Fred Redis library? The complete server with some unnecessary stuff removed is the following:
use actix_web::{get, post, web, App, HttpResponse, HttpServer, Responder};
use actix_web::http::header::ContentType;
use serde::Deserialize;
use std::env;
use std::sync::Arc;
use fred::prelude::*;
use fred::pool::RedisPool;
use futures::future::join_all;
//
// Server configuration
//
fn get_redis_url() -> String {
match env::var("REDIS_URL") {
Err(_) => panic!("REDIS_URL environment variable not set."),
Ok(s) => s
}
}
struct AppState {
redis: Arc<RedisPool>
}
//
// Request parsing
//
fn get_redis_keys_from_accounts(accounts: Option<&String>) -> Vec<String> {
match accounts {
None => vec!{},
Some(s) => s.split(",").map(|s| "sid:".to_owned() + s).collect()
}
}
fn get_redis_keys_from_app_accounts(accounts: Option<&String>) -> Vec<String> {
match accounts {
None => vec!{},
Some(s) => s.split(",").map(|s| "aid:".to_owned() + s).collect()
}
}
fn get_redis_keys(req: web::Form<RouteRequest>) -> Vec<String> {
let ids = get_redis_keys_from_accounts(req.ids.as_ref());
let accounts = get_redis_keys_from_app_accounts(req.accounts.as_ref());
let aliases = get_redis_keys_from_app_accounts(req.aliases.as_ref());
ids.into_iter().chain(accounts.into_iter()).chain(aliases.into_iter()).collect()
}
//
// Request handling
//
#[derive(Debug, Deserialize)]
struct RouteRequest {
ids: Option<String>,
accounts: Option<String>,
aliases: Option<String>
}
#[post("/v1/route")]
async fn route(state: web::Data<AppState>, req: web::Form<RouteRequest>) -> impl Responder {
let keys = get_redis_keys(req);
// TODO: Fix this!
let resp_futures = vec!{};
for key in keys.iter() {
resp_futures.push(state.redis.get(key));
}
let resps = join_all(resp_futures).await;
HttpResponse::Ok().content_type(ContentType::json()).body(r#"{"status": "ok"}"#)
}
#[actix_web::main]
async fn main() -> std::io::Result<()> {
println!("Connecting to Redis backend");
let url = get_redis_url();
let config = match RedisConfig::from_url(&url) {
Ok(x) => x,
Err(_) => panic!("Invalid redis URL")
};
let policy = ReconnectPolicy::default();
let redis = Arc::new(match RedisPool::new(config, 5) {
Ok(x) => x,
Err(_) => panic!("Unable to create Redis connection pool")
});
let _ = redis.connect(Some(policy));
println!("Starting HTTP server");
HttpServer::new(move || {
App::new()
.app_data(web::Data::new(AppState {redis: redis.clone()}))
.service(route)
})
.bind(("0.0.0.0", 8080))?
.run()
.await
}
Output of cargo check is :
error[E0698]: type inside `async fn` body must be known in this context
--> src/main.rs:76:39
|
76 | resp_futures.push(state.redis.get(key));
| ^^^ cannot infer type for type parameter `R` declared on the associated function `get`
|
note: the type is part of the `async fn` body because of this `await`
--> src/main.rs:78:39
|
78 | let resps = join_all(resp_futures).await;
| ^^^^^^

At line 70 you have to give a hint about the type which should be used. For a String the following line should be used:
resp_futures.push(state.redis.get::<String, _>(key));

Related

How to call struct method from axum server route?

I want to instantiate a struct instance and then call a method of that instance in an api route. Here's an example of what I want, but it results in an error:
use axum::{http::StatusCode, routing::get, Router, Server};
#[derive(Clone)]
struct Api {
name: String
}
impl Api {
async fn hello(&self) -> Result<String, StatusCode> {
Ok(format!("Hello {}!", self.name))
}
}
#[tokio::main]
async fn main() {
let api = Api { name: "Alice".to_owned() };
let app = Router::new()
.route("/hello-user", get(api.hello));
Server::bind(&([127, 0, 0, 1], 3000).into())
.serve(app.into_make_service())
.await
.unwrap();
}
error[E0615]: attempted to take value of method `hello` on type `Api`
--> src\main.rs:19:39
|
19 | .route("/hello-user", get(api.hello));
| ^^^^^ method, not a field
I have attempted to get around this by defining a function which calls the instance method:
let hello_method = move || async {
match api.hello().await {
Ok(response) => response,
Err(_) => "error".to_owned(),
}
};
let app = Router::new()
.route("/hello-user", get(hello_method));
However, with this I get a "lifetime may not live long enough" error. How should I go about calling an instance method from an axum server route?
You can move api into the closure and then into the future:
let hello_method = move || async move {
match api.hello().await {
Ok(response) => response,
Err(_) => "error".to_owned(),
}
};
or with using of nightly feature #![feature(async_closure)]:
let hello_method = async move || {
match api.hello().await {
Ok(response) => response,
Err(_) => "error".to_owned(),
}
};

How can I get an Axum Handler function to return a Vec?

I'm learning how to use Axum with SQLx starting with this example. The basic example works, but I have problem trying to move forward. I am working with a simple database table as shown below:
todo | description
--------+--------------
todo_1 | doing todo 1
todo_2 | doing todo 2
todo_3 | doing todo 3
I am trying to simply get back "SELECT * FROM todos", but I am getting an error. I think I am getting the return of the Result type wrong but I am not sure what to do next. The entirety of main.rs is shown below.
//! Example of application using <https://github.com/launchbadge/sqlx>
//!
//! Run with
//!
//! ```not_rust
//! cd examples && cargo run -p example-sqlx-postgres
//! ```
//!
//! Test with curl:
//!
//! ```not_rust
//! curl 127.0.0.1:3000
//! curl -X POST 127.0.0.1:3000
//! ```
use axum::{
async_trait,
extract::{Extension, FromRequest, RequestParts},
http::StatusCode,
routing::get,
Router,
};
use sqlx::postgres::{PgPool, PgPoolOptions, PgRow};
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};
use std::{net::SocketAddr, time::Duration};
#[tokio::main]
async fn main() {
tracing_subscriber::registry()
.with(tracing_subscriber::EnvFilter::new(
std::env::var("RUST_LOG").unwrap_or_else(|_| "example_tokio_postgres=debug".into()),
))
.with(tracing_subscriber::fmt::layer())
.init();
let db_connection_str = std::env::var("DATABASE_URL")
.unwrap_or_else(|_| "postgres://postgres:postgres#localhost".to_string());
// setup connection pool
let pool = PgPoolOptions::new()
.max_connections(5)
.connect_timeout(Duration::from_secs(3))
.connect(&db_connection_str)
.await
.expect("can connect to database");
// build our application with some routes
let app = Router::new()
.route(
"/",
get(using_connection_pool_extractor).post(using_connection_extractor),
)
.layer(Extension(pool));
// run it with hyper
let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
tracing::debug!("listening on {}", addr);
axum::Server::bind(&addr)
.serve(app.into_make_service())
.await
.unwrap();
}
// we can extract the connection pool with `Extension`
async fn using_connection_pool_extractor(
Extension(pool): Extension<PgPool>,
) -> Result<Vec<String>, (StatusCode, String)> {
sqlx::query_scalar("select * from todos")
.fetch_one(&pool)
.await
.map_err(internal_error)
}
// we can also write a custom extractor that grabs a connection from the pool
// which setup is appropriate depends on your application
struct DatabaseConnection(sqlx::pool::PoolConnection<sqlx::Postgres>);
#[async_trait]
impl<B> FromRequest<B> for DatabaseConnection
where
B: Send,
{
type Rejection = (StatusCode, String);
async fn from_request(req: &mut RequestParts<B>) -> Result<Self, Self::Rejection> {
let Extension(pool) = Extension::<PgPool>::from_request(req)
.await
.map_err(internal_error)?;
let conn = pool.acquire().await.map_err(internal_error)?;
Ok(Self(conn))
}
}
async fn using_connection_extractor(
DatabaseConnection(conn): DatabaseConnection,
) -> Result<String, (StatusCode, String)> {
let mut conn = conn;
sqlx::query_scalar("select 'hello world from pg'")
.fetch_one(&mut conn)
.await
.map_err(internal_error)
}
/// Utility function for mapping any error into a `500 Internal Server Error`
/// response.
fn internal_error<E>(err: E) -> (StatusCode, String)
where
E: std::error::Error,
{
(StatusCode::INTERNAL_SERVER_ERROR, err.to_string())
}
Compared to the example, I changed this function so that it returns a Vec<String> instead of a plain String, but I get a compiler error:
async fn using_connection_pool_extractor(
Extension(pool): Extension<PgPool>,
) -> Result<Vec<String>, (StatusCode, String)> {
sqlx::query_scalar("select * from todos")
.fetch_one(&pool)
.await
.map_err(internal_error)
}
error[E0277]: the trait bound `fn(Extension<Pool<sqlx::Postgres>>) -> impl Future<Output = Result<Vec<String>, (StatusCode, String)>> {using_connection_pool_extractor}: Handler<_, _>` is not satisfied
--> src/main.rs:52:17
|
52 | get(using_connection_pool_extractor).post(using_connection_extractor),
| --- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Handler<_, _>` is not implemented for `fn(Extension<Pool<sqlx::Postgres>>) -> impl Future<Output = Result<Vec<String>, (StatusCode, String)>> {using_connection_pool_extractor}`
| |
| required by a bound introduced by this call
|
= help: the trait `Handler<T, ReqBody>` is implemented for `axum::handler::Layered<S, T>`
note: required by a bound in `axum::routing::get`
I am not sure what this error is suggesting or if it is even related to the actual problem.
Try using axum::Json:
async fn using_connection_pool_extractor(
Extension(pool): Extension<PgPool>,
) -> Result<axum::Json<Vec<String>>, (StatusCode, String)> {
sqlx::query_scalar("select * from todos")
.fetch_one(&pool)
.await
.map(|todos| axum::Json(todos))
.map_err(internal_error)
}
The reason why is that there's no implementation of the IntoResponse trait for Vec<T>. Here's a longer answer by Axum's author: https://users.rust-lang.org/t/axum-error-handling-trait-question/65530

What lifetimes and bounds are needed to generalize this async code? [duplicate]

This question already has answers here:
How to fix lifetime error when function returns a serde Deserialize type?
(2 answers)
Why do Rust lifetimes matter when I move values into a spawned Tokio task?
(1 answer)
Closed 1 year ago.
I have this websocket code that uses tokio and serde here:
use async_once::AsyncOnce;
use common_wasm::models::status::{CommandMessage, StatusMessage};
use futures_util::{SinkExt, StreamExt};
use lazy_static::lazy_static;
use std::{collections::VecDeque, net::SocketAddr};
use tokio::{
net::{TcpListener, TcpStream}, sync::{broadcast, mpsc}
};
use tokio_tungstenite::{
accept_async, tungstenite::{Error, Message, Result}
};
use tracing::*;
// https://stackoverflow.com/questions/67650879/rust-lazy-static-with-async-await
lazy_static! {
pub static ref STATUS_REPORTER: AsyncOnce<StatusWs> = AsyncOnce::new(async {
info!("Init lazy static WS");
let server = StatusWs::init("ws://localhost:44444").await;
server
});
}
use StatusMessage as SenderType;
use CommandMessage as ReceiveType;
pub struct StatusWs {
buf: VecDeque<ReceiveType>,
rx_client_msg: mpsc::Receiver<ReceiveType>,
tx_server_msg: broadcast::Sender<SenderType>,
}
impl StatusWs {
pub async fn init(addr: &str) -> StatusWs {
info!("Init Status WS on {}", addr);
let listener = TcpListener::bind(&addr).await.expect("Can't listen");
// Clients producting to server, they use the tx to send and server uses the rx to read
let (tx_client_msg, rx_client_msg) = mpsc::channel::<ReceiveType>(32);
// spmc for server to broadcast status to listeners. Server uses tx to send and client uses rx to read
let (tx_server_msg, _rx_server_msg) = broadcast::channel::<SenderType>(10);
let tx_server_2 = tx_server_msg.clone();
tokio::spawn(async move {
while let Ok((stream, peer)) = listener.accept().await {
info!("Peer address connected: {}", peer);
let tx_client = tx_client_msg.clone();
let rx_server = tx_server_msg.subscribe();
tokio::spawn(async move {
accept_connection(peer, stream, tx_client, rx_server).await;
});
}
});
StatusWs { buf: VecDeque::new(), rx_client_msg, tx_server_msg: tx_server_2 }
}
pub async fn reportinfo(&self, msg: &SenderType) {
let my_msg = msg.clone();
match &self.tx_server_msg.send(my_msg) {
Ok(_size) => {
//trace!("Server Sending OK {}", size)
},
Err(_err) => {
//trace!("Server Sending ERR {:?}", err)
},
}
}
pub async fn next(&mut self) -> Result<Option<ReceiveType>> {
loop {
// If buffer contains data, we can directly return it.
if let Some(data) = self.buf.pop_front() {
return Ok(Some(data));
}
// Fetch new response if buffer is empty.
let response = self.next_response().await?;
// Handle the response, possibly adding to the buffer
self.handle_response(response)?;
}
}
async fn next_response(&mut self) -> Result<ReceiveType> {
loop {
tokio::select! { // TODO don't need select if there's only one thing?
Some(msg) = self.rx_client_msg.recv() => {
return Ok(msg)
},
}
}
}
fn handle_response(&mut self, response: ReceiveType) -> Result<()> {
self.buf.push_back(response);
Ok(())
}
}
async fn accept_connection(peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, rx_server: broadcast::Receiver<SenderType>) {
info!("Accepting connection from {}", peer);
if let Err(e) = handle_connection(peer, stream, tx_client, rx_server).await {
match e {
Error::ConnectionClosed | Error::Protocol(_) | Error::Utf8 => error!("Connection closed"),
err => error!("Error processing connection: {}", err),
}
}
}
async fn handle_connection(
_peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, mut rx_server: broadcast::Receiver<SenderType>,
) -> Result<()> {
let ws_stream = accept_async(stream).await.expect("Failed to accept");
let (mut ws_sender, mut ws_receiver) = ws_stream.split();
loop {
tokio::select! {
remote_msg = ws_receiver.next() => {
match remote_msg {
Some(msg) => {
let msg = msg?;
match msg {
Message::Text(resptxt) => {
match serde_json::from_str::<ReceiveType>(&resptxt) {
Ok(cmd) => { let _ = tx_client.send(cmd).await; },
Err(err) => error!("Error deserializing: {}", err),
}
},
Message::Close(_) => break,
_ => { },
}
}
None => break,
}
}
Ok(msg) = rx_server.recv() => {
match serde_json::to_string(&msg) {
Ok(txt) => ws_sender.send(Message::Text(txt)).await?,
Err(_) => todo!(),
}
}
}
}
Ok(())
}
The sender and receiver types are simple (simple types all the way down):
use std::{collections::BTreeMap, fmt::Debug};
use serde::{Deserialize, Serialize};
#[derive(Default, Clone, Debug, Serialize, Deserialize)]
pub struct StatusMessage {
pub name: String,
pub entries: BTreeMap<i32, GuiEntry>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CommandMessage {
pub sender: String,
pub entryid: i32,
pub command: GuiValue,
}
Now I want to generalize the code so that I can create a struct that takes some other kind of Sender and Receiver type. Yes, I could just change the aliases, but I want to be able to use the generic type arguments rather than duplicate the whole file. The problem is as I follow the suggestions from the compiler, I end up in a place where I don't know what to do next. It's telling me resptext does not live long enough:
`resptxt` does not live long enough
borrowed value does not live long enoughrust cE0597
status_ws.rs(133, 29): `resptxt` dropped here while still borrowed
status_ws.rs(115, 28): lifetime `'a` defined here
status_ws.rs(129, 39): argument requires that `resptxt` is borrowed for `'a`
Here's what I have thus far:
use async_once::AsyncOnce;
use common_wasm::models::status::{CommandMessage, StatusMessage};
use futures_util::{SinkExt, StreamExt};
use lazy_static::lazy_static;
use serde::{Serialize, Deserialize};
use std::{collections::VecDeque, net::SocketAddr};
use tokio::{
net::{TcpListener, TcpStream}, sync::{broadcast, mpsc}
};
use tokio_tungstenite::{
accept_async, tungstenite::{Error, Message, Result}
};
use tracing::*;
// https://stackoverflow.com/questions/67650879/rust-lazy-static-with-async-await
lazy_static! {
pub static ref STATUS_REPORTER: AsyncOnce<StatusWs<CommandMessage, StatusMessage>> = AsyncOnce::new(async {
info!("Init lazy static WS");
let server = StatusWs::init("ws://localhost:44444").await;
server
});
}
// use StatusMessage as SenderType;
// use CommandMessage as ReceiveType;
pub struct StatusWs<ReceiveType, SenderType> {
buf: VecDeque<ReceiveType>,
rx_client_msg: mpsc::Receiver<ReceiveType>,
tx_server_msg: broadcast::Sender<SenderType>,
}
impl <'a, ReceiveType: Deserialize<'a> + Send, SenderType: Serialize + Clone + Send + Sync> StatusWs <ReceiveType, SenderType> {
pub async fn init(addr: &str) -> StatusWs<ReceiveType, SenderType> {
info!("Init Status WS on {}", addr);
let listener = TcpListener::bind(&addr).await.expect("Can't listen");
// Clients producting to server, they use the tx to send and server uses the rx to read
let (tx_client_msg, rx_client_msg) = mpsc::channel::<ReceiveType>(32);
// spmc for server to broadcast status to listeners. Server uses tx to send and client uses rx to read
let (tx_server_msg, _rx_server_msg) = broadcast::channel::<SenderType>(10);
let tx_server_2 = tx_server_msg.clone();
tokio::spawn(async move {
while let Ok((stream, peer)) = listener.accept().await {
info!("Peer address connected: {}", peer);
let tx_client = tx_client_msg.clone();
let rx_server = tx_server_msg.subscribe();
tokio::spawn(async move {
accept_connection(peer, stream, tx_client, rx_server).await;
});
}
});
StatusWs { buf: VecDeque::new(), rx_client_msg, tx_server_msg: tx_server_2 }
}
pub async fn reportinfo(&self, msg: &SenderType) {
let my_msg = msg.clone();
match &self.tx_server_msg.send(my_msg) {
Ok(_size) => {
//trace!("Server Sending OK {}", size)
},
Err(_err) => {
//trace!("Server Sending ERR {:?}", err)
},
}
}
pub async fn next(&mut self) -> Result<Option<ReceiveType>> {
loop {
// If buffer contains data, we can directly return it.
if let Some(data) = self.buf.pop_front() {
return Ok(Some(data));
}
// Fetch new response if buffer is empty.
let response = self.next_response().await?;
// Handle the response, possibly adding to the buffer
self.handle_response(response)?;
}
}
async fn next_response(&mut self) -> Result<ReceiveType> {
loop {
tokio::select! { // TODO don't need select if there's only one thing?
Some(msg) = self.rx_client_msg.recv() => {
return Ok(msg)
},
}
}
}
fn handle_response(&mut self, response: ReceiveType) -> Result<()> {
self.buf.push_back(response);
Ok(())
}
}
async fn accept_connection<'a, ReceiveType: Deserialize<'a>, SenderType: Clone + Serialize>(peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, rx_server: broadcast::Receiver<SenderType>) {
info!("Accepting connection from {}", peer);
if let Err(e) = handle_connection(peer, stream, tx_client, rx_server).await {
match e {
Error::ConnectionClosed | Error::Protocol(_) | Error::Utf8 => error!("Connection closed"),
err => error!("Error processing connection: {}", err),
}
}
}
async fn handle_connection<'a, ReceiveType: Deserialize<'a>, SenderType: Clone + Serialize>(
_peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, mut rx_server: broadcast::Receiver<SenderType>,
) -> Result<()> {
let ws_stream = accept_async(stream).await.expect("Failed to accept");
let (mut ws_sender, mut ws_receiver) = ws_stream.split();
loop {
tokio::select! {
remote_msg = ws_receiver.next() => {
match remote_msg {
Some(msg) => {
let msg = msg?;
match msg {
Message::Text(resptxt) => {
match serde_json::from_str::<ReceiveType>(&resptxt) {
Ok(cmd) => { let _ = tx_client.send(cmd).await; },
Err(err) => error!("Error deserializing: {}", err),
}
},
Message::Close(_) => break,
_ => { },
}
}
None => break,
}
}
Ok(msg) = rx_server.recv() => {
match serde_json::to_string(&msg) {
Ok(txt) => ws_sender.send(Message::Text(txt)).await?,
Err(_) => todo!(),
}
}
}
}
Ok(())
}
I think there's some confusion about the necessary lifetimes and bounds, in particular the lifetime on the Deserializer from Serde and the Send/Sync auto trait markers on the message types.
In any case, it seems a bit brute force to just copy the whole original file and change out the aliases, which would definitely work, when it seems there's some sort of useful lesson here.
You should use serde::de::DeserializeOwned instead of Deserialize<'a>.
The Deserialize trait takes a lifetime parameter to support zero-cost deserialization, but you can't take advantage of that since the source, resptxt, is a transient value that isn't persisted anywhere. The DeserializeOwned trait can be used to constrain that the deserialized type does not keep references to the source and can therefore be used beyond it.
After fixing that, you'll get errors that ReceiveType and SenderType must be 'static to be used in a tokio::spawn'd task. Adding that constraint finally makes your code compile.
See the full compiling code on the playground for brevity.

How to print a response body in actix_web middleware?

I'd like to write a very simple middleware using actix_web framework but it's so far beating me on every front.
I have a skeleton like this:
let result = actix_web::HttpServer::new(move || {
actix_web::App::new()
.wrap_fn(move |req, srv| {
srv.call(req).map(move|res| {
println!("Got response");
// let s = res.unwrap().response().body();
// ???
res
})
})
})
.bind("0.0.0.0:8080")?
.run()
.await;
and I can access ResponseBody type via res.unwrap().response().body() but I don't know what can I do with this.
Any ideas?
This is an example of how I was able to accomplish this with 4.0.0-beta.14:
use std::cell::RefCell;
use std::pin::Pin;
use std::rc::Rc;
use std::collections::HashMap;
use std::str;
use erp_contrib::{actix_http, actix_web, futures, serde_json};
use actix_web::dev::{Service, ServiceRequest, ServiceResponse, Transform};
use actix_web::{HttpMessage, body, http::StatusCode, error::Error ,HttpResponseBuilder};
use actix_http::{h1::Payload, header};
use actix_web::web::{BytesMut};
use futures::future::{ok, Future, Ready};
use futures::task::{Context, Poll};
use futures::StreamExt;
use crate::response::ErrorResponse;
pub struct UnhandledErrorResponse;
impl<S: 'static> Transform<S, ServiceRequest> for UnhandledErrorResponse
where
S: Service<ServiceRequest, Response = ServiceResponse, Error = Error>,
S::Future: 'static,
{
type Response = ServiceResponse;
type Error = Error;
type Transform = UnhandledErrorResponseMiddleware<S>;
type InitError = ();
type Future = Ready<Result<Self::Transform, Self::InitError>>;
fn new_transform(&self, service: S) -> Self::Future {
ok(UnhandledErrorResponseMiddleware { service: Rc::new(RefCell::new(service)), })
}
}
pub struct UnhandledErrorResponseMiddleware<S> {
service: Rc<RefCell<S>>,
}
impl<S> Service<ServiceRequest> for UnhandledErrorResponseMiddleware<S>
where
S: Service<ServiceRequest, Response = ServiceResponse, Error = Error> + 'static,
S::Future: 'static,
{
type Response = ServiceResponse;
type Error = Error;
type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>>>>;
fn poll_ready(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.service.poll_ready(cx)
}
fn call(&self, mut req: ServiceRequest) -> Self::Future {
let svc = self.service.clone();
Box::pin(async move {
/* EXTRACT THE BODY OF REQUEST */
let mut request_body = BytesMut::new();
while let Some(chunk) = req.take_payload().next().await {
request_body.extend_from_slice(&chunk?);
}
let mut orig_payload = Payload::empty();
orig_payload.unread_data(request_body.freeze());
req.set_payload(actix_http::Payload::from(orig_payload));
/* now process the response */
let res: ServiceResponse = svc.call(req).await?;
let content_type = match res.headers().get("content-type") {
None => { "unknown"}
Some(header) => {
match header.to_str() {
Ok(value) => {value}
Err(_) => { "unknown"}
}
}
};
return match res.response().error() {
None => {
Ok(res)
}
Some(error) => {
if content_type.to_uppercase().contains("APPLICATION/JSON") {
Ok(res)
} else {
let error = error.to_string();
let new_request = res.request().clone();
/* EXTRACT THE BODY OF RESPONSE */
let _body_data =
match str::from_utf8(&body::to_bytes(res.into_body()).await?){
Ok(str) => {
str
}
Err(_) => {
"Unknown"
}
};
let mut errors = HashMap::new();
errors.insert("general".to_string(), vec![error]);
let new_response = match ErrorResponse::new(&false, errors) {
Ok(response) => {
HttpResponseBuilder::new(StatusCode::BAD_REQUEST)
.insert_header((header::CONTENT_TYPE, "application/json"))
.body(serde_json::to_string(&response).unwrap())
}
Err(_error) => {
HttpResponseBuilder::new(StatusCode::BAD_REQUEST)
.insert_header((header::CONTENT_TYPE, "application/json"))
.body("An unknown error occurred.")
}
};
Ok(ServiceResponse::new(
new_request,
new_response
))
}
}
}
})
}
}
The extraction of the Request Body is straightforward and similar to how Actix example's illustrate. However, with the update to version Beta 14, pulling the bytes directly from AnyBody has changed with the introduction of BoxedBody. Fortunately, I found a utility function body::to_bytes (use actix_web::body::to_bytes) which does a good job. It's current implementation looks like this:
pub async fn to_bytes<B: MessageBody>(body: B) -> Result<Bytes, B::Error> {
let cap = match body.size() {
BodySize::None | BodySize::Sized(0) => return Ok(Bytes::new()),
BodySize::Sized(size) => size as usize,
// good enough first guess for chunk size
BodySize::Stream => 32_768,
};
let mut buf = BytesMut::with_capacity(cap);
pin!(body);
poll_fn(|cx| loop {
let body = body.as_mut();
match ready!(body.poll_next(cx)) {
Some(Ok(bytes)) => buf.extend_from_slice(&*bytes),
None => return Poll::Ready(Ok(())),
Some(Err(err)) => return Poll::Ready(Err(err)),
}
})
.await?;
Ok(buf.freeze())
}
which I believe should be fine to extract the body in this way, as the body is extracted from the body stream by to_bytes().
If someone has a better way, let me know, but it was a little bit of pain, and I only had recently determined how to do it in Beta 13 when it switched to Beta 14.
This particular example intercepts errors and rewrites them to JSON format if they're not already json format. This would be the case, as an example, if an error occurs outside of a handler, such as parsing JSON in the handler function itself _data: web::Json<Request<'a, LoginRequest>> and not in the handler body. Extracting the Request Body and Response Body is not necessary to accomplish the goal, and is just here for illustration.

How do I convert an async / standard library future to futures 0.1?

I want to use the async function to parse the inbound stream progressively, but actix-web requires impl Future<Item = HttpResponse, Error = Error> as the return value.
How can I convert the future returned by async function to what actix-web requires?
I'm using Rust 1.39 nightly and actix-web 1.0.7.
http_srv.rs :
use futures::compat::Stream01CompatExt;
use futures::future::{FutureExt, TryFutureExt};
use futures::stream::TryStreamExt;
use futures01::future::Future;
use futures01::stream::Stream;
use futures01::sync::mpsc; // for `try_next`
use actix_web::*;
use bytes::Bytes;
use futures_timer::Delay;
use std::time::Duration;
fn inbound(
req: HttpRequest,
stream: web::Payload,
) -> impl Future<Item = HttpResponse, Error = Error> {
let fut = async_inbound(&req, &stream);
fut.unit_error().boxed_local().compat() // <--- compliation error here.
}
async fn async_inbound(req: &HttpRequest, stream: &web::Payload) -> HttpResponse {
let mut compat_stream = stream.compat();
loop {
let result = compat_stream.try_next().await;
if let Err(e) = result {
warn!("Failed to read stream from {} : {}", req.path(), e);
break;
}
if let Ok(option) = result {
match option {
None => {
info!("Request ends");
break;
}
Some(data) => {
println!("{:?}", data);
}
}
}
}
HttpResponse::Ok().content_type("text/html").body("RESP")
}
pub fn start(port: u16) {
info!("Starting HTTP server listening at port {} ...", port);
let _ = HttpServer::new(|| {
App::new()
.wrap(middleware::DefaultHeaders::new().header(http::header::CACHE_CONTROL, "no-cache"))
.wrap(middleware::Logger::default())
.service(web::resource("/").route(web::put().to_async(inbound)))
})
.bind(format!("0.0.0.0:{}", port))
.expect(&format!("Unable to bind on port {}", port))
.run()
.expect("Failed to start HTTP server");
}
Cargo.toml:
dependencies]
log = "0.4.8"
env_logger = "0.6.2"
chrono = "0.4.8"
actix = "0.8.3"
bytes = "0.4.12"
actix-utils = "0.4.5"
futures-timer = "0.3"
futures01 = { package = "futures", version = "0.1", optional = false }
[dependencies.actix-web]
version = "1.0.7"
features = ["ssl"]
# https://rust-lang-nursery.github.io/futures-rs/blog/2019/04/18/compatibility-layer.html
# Rust’s futures ecosystem is currently split in two:
# On the one hand we have the vibrant ecosystem built around futures#0.1 with its many libraries working on stable Rust
# and on the other hand there’s std::future ecosystem with support for the ergonomic and powerful async/await language feature.
# To bridge the gap between these two worlds we have introduced a compatibility layer as part of the futures#0.3 extension to std::future.
[dependencies.futures-preview]
version = "0.3.0-alpha.18"
default-features = false
features = ["compat", "async-await", "nightly"]
Compilation Error:
error[E0271]: type mismatch resolving `<std::pin::Pin<std::boxed::Box<dyn core::future::future::Future<Output = std::result::Result<actix_http::response::Response, ()>>>> as core::future::future::Future>::Output == std::result::Result<_, actix_http::error::Error>`
--> src/http_server.rs:39:55
|
39 | fn inbound(req: HttpRequest, stream: web::Payload) -> impl Future<Item=HttpResponse, Error=Error> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected (), found struct `actix_http::error::Error`
|
= note: expected type `std::result::Result<actix_http::response::Response, ()>`
found type `std::result::Result<_, actix_http::error::Error>`
= note: required because of the requirements on the impl of `futures_core::future::TryFuture` for `std::pin::Pin<std::boxed::Box<dyn core::future::future::Future<Output = std::result::Result<actix_http::response::Response, ()>>>>`
= note: the return type of a function must have a statically known size
std::future -> future#0.1 conversion steps:
The future needs to be TryFuture (Output = Result<T, E>)
The future needs to be Unpin (you can use boxed combinator)
Finally, you can call the compat combinator
Your inbound function:
fn inbound(
req: HttpRequest,
stream: web::Payload,
) -> impl Future<Item = HttpResponse, Error = Error> {
let fut = async_inbound(&req, &stream);
fut.unit_error().boxed_local().compat()
}
The inbound function signature is fine, but the conversion isn't.
The async_inbound function isn't TryFuture (because of -> HttpResponse). You're trying to convert it with the unit_error combinator, but the result is Result<HttpResponse, ()> and you want Result<HttpResponse, Error>. Fixed inbound function:
fn inbound(
req: HttpRequest,
stream: web::Payload,
) -> impl Future<Item = HttpResponse, Error = Error> {
let fut = async_inbound(req, stream);
fut.boxed_local().compat()
}
Your async_inbound function:
async fn async_inbound(req: &HttpRequest, stream: &web::Payload) -> HttpResponse {
// ...
}
The first issue here is to replace -> HttpResponse with -> Result<HttpResponse>. Another problem is that you're passing reg and stream by reference. Move them as there's no need to take a reference and you'll need 'static. Fixed async_inbound function:
async fn async_inbound(req: HttpRequest, stream: web::Payload) -> Result<HttpResponse> {
let mut compat_stream = stream.compat();
while let Some(data) = compat_stream.try_next().await? {
println!("{:?}", data);
}
Ok(HttpResponse::Ok().content_type("text/html").body("RESP"))
}

Resources