Rust Iron Web Framework gives some phantom error - rust

I am using iron. Most of time like 99.* % all is good. But sometimes I get error like Error was: ErrorImpl { code: EofWhileParsingString/List/Object, line: 1, column: 8186 } or InvalidUnicodeCodePoint. I am printing request in log and when i try that request every thing goes well. I also have server written in Golang receiving same request and they never have parsing or json to MyStruct conversion problem.Please note Code would not compile as it is, missing imports, error::from and structure definition. Can not provide reproducible request logs as it only happens when serving lots on concurrent request but if single request is taken it works fine.
I have tried serde_json::from_reader, bodyparser crate and all have same issue.
extern crate serde;
extern crate serde_json;
extern crate iron;
use self::iron;
use self::iron::prelude::*;
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct MyStruct {
}
struct ResponseTime;
impl typemap::Key for ResponseTime {
type Value = u64;
}
#[derive(Debug)]
struct RequestBody;
impl typemap::Key for RequestBody {
type Value = RefCell<Vec<u8>>;
}
impl BeforeMiddleware for ResponseTime {
fn before(&self, req: &mut Request) -> IronResult<()> {
req.extensions.insert::<RequestBody>(RefCell::new(Vec::new()));
req.extensions.insert::<ResponseTime>(precise_time_ns());
Ok(())
}
}
impl AfterMiddleware for ResponseTime {
fn after(&self, req: &mut Request, res: Response) -> IronResult<Response> {
Ok(res)
}
fn catch(&self, req : &mut Request, err : IronError) -> IronResult<Response> {
let ref byte_req = *req.extensions.get::<RequestBody>()
.unwrap()
.borrow();
//just to make sure uft8 is not causing some issue.
let payload = unsafe {
str::from_utf8_unchecked(&byte_req)
};
//but when i send request body all comes good
error!("Error {} for Body {}", err, payload);
Err(err)
}
}
fn iron_handler(req : &mut Request) -> Result<Response, CustomError>{
let mut buffer = req.extensions.get::<server::RequestBody>()
.unwrap()
.borrow_mut();
req.body.read_to_end(&mut buffer)?;
// not seeing InvalidUnicodeCodePoint after this.
let payload = String::from_utf8_lossy(&buffer);
//some request throw error
let my_struct_obj : MyStruct = serde_json::from_str(&payload)?;
Ok(Response::with((iron::status::Ok, "Final Response")))
}
Need help to figure out how to identify problem. Intent of posting here is to see if someone had same issue or can see obvious problem with this. Appreciate everyone'e time do not expect to build and run with examples as can not provide them because of privacy.

Related

type mismatch resolving <impl std::future::Future as std::future::Future>::Output == std::result::Result<wasm_bindgen::JsValue, wasm_bindgen::JsValue>

I am trying to implement an API class using wasm_bindgen with asynchronous calls.
#![allow(non_snake_case)]
use std::future::Future;
use serde::{Deserialize, Serialize};
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsCast;
use wasm_bindgen_futures::JsFuture;
use js_sys::Promise;
use web_sys::{Request, RequestInit, RequestMode, Response};
use wasm_bindgen_futures::future_to_promise;
use crate::fetch_example::*;
#[wasm_bindgen]
#[derive(Debug, Serialize, Deserialize)]
pub struct API {
root: String,
}
#[wasm_bindgen]
impl API {
pub fn new(root: &str) -> Self {
Self {
root: root.to_string(),
}
}
pub fn getVersion(&self) -> Promise {
let url = format!("{}/version", self.root);
future_to_promise(async move {
_request_json(&url, "GET")
})
}
// other methods ...
}
// What is the correct returned type instead of Result<JsValue, JsValue> ???
async fn _request_json(url: &str, method: &str) -> Result<JsValue, JsValue> {
let mut opts = RequestInit::new();
opts.method(method);
opts.mode(RequestMode::Cors);
let request = Request::new_with_str_and_init(&url, &opts)?;
request.headers().set("Accept", "application/json")?;
let window = web_sys::window().unwrap();
let resp_value = JsFuture::from(window.fetch_with_request(&request)).await?;
let resp: Response = resp_value.dyn_into().unwrap();
let json = JsFuture::from(resp.json()?).await?;
Ok(json)
}
As it is seen, I took out the HTTP call into a separate private function _request_json to avoid code duplication in different API methods (they are supposed to be several, not getVersion only).
I took the base example of an HTTP call from here: https://rustwasm.github.io/wasm-bindgen/examples/fetch.html
Also I found that asynchronous calls of struct methods should be implemented with the help of future_to_promise: https://github.com/rustwasm/wasm-bindgen/issues/1858
So the issue is with the returned type by _request_json. I cannot guess it correctly.
When I compile the project I get the error:
type mismatch resolving `<impl std::future::Future as std::future::Future>::Output == std::result::Result<wasm_bindgen::JsValue, wasm_bindgen::JsValue>`
If I copy the body of _request_json inside of future_to_promise everything works fine.
What is the correct returned expression for _request_json to make the code works?
This construction:
async move {
_request_json(&url, "GET")
}
has the type impl Future<Output = impl Future<Output = Result<JsValue, JsValue>>>.
You can fix it like this:
future_to_promise(async move {
_request_json(&url, "GET").await
})
and probably also like this:
future_to_promise(_request_json(&url, "GET"))

How to solve this return type function error in Rust

I am creating a rust web application. I am new to this thing and I am trying to make API requests and pass a result form a request as Response to the web view. There are main.rs , route.rs and common.rs files. Basically main.rs file call relevant route and then route will call the function.
But, there is an error with a function in the route.rs file.
error[E0277]: the trait bound `fn() -> impl std::future::Future {<search::routes::get_token as actix_web::service::HttpServiceFactory>::register::get_token}: actix_web::handler::Factory<_, _, _>` is not satisfied
--> src\search\routes.rs:20:10
|
20 | async fn get_token() -> Result<String, reqwest::Error> {
| ^^^^^^^^^ the trait `actix_web::handler::Factory<_, _, _>` is not implemented for `fn() -> impl std::future::Future {<search::routes::get_token as actix_web::service::HttpServiceFactory>::register::get_token}`
How can I Fix this ??
route.rs
use crate::search::User;
use actix_web::{get, post, put, delete, web, HttpResponse, Responder};
use serde_json::json;
extern crate reqwest;
extern crate serde;
mod bfm;
mod common;
#[get("/token")]
async fn get_token() -> Result<String, reqwest::Error> {
let set_token = common::authenticate();
// let set_token = common::get_rust_posts();
return set_token.await;
//HttpResponse::Ok().json(set_token)
}
common.rs
extern crate reqwest;
use reqwest::header::HeaderMap;
use reqwest::header::AUTHORIZATION;
use reqwest::header::CONTENT_TYPE;
pub async fn authenticate() -> Result<String, reqwest::Error> {
fn construct_headers() -> HeaderMap {
let mut headers = HeaderMap::new();
headers.insert(AUTHORIZATION, "Basic bghythjuikyt==".parse().unwrap());
headers.insert(CONTENT_TYPE, "application/x-www-form-urlencoded".parse().unwrap());
assert!(headers.contains_key(AUTHORIZATION));
assert!(!headers.contains_key(CONTENT_TYPE));
headers
}
let client = reqwest::Client::new();
let reszr = client.post("https://api.test.com/auth/token")
.headers(construct_headers())
.body("grant_type=client_credentials")
.send()
.await?
.text()
.await;
return reszr;
}
First of all, please read: https://actix.rs/docs/errors/
--
Do you have implemented ResponseError for reqwest::Error?
You have to implement ResponseError trait because framework have to response with something in case when Result is Err.
Here is an example
impl ResponseError for Error {
fn error_response(&self) -> HttpResponse {
match self.kind_ref() {
ErrorKind::HttpResponse { msg, code, real } => {
println!("{}", real);
HttpResponse::build(code.to_owned()).json(json!({
"status": false,
"msg": msg
}))
}
_ => {
println!("{}", self);
HttpResponse::build(StatusCode::INTERNAL_SERVER_ERROR).finish()
}
}
}
}
Error in this case is my custom error that contains msg, code, real, but the logic should be similar in your case.
--
UPDATE: after #Masklinn comment about error[E0117]: only traits defined in the current crate can be implemented for arbitrary types
Code above works only on custom Error you own.
What you can to do is:
map reqwest::Error to Error that already implements ResponseError. You can check list of implementations here: https://docs.rs/actix-web/3.0.0-alpha.1/actix_web/trait.ResponseError.html#implementors
create custom Error that wraps reqwest::Error (and other errors in your code) and implement ResponseError on it (this is basicaly same/similar as first)

Futures - expected (), found struct futures::Map

I'm attempting to send a request off with Hyper, then deserialize it via JSON through Serde, but I cannot seem to wrap my head around futures and I'm receiving type mismatch errors stating expected (), found struct [put some odd struct here]. Nor can I wrap my head around the incredibly long and confusing error messages they spit out on each change. Here's my code:
extern crate futures;
extern crate hyper;
extern crate serde;
extern crate serde_json;
use futures::{
Future,
Stream,
future
};
use hyper::{
Body,
Client,
Response,
StatusCode,
Uri,
client::HttpConnector,
};
use serde::{ Deserialize };
use std::error::{ Error };
enum JsonError
{
RequestError(hyper::Error),
ResponseError(StatusCode),
DeserializeError(serde_json::Error),
}
fn get_json
<'t, T, F>
(client: &Client<HttpConnector>, uri: Uri)
-> impl Future<Item = T, Error = JsonError>
where
T : Deserialize<'t>
{
let f = client
.get(uri)
.map(|response|
{
let (parts, body) = response.into_parts();
if parts.status.is_success()
{
body
.fold(
vec![],
|mut accum, chunk|
{
accum.extend_from_slice(&*chunk);
Ok(accum)
}
)
.map(|v|
{
serde_json::from_slice::<T>(&v)
.map_err(|err| JsonError::DeserializeError(err))
})
}
future::err(JsonError::ResponseError(parts.status))
})
.map_err(|err| JsonError::RequestError(err));
return f;
}
I am completely lost and I think any advice could help at this point.
You have multiple errors stemming from logic issues when chaining futures. I have fixed its implementation, and it is available on the playground but I would strongly recommend you walk with me through what I changed.
But first, a recurring trend in your code:
#1: Future::map()'s return type
Future::map() allows you to change a future result of type T into type R, and it does so with the assumption that the transformation cannot fail (i.e. Fn(T) -> R). You've used map() multiple times in your code while returning either another Future, or a Result. Both are incorrect.
For Future chaining, and_then() allows you to perform the mapping fn(T) -> IntoFuture<Item = R> with error types remaining the same
For Result, convert them via future::result() into an already-executed future so you can also and_then()
#2: Errors
Errors do not convert themselves, especially not if you do not define their conversion methods. For this purpose, I've implemented From<hyper::Error> for your error type:
impl From<hyper::Error> for JsonError {
fn from(s: hyper::Error) -> JsonError {
JsonError::RequestError(s)
}
}
This allows you to use into() wherever the typechecker notices that it is possible to do so.
Do note, however, that because of hyper's own response type, err_into() gets the type checker confused, hence the explicit map(|r| r.into()).
#3: Deserialize and lifetimes
Deserialize<'t> is not the exact trait you were looking for. This trait implies that the entire object needs to live for lifetime 't. In a non-futures world, this would probably pass, but not in this case where the return object needs to be owned (hence the lifetime error you are getting).
for<'t> Deserialize<'t> is a completely different beast, and tells the compiler that this trait will have a lifetime 't, but will then be an owned object, or in other words, that the slice used to create the object will need to live for lifetime 't, not the entire returned object. Just what we need!
An additional nitpick: you really ought to separate the response parsing from the HTTP extraction in this function. As it stands right now, if I make a request over HTTPS, I will not be able to use your get_json() function, as my connector for hyper would then be TlsConnector<HttpConnector>. Problematic ;-)
The code:
use futures::{future, Future, Stream};
use hyper::{client::HttpConnector, Client, StatusCode, Uri};
use serde::Deserialize;
enum JsonError {
RequestError(hyper::Error),
ResponseError(StatusCode),
DeserializeError(serde_json::Error),
}
impl From<hyper::Error> for JsonError {
fn from(s: hyper::Error) -> JsonError {
JsonError::RequestError(s)
}
}
fn get_json<T, F>(
client: &Client<HttpConnector>,
uri: Uri,
) -> impl Future<Item = T, Error = JsonError>
where
T: for<'t> Deserialize<'t> + 'static,
{
client.get(uri).map_err(|e| e.into()).and_then(
|response| -> Box<dyn Future<Item = T, Error = JsonError>> {
let (parts, body) = response.into_parts();
match parts.status.is_success() {
true => Box::new(body
.map_err(|e| e.into())
.fold(
vec![],
|mut accum, chunk| -> Box<dyn Future<Item = Vec<u8>, Error = JsonError>>
{
accum.extend_from_slice(&*chunk);
Box::new(future::ok(accum))
}
)
.and_then(|v|
{
future::result(serde_json::from_slice::<T>(&v))
.map_err(|err| JsonError::DeserializeError(err))
})),
false => Box::new(future::err(JsonError::ResponseError(parts.status)))
}
},
)
}

Stop Rust from enforcing the serde::Deserialize trait on an error type

The code below is the beginnings of a small library I'm writing to talk to a web API. Users of the library will instantiate a client MyClient and access the web API through it. Here, I'm trying to get an access token from the API before making requests to it.
In get_new_access() I'm able to make the request and receive the JSON response. I then try to use serde to turn the response into an Access struct, and this is where the problems start.
I've created a library specific error enum MyError which can represent the JSON deserializing and reqwest errors that could occur within get_new_access(). However, when I go to compile I get the trait serde::Deserialize<'_> is not implemented for MyError. My understanding is that this is happening because in the case that I get one of the aforementioned errors, serde does not know how to deserialize it into an Access struct. Of course, I don't want it to do that at all, so my question is what should I do?
I've looked at various serde deserialize examples, but all of them seem to assume that they are running in a main function that can only return a serde error. If I put #[derive(Deserialize)] above MyError's declaration, then I get the same error, but it shifts to reqwest::Error and serde_json::Error instead.
use std::error;
use std::fmt;
extern crate chrono;
extern crate reqwest;
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;
use chrono::prelude::*;
use reqwest::Client;
pub struct MyClient {
access: Access,
token_expires: DateTime<Utc>,
}
#[derive(Deserialize, Debug)]
struct Access {
access_token: String,
expires_in: i64,
token_type: String,
}
fn main() {
let sc: MyClient = MyClient::new();
println!("{:?}", &sc.access);
}
impl MyClient {
pub fn new() -> MyClient {
let a: Access = MyClient::get_new_access().expect("Couldn't get Access");
let e: DateTime<Utc> = chrono::Utc::now(); //TODO
MyClient {
access: a,
token_expires: e,
}
}
fn get_new_access() -> Result<Access, MyError> {
let params = ["test"];
let client = Client::new();
let json = client
.post(&[""].concat())
.form(&params)
.send()?
.text()
.expect("Couldn't get JSON Response");
println!("{}", &json);
serde_json::from_str(&json)?
//let a = Access {access_token: "Test".to_string(), expires_in: 3600, token_type: "Test".to_string() };
//serde_json::from_str(&json)?
}
}
#[derive(Debug)]
pub enum MyError {
WebRequestError(reqwest::Error),
ParseError(serde_json::Error),
}
impl fmt::Display for MyError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "eRROR")
}
}
impl error::Error for MyError {
fn description(&self) -> &str {
"API internal error"
}
fn cause(&self) -> Option<&error::Error> {
// Generic error, underlying cause isn't tracked.
None
}
}
impl From<serde_json::Error> for MyError {
fn from(e: serde_json::Error) -> Self {
MyError::ParseError(e)
}
}
impl From<reqwest::Error> for MyError {
fn from(e: reqwest::Error) -> Self {
MyError::WebRequestError(e)
}
}
Playground link here.
Your first problem is that your fn get_new_access() -> Result<Access, MyError> expects a Result. But in here:
//...
serde_json::from_str(&json)?
}
because of using ?(try macro), you are trying to return Result's unwrapped value which is a subtype of serde::Deserialize<'_>. The compiler warns you about this Deserialize is not a Result. What you should do is just return the result without unwrapping it:
//...
serde_json::from_str(&json)
}
Or
//...
let access = serde_json::from_str(&json)?; // gets access or propagates error
Ok(access) //if no error return access in a Result
}
Then you will have a second problem because your function expects MyError in the Result while you are returning serde_json::Error with this call serde_json::from_str(&json). Luckily Result has the function map_err which maps the actual error type to your custom error type.
This code will solve your problem:
//...
serde_json::from_str(&json).map_err(MyError::ParseError)
}
For the request in the comment :
For example, if I change the web request line to let json = client.post("").form(&params).send().map_err(MyError::WebRequestError)?.text()?;,
is that better practice at all?
Yes but since text() returns a Result you need to map it's error as MyError too. Since both send and text has same error type(reqwest::Error) you can combine the results with and_then :
let json = client
.post(&[""].concat())
.form(&params)
.send()
.and_then(Response::text) //use reqwest::Response;
.map_err(MyError::WebRequestError)?;

Creating a hyper service with custom error type

I am trying to create a REST server using hyper. For robust error handling, I would prefer to have the service return a future with a custom error type that wraps hyper, Diesel, and other errors. Unfortunately, hyper::Response seems to hard-code a stream with error type hyper::error::Error, which conflicts with the error type I've defined for my service. I see a couple possible solutions:
Make my service return my custom error type by modifying hyper::Response, which seems hard.
Wrap non-hyper errors in a hyper::error::Error. This seems hacky.
Something else. It seems like I'm missing the "right" way to do this.
The following code shows what I think I want to do:
extern crate diesel;
extern crate futures;
extern crate hyper;
use futures::future::{ok, Future};
use hyper::StatusCode;
use hyper::server::{Request, Response, Service};
fn main() {
let address = "127.0.0.1:8080".parse().unwrap();
let server = hyper::server::Http::new()
.bind(&address, move || Ok(ApiService {}))
.unwrap();
server.run().unwrap();
}
pub struct ApiService;
impl Service for ApiService {
type Request = Request;
type Response = Response;
type Error = Error;
type Future = Box<Future<Item = Self::Response, Error = Self::Error>>;
fn call(&self, request: Request) -> Self::Future {
Box::new(ok(Response::new().with_status(StatusCode::Ok)))
}
}
#[derive(Debug)]
pub enum Error {
Request(hyper::Error),
DatabaseResult(diesel::result::Error),
DatabaseConnection(diesel::ConnectionError),
Other(String),
}
// omitted impl of Display, std::error::Error for brevity
This code results in a compiler error which I believe is because the bind function requires that the response type have a body that is a stream with error type hyper::error::Error:
error[E0271]: type mismatch resolving `<ApiService as hyper::client::Service>::Error == hyper::Error`
--> src/main.rs:14:10
|
14 | .bind(&address, move || Ok(ApiService {}))
| ^^^^ expected enum `Error`, found enum `hyper::Error`
|
= note: expected type `Error`
found type `hyper::Error`
Because the ultimate goal of the server is to return a response to the user, I found an acceptable solution to be to create a finalize function that converts errors encountered while processing a request into correctly formed responses and treats those errors as non-errors from hyper's perspective. I will need to flesh this idea out some (e.g. By passing along hyper errors as errors), but I believe the basic idea is sound.
The following code modifies the code in the question to do this:
extern crate diesel;
extern crate futures;
extern crate hyper;
#[macro_use]
extern crate serde_derive;
use futures::future::{ok, Future};
use hyper::StatusCode;
use hyper::server::{Request, Response, Service};
fn main() {
let address = "127.0.0.1:8080".parse().unwrap();
let server = hyper::server::Http::new()
.bind(&address, move || Ok(ApiService {}))
.unwrap();
server.run().unwrap();
}
fn finalize(result: Result<Response, Error>) -> FutureResult<Response, hyper::Error> {
match result {
Ok(response) => ok(response),
Err(error) => {
let response_body =
json!({"status": 500, "description": error.description()}).to_string();
ok(Response::new()
.with_status(StatusCode::InternalServerError)
.with_header(ContentLength(response_body.len() as u64))
.with_body(response_body))
}
}
}
pub struct ApiService;
impl Service for ApiService {
type Request = Request;
type Response = Response;
type Error = hyper::Error;
type Future = Box<Future<Item = Self::Response, Error = Self::Error>>;
fn call(&self, request: Request) -> Self::Future {
let response = Ok(Response::new().with_status(StatusCode::Ok));
Box::new(finalize(response))
}
}
#[derive(Debug)]
pub enum Error {
Request(hyper::Error),
DatabaseResult(diesel::result::Error),
DatabaseConnection(diesel::ConnectionError),
Other(String),
}
// omitted impl of Display, std::error::Error for brevity
You can implement the std::convert::From trait for your Error type. E.g. for the hyper::Error case:
impl From<hyper::Error> for Error {
fn from(error: hyper::Error) -> Self {
Error::Request(error)
}
}

Resources