Rust: Response error handling using reqwest - rust

Ok, so I'm very new to Rust and I'm trying to clumsily piece together a little CLI tool that makes http requests and handles the responses, by using tokio, clap, reqwest and serde.
The tool accepts a customer number as input and then it tries to fetch information about the customer. The customer may or may not have a FooBar in each country.
My code currently only works if I get a nice 200 response containing a FooBar. If I don't, the deserialization fails (naturally). (Edit: Actually, this initial assumption about the problem seems to be false, see comments below)
My aim is to only attempt the deserialization if I actually get a valid response.
How would I do that? I feel the need to see the code of a valid approach to understand this better.
Below is the entirety of my program.
use clap::Parser;
use reqwest::Response;
use serde::{Deserialize, Serialize};
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
let args: Cli = Cli::parse();
let client = reqwest::Client::new();
let countries = vec!["FR", "GB", "DE", "US"];
for country in countries.iter() {
let foo_bar : FooBar = client.get(
format!("http://example-service.com/countries/{}/customers/{}/foo_bar", country, args.customer_number))
.send()
.await?
.json()
.await?;
println!("{}", foo_bar.a_value);
}
Ok(())
}
#[derive(Debug, Serialize, Deserialize)]
struct FooBar {
a_value: String,
}
#[derive(Parser, Debug)]
struct Cli {
customer_number: i32,
}

There are a few ways to approach this issue, first of all you can split the json() deserialization from send().await, i.e.:
for country in countries.iter() {
let resp: reqwest::Response = client.get(
format!("http://example-service.com/countries/{}/customers/{}/foo_bar", country, args.customer_number))
.send()
.await?;
if resp.status() != reqwest::StatusCode::OK {
eprintln!("didn't get OK status: {}", resp.status());
} else {
let foo_bar = resp.json().await?;
println!("{}", foo_bar.a_value);
}
}
If you want to keep the response body around, you can extract it through let bytes = resp.bytes().await?; and pass bytes to serde_json::from_slice(&*bytes) for the deserialization attempt.
This can be useful if you have a set of expected error response bodies.

Related

Testing with Rocket and Diesel

I am trying to work on a web app with Diesel and Rocket, by following the rocket guide. I have not able to understand how to do testing of this app.
//in main.rs
#[database("my_db")]
struct PgDbConn(diesel::PgConnection);
#[post("/users", format="application/json", data="<user_info>")]
fn create_new_user(conn: PgDbConn, user_info: Json<NewUser>) {
use schema::users;
diesel::insert_into(users::table).values(&*user_info).execute(&*conn).unwrap();
}
fn main() {
rocket::ignite()
.attach(PgDbConn::fairing())
.mount("/", routes![create_new_user])
.launch();
}
// in test.rs
use crate::PgDbConn;
#[test]
fn test_user_creation() {
let rocket = rocket::ignite().attach(PgDbConn::fairing());
let client = Client::new(rocket).unwrap();
let response = client
.post("/users")
.header(ContentType::JSON)
.body(r#"{"username": "xyz", "email": "temp#abc.com"}"#)
.dispatch();
assert_eq!(response.status(), Status::Ok);
}
But this modifies the database. How can I make sure that the test does not alter the database.
I tried to create two database and use them in the following way(I am not sure if this is recommended)
#[cfg(test)]
#[database("test_db")]
struct PgDbConn(diesel::PgConnection);
#[cfg(not(test))]
#[database("live_db")]
struct PgDbConn(diesel::PgConnection);
Now I thought I can use the test_transaction method of the diesel::connection::Connection trait in the following way:-
use crate::PgDbConn;
#[test]
fn test_user_creation() {
// !!This statment is wrong as PgDbConn is an Fn object instead of a struct
// !!I am not sure how it works but it seems that this Fn object is resolved
// !!into struct only when used as a Request Guard
let conn = PgDbConn;
// Deref trait for PgDbConn is implemented, So I thought that dereferencing
// it will return a diesel::PgConnection
(*conn).test_transaction::<_, (), _>(|| {
let rocket = rocket::ignite().attach(PgDbConn::fairing());
let client = Client::new(rocket).unwrap();
let response = client
.post("/users")
.header(ContentType::JSON)
.body(r#"{"username": "Tushar", "email": "temp#abc.com"}"#)
.dispatch();
assert_eq!(response.status(), Status::Ok);
Ok(())
});
}
The above code obviously fails to compile. Is there a way to resolve this Fn object into the struct and obtain the PgConnection in it. And I am not even sure if this is the right to way to do things.
Is there a recommended way to do testing while using both Rocket and Diesel?
This will fundamentally not work as you imagined there, as conn will be a different connection than whatever rocket generates for you. The test_transaction pattern assumes that you use the same connection for everything.

Where does a variable passed to Reqwest's Result::read_to_string get the data from?

I am learning Rust and have been playing around with this example to perform an HTTP GET request and then display the data:
extern crate reqwest;
use std::io::Read;
fn run() -> Result<()> {
let mut res = reqwest::get("http://httpbin.org/get")?;
let mut body = String::new();
res.read_to_string(&mut body)?;
println!("Status: {}", res.status());
println!("Headers:\n{:#?}", res.headers());
println!("Body:\n{}", body);
Ok(())
}
I cannot understand how the variable body is actually ending up with the correct data. For headers and status, I can see the associated functions but for the body data it just uses read_to_string for the whole data?
The res object has a read_to_string() method which stores the response into the String that you pass it in
res.read_to_string(&mut body);
Edit: imported from my comment:
reqwest::Response 0.6.2 documentation states for Read for Response:
Read the body of the Response
which somehow seems missing from the documentation of the current version.

What's the easiest way to get the HTML output of an actix-web endpoint handler to be rendered properly?

I've defined an endpoint with actix-web like so:
#[derive(Deserialize)]
struct RenderInfo {
filename: String,
}
fn render(info: actix_web::Path<RenderInfo>) -> Result<String> {
// ...
}
App::new()
.middleware(middleware::Logger::Default())
.resource("/{filename}", |r| r.get().with(render))
The problem I've run into is that the raw HTML gets displayed in the browser rather than being rendered. I assume the content-type is not being set properly.
Most of the actix-web examples I saw used impl Responder for the return type, but I wasn't able to figure out how to fix the type inference issues that created. The reason seems to have something to do with file operations returning a standard failure::Error-based type. It looks like actix_web requires implementation of a special WebError to block unintended propagation of errors. For this particular instance, I don't really care, because it's more of an internal tool.
From the actix-web examples, use HttpResponse:
fn welcome(req: &HttpRequest) -> Result<HttpResponse> {
println!("{:?}", req);
// session
let mut counter = 1;
if let Some(count) = req.session().get::<i32>("counter")? {
println!("SESSION value: {}", count);
counter = count + 1;
}
// set counter to session
req.session().set("counter", counter)?;
// response
Ok(HttpResponse::build(StatusCode::OK)
.content_type("text/html; charset=utf-8")
.body(include_str!("../static/welcome.html")))
}

How do I access HttpRequest data inside a Future in Actix-web?

I'd like to have an Actix Web handler which responds to a POST request by printing the POST body to the console and constructing an HTTP response that contains the current URL from the request object.
When reading the request's POST body, futures seem to need to be involved. The closest I've gotten so far is:
fn handler(req: HttpRequest) -> FutureResponse<HttpResponse> {
req.body()
.from_err()
.and_then(|bytes: Bytes| {
println!("Body: {:?}", bytes);
let url = format!("{scheme}://{host}",
scheme = req.connection_info().scheme(),
host = req.connection_info().host());
Ok(HttpResponse::Ok().body(url).into())
}).responder()
}
This won't compile because the future outlives the handler, so my attempts to read req.connection_info() are illegal. The compiler error suggests I use the move keyword into the closure definition, i.e. .and_then(move |bytes: Bytes| {. This also won't compile because req gets moved on the req.body() call and is then captured after the move in the references constructing url.
What is a reasonable way of constructing a scope in which I have access to data attached to the request object (e.g. the connection_info) at the same time as access to the POST body?
The simplest solution is to not access it inside the future at all:
extern crate actix_web; // 0.6.14
extern crate bytes; // 0.4.8
extern crate futures; // 0.1.21
use actix_web::{AsyncResponder, FutureResponse, HttpMessage, HttpRequest, HttpResponse};
use bytes::Bytes;
use futures::future::Future;
fn handler(req: HttpRequest) -> FutureResponse<HttpResponse> {
let url = format!(
"{scheme}://{host}",
scheme = req.connection_info().scheme(),
host = req.connection_info().host(),
);
req.body()
.from_err()
.and_then(move |bytes: Bytes| {
println!("Body: {:?}", bytes);
Ok(HttpResponse::Ok().body(url).into())
})
.responder()
}
In case this is more than a quick hack for demonstration purposes, constructing URLs by concatenating strings is a terrible idea as it doesn't properly escape the values. You should be using a type that does that for you.

Rust Iron Web Framework gives some phantom error

I am using iron. Most of time like 99.* % all is good. But sometimes I get error like Error was: ErrorImpl { code: EofWhileParsingString/List/Object, line: 1, column: 8186 } or InvalidUnicodeCodePoint. I am printing request in log and when i try that request every thing goes well. I also have server written in Golang receiving same request and they never have parsing or json to MyStruct conversion problem.Please note Code would not compile as it is, missing imports, error::from and structure definition. Can not provide reproducible request logs as it only happens when serving lots on concurrent request but if single request is taken it works fine.
I have tried serde_json::from_reader, bodyparser crate and all have same issue.
extern crate serde;
extern crate serde_json;
extern crate iron;
use self::iron;
use self::iron::prelude::*;
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct MyStruct {
}
struct ResponseTime;
impl typemap::Key for ResponseTime {
type Value = u64;
}
#[derive(Debug)]
struct RequestBody;
impl typemap::Key for RequestBody {
type Value = RefCell<Vec<u8>>;
}
impl BeforeMiddleware for ResponseTime {
fn before(&self, req: &mut Request) -> IronResult<()> {
req.extensions.insert::<RequestBody>(RefCell::new(Vec::new()));
req.extensions.insert::<ResponseTime>(precise_time_ns());
Ok(())
}
}
impl AfterMiddleware for ResponseTime {
fn after(&self, req: &mut Request, res: Response) -> IronResult<Response> {
Ok(res)
}
fn catch(&self, req : &mut Request, err : IronError) -> IronResult<Response> {
let ref byte_req = *req.extensions.get::<RequestBody>()
.unwrap()
.borrow();
//just to make sure uft8 is not causing some issue.
let payload = unsafe {
str::from_utf8_unchecked(&byte_req)
};
//but when i send request body all comes good
error!("Error {} for Body {}", err, payload);
Err(err)
}
}
fn iron_handler(req : &mut Request) -> Result<Response, CustomError>{
let mut buffer = req.extensions.get::<server::RequestBody>()
.unwrap()
.borrow_mut();
req.body.read_to_end(&mut buffer)?;
// not seeing InvalidUnicodeCodePoint after this.
let payload = String::from_utf8_lossy(&buffer);
//some request throw error
let my_struct_obj : MyStruct = serde_json::from_str(&payload)?;
Ok(Response::with((iron::status::Ok, "Final Response")))
}
Need help to figure out how to identify problem. Intent of posting here is to see if someone had same issue or can see obvious problem with this. Appreciate everyone'e time do not expect to build and run with examples as can not provide them because of privacy.

Resources