I am trying to work on a web app with Diesel and Rocket, by following the rocket guide. I have not able to understand how to do testing of this app.
//in main.rs
#[database("my_db")]
struct PgDbConn(diesel::PgConnection);
#[post("/users", format="application/json", data="<user_info>")]
fn create_new_user(conn: PgDbConn, user_info: Json<NewUser>) {
use schema::users;
diesel::insert_into(users::table).values(&*user_info).execute(&*conn).unwrap();
}
fn main() {
rocket::ignite()
.attach(PgDbConn::fairing())
.mount("/", routes![create_new_user])
.launch();
}
// in test.rs
use crate::PgDbConn;
#[test]
fn test_user_creation() {
let rocket = rocket::ignite().attach(PgDbConn::fairing());
let client = Client::new(rocket).unwrap();
let response = client
.post("/users")
.header(ContentType::JSON)
.body(r#"{"username": "xyz", "email": "temp#abc.com"}"#)
.dispatch();
assert_eq!(response.status(), Status::Ok);
}
But this modifies the database. How can I make sure that the test does not alter the database.
I tried to create two database and use them in the following way(I am not sure if this is recommended)
#[cfg(test)]
#[database("test_db")]
struct PgDbConn(diesel::PgConnection);
#[cfg(not(test))]
#[database("live_db")]
struct PgDbConn(diesel::PgConnection);
Now I thought I can use the test_transaction method of the diesel::connection::Connection trait in the following way:-
use crate::PgDbConn;
#[test]
fn test_user_creation() {
// !!This statment is wrong as PgDbConn is an Fn object instead of a struct
// !!I am not sure how it works but it seems that this Fn object is resolved
// !!into struct only when used as a Request Guard
let conn = PgDbConn;
// Deref trait for PgDbConn is implemented, So I thought that dereferencing
// it will return a diesel::PgConnection
(*conn).test_transaction::<_, (), _>(|| {
let rocket = rocket::ignite().attach(PgDbConn::fairing());
let client = Client::new(rocket).unwrap();
let response = client
.post("/users")
.header(ContentType::JSON)
.body(r#"{"username": "Tushar", "email": "temp#abc.com"}"#)
.dispatch();
assert_eq!(response.status(), Status::Ok);
Ok(())
});
}
The above code obviously fails to compile. Is there a way to resolve this Fn object into the struct and obtain the PgConnection in it. And I am not even sure if this is the right to way to do things.
Is there a recommended way to do testing while using both Rocket and Diesel?
This will fundamentally not work as you imagined there, as conn will be a different connection than whatever rocket generates for you. The test_transaction pattern assumes that you use the same connection for everything.
Related
After reading the Rust book, I've decided to give it a try with Web Assembly. I'm creating a simple tracker script to practice and learn more about it. There are a couple of methods that need to access the window, navigator or cookie API. Every time I have to access any of those there are a lot of boiler plate code involved:
pub fn start() {
let window = web_sys::window().unwrap();
let document = window.document().unwrap();
let html = document.dyn_into::<web_sys::HtmlDocument>().unwrap();
let cookie = html_document.cookie().unwrap();
}
That's unpractical and bothers me. Is there a smart way to solve this? I've in fact tried to use lazy_static to have all of this in a global.rs file:
#[macro_use]
extern crate lazy_static;
use web_sys::*;
lazy_static! {
static ref WINDOW: window = {
web_sys::window().unwrap()
};
}
But the compile fails with: *mut u8 cannot be shared between threads safely`.
You could use the ? operator instead of unwrapping.
Instead of writing
pub fn start() {
let window = web_sys::window().unwrap();
let document = window.document().unwrap();
let html = document.dyn_into::<web_sys::HtmlDocument>().unwrap();
let cookie = html_document.cookie().unwrap();
}
You can write
pub fn start() -> Result<()> {
let cookie = web_sys::window()?
.document()?
.dyn_into<web_sys::HtmlDocument>()?
.cookie()?;
Ok(())
}
It's the same number of lines, but less boilerplate and for simpler cases a one-liner.
If you really don't want to return a result you can wrap the whole thing in a lambda, (or a try block if you're happy using unstable features).
pub fn start() {
let cookie = (|| Result<Cookie)> {
web_sys::window()?
.document()?
.dyn_into<web_sys::HtmlDocument>()?
.cookie()
}).unwrap();
}
if you find you don't like repeating this frequently - you can use functions
fn document() -> Result<Document> {
web_sys::window()?.document()
}
fn html() -> Result<web_sys::HtmlDocument> {
document()?.dyn_into<web_sys::HtmlDocument>()
}
fn cookie() -> Result<Cookie> {
html()?.cookie()
}
pub fn start() {
let cookie = cookie()?;
}
That's unpractical and bothers me.
Unsure what's your issue here, but if you access the same cookie again and again in your application, perhaps you can save it in a struct and just use that struct? In my recent WebAssembly project I saved some of the elements I've used in a struct and used them by passing it around.
I also think that perhaps explaining your specific use case might lead to more specific answers :)
I have an object that I know that is inside an Arc because all the instances are always Arced. I would like to be able to pass a cloned Arc of myself in a function call. The thing I am calling will call me back later on other threads.
In C++, there is a standard mixin called enable_shared_from_this. It enables me to do exactly this
class Bus : public std::enable_shared_from_this<Bus>
{
....
void SetupDevice(Device device,...)
{
device->Attach(shared_from_this());
}
}
If this object is not under shared_ptr management (the closest C++ has to Arc) then this will fail at run time.
I cannot find an equivalent.
EDIT:
Here is an example of why its needed. I have a timerqueue library. It allows a client to request an arbitrary closure to be run at some point in the future. The code is run on a dedicated thread. To use it you must pass a closure of the function you want to be executed later.
use std::time::{Duration, Instant};
use timerqueue::*;
use parking_lot::Mutex;
use std::sync::{Arc,Weak};
use std::ops::{DerefMut};
// inline me keeper cos not on github
pub struct MeKeeper<T> {
them: Mutex<Weak<T>>,
}
impl<T> MeKeeper<T> {
pub fn new() -> Self {
Self {
them: Mutex::new(Weak::new()),
}
}
pub fn save(&self, arc: &Arc<T>) {
*self.them.lock().deref_mut() = Arc::downgrade(arc);
}
pub fn get(&self) -> Arc<T> {
match self.them.lock().upgrade() {
Some(arc) => return arc,
None => unreachable!(),
}
}
}
// -----------------------------------
struct Test {
data:String,
me: MeKeeper<Self>,
}
impl Test {
pub fn new() -> Arc<Test>{
let arc = Arc::new(Self {
me: MeKeeper::new(),
data: "Yo".to_string()
});
arc.me.save(&arc);
arc
}
fn task(&self) {
println!("{}", self.data);
}
// in real use case the TQ and a ton of other status data is passed in the new call for Test
// to keep things simple here the 'container' passes tq as an arg
pub fn do_stuff(&self, tq: &TimerQueue) {
// stuff includes a async task that must be done in 1 second
//.....
let me = self.me.get().clone();
tq.queue(
Box::new(move || me.task()),
"x".to_string(),
Instant::now() + Duration::from_millis(1000),
);
}
}
fn main() {
// in real case (PDP11 emulator) there is a Bus class owning tons of objects thats
// alive for the whole duration
let tq = Arc::new(TimerQueue::new());
let test = Test::new();
test.do_stuff(&*tq);
// just to keep everything alive while we wait
let mut input = String::new();
std::io::stdin().read_line(&mut input).unwrap();
}
cargo toml
[package]
name = "tqclient"
version = "0.1.0"
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
timerqueue = { git = "https://github.com/pm100/timerqueue.git" }
parking_lot = "0.11"
There is no way to go from a &self to the Arc that self is stored in. This is because:
Rust references have additional assumptions compared to C++ references that would make such a conversion undefined behavior.
Rust's implementation of Arc does not even expose the information necessary to determine whether self is stored in an Arc or not.
Luckily, there is an alternative approach. Instead of creating a &self to the value inside the Arc, and passing that to the method, pass the Arc directly to the method that needs to access it. You can do that like this:
use std::sync::Arc;
struct Shared {
field: String,
}
impl Shared {
fn print_field(self: Arc<Self>) {
let clone: Arc<Shared> = self.clone();
println!("{}", clone.field);
}
}
Then the print_field function can only be called on an Shared encapsulated in an Arc.
having found that I needed this three times in recent days I decided to stop trying to come up with other designs. Maybe poor data design as far as rust is concerned but I needed it.
Works by changing the new function of the types using it to return an Arc rather than a raw self. All my objects are arced anyway, before they were arced by the caller, now its forced.
mini util library called mekeeper
use parking_lot::Mutex;
use std::sync::{Arc,Weak};
use std::ops::{DerefMut};
pub struct MeKeeper<T> {
them: Mutex<Weak<T>>,
}
impl<T> MeKeeper<T> {
pub fn new() -> Self {
Self {
them: Mutex::new(Weak::new()),
}
}
pub fn save(&self, arc: &Arc<T>) {
*self.them.lock().deref_mut() = Arc::downgrade(arc);
}
pub fn get(&self) -> Arc<T> {
match self.them.lock().upgrade() {
Some(arc) => return arc,
None => unreachable!(),
}
}
}
to use it
pub struct Test {
me: MeKeeper<Self>,
foo:i8,
}
impl Test {
pub fn new() -> Arc<Self> {
let arc = Arc::new(Test {
me: MeKeeper::new(),
foo:42
});
arc.me.save(&arc);
arc
}
}
now when an instance of Test wants to call a function that requires it to pass in an Arc it does:
fn nargle(){
let me = me.get();
Ooddle::fertang(me,42);// fertang needs an Arc<T>
}
the weak use is what the shared_from_this does so as to prevent refcount deadlocks, I stole that idea.
The unreachable path is safe because the only place that can call MeKeeper::get is the instance of T (Test here) that owns it and that call can only happen if the T instance is alive. Hence no none return from weak::upgrade
The code below is the beginnings of a small library I'm writing to talk to a web API. Users of the library will instantiate a client MyClient and access the web API through it. Here, I'm trying to get an access token from the API before making requests to it.
In get_new_access() I'm able to make the request and receive the JSON response. I then try to use serde to turn the response into an Access struct, and this is where the problems start.
I've created a library specific error enum MyError which can represent the JSON deserializing and reqwest errors that could occur within get_new_access(). However, when I go to compile I get the trait serde::Deserialize<'_> is not implemented for MyError. My understanding is that this is happening because in the case that I get one of the aforementioned errors, serde does not know how to deserialize it into an Access struct. Of course, I don't want it to do that at all, so my question is what should I do?
I've looked at various serde deserialize examples, but all of them seem to assume that they are running in a main function that can only return a serde error. If I put #[derive(Deserialize)] above MyError's declaration, then I get the same error, but it shifts to reqwest::Error and serde_json::Error instead.
use std::error;
use std::fmt;
extern crate chrono;
extern crate reqwest;
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;
use chrono::prelude::*;
use reqwest::Client;
pub struct MyClient {
access: Access,
token_expires: DateTime<Utc>,
}
#[derive(Deserialize, Debug)]
struct Access {
access_token: String,
expires_in: i64,
token_type: String,
}
fn main() {
let sc: MyClient = MyClient::new();
println!("{:?}", &sc.access);
}
impl MyClient {
pub fn new() -> MyClient {
let a: Access = MyClient::get_new_access().expect("Couldn't get Access");
let e: DateTime<Utc> = chrono::Utc::now(); //TODO
MyClient {
access: a,
token_expires: e,
}
}
fn get_new_access() -> Result<Access, MyError> {
let params = ["test"];
let client = Client::new();
let json = client
.post(&[""].concat())
.form(¶ms)
.send()?
.text()
.expect("Couldn't get JSON Response");
println!("{}", &json);
serde_json::from_str(&json)?
//let a = Access {access_token: "Test".to_string(), expires_in: 3600, token_type: "Test".to_string() };
//serde_json::from_str(&json)?
}
}
#[derive(Debug)]
pub enum MyError {
WebRequestError(reqwest::Error),
ParseError(serde_json::Error),
}
impl fmt::Display for MyError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "eRROR")
}
}
impl error::Error for MyError {
fn description(&self) -> &str {
"API internal error"
}
fn cause(&self) -> Option<&error::Error> {
// Generic error, underlying cause isn't tracked.
None
}
}
impl From<serde_json::Error> for MyError {
fn from(e: serde_json::Error) -> Self {
MyError::ParseError(e)
}
}
impl From<reqwest::Error> for MyError {
fn from(e: reqwest::Error) -> Self {
MyError::WebRequestError(e)
}
}
Playground link here.
Your first problem is that your fn get_new_access() -> Result<Access, MyError> expects a Result. But in here:
//...
serde_json::from_str(&json)?
}
because of using ?(try macro), you are trying to return Result's unwrapped value which is a subtype of serde::Deserialize<'_>. The compiler warns you about this Deserialize is not a Result. What you should do is just return the result without unwrapping it:
//...
serde_json::from_str(&json)
}
Or
//...
let access = serde_json::from_str(&json)?; // gets access or propagates error
Ok(access) //if no error return access in a Result
}
Then you will have a second problem because your function expects MyError in the Result while you are returning serde_json::Error with this call serde_json::from_str(&json). Luckily Result has the function map_err which maps the actual error type to your custom error type.
This code will solve your problem:
//...
serde_json::from_str(&json).map_err(MyError::ParseError)
}
For the request in the comment :
For example, if I change the web request line to let json = client.post("").form(¶ms).send().map_err(MyError::WebRequestError)?.text()?;,
is that better practice at all?
Yes but since text() returns a Result you need to map it's error as MyError too. Since both send and text has same error type(reqwest::Error) you can combine the results with and_then :
let json = client
.post(&[""].concat())
.form(¶ms)
.send()
.and_then(Response::text) //use reqwest::Response;
.map_err(MyError::WebRequestError)?;
I am using iron. Most of time like 99.* % all is good. But sometimes I get error like Error was: ErrorImpl { code: EofWhileParsingString/List/Object, line: 1, column: 8186 } or InvalidUnicodeCodePoint. I am printing request in log and when i try that request every thing goes well. I also have server written in Golang receiving same request and they never have parsing or json to MyStruct conversion problem.Please note Code would not compile as it is, missing imports, error::from and structure definition. Can not provide reproducible request logs as it only happens when serving lots on concurrent request but if single request is taken it works fine.
I have tried serde_json::from_reader, bodyparser crate and all have same issue.
extern crate serde;
extern crate serde_json;
extern crate iron;
use self::iron;
use self::iron::prelude::*;
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct MyStruct {
}
struct ResponseTime;
impl typemap::Key for ResponseTime {
type Value = u64;
}
#[derive(Debug)]
struct RequestBody;
impl typemap::Key for RequestBody {
type Value = RefCell<Vec<u8>>;
}
impl BeforeMiddleware for ResponseTime {
fn before(&self, req: &mut Request) -> IronResult<()> {
req.extensions.insert::<RequestBody>(RefCell::new(Vec::new()));
req.extensions.insert::<ResponseTime>(precise_time_ns());
Ok(())
}
}
impl AfterMiddleware for ResponseTime {
fn after(&self, req: &mut Request, res: Response) -> IronResult<Response> {
Ok(res)
}
fn catch(&self, req : &mut Request, err : IronError) -> IronResult<Response> {
let ref byte_req = *req.extensions.get::<RequestBody>()
.unwrap()
.borrow();
//just to make sure uft8 is not causing some issue.
let payload = unsafe {
str::from_utf8_unchecked(&byte_req)
};
//but when i send request body all comes good
error!("Error {} for Body {}", err, payload);
Err(err)
}
}
fn iron_handler(req : &mut Request) -> Result<Response, CustomError>{
let mut buffer = req.extensions.get::<server::RequestBody>()
.unwrap()
.borrow_mut();
req.body.read_to_end(&mut buffer)?;
// not seeing InvalidUnicodeCodePoint after this.
let payload = String::from_utf8_lossy(&buffer);
//some request throw error
let my_struct_obj : MyStruct = serde_json::from_str(&payload)?;
Ok(Response::with((iron::status::Ok, "Final Response")))
}
Need help to figure out how to identify problem. Intent of posting here is to see if someone had same issue or can see obvious problem with this. Appreciate everyone'e time do not expect to build and run with examples as can not provide them because of privacy.
I'm trying to do a simple extension to the comments example by creating a REST API and committing the post to the database. I'm creating the connection outside the scope of the handler itself which I'm assuming is where my problem lies. I'm just not sure how to fix it.
This is the code for the post handler:
server.get("/comments", middleware! {
let mut stmt = conn.prepare("SELECT * FROM comment").unwrap();
let mut iter = stmt.query_map(&[], |row| {
Comment { id: row.get(0), author: row.get(1), text: row.get(2) }
}).unwrap();
let mut out: Vec<Comment> = Vec::new();
for comment in iter {
out.push(comment.unwrap());
}
json::encode(&out).unwrap()
});
This is the error I get:
<nickel macros>:22:50: 22:66 error: the trait `core::marker::Sync` is not implemented for the type `core::cell::UnsafeCell<rusqlite::InnerConnection>` [E0277]
I assume the error is because I have created the instance and then tried to use it in a closure and that variable is probably destroyed once my main function completes.
Here's an MCVE that reproduces the problem (you should provide these when asking questions):
extern crate rusqlite;
#[macro_use]
extern crate nickel;
use nickel::{Nickel, HttpRouter};
use rusqlite::Connection;
fn main() {
let mut server = Nickel::new();
let conn = Connection::open_in_memory().unwrap();
server.get("/comments", middleware! {
let _stmt = conn.prepare("SELECT * FROM comment").unwrap();
""
});
server.listen("127.0.0.1:6767");
}
The Sync trait says:
Types that are not Sync are those that have "interior mutability" in a non-thread-safe way, such as Cell and RefCell
Which matches with the error message you get. Something inside the Connection has interior mutability which means that the compiler cannot automatically guarantee that sharing it across threads is safe. I had a recent question that might be useful to the implementor of Connection, if they can guarantee it's safe to share (perhaps SQLite itself makes guarantees).
The simplest thing you can do is to ensure that only one thread has access to the database object at a time:
use std::sync::Mutex;
fn main() {
let mut server = Nickel::new();
let conn = Mutex::new(Connection::open_in_memory().unwrap());
server.get("/comments", middleware! {
let conn = conn.lock().unwrap();
let _stmt = conn.prepare("SELECT * FROM comment").unwrap();
""
});
server.listen("127.0.0.1:6767");
}