I am using Diesel to query a DB in PostgreSQL using JOINs:
let product_id = 1;
sales::table
.inner_join(product::table)
.select((
product::description,
sales::amount,
sales::date_sale
))
.filter(sales::product_id.eq(product_id))
.load(&diesel::PgConnection)
My model:
pub struct Sales {
pub id: i32,
pub product_id: Option<i32>,
pub amount: Option<BigDecimal>,
pub date_sale: Option<NaiveDateTime>
}
The result is as expected, but I need to give a date format to the field sales::date_sale that in pgadmin I do it with to_char(date_sale, 'dd/mm/YYYY').
Is it possible to use to_char in Diesel or in what way can I modify the data that the Diesel ORM brings me?
In addition to the answers provided by harmic there are two more possibilities to solve this problem.
sql_function!
Diesel provides an interface to easily define query ast nodes for sql functions that are not provided by diesel itself. Users are encouraged to use this functionality to define missing functions on their own. (In fact diesel uses internally the same method to define query ast nodes for sql functions provided out of the box). The defined query ast node is usable in every context where the types of its expression are valid, so it can be used in select and where clauses.
(This is basically the type safe version of the raw sql solution above)
For the given question something like this should work:
#[derive(Queryable)]
pub struct Sales {
pub id: i32,
pub product_id: Option<i32>,
pub amount: Option<BigDecimal>,
pub date_sale: Option<String>,
}
sql_function! {
fn to_char(Nullable<Timestamp>, Text) -> Nullable<Text>;
}
let product_id = 1;
sales::table
.inner_join(product::table)
.select((
product::description,
sales::amount,
to_char(sales::date_sale, "dd/mm/YYYY")
))
.filter(sales::product_id.eq(product_id))
.load(&diesel::PgConnection);
#[derive(Queryable)] + #[diesel(deserialize_as = "Type")]
Diesels Queryable derive provides a way to apply certain type manipulations at load time via a custom attribute. In combination with the chrono solution provided by harmic this gives the following variant:
#[derive(Queryable)]
pub struct Sales {
pub id: i32,
pub product_id: Option<i32>,
pub amount: Option<BigDecimal>,
#[diesel(deserialize_as = "MyChronoTypeLoader")]
pub date_sale: Option<String>
}
struct MyChronoTypeLoader(Option<String>);
impl Into<Option<String>> for MyChronoTypeLoader {
fn into(self) -> String {
self.0
}
}
impl<DB, ST> Queryable<ST, DB> for MyChronoTypeLoader
where
DB: Backend,
Option<NaiveDateTime>: Queryable<ST, DB>,
{
type Row = <Option<NaiveDateTime> as Queryable<ST, DB>>::Row;
fn build(row: Self::Row) -> Self {
MyChronoTypeLoader(Option::<NaiveDateTime>::build(row).map(|d| d.format("%d/%m/%Y").to_string()))
}
}
When using an ORM, data is extracted from the database in the most suitable representation for interpretation by the ORM; after which you would manipulate it in the target domain (rust in this case).
Since date_sale is an Option<NaiveDateTime>, you can use the formatting options provided by chrono:
sale.date_sale.unwrap().format("%d/%m/%Y").to_string()
(Your real code would not use unwrap(), of course!)
Alternatively, if you really need the database to do the formatting, you could use a [sql][2] to insert some raw SQL into your query:
let results = sales::table.select((sales::id, sql("to_char(date_sale, 'dd/mm/YYYY')")))
.load::<(i32, String)>(&conn);
This would be more useful if you were trying to implement some logic that would be more easy or efficient to implement in SQL.
An example would be a filtering condition: suppose you wanted to include rows from your table where the week number was an even number. While you could load the whole table and then filter it in the rust domain, that would not be very efficient. Instead you could do this:
let results = sales::table
.filter(sql("extract(week from date_sale)::smallint % 2=0"))
.load::<Sales>(&conn);
Related
I am building a restful api using rocket.rs. I am a Rust beginner and learning rocket.rs. I have two database tables: chair and table. I use Rust Diesel ORM. The chair is represented by the Chair struct and table by the Table struct. A chair can refer to a table having Some(table_id) or to no table if None.
I am trying to create a POST endpoint that uses a chair id as parameter and transforms it into a Chair struct. Also, I want to validate that this chair is associated to a table. If the provided chair id is not associated to a table, then an error is sent. I have an implementation working already, but I feel that this is not a clean solution.
#[derive(Queryable, Serialize)]
pub struct Chair {
pub id: i32,
pub color: String,
pub table_id: Option<i32>
}
#[derive(Queryable, Serialize)]
pub struct ChairWithTable {
pub id: i32,
pub color: String,
pub table: i32
}
impl TryFrom<Chair> for ChairWithTable {
type Error = ChairError;
fn try_from(chair: Chair) -> Result<Self, Self::Error> {
if chair.table_id.is_none() {
Err(ChairError::WithoutTable)
} else {
Ok(ChairWithTable {
id: chair.id,
color: chair.color,
table_id: chair.table_id.unwrap()
})
}
}
}
pub struct Table {
pub id: i32,
// ... whatever other fields. this is not important in the example
}
impl<'a> FromParam<'a> for ChairWithTable {
// implemention details omitted
}
impl<'a> FromParam<'a> for Chair {
// implemention details omitted
}
Here we pretend we can add pieces (could be anything else, not very relevant to the case anyhow) to a chair but only if the chair is bound to a table. Does not make a lot of sense, but I changed the table names so bear with me.
#[post("/chairs/<chair>/pieces", rank = 1)]
async fn pieces(chair: ChairWithTable, conn: DatabaseConnection) -> Result<Json<[/*irrelevent*/]>, Forbidden<Option<String>>> {
// implementation details omitted
}
So what I feel is wrong about my solution, is that there is a lot of duplication between Chair and ChairWithTable. Actually I copied all the fields and removed the Option wrapping the i32. I also had to impl TryFrom<Chair> for ChairWithTable. This is already verbose and it would get much more complicated if I had even just two Option fields on the same model. I could also omit the ChairWithTable struct and only keep Chair struct. But then when I get in my controller method, I have to check if table_id is Some or None every time and throw an error mid way in the controller code instead of using the param guards as validation.
Coming from web dev background in php and TypeScript, my brain would try to solve this with TypeScript's utility types. I would do something like take the Chair struct and make the field required with something that looks like this type ChairWithTabl = Pick<Chair, 'id'|'color'> & Required<Pick<Chair, 'table_id'>>. Another way of solving it would for me to go with a library like Zod that creates types on the fly with type inference.
Think that one solution in pure Rust would be to aim for struct composition like this answer for this question. But as far as I know, it is not possible to use composition on diesel models
How would you recommend to tackle this kind problem in the context of Rust, Rocket.rs and Diesel?
I am new to Rust, and am attempting to take a struct returned from a library (referred to as source struct) and convert it into protobuf message using prost. The goal is to take the source struct, map the source struct types to the protobuf message types (or rather, the appropriate types for prost-generated struct, referred to as message struct), and populate the message struct fields using fields from the source struct. The source struct fields are a subset of message struct fields. For example:
pub struct Message {
pub id: i32,
pub email: String,
pub name: String,
}
pub struct Source {
pub email: Email,
pub name: Name,
}
So, I would like to take fields from from Source, map the types to corresponding types in Message, and populate the fields of Message using Source (fields have the same name). Currently, I am manually assigning the values by creating a Message struct and doing something like message_struct.email = source_struct.email.to_string();. Except I have multiple Message structs based on protobuf, some having 20+ fields, so I'm really hoping to find a more efficient way of doing this.
If I understand you correctly you want to generate define new struct based on fields from another. In that case you have to use macros.
https://doc.rust-lang.org/book/ch19-06-macros.html
Also this question (and answer) could be useful for you Is it possible to generate a struct with a macro?
To convert struct values from one to another struct best way is to use From<T> or Into<T> trait.
https://doc.rust-lang.org/std/convert/trait.From.html
This is called FRU (functional record update).
This currently works only for structs with the same type and structs with the same type modulo generic parameters.
The RFC-2528 talks about a generalization that would make this work:
struct Foo {
field1: &'static str,
field2: i32,
}
struct Bar {
field1: f64,
field2: i32,
}
let foo = Foo { field1: "hi", field2: 1 };
let bar = Bar { field1: 3.14, ..foo };
Unfortunately, this has not yet been implemented.
You could make a method to create a Message from a Source like this.
impl Message {
pub fn from_source(source: &Source, id: i32) -> Self {
Message {
id: id,
email: source.email.to_string(),
name: source.name.to_string(),
}
}
}
And then,
let source = // todo
let id = // todo
let message = Message::from_source(&source, id);
I have three models with these relations:
Artist 1<->n Song 1<->n Play
I want to load Plays and eagerly load its Songs and its Artists.
I did this in PHP framework Eloquent easily. And it takes only 3 DB requests (one for each model).
Is this some how possible with diesel?
I think the Play model should look something like this:
#[derive(Queryable, Serialize, Deserialize)]
pub struct Play {
pub id: u64,
pub song_id: u64,
pub song: Song, // <-- ????
pub date: NaiveDateTime,
pub station_id: u64,
}
And loading something like this:
// loads one model only at the moment
// it should load the related models Song and Artist too
let items = plays.filter(date.between(date_from, date_to)).load(&*conn)?;
The resulting struct should be JSON serialized to be used by the REST API. But I can't find a way to get the needed struct in one simple and efficient way.
One way to achieve this is to join the two other tables. I did not figure out how to avoid enumerating all table fields unfortunately. This is not an elegant solution, but it should work.
fn select_plays_with_song_and_artist(
conn: &Conn,
date_from: NaiveDate,
date_to: NaiveDate,
) -> Result<Vec<(models::Play, models::Song, models::Artist)>> {
plays::table
.filter(plays::date.between(date_from, date_to))
.join(songs::table.join(artists::table))
.select((
(
plays::id,
plays::song_id,
plays::date,
plays::station_id,
),
(
songs::id,
// other relevant fields...
),
(
artist::id,
// other relevant fields...
),
))
.load::<(models::Play, models::Song, models::Artist)>(conn)
}
First of all: If you post a question about diesel online, please always include the relevant part of your schema, otherwise others need to guess how it looks like.
I will assume the following schema for your question:
table! {
plays (id)
id -> BigInt,
song_id -> BigInt,
date -> Timestamp,
station_id -> BigInt,
}
}
table! {
songs(id) {
id -> BigInt,
name -> Text,
}
}
joinable!(plays -> songs (song_id));
Now there is some issue with your Play struct. u64 is not a type supported by diesel. Checkout this documentation about which types are compatible with a BigInt SQL type.
That means I will assume the following structs implementing Queryable:
#[derive(Queryable, Serialize, Deserialize)]
pub struct Play {
pub id: i64,
// pub song_id: u64, leave this out as it is a duplicate with `song` below
pub song: Song,
pub date: NaiveDateTime,
pub station_id: i64,
}
#[derive(Queryable, Serialize, Deserialize)]
pub struct Song {
pub id: i64,
pub name: String,
}
Now to come to your core question: At first, diesel does not have the concept of a single model like known from other ORM's. If you have ever used such a thing, don't try to apply that knowledge to diesel, it won't work. There are a set of traits describing a capability of each struct. As we are talking about querying here I will concentrate on Queryable. Queryable is designed to map the result of a query (which could include zero, one or more tables) to a rust data structure. By default (by using the derive) it does this by mapping fields structural, that means your select clause needs to match the structure deriving Queryable.
Now back to your concrete problem. It means that we must construct a query that returns (i64, (i64, String), NaiveDateTime, i64) (or better the fields with the corresponding SQL types).
For this case this is as easy as using a single join:
plays::table.inner_join(songs::table)
.select((plays::id, songs::all_columns, plays::date, plays::station_id))
The only important thing here for the mapping is the select clause. You can append/prepend whatever other QueryDsl method you like.
I couldn't find anything elegant way of doing this. Here's the snippet, if you want to do it in two steps.
First, get the both models using join on both table.
impl Comment {
pub fn list(conn: &PgConnection, page_id: i32) -> Result<Vec<(Comment, User)>, Error> {
use crate::schema::users;
comment::table
.inner_join(users::table)
.select((comment::all_columns, users::all_columns))
.filter(comment::page_id.eq(page_id))
.load(conn)
}
}
Now, iterate over the result to make it fit into your own api response structure.
#[derive(Serialize)]
pub struct CommentListResponse {
#[serde(flatten)]
pub comment: Comment,
pub user: UserResponse
}
let result = Comment::list(&connection, page.id)?;
let mut comment_list = vec![];
for (comment, user) in result {
let cr = CommentListResponse {
comment,
user: UserResponse::from(user),
};
comment_list.push(cr);
}
Ok(HttpResponse::Ok().json(comment_list))
I'm using Actix and Diesel. The way I set it up is that I have a DbExecutor - an async worker which takes messages and returns the results of the queries they represent, like so:
#[derive(Message)]
#[rtype(result = "Result<User>")]
pub struct RetrieveUser {
pub uid: i64,
}
impl Handler<RetrieveUser> for DbExecutor {
// ...
fn handle(&mut self, msg: InsertFile, _: &mut Self::Context) -> Self::Result {
use crate::schema::users::dsl::*;
// ...
Ok(users.filter(uid.eq(msg.uid)).execute(connection))
}
}
fn my_route_handler() {
db_executor.send(RetrieveUser{ uid: 123 });
//...
}
However, there is a table which doesn't fit neatly into these simple queries, and I would like to specify the filters from the side of the message sender. I was thinking I could send a message containing a closure which takes the users object, and then execute it on the side of the DbExecutor, returning the results:
fn my_ideal_route_handler(){
db_executor.send(RetrieveUsers { predicate: |users| users.filter(uid.eq(123)) });
}
However, I am stuck, seeing as all Messages have to implement Copy, and the Diesel types are confusing - I am not sure how to specify the type I need generically enough, while getting it to compile, eg.:
// FAILS
#[derive(Message)]
#[rtype(result = "Result<Vec<User>>")]
pub struct RetrieveUsers {
pub predicate: dyn FnOnce(dyn QueryDsl) -> dyn RunQueryDsl<PgConnection>,
}
I am also not sure whether this approach is even sensible at all, so I am open to other suggestions which would:
allow me to query the table dynamically
still keep most of the database handling separate and not all across the project as if I were passing the plain database connection around
(ideally) utilize concurrent access to the database
I have a data structure Document, which I would like to serialize other Rust structs to. It is basically a HashMap for the fields internally, however it interacts with a database API, so I will definitely want to convert other types into those Documents.
For example this struct
struct Entry {
id: String,
user: String,
duration: u32,
location: (f64, f64),
}
I already have a conversion to the Document type using the From trait, however this is an extra place I have to modify when the Entry struct changes. The implementation uses a DocumentBuilder and looks like this:
impl From<Entry> for Document {
fn from(entry: Entry) -> Self {
Document::builder()
.name(&entry.id) // set the name of the document
.field("user", entry.user) // add fields ...
.field("duration", entry.duration)
.field("location", entry.location)
.build() // build a Document
}
}
The field method can assign any value which can be converted to a FieldValue to a key. So the signature of field is:
impl DocumentBuilder {
// ...
pub fn field<T: Into<FieldValue>>(mut self, key: &str, value: T) -> Self { ... }
// ...
}
I would like to use serde and its derive feature to automatically serialize the struct and its fields into a Document. How would I go about doing this? I looked at the wiki for Implementing a Serializer but the example shown writes to a string and I would like to know how I can serialize to a data structure using the builder pattern.
The simplest way to do this would be to use serde_json::from_value (applicable even if you're not using JSON, but requires all fields to be valid JSON [e.g. no non-string keys in hashmaps]):
let entry = Entry {
a: 24,
b: 42,
c: "nice".to_string()
};
let v = serde_json::to_value(&entry).unwrap();
let document: Document = serde_json::from_value(v).unwrap();
Caveat: the value type of Document will have to implement Deserialize, and in a way that can deserialize any value into the correct argument. This can be done by using #[serde(untagged)], but may be prone to certain types being wrong, such as u8 being converted to u64.
Full playground example
A more sophisticated method that doesn't involve any unnecessary copies will require you to write a custom (De)serializer, and a good one to look at would be serde_transcode::transcode, which does the inverse of the thing you want - it converts between two data formats.