I'm trying to construct some filters at runtime which can be applied either to a table tunnel or to tunnel LEFT OUTER JOIN connection ON (tunnel.id = connection.tunnel_id).
The tables are defined like this:
// Define the tunnel table and struct
table! {
#[allow(unused_imports)]
use diesel::sql_types::*;
tunnel (id) {
id -> BigInt,
name -> Text,
}
}
#[derive(Queryable, Identifiable, Clone, Debug, PartialEq, Eq)]
#[table_name = "tunnel"]
pub struct Tunnel {
pub id: i64,
pub name: String,
}
// Define the connection table and struct
table! {
#[allow(unused_imports)]
use diesel::sql_types::*;
connection(id) {
id -> BigInt,
tunnel_id -> BigInt,
}
}
#[derive(Debug, Associations, Identifiable, Queryable)]
#[table_name = "connection"]
#[primary_key(id)]
#[belongs_to(Tunnel)]
pub struct Connection {
pub id: i64,
pub tunnel_id: i64,
}
joinable!(connection -> tunnel(tunnel_id));
allow_tables_to_appear_in_same_query!(connection, tunnel);
I can write a function that constructs dynamic for either the single table:
fn filters_t(
name: &'static str,
) -> Vec<Box<dyn BoxableExpression<tunnel::table, Pg, SqlType = Bool>>> {
let mut wheres: Vec<Box<dyn BoxableExpression<tunnel::table, Pg, SqlType = Bool>>> = Vec::new();
wheres.push(Box::new(tunnel::name.eq(name)));
wheres
}
Or for the join:
pub type TunnelJoinConnection = JoinOn<
Join<tunnel::table, connection::table, LeftOuter>,
Eq<Nullable<connection::columns::tunnel_id>, Nullable<tunnel::columns::id>>,
>;
fn filters_j(
name: &'static str,
) -> Vec<Box<dyn BoxableExpression<TunnelJoinConnection, Pg, SqlType = Bool>>> {
let mut wheres: Vec<Box<dyn BoxableExpression<TunnelJoinConnection, Pg, SqlType = Bool>>> =
Vec::new();
wheres.push(Box::new(tunnel::name.eq(name)));
wheres
}
Note that the two filter functions have the exact same function body, so I should be able to make a generic function that achieves both of them. But I get an error when I try to make it generic.
fn filters<T>(name: &'static str) -> Vec<Box<dyn BoxableExpression<T, Pg, SqlType = Bool>>>
where
T: AppearsInFromClause<tunnel::table, Count = Once>,
{
vec![Box::new(tunnel::name.eq(name))]
}
The error is
|
85 | vec![Box::new(tunnel::name.eq(name))]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `diesel::SelectableExpression<T>` is not implemented for `tunnel::columns::name`
|
= note: required because of the requirements on the impl of `diesel::SelectableExpression<T>` for `diesel::expression::operators::Eq<tunnel::columns::name, diesel::expression::bound::Bound<diesel::sql_types::Text, &str>>`
= note: required because of the requirements on the impl of `diesel::BoxableExpression<T, diesel::pg::Pg>` for `diesel::expression::operators::Eq<tunnel::columns::name, diesel::expression::bound::Bound<diesel::sql_types::Text, &str>>`
= note: required for the cast to the object type `dyn diesel::BoxableExpression<T, diesel::pg::Pg, SqlType = diesel::sql_types::Bool>`
Minimal example here, you can clone that minimal example and run cargo check yourself to see the error.
This can be be fixed with one small change:
fn filters<T>(name: &'static str) -> Vec<Box<dyn BoxableExpression<T, Pg, SqlType = Bool>>>
where
diesel::dsl::Eq<tunnel::name, &'static str>: BoxableExpression<T, Pg, SqlType = Bool>,
{
vec![Box::new(tunnel::name.eq(name))]
}
Basically you need to assert at compile time that your boxed expression really implements BoxableExpression<T, Pg, SqlType = Bool> for all possible T's. If there is only a specific T that is checked by rustc, for the generic case this needs to be written out explicitly. The diesel::dsl::Eq helper type is a type level constructor for the type returned by tunnel::name.eq(name). This implies you will need a similar clause for each expression you add to the list.
Another unrelated note:
pub type TunnelJoinConnection = JoinOn<
Join<tunnel::table, connection::table, LeftOuter>,
Eq<Nullable<connection::columns::tunnel_id>, Nullable<tunnel::columns::id>>,
>;
uses types that are not considered to be part of the public API of diesel. This means such a expression can break with any update. The correct way to write this type using the public API is
pub type TunnelJoinConnection = diesel::dsl::LeftJoin<tunnel::table, connection::table>;`
Related
I have the following code:
// cargo.toml:
// serde = { version = "1.0", features = ["derive"] }
// uuid = { version = "1.2", features = ["serde", "v4"] }
// sqlx = { version = "0.6.2", features = ["runtime-async-std-native-tls", "sqlite", "postgres", "chrono", "uuid", "macros"]}
use uuid::{Uuid, fmt::Hyphenated};
use serde::{Deserialize, Serialize};
#[derive(Debug, Serialize, Deserialize, FromRow, PartialEq)]
pub struct Transaction {
#[sqlx(try_from = "Hyphenated")]
pub t_id: Uuid,
#[sqlx(try_from = "Option<Hyphenated>")]
pub corr_id: Option<Uuid>,
}
In SQLite database, id stored in hyphenated format like "550e8400-e29b-41d4-a716-446655440000". t_id not null, corr_id nullable.
macro #[sqlx(try_from = "Hyphenated")] works fine, but I cant figure out, how to use it with Option for corr_id field. Given code panics. Any help is greatly appreciated.
The compiler tells us that it cannot implicitly convert Hyphenated to an Option<Uuid>,
| #[derive(Debug, Serialize, Deserialize, FromRow, PartialEq)]
| ^^^^^^^ the trait `From<Hyphenated>` is not implemented for `std::option::Option<Uuid>`
and we can't implement external traits for external types. The only choices left seem to be
1. Implement the FromRow trait yourself
Since we know that we'll be using SQLite and the corr_id is a nullable text column, we can implement FromRow for sqlx::sqlite::SqliteRows. If your struct (row) only has these two fields, this is fine but when extending it with additional fields, you'll need to update your FromRow implementation as well.
use sqlx::{sqlite::SqliteRow, FromRow, Row};
use uuid::{fmt::Hyphenated, Uuid};
pub struct Transaction {
pub t_id: Uuid,
pub corr_id: Option<Uuid>,
}
impl<'r> FromRow<'r, SqliteRow> for Transaction {
fn from_row(row: &'r SqliteRow) -> Result<Self, sqlx::Error> {
let t_id: Hyphenated = row.try_get("t_id")?;
let corr_id: &str = row.try_get("corr_id")?;
let corr_id = if corr_id.is_empty() {
None
} else {
let uuid = Uuid::try_parse(&corr_id).map_err(|e| sqlx::Error::ColumnDecode {
index: "corr_id".to_owned(),
source: Box::new(e),
})?;
Some(uuid)
};
Ok(Transaction {
t_id: t_id.into(),
corr_id,
})
}
}
2. Use a newtype
This way, you can reuse your "nullable" type in other structs if necessary, and can even implement Deref, if you want to make extracting the inner UUID easier. It does come with some extra allocations though, since the incoming bytes are converted first to String, then parsed into Uuid.
use std::ops::Deref;
use sqlx::FromRow;
#[derive(FromRow)]
pub struct Transaction {
#[sqlx(try_from = "Hyphenated")]
pub t_id: Uuid,
#[sqlx(try_from = "String")]
pub corr_id: NullableUuid,
}
pub struct NullableUuid(Option<Uuid>);
impl TryFrom<String> for NullableUuid {
type Error = uuid::Error;
fn try_from(value: String) -> Result<Self, Self::Error> {
let inner = if value.is_empty() {
None
} else {
let uuid = Uuid::try_parse(&value)?;
Some(uuid)
};
Ok(NullableUuid(inner))
}
}
impl Deref for NullableUuid {
type Target = Option<Uuid>;
fn deref(&self) -> &Self::Target {
&self.0
}
}
I have multiple structs that correspond to serialized/deserialized objects only known during runtime, example:
#[derive(Serialize, Deserialize)]
struct Car{
model: i32,
year: i32
}
#[derive(Serialize, Deserialize)]
struct Person{
name: String,
age: i32
}
Then I have functions to serialize and deserialize:
fn deserialize(data: Vec<u8>){
let msg = str::from_utf8(data);
serde_json::from_str(msg);
}
fn serialize(&self, object: Car) -> String{
let msg = serde_json::to_string(&object).unwrap();
return msg;
}
How can I make the deserialize function deserialize to both Car and Person (and possibly many other different types) and return the object? And how can I make the serialize function do the same thing: serialize Car, Person, and other objects (accepting these types in the attributes of the function)?
You want to use generic functions to allow different types to be passed in, and set trait bounds to make sure the objects are able to be serialized/deserialized. When calling serialize, the type will be inferred by the type of the parameter, but when calling deserialize, you need to use the turbofish (::<>) to specify the type, if it can't be inferred.
use serde::{Serialize, Deserialize};
use std::str;
#[derive(Serialize, Deserialize)]
struct Car {
model: i32,
year: i32
}
#[derive(Serialize, Deserialize)]
struct Person {
name: String,
age: i32
}
// constrain output types to have the `Deserialize` trait
fn deserialize<'a, T>(data: &'a [u8]) -> T where T: Deserialize<'a> {
let msg = str::from_utf8(data).unwrap();
serde_json::from_str::<T>(msg).unwrap()
}
// shorthand for the above when `T` isn't needed in the function body
fn serialize(object: &impl Serialize) -> String {
let msg = serde_json::to_string(object).unwrap();
return msg;
}
fn main() {
let car = Car { model: 7, year: 2077 };
let person = Person { name: "Bob".to_string(), age: 42 };
// types are infrerred from the parameters
let car_json = serialize(&car);
let person_json = serialize(&person);
let _: Car = deserialize(car_json.as_bytes()); // output type can be inferred
let _ = deserialize::<Car>(car_json.as_bytes()); // requres turbofish
let _: Person = deserialize(person_json.as_bytes()); // works for `Person` too
}
You can make deserialize generic over the Deserialize trait:
fn deserialize<'a, T: Deserialize<'a>>(data: &'a [u8]) -> T {
let msg = str::from_utf8(data).unwrap();
serde_json::from_str(msg).unwrap()
}
note that you need some lifetimes, because some types needs to borrow from the deserialized string, doc.
You can make serialize generic too:
fn serialize<T: Serialize>(object: &T) -> String {
serde_json::to_string(object).unwrap()
}
playground
I have the following code (constrained example). There is a serial protocol, modelled with Command and Responce enums using enum_dispatch crate - every variant in enum is represented with a struct. Transport struct taskes care of serialization, execution and deserialization of Commands, returning TransportResults back with generic Responce enum(generic impl) or specific struct(templated impl). The generic implementation works just fine, but the templated one fails to compile.
use enum_dispatch::enum_dispatch;
use thiserror::Error;
trait Value{/**/}
trait CommandResponce {type Responce}
#[enum_dipatch(Value)]
enum Command{
Cmd1(Cmd1)
// ...
}
struct Cmd1{}
impl Value for Cmd1{ /**/ }
#[enum_dipatch(Value)]
enum Responce{
Resp1(Resp1)
// ...
}
struct Resp1{}
impl Value for Resp1{/**/}
impl CommandResponce for Cmd1{ type Responce=Resp1 }
#[derive(Error, Debug)]
pub enum ProtocolError{/**/}
type ProtocolResult<T> = Result<T, ProtocolError>;
struct Transport {/**/}
impl Transport {
// generic
pub fn command_generic_with_addr(
&mut self, addr: &mut u8, c: Command
) -> ProtocolResult<Responce>{ /**/ }
// templated
pub fn command_with_addr<T: SerializeCommand + CommandResponce>(
&mut self, addr: &mut u8, c: T) -> ProtocolResult<T::Responce>
where
Command: From<T>, Responce: TryInto<T::Responce, Error=&'static str>,
Responce: TryInto<T::Responce, Error= CommandErrors> {
let resp: Responce = self.command_generic_with_addr(addr, Command::from(c))?;
let ret: T::Responce = resp.try_into()?;
Ok(ret)
}
}
fn main() -> eyre::Result<()>{
let t = Transport::new();
let addr : u8 = 0xFF;
t.command_with_addr(&mut addr, Cmd1{/**/})
}
When I try to compile code, identical to the one above, I get the following error:
error[E0284]: type annotations needed: cannot satisfy `<Responce as TryInto<<T as CommandResponce>::Responce>>::Error == _`
-->
|
85 | let ret: T::Responce = match resp.try_into() {
| ^^^^^^^^ cannot satisfy `<Responce as TryInto<<T as CommandResponce>::Responce>>::Error == _`
I can't understand, what is the error here - I thought I've stated all the neccessary type annotations in the Transport::command_with_addr member function.
Note, enum_dispatch uses the following code to generate try_into conversion, used in the code above:
impl #impl_generics core::convert::TryInto<#variant_type> for #enumname #ty_generics #where_clause {
type Error = &'static str;
fn try_into(self) -> ::core::result::Result<#variant_type, <Self as core::convert::TryInto<#variant_type>>::Error> {
//..
}
}
The relevant bits of my model:
table! {
server_website {
id -> Integer,
server_database_id -> Nullable<Integer>,
server_id -> Integer,
}
}
table! {
server_database {
id -> Integer,
server_id -> Integer,
}
}
table! {
server {
id -> Integer,
}
}
joinable!(server_website -> server_database (server_database_id));
allow_tables_to_appear_in_same_query!(server_website, server_database);
#[derive(Queryable, Debug, Clone, PartialEq, Eq)]
pub struct Server {
pub id: i32,
}
#[derive(Queryable, Debug, Clone, PartialEq, Eq)]
pub struct ServerWebsite {
pub id: i32,
pub server_database_id: Option<i32>,
pub server_id: i32,
}
#[derive(Queryable, Debug, Clone, PartialEq, Eq)]
pub struct ServerDatabase {
pub id: i32,
pub server_id: i32,
}
I'm trying to express the following working SQLite query with Diesel:
select * from server_website where server_database_id in (
select id from server_database where server_id = 83
);
And here is my diesel query code:
use projectpadsql::schema::server::dsl as srv;
use projectpadsql::schema::server_database::dsl as db;
use projectpadsql::schema::server_website::dsl as srvw;
let database_ids_from_server = db::server_database
.filter(db::server_id.eq(83))
.select(db::id);
let used_websites = srvw::server_website
.filter(srvw::server_database_id.eq_any(database_ids_from_server))
.load::<ServerWebsite>(sql_conn)
.unwrap();
This fails with the error:
error[E0277]: the trait bound `diesel::query_builder::SelectStatement<projectpadsql::schema::server_database::table, diesel::query_builder::select_clause::SelectClause<projectpadsql::schema::server_database::columns::id>, diesel::query_builder::distinct_clause::NoDistinctClause, diesel::query_builder::where_clause::WhereClause<diesel::expression::operators::Eq<projectpadsql::schema::server_database::columns::server_id, diesel::expression::bound::Bound<diesel::sql_types::Integer, i32>>>>: diesel::expression::array_comparison::AsInExpression<diesel::sql_types::Nullable<diesel::sql_types::Integer>>` is not satisfied
I see that the compiler would like the inner query to have the type AsInExpression<Nullable<Integer>> which looks good to me (besides the nullable).
The inner query has more or less the type SelectStatement<server_database, SelectClause<server_database::id>>, which looks good to me since the id is Integer just like the outer query wants.
I have the feeling I'm quite close, but I can't put my finger on what the issue is.
I needed to use nullable() to get the types matching. Notice the db::id.nullable():
let database_ids_from_server = db::server_database
.filter(db::server_id.eq(server_id))
.select(db::id.nullable());
let used_websites = srvw::server_website
.filter(srvw::server_database_id.eq_any(database_ids_from_server))
.load::<ServerWebsite>(sql_conn)
.unwrap();
Georg Semmler - #weiznich on the Diesel Gitter channel answered this question.
I am using the Diesel crate to perform some database work. In some tables, two columns of the table should be treated together as a single key.
This pattern is repeated in many places in the database, so it would be nice to avoid a heap of repeated copy-paste code to handle this. However, I can't convince Diesel to automatically produce a type I can use in queries or inserts.
Consider the table
table! {
records (iid) {
iid -> Integer,
id_0 -> BigInt,
id_1 -> BigInt,
data -> Text,
}
}
and the ideal types
#[derive(Debug, Copy, Clone, FromSqlRow)]
pub struct RecordId {
id_0: i64,
id_1: i64,
}
#[derive(Queryable, Debug)]
pub struct Record {
pub iid: i32,
pub id: RecordId,
pub data: String,
}
This code compiles OK, but when I try to use it I get an error, for example:
pub fn find(connection: &SqliteConnection) -> types::Record {
records
.find(1)
.get_result::<types::Record>(connection)
.unwrap()
}
produces:
error[E0277]: the trait bound `(i32, types::RecordId, std::string::String): diesel::Queryable<(diesel::sql_types::Integer, diesel::sql_types::BigInt, diesel::sql_types::BigInt, diesel::sql_types::Text), _>` is not satisfied
--> src/main.rs:76:21
|
76 | records.find(1).get_result::<types::Record>(connection).unwrap()
| ^^^^^^^^^^ the trait `diesel::Queryable<(diesel::sql_types::Integer, diesel::sql_types::BigInt, diesel::sql_types::BigInt, diesel::sql_types::Text), _>` is not implemented for `(i32, types::RecordId, std::string::String)`
|
= help: the following implementations were found:
<(A, B, C) as diesel::Queryable<(SA, SB, SC), __DB>>
<(A, B, C) as diesel::Queryable<diesel::sql_types::Record<(SA, SB, SC)>, diesel::pg::Pg>>
= note: required because of the requirements on the impl of `diesel::Queryable<(diesel::sql_types::Integer, diesel::sql_types::BigInt, diesel::sql_types::BigInt, diesel::sql_types::Text), _>` for `types::Record`
= note: required because of the requirements on the impl of `diesel::query_dsl::LoadQuery<_, types::Record>` for `diesel::query_builder::SelectStatement<types::records::table, diesel::query_builder::select_clause::DefaultSelectClause, diesel::query_builder::distinct_clause::NoDistinctClause, diesel::query_builder::where_clause::WhereClause<diesel::expression::operators::Eq<types::records::columns::iid, diesel::expression::bound::Bound<diesel::sql_types::Integer, i32>>>>`
If I create a version which does not contain the RecordId but has the sub-pieces directly, then there is no error:
pub struct RecordDirect {
pub iid: i32,
pub id_0: i64,
pub id_1: i64,
pub data: String,
}
// ...
pub fn find_direct(connection: &SqliteConnection) -> types::RecordDirect {
records
.find(1)
.get_result::<types::RecordDirect>(connection)
.unwrap()
}
Similarly, I can manually implement the Queryable trait and that works OK too,
#[derive(Debug)]
pub struct RecordManual {
pub iid: i32,
pub id: RecordId,
pub data: String,
}
impl Queryable<records::SqlType, diesel::sqlite::Sqlite> for RecordManual {
type Row = (i32, i64, i64, String);
fn build(row: Self::Row) -> Self {
RecordManual {
iid: row.0,
id: RecordId {
id_0: row.1,
id_1: row.2,
},
data: row.3,
}
}
}
// ...
pub fn find_manual(connection: &SqliteConnection) -> types::RecordManual {
records
.find(1)
.get_result::<types::RecordManual>(connection)
.unwrap()
}
This case is ugly to maintain and I could not work out how to get it working for insertion — manually implementing Insertable seems a little trickier than Queryable.
To make this easier for anyone looking at it to play with, I've created a repository containing an almost compiling small reproducer containing the code blocks from this post. (Normally I'd put it up on the rust-playground, but that doesn't support diesel). You can find that code at https://github.com/mikeando/diesel_custom_type_demo.
Is there a way to make the #[derive(Queryable)] (and #[derive(Insertable)]) work for these kinds of cases?
A minimal reproducer for the failing initial case is:
#[macro_use]
extern crate diesel;
use diesel::prelude::*;
mod types {
use diesel::deserialize::Queryable;
use diesel::sqlite::SqliteConnection;
table! {
records (iid) {
iid -> Integer,
id_0 -> BigInt,
id_1 -> BigInt,
data -> Text,
}
}
#[derive(Debug, Copy, Clone, FromSqlRow)]
pub struct RecordId {
id_0: i64,
id_1: i64,
}
// Using a RecordId in a Record compiles, but
// produces an error when used in an actual query
#[derive(Queryable, Debug)]
pub struct Record {
pub iid: i32,
pub id: RecordId,
pub data: String,
}
}
use types::records::dsl::*;
pub fn find(connection:&SqliteConnection) -> types::Record {
records.find(1).get_result::<types::Record>(connection).unwrap()
}
Is there a way to make the #[derive(Queryable)] (and #[derive(Insertable)]) work for these kinds of cases?
For #[derive(Insertable)] this is simply possible by adding a #[diesel(embedded)] to your id field and a #[derive(Insertable)] on both structs. See the documentation of Insertable for details.
For #[derive(Queryable)] this is not possible because Queryable is supposed to be a plain mapping from query result to struct, with the basic assumption that the "shape" of the output remains the same (at least for the derive).