Custom diesel type - rust

I'm trying to implement a custom Diesel type using ToSql/FromSql traits.
My code now looks like this:
use diesel::{
backend::Backend,
deserialize,
serialize,
sql_types::VarChar,
};
#[derive(AsExpression, FromSqlRow, Debug)]
#[sql_type = "VarChar"]
pub struct File {
id: String,
}
impl File {
pub fn new(id: String) -> Self {
Self { id }
}
}
impl<B: Backend> serialize::ToSql<VarChar, B> for File {
fn to_sql(&self, out: &mut serialize::Output<B>) -> serialize::Result {
<String as serialize::ToSql<VarChar, B>>::to_sql(&self.id, out)
}
}
impl<B: Backend<RawValue=[u8]>> deserialize::FromSql<VarChar, B> for File {
fn from_sql(bytes: Option<&B::RawValue>) -> deserialize::Result<Self> {
<String as deserialize::FromSql<VarChar, B>>::from_sql(bytes).map(|id| File::new(id))
}
}
When I try to compile it, I receive a bunch of errors.
A couple of them are related to inability to detect names.
error: cannot find derive macro `AsExpression` in this scope
--> src\file.rs:8:10
|
8 | #[derive(AsExpression, FromSqlRow, Debug)]
| ^^^^^^^^^^^^
|
= note: consider importing this derive macro:
diesel::AsExpression
error: cannot find derive macro `FromSqlRow` in this scope
--> src\file.rs:8:24
|
8 | #[derive(AsExpression, FromSqlRow, Debug)]
| ^^^^^^^^^^
|
= note: consider importing one of these items:
crate::file::deserialize::FromSqlRow
diesel::FromSqlRow
error: cannot find attribute `sql_type` in this scope
--> src\file.rs:9:3
|
9 | #[sql_type = "VarChar"]
| ^^^^^^^^
I thought the problem was because I hadn't added Diesel's prelude to my scope. But, unfortunately, when I do this, I encounter the same problem.
The second problem looks like this:
error[E0212]: cannot use the associated type of a trait with uninferred generic
parameters
--> src\file.rs:30:32
|
30 | fn from_sql(bytes: Option<&B::RawValue>) -> deserialize::Result<Self> {
| ^^^^^^^^^^^
|
help: use a fully qualified path with inferred lifetimes
|
30 | fn from_sql(bytes: Option<&<B as backend::private::HasRawValue<'_>>::Ra
wValue>) -> deserialize::Result<Self> {
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The compiler requires a fully qualified syntax although each example of custom types shows almost the same code.

It is not clear to me what version of Diesel you are using; there were plenty of changes between v1.4.8 and v2.0.0 that are relevant here. It also seems like you are using aspects of each version, but for completeness' sake, I'll answer for both.
Diesel 1.4.8
Since this version was created back with Rust 2015 edition, it requires you to import macros like this at the root of your crate with #[macro_use] (as shown in the documentation for the table! macro for example).
The method ToSql::to_sql uses an additional W: std::io::Write type parameter that you omitted (maybe from looking at the newest version).
The complete example:
#[macro_use]
extern crate diesel;
use diesel::{backend::Backend, deserialize, serialize, sql_types::VarChar};
#[derive(AsExpression, FromSqlRow, Debug)]
#[sql_type = "VarChar"]
pub struct File {
id: String,
}
impl File {
pub fn new(id: String) -> Self {
Self { id }
}
}
impl<B: Backend> serialize::ToSql<VarChar, B> for File {
fn to_sql<W: std::io::Write>(&self, out: &mut serialize::Output<W, B>) -> serialize::Result {
<String as serialize::ToSql<VarChar, B>>::to_sql(&self.id, out)
}
}
impl<B: Backend<RawValue = [u8]>> deserialize::FromSql<VarChar, B> for File {
fn from_sql(bytes: Option<&B::RawValue>) -> deserialize::Result<Self> {
<String as deserialize::FromSql<VarChar, B>>::from_sql(bytes).map(|id| File::new(id))
}
}
Diesel 2.0.0
This version, the crate was updated to Rust 2018 edition, meaning you no longer needed to use #[macro_use] but you still have to import them as normal items.
The macro attributes all changes to be wrapped by diesel() to avoid inter-crate ambiguity. So #[sql_type = "VarChar"] was changed to #[diesel(sql_type = VarChar)].
The method FromSql::from_sql changed its parameter from Option<&B::RawValue> to RawValue<B>, and the ToSql::to_sql method sprouted some lifetime annotations. Both also needed an additional constraint since I don't think its guaranteed that a B: Backend has a String: ToSql/FromSql implementation.
The complete example:
use diesel::{
backend::{Backend, RawValue},
deserialize, serialize,
sql_types::VarChar,
AsExpression, FromSqlRow,
};
#[derive(AsExpression, FromSqlRow, Debug)]
#[diesel(sql_type = VarChar)]
pub struct File {
id: String,
}
impl File {
pub fn new(id: String) -> Self {
Self { id }
}
}
impl<B: Backend> serialize::ToSql<VarChar, B> for File
where
String: serialize::ToSql<VarChar, B>,
{
fn to_sql<'b>(&'b self, out: &mut serialize::Output<'b, '_, B>) -> serialize::Result {
<String as serialize::ToSql<VarChar, B>>::to_sql(&self.id, out)
}
}
impl<B: Backend> deserialize::FromSql<VarChar, B> for File
where
String: deserialize::FromSql<VarChar, B>,
{
fn from_sql(bytes: RawValue<B>) -> deserialize::Result<Self> {
<String as deserialize::FromSql<VarChar, B>>::from_sql(bytes).map(|id| File::new(id))
}
}

Related

Rust compile error - cannot infer type for type parameter

I am trying to write a simple cache implementation in rust.
The general idea is to fetch serialized object from remote location, deserialize it, and save it the deserialized object in the local cache. I want to use the DeserializeOwned trait in the cache because the cached object lifetime is longer than the raw data.
I started with the following code:
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
#[derive(Deserialize, Serialize)]
struct Demo{
s: u32
}
pub struct Cache<T : DeserializeOwned> {
cache: HashMap<String , T>,
}
impl<T: DeserializeOwned> Cache<T> {
pub fn new<K :DeserializeOwned>() -> Cache<K> {
let cache = HashMap::new();
Cache {
cache,
}
}
}
fn main() {
let cache = Cache::new::<Demo>();
}
and got the following error :
error[E0283]: type annotations needed
--> src/main.rs:96:17
|
84 | pub fn new<K :DeserializeOwned>() -> Cache<K> {
| ---------------- required by this bound in `Cache::<T>::new`
...
96 | let cache = Cache::new::<Demo>();
| ^^^^^^^^^^^^^^^^^^ cannot infer type for type parameter `T`
|
= note: cannot satisfy `_: DeserializeOwned`
But if i try to implement DeserializeOwned for Demo object the compiler complains about conflicting about conflicting implementation.
any help would be appreciated :)
You have two separate generic type parameters for Cache::new, and the fully qualified syntax with your example would be Cache::<Demo>::new::<Demo>(). This is probably not what you intended, since the type parameter on Cache itself is completely unused in Cache::new. Instead, I'd remove the type parameter from the function and instead use the existing type parameter from Cache, like so:
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
#[derive(Deserialize, Serialize)]
struct Demo{
s: u32
}
pub struct Cache<T : DeserializeOwned> {
cache: HashMap<String , T>,
}
impl<T: DeserializeOwned> Cache<T> {
pub fn new() -> Cache<T> {
let cache = HashMap::new();
Cache {
cache,
}
}
}
fn main() {
let cache = Cache::<Demo>::new();
}
Playground link

How to use a custom serde deserializer for chrono timestamps?

I'm trying to parse JSON into a struct which has a chrono::DateTime field. The JSON has the timestamps saved in a custom format that I wrote a deserializer for.
How do I connect the two and get it working using #[serde(deserialize_with)]?
I'm using NaiveDateTime for simpler code
extern crate serde;
extern crate serde_json;
use serde::Deserialize;
extern crate chrono;
use chrono::NaiveDateTime;
fn from_timestamp(time: &String) -> NaiveDateTime {
NaiveDateTime::parse_from_str(time, "%Y-%m-%dT%H:%M:%S.%f").unwrap()
}
#[derive(Deserialize, Debug)]
struct MyJson {
name: String,
#[serde(deserialize_with = "from_timestamp")]
timestamp: NaiveDateTime,
}
fn main() {
let result: MyJson =
serde_json::from_str(r#"{"name": "asdf", "timestamp": "2019-08-15T17:41:18.106108"}"#)
.unwrap();
println!("{:?}", result);
}
I'm getting three different compile errors:
error[E0308]: mismatched types
--> src/main.rs:11:10
|
11 | #[derive(Deserialize, Debug)]
| ^^^^^^^^^^^ expected reference, found type parameter
|
= note: expected type `&std::string::String`
found type `__D`
error[E0308]: mismatched types
--> src/main.rs:11:10
|
11 | #[derive(Deserialize, Debug)]
| ^^^^^^^^^^-
| | |
| | this match expression has type `chrono::NaiveDateTime`
| expected struct `chrono::NaiveDateTime`, found enum `std::result::Result`
| in this macro invocation
|
= note: expected type `chrono::NaiveDateTime`
found type `std::result::Result<_, _>`
error[E0308]: mismatched types
--> src/main.rs:11:10
|
11 | #[derive(Deserialize, Debug)]
| ^^^^^^^^^^-
| | |
| | this match expression has type `chrono::NaiveDateTime`
| expected struct `chrono::NaiveDateTime`, found enum `std::result::Result`
| in this macro invocation
|
= note: expected type `chrono::NaiveDateTime`
found type `std::result::Result<_, _>`
I'm pretty sure the from_timestamp function is returning a DateTime struct and not a Result, so I don't know what "expected struct chrono::NaiveDateTime, found enum std::result::Result" may mean.
While #edwardw's answer is technically correct it IMHO contains too much boilerplate.
NaiveDataTime implements FromStr which means you can write a reusable generic deserializer function.
A convoluted example - did add the age field (u8) represented as string in the JSON. Just to demonstrate that you can use it for anything that implements FromStr.
use std::fmt::Display;
use std::str::FromStr;
use chrono::NaiveDateTime;
use serde::{de, Deserialize, Deserializer};
#[derive(Deserialize, Debug)]
struct MyJson {
name: String,
#[serde(deserialize_with = "deserialize_from_str")]
timestamp: NaiveDateTime,
#[serde(deserialize_with = "deserialize_from_str")]
age: u8,
}
// You can use this deserializer for any type that implements FromStr
// and the FromStr::Err implements Display
fn deserialize_from_str<'de, S, D>(deserializer: D) -> Result<S, D::Error>
where
S: FromStr, // Required for S::from_str...
S::Err: Display, // Required for .map_err(de::Error::custom)
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
S::from_str(&s).map_err(de::Error::custom)
}
fn main() {
let result: MyJson = serde_json::from_str(
r#"{"name": "asdf", "timestamp": "2019-08-15T17:41:18.106108", "age": "11"}"#,
)
.unwrap();
println!("{:?}", result);
}
It's even easier if you want to specify format (use NaiveDateTime::parse_from_str):
use chrono::NaiveDateTime;
use serde::{de, Deserialize, Deserializer};
#[derive(Deserialize, Debug)]
struct MyJson {
name: String,
#[serde(deserialize_with = "naive_date_time_from_str")]
timestamp: NaiveDateTime,
}
fn naive_date_time_from_str<'de, D>(deserializer: D) -> Result<NaiveDateTime, D::Error>
where
D: Deserializer<'de>,
{
let s: String = Deserialize::deserialize(deserializer)?;
NaiveDateTime::parse_from_str(&s, "%Y-%m-%dT%H:%M:%S.%f").map_err(de::Error::custom)
}
fn main() {
let result: MyJson =
serde_json::from_str(r#"{"name": "asdf", "timestamp": "2019-08-15T17:41:18.106108"}"#)
.unwrap();
println!("{:?}", result);
}
#[serde(deserialize_with = "path")] documentation:
Deserialize this field using a function that is different from its implementation of Deserialize. The given function must be callable as fn<'de, D>(D) -> Result<T, D::Error> where D: Deserializer<'de>, although it may also be generic over T. Fields used with deserialize_with are not required to implement Deserialize.
This is rather involved, but the following works:
use chrono::NaiveDateTime;
use serde::de;
use serde::Deserialize;
use std::fmt;
struct NaiveDateTimeVisitor;
impl<'de> de::Visitor<'de> for NaiveDateTimeVisitor {
type Value = NaiveDateTime;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
write!(formatter, "a string represents chrono::NaiveDateTime")
}
fn visit_str<E>(self, s: &str) -> Result<Self::Value, E>
where
E: de::Error,
{
match NaiveDateTime::parse_from_str(s, "%Y-%m-%dT%H:%M:%S.%f") {
Ok(t) => Ok(t),
Err(_) => Err(de::Error::invalid_value(de::Unexpected::Str(s), &self)),
}
}
}
fn from_timestamp<'de, D>(d: D) -> Result<NaiveDateTime, D::Error>
where
D: de::Deserializer<'de>,
{
d.deserialize_str(NaiveDateTimeVisitor)
}
#[derive(Deserialize, Debug)]
struct MyJson {
name: String,
#[serde(deserialize_with = "from_timestamp")]
timestamp: NaiveDateTime,
}
fn main() {
let result: MyJson =
serde_json::from_str(r#"{"name": "asdf", "timestamp": "2019-08-15T17:41:18.106108"}"#)
.unwrap();
println!("{:?}", result);
}

Trying to apply polymorphism with Box<_> has the error "cannot move a value ... the size cannot be statically determined"

I am applying the polymorphism solution in Rust to my problem. I would like to use this solution with Box<_> as it seems the most straight and simple, but it doesn't work.
#[derive(Clone, Copy)]
pub struct NewPost;
#[derive(Clone, Copy)]
pub struct Post;
#[derive(Clone, Copy)]
pub struct PgConnection;
#[derive(Clone, Copy)]
pub struct DBPost;
pub trait DBAdapter {
fn create(self, post: NewPost) -> Post;
fn read(self) -> Vec<Post>;
}
impl DBPost {
// DATABASE classes
pub fn establish_connection(self) -> PgConnection {
unimplemented!()
}
}
impl DBAdapter for DBPost {
fn create(self, _post: NewPost) -> Post {
unimplemented!()
}
fn read(self) -> Vec<Post> {
unimplemented!()
}
}
struct GetPostsCase {
db: Box<dyn DBAdapter>,
}
impl GetPostsCase {
pub fn new(db: Box<dyn DBAdapter>) -> GetPostsCase {
GetPostsCase { db: db }
}
pub fn run(&self) -> Vec<Post> {
let result = self.db.read();
result
}
}
The error is:
error[E0161]: cannot move a value of type dyn DBAdapter: the size of dyn DBAdapter cannot be statically determined
--> src/lib.rs:45:22
|
45 | let result = self.db.read();
| ^^^^^^^
error[E0507]: cannot move out of `*self.db` which is behind a shared reference
--> src/lib.rs:45:22
|
45 | let result = self.db.read();
| ^^^^^^^ move occurs because `*self.db` has type `dyn DBAdapter`, which does not implement the `Copy` trait
Your read method takes the (unsized) value instead of taking a reference (whose size is always the same).
You can solve the problem by changing the contract of DBAdapter
from
fn read(self) -> Vec<Post> {
to
fn read(&self) -> Vec<Post> {
// ^--- added the ampersand
(depending on your implementation you might need &mut)

How can I mass implement Deserialize for all types that implement a specific trait?

I am deserializing a YAML config file with Serde. For most structs I deserialize into, things are quite simple — there's a one-to-one relationship between the fields of the structs and the properties in my YAML file.
In a few cases, things are a bit more complicated. For these, the properties in the YAML file are better viewed as parameters to the constructor. The actual struct will have different fields, calculated from those.
For these cases, I have written separate config structs that I deserialize into. For simplicity, consider this silly example:
struct Message {
text: String,
}
impl Message {
fn from_config(config: MessageConfig) -> Message {
Message {
text: format!("{} {}", config.first_half, config.second_half),
}
}
}
#[derive(Deserialize)]
struct MessageConfig {
first_half: String,
second_half: String,
}
To have Serde do the conversion from MessageConfig to Message for me, I implemented Deserialize for Message:
impl<'de> Deserialize<'de> for Message {
fn deserialize<D>(deserializer: D) -> Result<Message, D::Error>
where
D: Deserializer<'de>,
{
MessageConfig::deserialize(deserializer).map(|config| Message::from_config(config))
}
}
This works, but there would be a lot of copy pasting of the deserialization code involved if I were to do this for every struct, so I figured I should make a trait out of it:
use serde::{Deserialize, Deserializer};
use serde_json;
#[macro_use]
extern crate serde_derive;
trait Configurable {
type Config;
fn from_config(config: Self::Config) -> Self;
}
impl<'de, T, C> Deserialize<'de> for T
where
T: Configurable<Config = C>,
C: Deserialize<'de>,
{
fn deserialize<D>(deserializer: D) -> Result<T, D::Error>
where
D: Deserializer<'de>,
{
Self::Config::deserialize(deserializer).map(|config| Self::from_config(config))
}
}
struct Message {
text: String,
}
impl<'de> Configurable for Message {
type Config = MessageConfig;
fn from_config(config: MessageConfig) -> Message {
Message {
text: format!("{} {}", config.first_half, config.second_half),
}
}
}
#[derive(Deserialize)]
struct MessageConfig {
first_half: String,
second_half: String,
}
However, the compiler is not happy about this:
error[E0119]: conflicting implementations of trait `_IMPL_DESERIALIZE_FOR_MessageConfig::_serde::Deserialize<'_>` for type `std::boxed::Box<_>`:
--> src/lib.rs:11:1
|
11 | / impl<'de, T, C> Deserialize<'de> for T
12 | | where
13 | | T: Configurable<Config = C>,
14 | | C: Deserialize<'de>,
... |
21 | | }
22 | | }
| |_^
|
= note: conflicting implementation in crate `serde`:
- impl<'de, T> _IMPL_DESERIALIZE_FOR_MessageConfig::_serde::Deserialize<'de> for std::boxed::Box<T>
where T: _IMPL_DESERIALIZE_FOR_MessageConfig::_serde::Deserialize<'de>;
= note: downstream crates may implement trait `Configurable` for type `std::boxed::Box<_>`
error[E0210]: type parameter `T` must be used as the type parameter for some local type (e.g. `MyStruct<T>`)
--> src/lib.rs:11:1
|
11 | / impl<'de, T, C> Deserialize<'de> for T
12 | | where
13 | | T: Configurable<Config = C>,
14 | | C: Deserialize<'de>,
... |
21 | | }
22 | | }
| |_^ type parameter `T` must be used as the type parameter for some local type
|
= note: only traits defined in the current crate can be implemented for a type parameter
The error messages make little sense to me. What does Box have to do with anything? And is it somehow possible to make this trait work?
I am not sure if there's a way to define such a broad trait without causing conflicting implementations. What you could do is use a macro to avoid repetition:
use serde::{Deserialize, Deserializer};
use serde_json;
use serde_json::Error;
#[macro_use]
extern crate serde_derive;
struct Message {
text: String,
}
#[derive(Deserialize)]
struct MessageConfig {
first_half: String,
second_half: String,
}
impl Message {
fn from_config(config: MessageConfig) -> Message {
Message {
text: format!("{} {}", config.first_half, config.second_half),
}
}
}
macro_rules! derive_configurable_serializer {
( $t:ident, $c:ident ) => {
impl<'de> Deserialize<'de> for $t {
fn deserialize<D>(deserializer: D) -> Result<$t, D::Error>
where
D: Deserializer<'de>,
{
$c::deserialize(deserializer).map(|config| $t::from_config(config))
}
}
};
}
derive_configurable_serializer!(Message, MessageConfig);
fn main() -> Result<(), Error> {
let data = r#"{ "first_half": "John", "second_half": "Doe" }"#;
let p: Message = serde_json::from_str(data)?;
println!("Hello, {}!", p.text);
Ok(())
}

Why is the failure::Fail trait bound not satisfied by my Result type alias?

I'm trying to implement event hooks as demonstrated by "simple event hooks in Rust" while also using the Error + ErrorKind pattern of the failure crate.
This is a stripped down version of my code:
#[macro_use]
extern crate failure;
use failure::{Backtrace, Context, Error, Fail};
use std::fmt;
#[derive(Debug)]
pub struct PortalError {
inner: Context<PortalErrorKind>,
}
impl Fail for PortalError {
fn cause(&self) -> Option<&Fail> {
self.inner.cause()
}
fn backtrace(&self) -> Option<&Backtrace> {
self.inner.backtrace()
}
}
impl fmt::Display for PortalError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
fmt::Display::fmt(&self.inner, f)
}
}
#[derive(Copy, Clone, PartialEq, Debug, Fail)]
pub enum PortalErrorKind {
#[fail(display = "Unknown Error")]
Unknown,
}
//----------------------------------------------------------
pub type PortalResult<T> = Result<PortalError, T>;
pub trait Portal {
fn get_something(&self) -> PortalResult<Vec<u32>>;
}
//----------------------------------------------------------
pub trait FeedApi<'a> {
type T: FeedApi<'a>;
fn new<P: Portal + 'a>(portal: P) -> Result<Self::T, Error>;
}
//----------------------------------------------------------
pub struct Feedly<'a> {
portal: Box<Portal + 'a>,
}
impl<'a> FeedApi<'a> for Feedly<'a> {
type T = Feedly<'a>;
fn new<P: Portal + 'a>(portal: P) -> Result<Self::T, Error> {
Ok(Feedly {
portal: Box::new(portal),
})
}
}
impl<'a> Feedly<'a> {
pub fn demo_function(&self) -> Result<(), Error> {
let _ = self.portal.get_something().context(PortalErrorKind::Unknown)?;
Ok(())
}
}
fn main() {
println!("Hello, world!");
}
[dependencies]
failure = "0.1.1"
In a method of 'Feedly' I want to use the portal:
self.portal.get_something().context(PortalErrorKind::Unknown)?
But I get the following error:
error[E0599]: no method named `context` found for type `std::result::Result<PortalError, std::vec::Vec<u32>>` in the current scope
--> src/main.rs:67:45
|
67 | let _ = self.portal.get_something().context(PortalErrorKind::Unknown)?;
| ^^^^^^^
|
= note: the method `context` exists but the following trait bounds were not satisfied:
`std::result::Result<PortalError, std::vec::Vec<u32>> : failure::Fail`
`&std::result::Result<PortalError, std::vec::Vec<u32>> : failure::Fail`
`&mut std::result::Result<PortalError, std::vec::Vec<u32>> : failure::Fail`
Looking through the docs the failure::Fail trait has a bound 'static. And the method context has a bound Self: Sized.
I'm not sure which trait is not satisfied here. The boxed Portal is neither Sized nor 'static, but the returned result should be, right?
This is the first time I'm handling boxes and lifetimes in Rust.
pub type PortalResult<T> = Result<PortalError, T>;
Result has two type parameters: the success type and the error type. You have transposed them; you want:
pub type PortalResult<T> = Result<T, PortalError>;

Resources