In Rust i am receiving data from a websocket. For simplicity it looks like this:
[1, {"a": ["1.2345", 5, "9.8765"]}]
The string i get from the websocket is indeed double-quoted 'floating point values' (thus in actuality strings), and unquoted integers.
I want to deserialize this object into a struct. But since the return array "a" is of mixed type, I can't use something like:
struct MyStruct {
id: i32,
a: [f64; 3],
}
So I thought let's define another struct:
struct Ask {
price: f64,
whole_lot_volume: i64,
lot_volume: f64
}
struct MyStruct {
id: i32,
a: Ask
}
But how should I write the deserializer for this? Looking at the serde docs I understand that I should write a Visitor for Ask:
impl<'de> Visitor<'de> for Ask {
type Value = ...
}
But what would be the Value type then?
So I'm sure I am not correctly understanding how the deserialization process works. Or is the fact that the Websocket returns an array of mixed types just incompatible with the serde deserialization process?
Serde can deserialize to a Rust struct from sequence-like structures as well as map-like ones.
Your structs are almost right, but there is an extra layer of hierarchy in your JSON. If your JSON was:
{
"id": 1,
"a": [1.2345, 5, 9.8765]
}
then this will just work, with the right serde annotations:
use serde::{Serialize, Deserialize};
#[derive(Deserialize)]
struct Ask {
price: f64,
whole_lot_volume: i64,
lot_volume: f64,
}
#[derive(Deserialize)]
struct MyStruct {
id: i32,
a: Ask,
}
If you can't change the JSON, you can use an extra layer of structs to match:
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize)]
struct Order {
price: f64,
whole_lot_volume: i64,
lot_volume: f64,
}
#[derive(Debug, Deserialize)]
struct Ask {
a: Order,
}
#[derive(Debug, Deserialize)]
struct MyStruct {
id: i32,
a: Ask,
}
It is rare that you need to implement your own Visitor; the Deserialize macro provided by serde is quite customisable. However, if you want to omit the extra struct, that's what you'd have to do.
You may need to do more work if some of the numbers are represented as JSON strings, but you can still do that without a custom Visitor implementation. See:
How to transform fields during deserialization using Serde?
How to transform fields during serialization using Serde?
Serde field attributes
Related
I know that there is a similar question here, but I've not been able to make it fit my use case.
I have a Model struct that's nested into other structs. The model can have two different types of Config objects, a ModelConfig or a SeedConfig. They are nearly identical save for a few fields. As it stands now, I need two concrete implementations of Model (SeedModel and ModelModel) in order to change the config field, resulting in duplication of all the methods and trait implementations.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct MetaModel {
pub model: Model
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct Model {
pub name: String,
pub config: Option<ModelConfig>
}
What I've tried:
Using Generics: This pushes the generic type up the chain and results in very complex definitions and areas where I don't have the context to create the parent struct (i.e. the MetaModel definition has no access to the Model definition at creation).
This eventually results in a the type parameter C is not constrained by the impl trait, self type, or predicates unconstrained type parameter error
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct MetaModel<C> {
pub model: Model<C>
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct Model<C> {
pub name: String,
pub config: Option<C>
}
Trait Objects: This doesn't work because serde cannot serialize trait objects
pub trait Config {}
pub struct ModelConfig;
impl Config for ModelConfig {}
pub struct SeedConfig;
impl Config for SeedConfig {}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct Model {
pub name: String,
pub config: Option<Box<dyn Config>
}
What I'd like to do:
impl OtherTrait for Model {
type Value = Model;
fn build_model(&self, m: DerivedMeta) -> Result<Self::Value, &'static str> {
Ok(Model {
// Either a SeedConfig or a ModelConfig
})
}
}
What I would do is use a combination of #[serde(flatten)] and #[serde(untagged)]:
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
struct Config {
members: i32,
shared_by: String,
both: i64,
#[serde(flatten)]
specific: SpecificConfig,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
#[serde(untagged)]
enum SpecificConfig {
SeedConfig {
some_members: i16,
unique_to: String,
seed_config: u64,
},
ModelConfig {
other_members: i8,
not_shared_with_above: u32,
},
}
Serde explanation of flatten:
Flatten the contents of this field into the container it is defined in.
This removes one level of structure between the serialized representation and the Rust data structure representation.
Serde explanation of untagged:
There is no explicit tag identifying which variant the data contains. Serde will try to match the data against each variant in order and the first one that deserializes successfully is the one returned.
By combining these two, we get the following behavior:
flatten allows all shared fields and specific fields to be on the same level in the config
untagged allows us to avoid adding an explicit tag in the config
all shared properties are directly accessible
only specific properties require matching the specific enum
I need to serialize a struct from a remote crate and all of the fields in the struct are private. There are getter's implemented in the remote struct to get those values. I am following this guidance and got it to work just fine for primitive types. However, I'm struggling with how to implement this for non-primitive types (ie: String) that the remote struct contains.
Below is a small piece of what I've implemented to frame the issue. The DataWrapper struct simply wraps Data, where Data is the remote struct.
#[derive(Serialize)]
pub struct DataWrapper {
#[serde(with = "DataDef")]
pub data: Data,
}
#[derive(Serialize)]
#[serde(remote = "remote_crate::data::Data")]
pub struct DataDef {
#[serde(getter = "Data::image_id")] // This works
image_id: u16,
#[serde(getter = "Data::description")] // This does not work
description: String,
}
The error I get when compiling this is
#[derive(Serialize)]
^^^^^^^^^ expected struct `std::string::String`, found `&str`
This makes sense, since the getter Data::description returns &str rather than a String. But, I'm not seeing a way in my code to coerce this so the compiler is happy.
If I change DataDef::description to be &str instead of String, then I have to implement lifetimes. But, when I do that, the compiler then says the remote "struct takes 0 lifetime arguments".
Appreciate any tips on how I can serialize this and other non-primitive types.
One approach you could do, so that you have full control of the serialization. Is to have the data wrapper be a copy of the struct fields you need, instead of the entire remote struct. Then you can implement From<remote_crate::data::Data> for DataWrapper and use serde without trying to coerce types.
#[derive(Serialize)]
pub struct Data {
image_id: u16,
description: String,
}
impl From<remote_crate::data::Data> for Data {
fn from(val: remote_crate::data::Data) -> Self {
Self {
image_id: val.image_id,
description: val.description.to_string(),
}
}
}
// Then you could use it like this:
// let my_data: Data = data.into();
I couldn't get it to work with an &str, but if you're OK with an allocation, you can write a custom getter:
mod local {
use super::remote::RemoteData;
use serde::{Deserialize, Serialize};
fn get_owned_description(rd: &RemoteData) -> String {
rd.description().into()
}
#[derive(Serialize)]
#[serde(remote = "RemoteData")]
pub struct DataDef {
#[serde(getter = "get_owned_description")]
description: String,
}
}
mod remote {
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize)]
pub struct RemoteData {
description: String,
}
impl RemoteData {
pub fn description(&self) -> &str {
&self.description
}
}
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e9c7c0b069d7e16b6faac2fa2b840c72
I use Serde to deserialize a custom configuration file written in YAML. The file can contain definitions of various kinds that I represent as internally tagged enums:
OfKindFoo:
kind: Foo
bar: bar;
baz: baz;
OfKindQux:
kind: Qux
quux: qux;
I represent it in Rust like this:
#[derive(Deserialize)]
#[serde(tag = "kind")]
enum Definition {
Foo(Foo),
Qux(Qux),
}
#[derive(Deserialize)]
struct Foo {
bar: String,
baz: String,
}
#[derive(Deserialize)]
struct Qux {
quux: String,
}
I want the user to be able to omit the kind field completely, and when it is omitted Serde should default to deserializing it as Foo.
I started to implement Deserialize on Definition. I'm trying to deserialize it as a map and look for the kind key and return a respective enum variant based on this key and whether it is present.
I need to somehow "forward" the deserialization of other map fields to Foo::deserialize or Bar::deserialize, respectively. fn deserialize only takes one argument which is Deserializer. Is there a way to "convert" the map into a deserializer or otherwise get a deserializer that "starts" on that particular map?
I cannot use #[serde(other)] because it returns Err for a missing tag. Even if it didn't, the documentation states that other can only be applied to a "unit variant", a variant not containing any data.
You can mark the main enum as untagged and add tags to the sub-structs that do have a tag (this feature is not documented, but was added deliberately and so seems likely to stay). The variant without a tag should be declared after the other ones though, as serde will try to deserialize the variants in declared order with #[serde(untagged)]. Also note that if in your actual code, the variants and the structs have different names, or you're using #[serde(rename)], with this, the names of the structs are what matters for (de)serialization, not the variant names. All that applied to your example:
#[derive(Deserialize)]
#[serde(untagged)]
enum Definition {
Qux(Qux),
Foo(Foo), // variant that doesn't have a tag is the last one
}
#[derive(Deserialize)]
struct Foo {
bar: String,
baz: String,
}
#[derive(Deserialize)]
#[serde(tag = "kind")]
// if you want the tag to be "qux" instead of "Qux", do
// #[serde(rename = "qux")]
// here (or add `, rename = "qux"` to the previous serde attribute)
struct Qux {
quux: String,
}
If structs have the same shape or all the fields are optional, accepted answer won't work and will be deserialized to the first kind. playground
With the monostate crate it can be fixed.
use monostate::MustBe;
use serde::{Deserialize, Serialize};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum Action {
Hi(Hi),
Bye(Bye),
Unknown(Unknown),
}
#[derive(Debug, PartialEq, Serialize, Deserialize)]
pub struct Hi {
kind: MustBe!("Hi"),
name: String,
}
#[derive(Debug, PartialEq, Serialize, Deserialize)]
pub struct Bye {
kind: MustBe!("Bye"),
name: String,
}
#[derive(Debug, PartialEq, Serialize, Deserialize)]
pub struct Unknown {
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tests() {
assert_eq!(
Action::Hi(Hi { kind: Default::default(), name: "John".to_string() }),
serde_json::from_str::<Action>(r#"{"kind": "Hi", "name": "John"}"#).unwrap()
);
assert_eq!(
Action::Bye(Bye { kind: Default::default(), name: "John".to_string() }),
serde_json::from_str::<Action>(r#"{ "kind": "Bye", "name": "John" }"#).unwrap()
);
assert_eq!(
Action::Unknown(Unknown { }),
serde_json::from_str::<Action>(r#"{ "name": "John" }"#).unwrap()
);
assert_eq!(
Action::Unknown(Unknown { }),
serde_json::from_str::<Action>(r#"{}"#).unwrap()
);
}
}
The serde_json::to_string() function will generate a string which may include null for an Option<T>, or 0 for a u32. This makes the output larger, so I want to ignore these sorts of values.
I want to simplify the JSON string output of the following structure:
use serde_derive::Serialize; // 1.0.82
#[derive(Serialize)]
pub struct WeightWithOptionGroup {
pub group: Option<String>,
pub proportion: u32,
}
When group is None and proportion is 0, the JSON string should be "{}"
Thanks for the answerHow do I change Serde's default implementation to return an empty object instead of null?, it can resolve Optionproblem, but for 0 there is none solution.
The link Skip serializing field give me the answer.
And the fixed code:
#[derive(Debug, Clone, Serialize, Deserialize, Default, PartialEq, Ord, PartialOrd, Eq)]
pub struct WeightWithOptionGroup {
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub group: Option<String>,
#[serde(skip_serializing_if = "is_zero")]
#[serde(default)]
pub proportion: u32,
}
/// This is only used for serialize
#[allow(clippy::trivially_copy_pass_by_ref)]
fn is_zero(num: &u32) -> bool {
*num == 0
}
There's a couple of ways you could do this:
Mark each of the fields with a skip_serialising_if attribute to say when to skip them. This is much easier, but you'll have to remember to do it for every field.
Write your own Serde serialiser that does this custom JSON form. This is more work, but shouldn't be too bad, especially given you can still use the stock JSON deserialiser.
For those searching how to skip serialization for some enum entries you can do this
#[derive(Serialize, Deserialize)]
enum Metadata<'a> {
App, // want this serialized
Ebook, // want this serialized
Empty // dont want this serialized
}
#[derive(Serialize, Deserialize)]
struct Request<'a> {
request_id: &'a str,
item_type: ItemType,
#[serde(skip_serializing_if = "metadata_is_empty")]
metadata: Metadata<'a>,
}
fn metadata_is_empty<'a>(metadata: &Metadata<'a>) -> bool {
match metadata {
Metadata::Empty => true,
_ => false
}
}
I have JSON that looks like:
[{"range": [1, 2]}, {"range": [2, 5]}]
The objects in array have fields other than range of course, but it doesn't matter.
Would it be possible to deserialize them into tuples that have two phantom types to indicate whether the start and end are inclusive or exclusive automatically?
This could also be solved with deserialzing numbers into some kind of tuples with phantom types.
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
use std::marker::PhantomData;
#[derive(Debug)]
struct Inclusive;
#[derive(Debug)]
struct Exclusive;
#[derive(Deserialize)]
struct Range<S, E, V: Ord>(Option<V>, Option<V>, PhantomData<S>, PhantomData<E>);
fn main() {
let data = "[1, 2]";
let r: Range<Inclusive, Exclusive, i32> = serde_json::from_str(data).expect("Error");
println!("Range from {:?} to {:?}", r.0, r.1);
}
This doesn't work because serde_json seems to be ignorant about PhantomData and expects arrays of size 4, which can be solved by implementing Deserializer manually, which is exactly the thing I'd like to avoid.
I don't have high hopes, but maybe this can be done and I don't know something.
You seem to want serde to ignore certain fields entirely. That can be done with #[serde(skip)]. Serde will fetch a default value from Default::default(), which is available for PhantomData.
#[derive(Debug, Deserialize)]
struct Range<S, E, V: Ord>(
Option<V>,
Option<V>,
#[serde(skip)] PhantomData<S>,
#[serde(skip)] PhantomData<E>,
);
Playground
On (possibly) a side note, if your types Inclusive and Exclusive are always unit-like and not singletons, you might consider holding them directly instead of PhantomData, since they will also be zero-sized.
#[derive(Debug, Default)]
struct Inclusive;
#[derive(Debug, Default)]
struct Exclusive;
#[derive(Deserialize)]
struct Range<S, E, V: Ord>(
Option<V>,
Option<V>,
#[serde(skip)] S,
#[serde(skip)] E,
);