serde: deserialize a field based on the value of another field - rust

Question

I would like to deserialize a wire format, like this JSON, into the Data structure below and I am failing to write the serde Deserialize implementations for the corresponding rust types.
{ "type": "TypeA", "value": { "id": "blah", "content": "0xa1b.." } }
enum Content {
TypeA(Vec<u8>),
TypeB(BigInt),
}
struct Value {
id: String,
content: Content,
}
struct Data {
typ: String,
value: Value,
}
The difficulty is selecting the correct value of the Content enumeration, which is based on the typ value.
As far as I know, deserialization in serde is stateless, an hence there is no way of either
knowing what the value of typ is at the time of deserialization of content (even though the deserialization order is guaranteed)
or injecting the value of typ in the deserializer then collecting it.
How can this be achieved with serde ?
I have looked at
serde_state but I cannot get the macros working and this library is wrapping serde, which worries me
DeserializeSeed but my undestanding is that it must be used in place of Deserialize for all types and my data model is big
The existing SO answers usually exploit the fact that the related fields are at the same level. This is not the case here: the actual data model is big, deep and the fields are "far apart"

Answer 1

Much simpler using tagging, but changing your data structure:
use serde::{Deserialize, Deserializer}; // 1.0.130
use serde_json; // 1.0.67
#[derive(Debug, Deserialize)]
#[serde(tag = "type", content = "value")]
enum Data {
TypeA(Value<String>),
TypeB(Value<u32>),
}
#[derive(Debug, Deserialize)]
struct Value<T> {
id: String,
content: T,
}
fn main() {
let input = r#"{"type": "TypeA", "value": { "id": "blah", "content": "0xa1b..."}}"#;
let data: Data = serde_json::from_str(input).unwrap();
println!("{:?}", data);
}
Playground
Also, you can write your own custom desializer using some intermediary serde_json::Value:
use serde::{Deserialize, Deserializer};// 1.0.130
use serde_json; // 1.0.67
#[derive(Debug)]
enum Content {
TypeA(String),
TypeB(String),
}
#[derive(Debug)]
struct Value {
id: String,
content: Content,
}
#[derive(Debug)]
struct Data {
typ: String,
value: Value,
}
impl<'de> Deserialize<'de> for Data {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let json: serde_json::value::Value = serde_json::value::Value::deserialize(deserializer)?;
let typ = json.get("type").expect("type").as_str().unwrap();
let value = json.get("value").expect("value");
let id = value.get("id").expect("id").as_str().unwrap();
let content = value.get("content").expect("content").as_str().unwrap();
Ok(Data {
typ: typ.to_string(),
value: Value {
id: id.to_string(),
content: {
match typ {
"TypeA" => Content::TypeA(content.to_string()),
"TypeB" => Content::TypeB(content.to_string()),
_ => panic!("Invalid type, but this should be an error not a panic"),
}
}
}
})
}
}
fn main() {
let input = r#"{"type": "TypeA", "value": { "id": "blah", "content": "0xa1b..."}}"#;
let data: Data = serde_json::from_str(input).unwrap();
println!("{:?}", data);
}
Playground
Disclaimer: I didn't handle error correctly and you could also extract the content matching into a function for example. The above code is just to illustrate the main idea.

Answer 2

There's a few different ways this can be solved, e.g. with a custom impl Deserialize for Data, then deserialize into a serde_json::Value, and then manually juggling between the types.
For a somewhat example of that, checkout this answer that I wrote in the past. It's not a one-to-one solution, but it might give some hints for implementing Deserialize manually, for what you want.
That being said. Personally, I prefer to minimize when I have to impl Deserialize manually, and instead deserialize into another type, and have it automatically convert using #[serde(from = "FromType")].
First, instead of type_: String, I'd suggest we introduce enum ContentType.
#[derive(Deserialize, Clone, Copy, Debug)]
enum ContentType {
TypeA,
TypeB,
TypeC,
TypeD,
}
Now, let's consider the types you introduced. I've added a few extra variants to Content, as you mentioned the variants can be different.
#[derive(Deserialize, Debug)]
#[serde(untagged)]
enum Content {
TypeA(Vec<u8>),
TypeB(Vec<u8>),
TypeC(String),
TypeD { foo: i32, bar: i32 },
}
#[derive(Deserialize, Debug)]
struct Value {
id: String,
content: Content,
}
#[derive(Deserialize, Debug)]
#[serde(try_from = "IntermediateData")]
struct Data {
#[serde(alias = "type")]
type_: ContentType,
value: Value,
}
Nothing crazy yet or much different. All the "magic" happens in the IntermediateData type, along with the impl TryFrom.
First, let's introduce a check_type(), which takes a ContentType and checks it against the Content. If the Content variant doesn't match the ContentType variant, then convert it.
In short, when using #[serde(untagged)] then when serde attempts to deserialize Content it will always return the first successful variant it can deserialize to if any. So if it can deserialize a Vec<u8>, then it will always result in Content::TypeA(). Knowing this, then in our check_type(), if the ContentType is TypeB and the Content is TypeA. Then we simply change it to TypeB.
impl Content {
// TODO: impl proper error type instead of `String`
fn check_type(self, type_: ContentType) -> Result<Self, String> {
match (type_, self) {
(ContentType::TypeA, content # Self::TypeA(_)) => Ok(content),
(ContentType::TypeB, Self::TypeA(content)) => Ok(Self::TypeB(content)),
(ContentType::TypeC | ContentType::TypeD, content) => Ok(content),
(type_, content) => Err(format!(
"unexpected combination of {:?} and {:?}",
type_, content
)),
}
}
}
Now all we need is the intermediate IntermediateData, along with a TryFrom conversion, which calls check_type() on the Content.
#[derive(Deserialize, Debug)]
struct IntermediateData {
#[serde(alias = "type")]
type_: ContentType,
value: Value,
}
impl TryFrom<IntermediateData> for Data {
// TODO: impl proper error type instead of `String`
type Error = String;
fn try_from(mut data: IntermediateData) -> Result<Self, Self::Error> {
data.value.content = data.value.content.check_type(data.type_)?;
Ok(Data {
type_: data.type_,
value: data.value,
})
}
}
That's all. Now we can test it against the following:
// serde = { version = "1", features = ["derive"] }
// serde_json = "1.0"
use std::convert::TryFrom;
use serde::Deserialize;
// ... all the previous code ...
fn main() {
let json = r#"{ "type": "TypeA", "value": { "id": "foo", "content": [0, 1, 2, 3] } }"#;
let data: Data = serde_json::from_str(json).unwrap();
println!("{:#?}", data);
let json = r#"{ "type": "TypeB", "value": { "id": "foo", "content": [0, 1, 2, 3] } }"#;
let data: Data = serde_json::from_str(json).unwrap();
println!("{:#?}", data);
let json = r#"{ "type": "TypeC", "value": { "id": "bar", "content": "foo" } }"#;
let data: Data = serde_json::from_str(json).unwrap();
println!("{:#?}", data);
let json = r#"{ "type": "TypeD", "value": { "id": "baz", "content": { "foo": 1, "bar": 2 } } }"#;
let data: Data = serde_json::from_str(json).unwrap();
println!("{:#?}", data);
}
Then it correctly results in Datas with Content::TypeA, Content::TypeB, Content::TypeC, and the last one Content::TypeD.
Lastly. There is issue #939 which talks about adding a #[serde(validate = "...")]. However, it was created in 2017, so I wouldn't hold my breath on it.

Answer 3

DeserializeSeed can be mixed with normal Deserialize code. It does not need to be used for all types. Here, it is enough to use it to deserialize Value.
Playground
use serde::de::{DeserializeSeed, IgnoredAny, MapAccess, Visitor};
use serde::*;
use std::fmt;
#[derive(Debug)]
enum ContentType {
A,
B,
Unknown,
}
#[derive(Debug)]
enum Content {
TypeA(String),
TypeB(i32),
}
#[derive(Debug)]
struct Value {
id: String,
content: Content,
}
#[derive(Debug)]
struct Data {
typ: String,
value: Value,
}
impl<'de> Deserialize<'de> for Data {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
struct DataVisitor;
impl<'de> Visitor<'de> for DataVisitor {
type Value = Data;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("struct Data")
}
fn visit_map<A>(self, mut access: A) -> Result<Self::Value, A::Error>
where
A: MapAccess<'de>,
{
let mut typ = None;
let mut value = None;
while let Some(key) = access.next_key()? {
match key {
"type" => {
typ = Some(access.next_value()?);
}
"value" => {
let seed = match typ.as_deref() {
Some("TypeA") => ContentType::A,
Some("TypeB") => ContentType::B,
_ => ContentType::Unknown,
};
value = Some(access.next_value_seed(seed)?);
}
_ => {
access.next_value::<IgnoredAny>()?;
}
}
}
Ok(Data {
typ: typ.unwrap(),
value: value.unwrap(),
})
}
}
deserializer.deserialize_map(DataVisitor)
}
}
impl<'de> DeserializeSeed<'de> for ContentType {
type Value = Value;
fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
where
D: Deserializer<'de>,
{
struct ValueVisitor(ContentType);
impl<'de> Visitor<'de> for ValueVisitor {
type Value = Value;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("struct Value")
}
fn visit_map<A>(self, mut access: A) -> Result<Self::Value, A::Error>
where
A: MapAccess<'de>,
{
let mut id = None;
let mut content = None;
while let Some(key) = access.next_key()? {
match key {
"id" => {
id = Some(access.next_value()?);
}
"content" => {
content = Some(match self.0 {
ContentType::A => Content::TypeA(access.next_value()?),
ContentType::B => Content::TypeB(access.next_value()?),
ContentType::Unknown => {
panic!("Should not happen if type happens to occur before value, but JSON is unordered.");
}
});
}
_ => {
access.next_value::<IgnoredAny>()?;
}
}
}
Ok(Value {
id: id.unwrap(),
content: content.unwrap(),
})
}
}
deserializer.deserialize_map(ValueVisitor(self))
}
}
fn main() {
let j = r#"{"type": "TypeA", "value": {"id": "blah", "content": "0xa1b.."}}"#;
dbg!(serde_json::from_str::<Data>(j).unwrap());
let j = r#"{"type": "TypeB", "value": {"id": "blah", "content": 666}}"#;
dbg!(serde_json::from_str::<Data>(j).unwrap());
let j = r#"{"type": "TypeB", "value": {"id": "blah", "content": "Foobar"}}"#;
dbg!(serde_json::from_str::<Data>(j).unwrap_err());
}
The main downside of this solution is that you lose the possibility of deriving the code.