#[derive(Deserialize, Serialize)]
enum En {
A(i32),
B(String)
}
#[derive(Deserialize, Serialize)]
struct Data {
data: Mutex<HashSet<En>>
}
When using this structure, serializing the struct outputs something like {"data": [{"A": 0}, {"B": "String"}]}, I'd like to remove the list from the json and combine the data into one map ({"data": {"A": 0, "B": "String"}}).
I'd like to keep the HashSet for unique data and to force each variant's type, is there a way to do that without having to implement my own serialize or deserialize methods.
I've tried using serde_as from the serde_with crate with the as type Mutex<HashMap<String, _>>, though I keep getting errors saying "the trait bound HashMap<Same, Same>: SerializeAs<HashSet<En>> is not satisfied"
I have a partial solution using a HashMap instead of a HashSet, though I'd prefer if it was possible to keep using a set.
Using the serde_with crate seems to work
#[serde_as]
#[derive(Deserialize, Serialize)]
struct Data {
#[serde_as(as = "serde_with::EnumMap")
data: Mutex<Vec<En>>
}
As a replacement for a HashSet I used the strum crate AsRefStr trait, implemented PartialEq and Eq using it and used Vec.retain() when I inserted data
Related
I try to make a web API with rocket to try out the framework. I managed to return paginated results with a special struct that implements serializable.
However, the API I try to build depends on arbitrary values in a special dictionary. The received values may be strings, integers, bools, or other complex objects. The problem now is, that I'm not able to create a struct that contains "any" since Any is not serializable.
The basic idea would be something like this:
#[derive(Debug, Serialize, Deserialize)]
pub struct Foobar<'a> {
pub id: Uuid,
pub data: HashMap<&'a str, ??????>,
}
Even with enums, the problem remains since there is an infinite count of variations. Let's say, I use an enum to determine strings, bools, integers. When the containing type is another type, I need the json representation of that specific type. Basically another map with string -> any.
The current idea would be to use:
#[derive(Debug, Serialize, Deserialize)]
pub struct Foobar {
pub id: Uuid,
pub data: HashMap<String, rocket::serde::json::Value>,
}
But I don't know how the API will fare when there are non json values (e.g. msgpack).
Has somebody accomplished such a feat with rust/rocket?
After several tries with traits and std::any::Any, I ended up with:
pub type DataObject = HashMap<String, DataValue>;
#[derive(Debug, Serialize, Deserialize, ToSchema)]
pub enum DataValue {
String(String),
Integer(i64),
Float(f64),
Boolean(bool),
Date(String),
Time(String),
DateTime(String),
Reference(DataObject),
ArrayReference(Vec<DataObject>),
}
This is - more or less - the same as serde_json::Value. It helps to further check the values and validate them against a possible type description of the objects.
I have an enum similar to:
#[derive(Debug, Deserialize, Serialize)]
pub enum Gender {
Male,
Female,
NonBinary
}
and I a have an axum handler function which expects it to be passed in as Query(gender): Query<Gender>
But when I request the endpoint like this:
http://localhost:3000/greet?gender=Male
The page returns the following error message:
Failed to deserialize query string: invalid type: map, expected enum Gender
If I switch the params to use a struct instead everything works fine, which is why I know this is specifically an issue with enum serialization.
Any ideas how to fix?
You should use a struct as your query parameter type:
#[derive(Debug, Deserialize, Serialize)]
pub enum Gender {
Male,
Female,
NonBinary
}
#[derive(Debug, Deserialize)]
pub struct GreetParams {
gender: Gender,
}
async fn greet_handler(Query(params): Query<GreetParams>) {}
The reason you get an error is because ?gender=Male is parsed as a mapping of key-value pairs. Therefore, you need to use a mapped data structure that has named keys: something like BTreeMap<String, _>, HashMap<String, _>, or a struct.
If I switch the params to use a struct instead everything works fine, which is why I know this is specifically an issue with enum serialization. Any ideas how to fix?
There is nothing to fix; this is working as intended.
I make a request to a CouchDB database and get a response. Here's the struct for it:
use couch_rs::{types::{document::DocumentId}, CouchDocument, error::CouchResult};
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, CouchDocument, Default, Debug)]
pub struct GuildEntry {
#[serde(skip_serializing_if = "String::is_empty")]
pub _id: DocumentId,
#[serde(skip_serializing_if = "String::is_empty")]
pub _rev: String,
pub embedColor: String
}
I want to remove _id and _rev fields, or at least create a struct instance with all of the fields, except for those two. I can't afford creating a new struct without those fields and transform it from the response, because there's going to be a lot of fields in the future.
What have I tried/looked into so far:
I've looked into utilizing conversion to serde_json Value and then back, but I'll need to declare 2 identical structs, but with one has 2 fields missing.
I've also seen people mentioning composition, but it's kinda ugly and I'd like to avoid using it.
I can't use Option because CouchDocument requires that _id and _rev must be String and not Option<String>.
How can I accomplish this? Is this even possible?
Composition is really the only way. Though you don't have to make two separate structures for each; you can use one generic structure:
extern crate serde;
#[derive(serde::Deserialize)]
struct WithID<T> {
id: String,
#[serde(flatten)]
inner: T,
}
#[derive(serde::Deserialize)]
struct Foo {
bar: String,
baz: String,
}
type FooWithId = WithID<Foo>;
Implementing Deref and DerefMut for WithID with Target = T would make things easier, as you won't have to reference inner as much.
Say I have this struct:
use serde::{Serialize, Deserialize};
#[derive(Deserialize)]
struct MyStruct {
field_1: Option<usize>, // should only have field_1 or field_2
field_2: Option<usize>, // should only have field_1 or field_2
other_field: String,
}
How can I deserialize this but only allow one of these fields to exist?
The suggestions in the comments to use an enum are likely your best bet. You don't need to replace your struct with an enum, instead you'd add a separate enum type to represent this constraint, e.g.:
use serde::{Serialize, Deserialize};
#[derive(Deserialize)]
enum OneOf {
F1(usize), F2(usize)
}
#[derive(Deserialize)]
struct MyStruct {
one_of_field: OneOf,
other_field: String,
}
Now MyStruct's one_of_field can be initialized with either an F1 or an F2.
#dimo414's answer is correct about the enum being necessary, but the code sample will not function in the way the question is described. This is caused by a couple factors related to how enums are deserialized. Mainly, it will not enforce mutual exclusion of the two variants and will silently pick the first variant that matches and ignore extraneous fields. Another issue is the enum will be treated as a separate structure within MyStruct (Ex: {"one_of_field":{"F1":123},"other_field":"abc"} in JSON).
Solution
For anyone wanting an easy solution, here it is. However keep in mind that variants of the mutually exclusive type can not contain #[serde(flatten)] fields (more information in the issue section). To accommodate neither field_1 or field_2, Option<MutuallyExclusive> can be used in MyStruct.
/// Enum containing mutually exclusive fields. Variants names will be used as the
/// names of fields unless annotated with `#[serde(rename = "field_name")]`.
#[derive(Deserialize)]
enum MutuallyExclusive {
field_1(usize),
field_2(usize),
}
#[derive(Deserialize)]
/// `deny_unknown_fields` is required. If not included, it will not error when both
/// `field_1` and `field_2` are both present.
#[serde(deny_unknown_fields)]
struct MyStruct {
/// Flatten makes it so the variants of MutuallyExclusive are seen as fields of
/// this struct. Without it, foo would be treated as a separate struct/object held
/// within this struct.
#[serde(flatten)]
foo: MutuallyExclusive,
other_field: String,
}
The Issue
TL;DR: It should be fine to use deny_unknown_fields with flatten in this way so long as types used in MutuallyExclusive do not use flatten.
If you read the serde documentation, you may notice it warns that using deny_unknown_fields in conjunction with flatten is unsupported. This is problematic as it throws the long-term reliability of the above code into question. As of writing this, serde will not produce any errors or warnings about this configuration and is able to handle it as intended.
The pull request adding this warning cited 3 issues when doing so:
#[serde(flatten)] error behavior is different to normal one
deny_unknown_fields incorrectly fails with flattened untagged enum
Structs with nested flattens cannot be deserialized if deny_unknown_fields is set
To be honest, I don't really care about the first one. I personally feel it is a bit overly pedantic. It simply states that the error message is not exactly identical between a type and a wrapper for that type using flatten for errors triggered by deny_unknown_fields. This should not have any effect on the functionality of the code above.
However, the other two errors are relevant. They both relate to nested flatten types within a deny_unknown_fields type. Technically the second issue uses untagged for the second layer of nesting, but it has the same effect as flatten in this context. The main idea is that deny_unknown_fields is unable to handle more than a single level of nesting without causing issues. The use case is in any way at fault, but the way deny_unknown_fields and flattened are handled makes it difficult to implement a workaround.
Alternative
However, if anyone still feels uncomfortable with using the above code, you can use this version instead. It will be a pain to work with if there are a lot of other fields, but sidesteps the warning in the documentation.
#[derive(Debug, Deserialize)]
#[serde(untagged, deny_unknown_fields)]
enum MyStruct {
WithField1 {
field_1: usize,
other_field: String,
},
WithField2 {
field_2: usize,
other_field: String,
},
}
You can deserialize your struct and then verify that all the invariants your type should uphold. You can implement Deserialize for your type to this while also relying on the derive macro to do the heavy lifting.
use serde::{Deserialize, Deserializer};
#[derive(Debug, Deserialize)]
#[serde(remote = "Self")]
struct MyStruct {
field_1: Option<usize>, // should only have field_1 or field_2
field_2: Option<usize>, // should only have field_1 or field_2
other_field: String,
}
impl<'de> Deserialize<'de> for MyStruct {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
use serde::de::Error;
let s = Self::deserialize(deserializer)?;
if s.field_1.is_some() && s.field_2.is_some() {
return Err(D::Error::custom("should only have field_1 or field_2"));
}
Ok(s)
}
}
fn main() -> () {
dbg!(serde_json::from_value::<MyStruct>(serde_json::json!({
"field_1": 123,
"other_field": "abc"
})));
dbg!(serde_json::from_value::<MyStruct>(serde_json::json!({
"field_2": 456,
"other_field": "abc"
})));
dbg!(serde_json::from_value::<MyStruct>(serde_json::json!({
"field_1": 123,
"field_2": 456,
"other_field": "abc"
})));
}
Playground
I have a struct like this:
#[derive(Serialize, Deserialize)]
struct Thing {
pub small_header: Header,
pub big_body: Body,
}
I want to serialize this Thing to send over the network. I already have a Body available but I can't move it (imagine I am doing something with it, and every now and then I receive a command to temporarily stop what I'm doing and send over whatever data I have now) and I can't copy it (it's too big, possibly hundreds of megabytes).
So I'd like Serde to just borrow the one I have for serializing it, since it shouldn't need to be moved into a struct for that. If I rewrite Thing to take a reference, I obviously can't derive Deserialize!
The workaround I've been using is just using an Arc<Body> in my code, so that I can work with the body in my normal logic and when I need to serialize it, I can just do a cheap clone and put the Arc<Body> into the struct for serialization. During deserialization Serde will make a new Arc with a refcount of 1.
This still involves scattering Arc all over my code, which isn't great, not to mention the unnecessary (though minor) runtime cost. What is the correct solution for this use case?
The funny thing is that if I didn't have to send a header, then this would be a non-issue since I could serialize by reference and deserialize by value, but the presence of a header makes this impossible. I feel like I'm missing something about how Serde borrows data here...
You could use a Cow, which will be Cow::Borrowed when serializing and will deserialize as Cow::Owned when deserializing.
use std::borrow::Cow;
#[derive(Serialize, Deserialize)]
struct Thing<'a> {
small_header: Header,
big_body: Cow<'a, Body>,
}
Alternatively, you could serialize and deserialize as two independent data structures.
#[derive(Serialize)]
struct SerializeThing<'a> {
small_header: Header,
big_body: &'a Body,
}
#[derive(Deserialize)]
struct DeserializeThing {
small_header: Header,
big_body: Body,
}
Keep in mind, that serde always allocate (as default) (https://github.com/serde-rs/serde/issues/1852).
To be sure that you are really borrowing, use the #[serde(borrow)].
As example:
#[derive(Serialize, Deserialize)]
struct Thing<'a> {
small_header: Header,
#[serde(borrow)]
big_body: Cow<'a, Body>,
}