Given this enum definition:
#[repr(u8)]
#[derive(Debug, Serialize)]
pub enum AnimalType {
#[serde(rename = "cat")]
Cat = 1,
#[serde(rename = "dog")]
Dog = 2,
#[serde(rename = "snake")]
Snake = 3,
#[serde(rename = "capybara")]
Capybara = 4,
}
I'd like to be able to return JSON defining the above enum similar to this:
{
"AnimalType": [
{ "cat": 1 },
{ "dog": 2 },
{ "snake": 3 },
{ "capybara": 4 }
]
}
Is it possible to do this programmatically in Rust, possibly with serde?
This question differs from How do I configure Serde to use an enum variant's discriminant rather than name? as I want to serialize the definition of the enum, not a value.
I ended up using strum and its EnumVariantNames macro: https://docs.rs/strum_macros/latest/strum_macros/derive.EnumVariantNames.html
#[repr(u8)]
#[derive(Serialize, EnumVariantNames)]
pub enum AnimalType {
#[serde(rename = "cat")]
#[strum(serialize = "cat")]
Cat = 1,
#[serde(rename = "dog")]
#[strum(serialize = "dog")]
Dog = 2,
#[serde(rename = "SNAKE")]
#[strum(serialize = "SNAKE")]
Snake = 3,
#[serde(rename = "capybara")]
#[strum(serialize = "capybara")]
Capybara = 4,
}
async fn get_animal_types() -> Result<impl warp::Reply, warp::Rejection> {
Ok(warp::reply::json(&AnimalType::VARIANTS))
}
Returning ["cat","dog","SNAKE","capybara"]
Related
An API with this internally tagged field structure, with "topic" being the tag:
{
"topic": "Car"
"name": "BMW"
"HP": 250
}
This can be deserialized with
#[derive(Serialize, Deserialize)]
#[serde(tag = "topic")]
pub enum catalog {
CarEntry(Car),
... (other types)
}
#[derive(Serialize, Deserialize)]
pub struct Car {
pub name: String
pub HP: i32
}
It turns out that instead of reporting the topic as just Car, the API actually sends Car.product1 or Car.product2 etc.
This breaks the deserialization, because the deserializer doesn't know what the type is based on the string. Is there a way to supply a function to chop off the type string so that the correct model is found?
I don't think serde provides a way to mangle the tag before using it (at least I don't see anything relevant). And the generated serializers for tagged enums are relatively complex, with internal caching if the tag isn't the first field, and whatnot, so I wouldn't want to reproduce that in a custom deserializer.
The cheapest (but not necessarily most efficient) shot at this is to deserialize to serde_json::Value first, manually process the tag, and then deserialize the serde_json::Values to whatever struct you want.
Do that in a custom deserializer, and it starts looking reasonable:
impl<'de> Deserialize<'de> for Catalog {
fn deserialize<D>(d: D) -> Result<Self, <D as Deserializer<'de>>::Error>
where
D: Deserializer<'de>,
{
use serde_json::{Map, Value};
#[derive(Deserialize)]
struct Pre {
topic: String,
#[serde(flatten)]
data: Map<String, Value>,
}
let v = Pre::deserialize(d)?;
// Now you can mangle Pre any way you want to get your final structs.
match v.topic.as_bytes() {
[b'C', b'a', b'r', b'.', _rest # ..] => Ok(Catalog::CarEntry(
serde_json::from_value(v.data.into()).map_err(de::Error::custom)?,
)),
[b'B', b'a', b'r', b'.', _rest # ..] => Ok(Catalog::BarEntry(
serde_json::from_value(v.data.into()).map_err(de::Error::custom)?,
)),
_ => return Err(de::Error::unknown_variant(&v.topic, &["Car.…", "Bar.…"])),
}
}
}
Playground
Btw, what do you want to do with the suffix of topic? Throw it away? How do you plan on handling serialization if you do throw it away?
You can directly use enum instead of defining extra struct type.
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "topic")]
pub enum Catalog {
Car { name: String, hp: i32 }
}
fn main() {
let car = Catalog::Car { name: String::from("BMW"), hp: 2000 };
// Convert the Car to a JSON string.
let serialized = serde_json::to_string(&car).unwrap();
// Prints serialized = {"topic":"Car","name":"BMW","hp":2000}
println!("serialized = {}", serialized);
// Convert the JSON string back to a Car.
let deserialized: Catalog = serde_json::from_str(&serialized).unwrap();
// Prints deserialized = Car { name: "BMW", hp: 2000 }
println!("deserialized = {:?}", deserialized);
}
Playground
You can use #[serde(rename()] to rename type in output
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "topic")]
pub enum Catalog {
#[serde(rename(serialize = "Car", deserialize = "CarEntry"))]
CarEntry(Car),
}
#[derive(Serialize, Deserialize, Debug)]
pub struct Car {
pub name: String,
pub hp: i32
}
fn main() {
let car = Car { name: String::from("BMW"), hp: 2000 };
let catalog = Catalog::CarEntry(car);
// Convert the Car to a JSON string.
let serialized = serde_json::to_string(&catalog).unwrap();
// Prints serialized = {"topic":"Car","name":"BMW","hp":2000}
println!("serialized = {}", serialized);
// Convert the JSON string back to a Car.
let deserialized: Car = serde_json::from_str(&serialized).unwrap();
// Prints deserialized = Car { name: "BMW", hp: 2000 }
println!("deserialized = {:?}", deserialized);
}
Playground
I have a data model that I would like to be deserialized from "camelCase" to the rust standard "snake_case" when reading from a source, X. But I'd like to leave it in "snake_case" when reading or writing to another source, Y.
For example, the following code,
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
struct Data {
foo_bar: String,
hello_word: String,
}
can only be encoded and decoded in camel case. Even if I manually defined my Serialize and Deserialize implementations, I can't define multiple for the same struct. I could define a second struct that's a copy/paste of the other and then derive but that method would get tedious with multiple large structs. What I would really like to do is specify that rename_all attribute at run-time. But I'm not seeing any way to do that in serde's API.
I think the best way sigh is to just write out one struct Data_ per #[serde(rename_all = ...)], then write one additional struct Data that will be the in-memory representation (which won't be serializable, to remove ambiguity), then implement From in both directions for the Data_s and Data so that they're interconvertible.
Thankfully, we can use a macro so that we only have to specify the fields once. (It is incredibly disgusting nonetheless.)
This playground available here.
use serde::{Deserialize, Serialize}; // 1.0.130
use serde_json; // 1.0.69
macro_rules! interconvertible {
($T:ident <-> $U:ident, $($field_name:ident),*) => {
impl From<$T> for $U {
fn from(t: $T) -> Self {
let $T { $($field_name),* } = t;
Self { $($field_name),* }
}
}
impl From<$U> for $T {
fn from(u: $U) -> Self {
let $U { $($field_name),* } = u;
Self { $($field_name),* }
}
}
};
}
macro_rules! create_data_structs {
($($field_name:ident: $field_type:ty),* $(,)?) => {
#[derive(Serialize, Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct DataX {
$($field_name: $field_type),*
}
#[derive(Serialize, Deserialize, Debug)]
#[serde(rename_all = "snake_case")]
struct DataY {
$($field_name: $field_type),*
}
#[derive(Debug)]
struct Data {
$($field_name: $field_type),*
}
interconvertible!(DataX <-> Data, $($field_name),*);
interconvertible!(DataY <-> Data, $($field_name),*);
}
}
create_data_structs!(foo_bar: String, hello_world: String);
fn main() -> serde_json::Result<()> {
let x1: DataX = serde_json::from_str(r#"{"fooBar": "a", "helloWorld": "b"}"#)?;
let y1: DataY = serde_json::from_str(r#"{"foo_bar": "a", "hello_world": "b"}"#)?;
println!("{:?}, {:?}", x1, y1);
let x2: Data = x1.into();
let y2: Data = y1.into();
println!("{:?}, {:?}", x2, y2);
let x_string = serde_json::to_string(&DataX::from(x2))?;
let y_string = serde_json::to_string(&DataY::from(y2))?;
println!("{:?}, {:?}", x_string, y_string);
Ok(())
}
The output is:
DataX { foo_bar: "a", hello_world: "b" }, DataY { foo_bar: "a", hello_world: "b" }
[Data { foo_bar: "a", hello_world: "b" }, Data { foo_bar: "a", hello_world: "b" }]
"{\"fooBar\":\"a\",\"helloWorld\":\"b\"}", "{\"foo_bar\":\"a\",\"hello_world\":\"b\"}"
Since I'm only every decoding from source X I can utilize the #[serde(alias = ???)] macro. So my above use case would be
#[derive(Serialize, Deserialize)]
struct Data {
#[serde(alias="fooBar")]
foo_bar: String,
#[serde(alias="helloWorld")]
hello_word: String,
}
It's still a little tedious but better than an intermediate struct. It won't work though if I want to decode or encode to different cases.
(I'm not going to mark this as an answer because it's a work-around for my specific use case. If anyone has a more generic solution feel free to answer.)
How can I pass data to a protobuf oneof in tonic?
I couldn't find any instruction or example in the docs.
Tonic should generate an enum and corresponding type for each oneof variant. You'll have to match on that.
I'm assuming tonic = "0.4.3" and prost = "0.7.0" or higher for this answer. Let's use the following .proto definition as an example:
syntax = "proto3";
package stack_overflow;
service StackOverflow {
rpc Question (HelpRequest) returns (HelpResponse);
}
message HelpRequest {
oneof foo {
FroozleType froozle = 1;
FrobnikType frobnik = 2;
}
}
message FroozleType {
int32 value = 1;
}
message FrobnikType {}
message HelpResponse {
oneof bar {
QuxType qux = 1;
BazType baz = 2;
}
}
message QuxType {
int32 answer = 1;
}
message BazType {
string comment = 1;
}
When built, this generates the following types and traits. For the service:
stack_overflow_server::StackOverflow: the server trait
For the requests:
HelpRequest: the struct for the request message
help_request::Foo: an enum for the oneof foo in the request having an Froozle and a Frobnik variant
FroozleType: the struct of the FroozleType message
FrobnikType: the struct of the FrobnikType message
For the replies:
HelpResponse: the struct for the response message
help_response::Bar: an enum for the oneof bar in the response having an Qux and a Baz variant
QuxType: the struct of the QuxType message
BazType: the struct of the BazType message
For example, the HelpRequest struct and help_request::Foo enum are defined as such:
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct HelpRequest {
#[prost(oneof = "help_request::Foo", tags = "1, 2")]
pub foo: ::core::option::Option<help_request::Foo>,
}
/// Nested message and enum types in `HelpRequest`.
pub mod help_request {
#[derive(Clone, PartialEq, ::prost::Oneof)]
pub enum Foo {
#[prost(message, tag = "1")]
Froozle(super::FroozleType),
#[prost(message, tag = "2")]
Frobnik(super::FrobnikType),
}
}
Sticking it all together, an example implementation of the above service could look like this:
use tonic::{Request, Response, Status};
mod grpc {
tonic::include_proto!("stack_overflow");
}
#[derive(Debug)]
pub struct StackOverflowService {}
#[tonic::async_trait]
impl grpc::stack_overflow_server::StackOverflow for StackOverflowService {
async fn question(
&self,
request: Request<grpc::HelpRequest>,
) -> Result<Response<grpc::HelpResponse>, Status> {
let request = request.into_inner();
let bar = match request.foo {
Some(grpc::help_request::Foo::Froozle(froozle)) => {
Some(grpc::help_response::Bar::Qux(grpc::QuxType {
answer: froozle.value + 42,
}))
}
Some(grpc::help_request::Foo::Frobnik(_)) => {
Some(grpc::help_response::Bar::Baz(grpc::BazType {
comment: "forty-two".into(),
}))
}
None => None,
};
let reply = grpc::HelpResponse { bar };
return Ok(Response::new(reply));
}
}
I'm quite new to rust and come from an OOP background. So, maybe I misunderstood some rust basics.
I want to parse a fixed json-structure with serde. This structure represents one of different messages types. Each message has a numeric type attribute to distinguish it. The exact structure of the individual message types varies mostly, but they can also be the same.
{"type": 1, "sender_id": 4, "name": "sender", ...}
{"type": 2, "sender_id": 5, "measurement": 3.1415, ...}
{"type": 3, "sender_id": 6, "measurement": 13.37, ...}
...
First of all, I defined an enum to distinguish between message types also a struct for each type of message without a field storing the type.
#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "type")]
enum Message {
T1(Type1),
T2(Type2),
T3(Type3),
// ...
}
#[derive(Debug, Serialize, Deserialize)]
struct Type1 {
sender_id: u32,
name: String,
// ...
}
#[derive(Debug, Serialize, Deserialize)]
struct Type2 {
sender_id: u32,
measurement: f64,
// ...
}
#[derive(Debug, Serialize, Deserialize)]
struct Type3 {
sender_id: u32,
measurement: f64,
// ...
}
// ...
When I try to turn a string to a Message object, I get an error.
let message = r#"{"type":1,"sender_id":123456789,"name":"sender"}"#;
let message: Message = serde_json::from_str(message)?; // error here
// Error: Custom { kind: InvalidData, error: Error("invalid type: integer `1`, expected variant identifier", line: 1, column: 9) }
So, as I understood, serde tries to figure out the type of the current message but it needs a string
for that. I also tried to write my own deserialize()-function. I tried to get the numerical value
of the corresponding type-key and wanted to create the specific object by the type value.
How I have to implement the deserialize() to extract the type of the message and create the specific message object? Is it possible to write this without writing a deserialize()-function for each Type1/2/3/... struct?
impl<'de> Deserialize<'de> for Message {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where D: Deserializer<'de>,
{
// which functions I have to call?
}
Or is there a better solution to achieve my deserialization?
I prepared a playground for this issue: Playground
Serde doesn't support integer tags yet (see issue #745).
If you're able to change what's producing the data, then if you're able to change type into a string, i.e. "1" instead of 1. Then you can get it working simply using #[serde(rename)].
#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "type")]
enum Message {
#[serde(rename = "1")]
T1(Type1),
#[serde(rename = "2")]
T2(Type2),
#[serde(rename = "3")]
T3(Type3),
// ...
}
If that's not an option, then you indeed need to create a custom deserializer. The shortest in terms of code, is likely to deserialize into a serde_json::Value, and then match on the type, and deserialize the serde_json::Value into the correct Type{1,2,3}.
use serde_json::Value;
impl<'de> serde::Deserialize<'de> for Message {
fn deserialize<D: serde::Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
let value = Value::deserialize(d)?;
Ok(match value.get("type").and_then(Value::as_u64).unwrap() {
1 => Message::T1(Type1::deserialize(value).unwrap()),
2 => Message::T2(Type2::deserialize(value).unwrap()),
3 => Message::T3(Type3::deserialize(value).unwrap()),
type_ => panic!("unsupported type {:?}", type_),
})
}
}
You'll probably want to perform some proper error handling, instead of unwrapping and panicking.
If you need serialization as well, then you will likewise need a custom serializer. For this you could create a new type to serialize into, as you cannot use Message.
use serde::Serializer;
impl Serialize for Message {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
#[derive(Serialize)]
#[serde(untagged)]
enum Message_<'a> {
T1(&'a Type1),
T2(&'a Type2),
T3(&'a Type3),
}
#[derive(Serialize)]
struct TypedMessage<'a> {
#[serde(rename = "type")]
t: u64,
#[serde(flatten)]
msg: Message_<'a>,
}
let msg = match self {
Message::T1(t) => TypedMessage { t: 1, msg: Message_::T1(t) },
Message::T2(t) => TypedMessage { t: 2, msg: Message_::T2(t) },
Message::T3(t) => TypedMessage { t: 3, msg: Message_::T3(t) },
};
msg.serialize(serializer)
}
}
When using #[serde(flatten)], then it uses serde::private::ser::FlatMapSerializer, which is hidden from the documentation. In place of creating new types, you could use SerializeMap and FlatMapSerializer.
However, be warned, given it's undocumented, then any future release of serde could break your code if you're using FlatMapSerializer directly.
use serde::{private::ser::FlatMapSerializer, ser::SerializeMap, Serializer};
impl Serialize for Message {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut s = serializer.serialize_map(None)?;
let type_ = &match self {
Message::T1(_) => 1,
Message::T2(_) => 2,
Message::T3(_) => 3,
};
s.serialize_entry("type", &type_)?;
match self {
Message::T1(t) => t.serialize(FlatMapSerializer(&mut s))?,
Message::T2(t) => t.serialize(FlatMapSerializer(&mut s))?,
Message::T3(t) => t.serialize(FlatMapSerializer(&mut s))?,
}
s.end()
}
}
I'm using toml and num-bigint with the serde feature to deserialize the following data:
[[trades]]
action = "Buy"
date_time = 2019-04-15T15:36:00+01:00
fee = [1, [44000000]]
id = "#1"
price = [-1, [20154500000]]
quantity = [1, [200000000]]
But I'm getting this error:
Error: Error { inner: ErrorInner { kind: Custom, line: Some(7), col: 14, at: Some(156), message: "invalid value: integer `20154500000`, expected u32", key: ["trades", "price"] } }
Of course, if I make the price value smaller than u32::MAX, the program compiles fine. But I want to use this high value, because I'm scaling numbers by 1e8 to avoid dealing with floating-point arithmetic.
Is it possible to make serde deserialize BigInts to u64 instead?
use num_bigint::BigInt;
use serde_derive::Deserialize;
use toml::from_str;
use toml::value::Datetime;
#[derive(Debug, Deserialize)]
pub struct Trade {
pub action: String,
pub date_time: Datetime,
pub fee: BigInt,
pub id: Option<String>,
pub price: BigInt,
pub quantity: BigInt,
}
#[derive(Debug, Deserialize)]
pub struct TradeTable {
pub trades: Vec<Trade>,
}
fn main() {
let trades_string: String = String::from("[[trades]]\naction = \"Buy\"\ndate_time = 2019-04-15T15:36:00+01:00\nexchange = \"Degiro\"\nfee = [1, [44000000]]\nid = \"#1\"\nprice = [-1, [20154500000]]\nquantity = [1, [200000000]]");
let trade_table: TradeTable = from_str(&trades_string).unwrap();
let trades: Vec<Trade> = trade_table.trades;
}
Also, here's a link to a Rust Playground. Note that you will need to copy the code to your local machine, because you need the serde feature from num-bigint:
Cargo.toml
[dependencies.num-bigint]
version = "0.2.6"
features = ["serde"]
How did you create this file -- did you make it by serializing a BigInt, or did you write it by hand?
I wrote the data by hand.
Your data is invalid, the following work:
use num_bigint::BigInt;
use std::str::FromStr;
#[derive(Debug, serde::Serialize, serde::Deserialize, PartialEq)]
pub struct Trade {
pub price: BigInt,
}
fn main() {
let price = BigInt::from_str("-201545000000000").unwrap();
let trade = Trade { price };
let j = serde_json::to_string(&trade).unwrap();
println!("{}", j);
let a: Trade = serde_json::from_str(&j).unwrap();
assert_eq!(trade, a);
let j = toml::to_string(&trade).unwrap();
println!("{}", j);
let b: Trade = toml::from_str(&j).unwrap();
assert_eq!(trade, b);
}
You are not supposed to create it by hand.