I have a struct containing data that's either a float or a one-dim ndarray:
#[derive(Serialize)]
pub struct Results {
pub data: Data,
}
#[derive(Serialize)]
#[serde(untagged)]
pub enum Data {
Float(f32),
Array1(ndarray::Array1<f32>),
}
When serializing Array1 with serde-json I want to discard the "v" and "dim" keys from ndarray and flatten the "data" key so the structure looks like this:
"{\"data\":[1.0,2.0,3.0]}"
How would you do that?
Related
I have three structs A, B and C that are being fed to a method process_data() as a list of JSON. All the three structs are serde serializable/deserializable.
They are defined below as follows:-
#[derive(Serialize, Deserialize)]
struct A {
pub a: u32,
}
#[derive(Serialize, Deserialize)]
struct B {
pub b: u32,
}
#[derive(Serialize, Deserialize)]
struct C {
pub c: u32,
}
The function signature looks like this
fn process_data(data: String) {}
data can have any of these structs but its guaranteed that one of A, B or C will be there
data = "A{a: 1}"
or data = "[A{a:1}, B{b:1}, C{c:1}]"
or data = "[B{b:1}, A{a:1}, C{c:1}]"
I am looking for a way to process the variable data through serde within process_data, such that I can extract the structs from the data stream.
What I have tried so far.
I tried defining a struct called Collect which holds all the structs like this:-
#[derive(Serialize, Deserialize)]
struct Collect {
pub a1: Option<A>
pub b1: Option<B>,
pub c1: Option<C>
}
and then process the data as follows:-
serde_json::from_str::<Collect>(data.as_str())
But the previous command throws an error. Also I am looking to preserve the order of the vector in which the data is coming
I am not sure if serde will work in this case.
I'll assume you wanted the following JSON data:
[{"a":1}, {"b":1}, {"c":1}]
So, you want to deserialize to Vec<Collect>:
{"a":1} only contains the subfield from struct A, no additional wrap for a1. Normally you handle missing levels by tagging with #[serde(flatten)]
{"a":1} doesn't contain the subfields from b1 or c1. Normally you handle missing fields by tagging (an Option) with #[serde(default)]
It seems that the combination of the two doesn't work on deserialization.
Instead, you can deserialize to an untagged enum:
#[derive(Serialize, Deserialize, Debug)]
#[serde(untagged)]
enum CollectSer {
A { a: u32 },
B { b: u32 },
C { c: u32 },
}
If you do absolutely want to use your Collect as is, with the Options, you can do that still:
#[derive(Serialize, Deserialize, Debug, Default)]
#[serde(from = "CollectSer")]
struct Collect {
#[serde(flatten)]
pub a1: Option<A>,
#[serde(flatten)]
pub b1: Option<B>,
#[serde(flatten)]
pub c1: Option<C>,
}
impl From<CollectSer> for Collect {
fn from(cs: CollectSer) -> Self {
match cs {
CollectSer::A { a } => Collect {
a1: Some(A { a }),
..Default::default()
},
CollectSer::B { b } => Collect {
b1: Some(B { b }),
..Default::default()
},
CollectSer::C { c } => Collect {
c1: Some(C { c }),
..Default::default()
},
}
}
}
I suggest you just stick with the enum though, it's a lot more rustic.
Playground
(Apologies if I misguessed the structure of your data, but if so, I suppose you can at least point out the difference with this?)
Trying to Deserialize the response from a rust get request. The following request seems to work fine. However, I was wondering if there is a good way to model the TradeResult. Right now it has XXBTZUSD hard coded. I want this to be any type of pair. Tried using a HashMap<String, TradeResultTypes> type on the Trades result; was hoping this would allow both the last property and the pair being retrieved, but can't seem to get this to work unless the pair is hard coded as it is below.
extern crate serde;
#[macro_use]
extern crate serde_derive;
use std::collections::HashMap;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let resp: Trades = reqwest::blocking::get("https://api.kraken.com/0/public/Trades?pair=XBTUSD")?.json()?;
println!("{:?}", resp);
Ok(())
}
#[derive(Debug, Deserialize)]
struct Trades {
result: TradeResult,
error: Vec<u32>,
}
// #[derive(Debug, Deserialize)]
// struct Trades {
// result: HashMap<String, TradeResultTypes>,
// error: Vec<u32>,
// }
#[derive(Debug, Deserialize)]
struct TradeResult {
last: String,
XXBTZUSD: TradeData,
}
// (price, volume, time, side, orderType, misc)`
type TradeData = Vec<(String, String, f64, String, String, String)>;
#[derive(Debug, Deserialize)]
enum TradeResultTypes {
String,
TradeData,
}
You can use the untagged attribute to deserialize either a String or a TradeData:
#[derive(Debug, Deserialize)]
struct Trades {
result: HashMap<String, TradeResultTypes>,
error: Vec<u32>,
}
// (price, volume, time, side, orderType, misc)`
type TradeData = Vec<(String, String, f64, String, String, String)>;
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum TradeResultTypes {
String (String),
TradeData (TradeData),
}
This way, serde will first attempt to deserialize a String. If it works, it will return a TradeResultTypes::String. If it doesn't work, it will try to deserialize an array of tuples. If that works it will return a TradeResultTypes::TradeData, otherwise it will return an error.
Note that you need to tell Rust the type of value that is to be stored in each variant of the enum. In other words when I write String (String) inside the enum, the first String is only a label (which serde will ignore due to the untagged attribute) and the second String is the type of data that it's supposed to contain. It would work as well with:
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum TradeResultTypes {
Last (String),
Quotations (TradeData),
}
In Rust i am receiving data from a websocket. For simplicity it looks like this:
[1, {"a": ["1.2345", 5, "9.8765"]}]
The string i get from the websocket is indeed double-quoted 'floating point values' (thus in actuality strings), and unquoted integers.
I want to deserialize this object into a struct. But since the return array "a" is of mixed type, I can't use something like:
struct MyStruct {
id: i32,
a: [f64; 3],
}
So I thought let's define another struct:
struct Ask {
price: f64,
whole_lot_volume: i64,
lot_volume: f64
}
struct MyStruct {
id: i32,
a: Ask
}
But how should I write the deserializer for this? Looking at the serde docs I understand that I should write a Visitor for Ask:
impl<'de> Visitor<'de> for Ask {
type Value = ...
}
But what would be the Value type then?
So I'm sure I am not correctly understanding how the deserialization process works. Or is the fact that the Websocket returns an array of mixed types just incompatible with the serde deserialization process?
Serde can deserialize to a Rust struct from sequence-like structures as well as map-like ones.
Your structs are almost right, but there is an extra layer of hierarchy in your JSON. If your JSON was:
{
"id": 1,
"a": [1.2345, 5, 9.8765]
}
then this will just work, with the right serde annotations:
use serde::{Serialize, Deserialize};
#[derive(Deserialize)]
struct Ask {
price: f64,
whole_lot_volume: i64,
lot_volume: f64,
}
#[derive(Deserialize)]
struct MyStruct {
id: i32,
a: Ask,
}
If you can't change the JSON, you can use an extra layer of structs to match:
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize)]
struct Order {
price: f64,
whole_lot_volume: i64,
lot_volume: f64,
}
#[derive(Debug, Deserialize)]
struct Ask {
a: Order,
}
#[derive(Debug, Deserialize)]
struct MyStruct {
id: i32,
a: Ask,
}
It is rare that you need to implement your own Visitor; the Deserialize macro provided by serde is quite customisable. However, if you want to omit the extra struct, that's what you'd have to do.
You may need to do more work if some of the numbers are represented as JSON strings, but you can still do that without a custom Visitor implementation. See:
How to transform fields during deserialization using Serde?
How to transform fields during serialization using Serde?
Serde field attributes
The serde_json::to_string() function will generate a string which may include null for an Option<T>, or 0 for a u32. This makes the output larger, so I want to ignore these sorts of values.
I want to simplify the JSON string output of the following structure:
use serde_derive::Serialize; // 1.0.82
#[derive(Serialize)]
pub struct WeightWithOptionGroup {
pub group: Option<String>,
pub proportion: u32,
}
When group is None and proportion is 0, the JSON string should be "{}"
Thanks for the answerHow do I change Serde's default implementation to return an empty object instead of null?, it can resolve Optionproblem, but for 0 there is none solution.
The link Skip serializing field give me the answer.
And the fixed code:
#[derive(Debug, Clone, Serialize, Deserialize, Default, PartialEq, Ord, PartialOrd, Eq)]
pub struct WeightWithOptionGroup {
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub group: Option<String>,
#[serde(skip_serializing_if = "is_zero")]
#[serde(default)]
pub proportion: u32,
}
/// This is only used for serialize
#[allow(clippy::trivially_copy_pass_by_ref)]
fn is_zero(num: &u32) -> bool {
*num == 0
}
There's a couple of ways you could do this:
Mark each of the fields with a skip_serialising_if attribute to say when to skip them. This is much easier, but you'll have to remember to do it for every field.
Write your own Serde serialiser that does this custom JSON form. This is more work, but shouldn't be too bad, especially given you can still use the stock JSON deserialiser.
For those searching how to skip serialization for some enum entries you can do this
#[derive(Serialize, Deserialize)]
enum Metadata<'a> {
App, // want this serialized
Ebook, // want this serialized
Empty // dont want this serialized
}
#[derive(Serialize, Deserialize)]
struct Request<'a> {
request_id: &'a str,
item_type: ItemType,
#[serde(skip_serializing_if = "metadata_is_empty")]
metadata: Metadata<'a>,
}
fn metadata_is_empty<'a>(metadata: &Metadata<'a>) -> bool {
match metadata {
Metadata::Empty => true,
_ => false
}
}
I want a compiler plugin to annotate a structure with some information. For example, the original struct has only one field:
struct X { x: i32 }
And I want to add another field:
struct X { x: i32, y: MARKTYPE }
As I looked into Rust compiler plugins, I decided to use the MultiModifier(SyntaxExtension) to do my work. enum ItemKind defines Struct(VariantData, Generics) and the VariantData stores the data fields:
#[derive(Clone, PartialEq, Eq, RustcEncodable, RustcDecodable, Hash, Debug)]
pub enum VariantData {
/// Struct variant.
///
/// E.g. `Bar { .. }` as in `enum Foo { Bar { .. } }`
Struct(Vec<StructField>, NodeId),
/// Tuple variant.
///
/// E.g. `Bar(..)` as in `enum Foo { Bar(..) }`
Tuple(Vec<StructField>, NodeId),
/// Unit variant.
///
/// E.g. `Bar = ..` as in `enum Foo { Bar = .. }`
Unit(NodeId),
}
StructField is defined as:
#[derive(Clone, PartialEq, Eq, RustcEncodable, RustcDecodable, Hash, Debug)]
pub struct StructField {
pub span: Span,
pub ident: Option<Ident>,
pub vis: Visibility,
pub id: NodeId,
pub ty: P<Ty>,
pub attrs: Vec<Attribute>,
}
I planned to insert a StructField, but I don't know how to make a Span for the field. Each Span contains a lo and a hi BytePos. The StructFields information looks like:
Fields 0: StructField {
span: Span {
lo: BytePos(432),
hi: BytePos(437),
expn_id: ExpnId(4294967295)
},
ident: Some(x#0),
vis: Inherited,
id: NodeId(4294967295),
ty: type(i32),
attrs: []
}
What is the proper way to insert the new field?
I know it would be easier to use a macro to do this job, but I want to know if it is plausible to insert the field by modifying the VariantData.