Trying to Deserialize the response from a rust get request. The following request seems to work fine. However, I was wondering if there is a good way to model the TradeResult. Right now it has XXBTZUSD hard coded. I want this to be any type of pair. Tried using a HashMap<String, TradeResultTypes> type on the Trades result; was hoping this would allow both the last property and the pair being retrieved, but can't seem to get this to work unless the pair is hard coded as it is below.
extern crate serde;
#[macro_use]
extern crate serde_derive;
use std::collections::HashMap;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let resp: Trades = reqwest::blocking::get("https://api.kraken.com/0/public/Trades?pair=XBTUSD")?.json()?;
println!("{:?}", resp);
Ok(())
}
#[derive(Debug, Deserialize)]
struct Trades {
result: TradeResult,
error: Vec<u32>,
}
// #[derive(Debug, Deserialize)]
// struct Trades {
// result: HashMap<String, TradeResultTypes>,
// error: Vec<u32>,
// }
#[derive(Debug, Deserialize)]
struct TradeResult {
last: String,
XXBTZUSD: TradeData,
}
// (price, volume, time, side, orderType, misc)`
type TradeData = Vec<(String, String, f64, String, String, String)>;
#[derive(Debug, Deserialize)]
enum TradeResultTypes {
String,
TradeData,
}
You can use the untagged attribute to deserialize either a String or a TradeData:
#[derive(Debug, Deserialize)]
struct Trades {
result: HashMap<String, TradeResultTypes>,
error: Vec<u32>,
}
// (price, volume, time, side, orderType, misc)`
type TradeData = Vec<(String, String, f64, String, String, String)>;
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum TradeResultTypes {
String (String),
TradeData (TradeData),
}
This way, serde will first attempt to deserialize a String. If it works, it will return a TradeResultTypes::String. If it doesn't work, it will try to deserialize an array of tuples. If that works it will return a TradeResultTypes::TradeData, otherwise it will return an error.
Note that you need to tell Rust the type of value that is to be stored in each variant of the enum. In other words when I write String (String) inside the enum, the first String is only a label (which serde will ignore due to the untagged attribute) and the second String is the type of data that it's supposed to contain. It would work as well with:
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum TradeResultTypes {
Last (String),
Quotations (TradeData),
}
Related
I need to serialize a struct from a remote crate and all of the fields in the struct are private. There are getter's implemented in the remote struct to get those values. I am following this guidance and got it to work just fine for primitive types. However, I'm struggling with how to implement this for non-primitive types (ie: String) that the remote struct contains.
Below is a small piece of what I've implemented to frame the issue. The DataWrapper struct simply wraps Data, where Data is the remote struct.
#[derive(Serialize)]
pub struct DataWrapper {
#[serde(with = "DataDef")]
pub data: Data,
}
#[derive(Serialize)]
#[serde(remote = "remote_crate::data::Data")]
pub struct DataDef {
#[serde(getter = "Data::image_id")] // This works
image_id: u16,
#[serde(getter = "Data::description")] // This does not work
description: String,
}
The error I get when compiling this is
#[derive(Serialize)]
^^^^^^^^^ expected struct `std::string::String`, found `&str`
This makes sense, since the getter Data::description returns &str rather than a String. But, I'm not seeing a way in my code to coerce this so the compiler is happy.
If I change DataDef::description to be &str instead of String, then I have to implement lifetimes. But, when I do that, the compiler then says the remote "struct takes 0 lifetime arguments".
Appreciate any tips on how I can serialize this and other non-primitive types.
One approach you could do, so that you have full control of the serialization. Is to have the data wrapper be a copy of the struct fields you need, instead of the entire remote struct. Then you can implement From<remote_crate::data::Data> for DataWrapper and use serde without trying to coerce types.
#[derive(Serialize)]
pub struct Data {
image_id: u16,
description: String,
}
impl From<remote_crate::data::Data> for Data {
fn from(val: remote_crate::data::Data) -> Self {
Self {
image_id: val.image_id,
description: val.description.to_string(),
}
}
}
// Then you could use it like this:
// let my_data: Data = data.into();
I couldn't get it to work with an &str, but if you're OK with an allocation, you can write a custom getter:
mod local {
use super::remote::RemoteData;
use serde::{Deserialize, Serialize};
fn get_owned_description(rd: &RemoteData) -> String {
rd.description().into()
}
#[derive(Serialize)]
#[serde(remote = "RemoteData")]
pub struct DataDef {
#[serde(getter = "get_owned_description")]
description: String,
}
}
mod remote {
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize)]
pub struct RemoteData {
description: String,
}
impl RemoteData {
pub fn description(&self) -> &str {
&self.description
}
}
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e9c7c0b069d7e16b6faac2fa2b840c72
Example struct that will be created and serialized often:
pub struct PriceMessage
{
pub message_type: String, // this will always be "price"
}
I want "message_type" to always equal e.g. "price" in order to save allocating a new string every time I create and serialize this struct using serde_json crate.
If you wanted to have a compile constant default string value for a field, that could at run time later be replaced by a heap allocated string, you could do something like this:
pub struct SomeStruct {
pub some_field: Cow<'static, str>,
}
impl SomeStruct {
const SOME_FIELD_DEFAULT: Cow<'static, str> = Cow::Borrowed("Foobar!");
pub fn new() -> Self {
Self {
some_field: Self::SOME_FIELD_DEFAULT,
}
}
}
However if you want to have an actually constant field, this doesn't really make much sense in rust, and you should consider just using an associated constant.
I am using rust rocket as the http server side. On the server side, I define a rust struct like this to receive the data from client side:
#[derive(Deserialize, Serialize)]
#[allow(non_snake_case)]
pub struct PlayRecordRequest {
pub id: i64,
pub title: String,
pub url: String,
pub mvId: i64,
pub album: MusicAlbum,
pub artist: Vec<Artist>,
}
and recieve in the http controller like this:
#[post("/user/v1/save-play-record", data = "<record>")]
pub fn save_play_record<'r>(
record: Json<PlayRecordRequest>,
) -> content::Json<String> {
info!("save play record,title:{}",record.title);
save_play_record_impl(record);
let res = ApiResponse {
result: "ok".to_string(),
..Default::default()
};
let result_json = serde_json::to_string(&res).unwrap();
return content::Json(result_json);
}
the problem is that when the client side did not have some fields, the code run into error. is it possible to auto fit the field for Deserialize, if the client have the field, Deserialize it normally, if the client did not contains some fields, just ignore it and do not run into error. I read the official document of serde and find the skip annotation. But the annotation just use to mark the field should be ignored, what I want is that the struct could auto fit all field exists or not. is it possible to do like this?
There are two ways to handle this.
First is with options. Options imply that the data may or may not exist. If the data is null or missing it convert to an Option::none value. You can preserve the lack of data on serialization if you add #[serde(skip_serializing_if = "Option::is_none")]
Second option is to apply defaults to the value if the data is missing. Although from your use case this doesn't seem to be ideal.
Here is a code snippet of the two cases that you can run on https://play.rust-lang.org/:
use::serde::{Deserialize, Serialize};
#[derive(Deserialize, Serialize, Debug)]
#[allow(non_snake_case)]
pub struct Foo {
#[serde(skip_serializing_if = "Option::is_none")]
pub bar: Option<i64>,
#[serde(default = "default_baz")]
pub baz: i64,
}
fn default_baz()-> i64{
3
}
fn main(){
let data = r#"{"bar": null, "baz": 1}"#;
let v: Foo = serde_json::from_str(data).unwrap();
println!("{:#?}", v);
let s = serde_json::to_string(&v).unwrap();
println!("Don't serialize None values: {:#?}", s);
let data = r#"{"missing_bar": null}"#;
let v: Foo = serde_json::from_str(data).unwrap();
println!("{:#?}", v)
}
I'm using Serde to deserialize an XML file which has the hex value 0x400 as a string and I need to convert it to the value 1024 as a u32.
Do I need to implement the Visitor trait so that I separate 0x and then decode 400 from base 16 to base 10? If so, how do I do that so that deserialization for base 10 integers remains intact?
The deserialize_with attribute
The easiest solution is to use the Serde field attribute deserialize_with to set a custom serialization function for your field. You then can get the raw string and convert it as appropriate:
use serde::{de::Error, Deserialize, Deserializer}; // 1.0.94
use serde_json; // 1.0.40
#[derive(Debug, Deserialize)]
struct EtheriumTransaction {
#[serde(deserialize_with = "from_hex")]
account: u64, // hex
amount: u64, // decimal
}
fn from_hex<'de, D>(deserializer: D) -> Result<u64, D::Error>
where
D: Deserializer<'de>,
{
let s: &str = Deserialize::deserialize(deserializer)?;
// do better hex decoding than this
u64::from_str_radix(&s[2..], 16).map_err(D::Error::custom)
}
fn main() {
let raw = r#"{"account": "0xDEADBEEF", "amount": 100}"#;
let transaction: EtheriumTransaction =
serde_json::from_str(raw).expect("Couldn't derserialize");
assert_eq!(transaction.amount, 100);
assert_eq!(transaction.account, 0xDEAD_BEEF);
}
playground
Note how this can use any other existing Serde implementation to decode. Here, we decode to a string slice (let s: &str = Deserialize::deserialize(deserializer)?). You can also create intermediate structs that map directly to your raw data, derive Deserialize on them, then deserialize to them inside your implementation of Deserialize.
Implement serde::Deserialize
From here, it's a tiny step to promoting it to your own type to allow reusing it:
#[derive(Debug, Deserialize)]
struct EtheriumTransaction {
account: Account, // hex
amount: u64, // decimal
}
#[derive(Debug, PartialEq)]
struct Account(u64);
impl<'de> Deserialize<'de> for Account {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let s: &str = Deserialize::deserialize(deserializer)?;
// do better hex decoding than this
u64::from_str_radix(&s[2..], 16)
.map(Account)
.map_err(D::Error::custom)
}
}
playground
This method allows you to also add or remove fields as the "inner" deserialized type can do basically whatever it wants.
The from and try_from attributes
You can also place the custom conversion logic from above into a From or TryFrom implementation, then instruct Serde to make use of that via the from or try_from attributes:
#[derive(Debug, Deserialize)]
struct EtheriumTransaction {
account: Account, // hex
amount: u64, // decimal
}
#[derive(Debug, PartialEq, Deserialize)]
#[serde(try_from = "IntermediateAccount")]
struct Account(u64);
#[derive(Deserialize)]
struct IntermediateAccount<'a>(&'a str);
impl<'a> TryFrom<IntermediateAccount<'a>> for Account {
type Error = std::num::ParseIntError;
fn try_from(other: IntermediateAccount<'a>) -> Result<Self, Self::Error> {
// do better hex decoding than this
u64::from_str_radix(&other.0[2..], 16).map(Self)
}
}
See also:
How to transform fields during serialization using Serde?
The serde_json::to_string() function will generate a string which may include null for an Option<T>, or 0 for a u32. This makes the output larger, so I want to ignore these sorts of values.
I want to simplify the JSON string output of the following structure:
use serde_derive::Serialize; // 1.0.82
#[derive(Serialize)]
pub struct WeightWithOptionGroup {
pub group: Option<String>,
pub proportion: u32,
}
When group is None and proportion is 0, the JSON string should be "{}"
Thanks for the answerHow do I change Serde's default implementation to return an empty object instead of null?, it can resolve Optionproblem, but for 0 there is none solution.
The link Skip serializing field give me the answer.
And the fixed code:
#[derive(Debug, Clone, Serialize, Deserialize, Default, PartialEq, Ord, PartialOrd, Eq)]
pub struct WeightWithOptionGroup {
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub group: Option<String>,
#[serde(skip_serializing_if = "is_zero")]
#[serde(default)]
pub proportion: u32,
}
/// This is only used for serialize
#[allow(clippy::trivially_copy_pass_by_ref)]
fn is_zero(num: &u32) -> bool {
*num == 0
}
There's a couple of ways you could do this:
Mark each of the fields with a skip_serialising_if attribute to say when to skip them. This is much easier, but you'll have to remember to do it for every field.
Write your own Serde serialiser that does this custom JSON form. This is more work, but shouldn't be too bad, especially given you can still use the stock JSON deserialiser.
For those searching how to skip serialization for some enum entries you can do this
#[derive(Serialize, Deserialize)]
enum Metadata<'a> {
App, // want this serialized
Ebook, // want this serialized
Empty // dont want this serialized
}
#[derive(Serialize, Deserialize)]
struct Request<'a> {
request_id: &'a str,
item_type: ItemType,
#[serde(skip_serializing_if = "metadata_is_empty")]
metadata: Metadata<'a>,
}
fn metadata_is_empty<'a>(metadata: &Metadata<'a>) -> bool {
match metadata {
Metadata::Empty => true,
_ => false
}
}