I have a custom field in my JSON which is coming dynamic and needs to be parsed to struct which has a HashMap field like following:
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;
use std::collections::HashMap;
#[derive(Serialize, Deserialize)]
struct MyStruct {
field1: String,
custom: HashMap<String, String>,
}
fn main() {
let json_string = r#"{"field1":"3","custom":{"custom1":"15000","custom2":"60"}}"#;
let my_struct = serde_json::from_str::<MyStruct>(json_string).unwrap();
println!("{}", serde_json::to_string(&my_struct).unwrap());
}
It works when my json string has string fields in custom field which can be easily parsed to string.
But the problem is my json string is:
let json_string_wrong = r#"{"field1":"3","custom":{"custom1":15000,"custom2":"60"}}"#; // Need to parse this
How to handle such castings in serde?
Serde provides serde_json::Value ( reference ) . It is an enum which contains data types like:
pub enum Value {
/// Represents a JSON null value.
Null,
/// Represents a JSON boolean.
Bool(bool),
/// Represents a JSON number, whether integer or floating point.
Number(Number),
/// Represents a JSON string.
String(String),
/// Represents a JSON array.
Array(Vec<Value>),
/// Represents a JSON object.
Object(Map<String, Value>),
}
You can use serde_json::Value as a value type for your HashMap. It is simply possible to pull data from serde_json::Value with using serde_json::from_value or use pattern matching. In your case i would use pattern matching, because only Integer types will be converted into a String and rest will be the same.
But you'll need to consider adding one more step after deserialize. Like
Creating shadow field for custom, will be filled after deserialization.
Or constructing new struct which contains custom as HashMap<String, String>.
Add a function to convert HashMap<String, Value> to HashMap<String, String>,
Implementation of this trait can solve your problem.
trait ToStringStringMap {
fn to_string_string_map(&self) -> HashMap<String, String>;
}
impl ToStringStringMap for HashMap<String, Value> {
fn to_string_string_map(&self) -> HashMap<String, String> {
self.iter()
.map(|(k, v)| {
let v = match v.clone() {
e # Value::Number(_) | e # Value::Bool(_) => e.to_string(),
Value::String(s) => s,
_ => {
println!(r#"Warning : Can not convert field : "{}'s value to String, It will be empty string."#, k);
"".to_string()
}
};
(k.clone(), v)
})
.collect()
}
}
Example: Playground
Note: Trait's name is not well chosen, suggestions are welcomed.
Related
I am delete a key from redis when using rust, this is part of the code looks like:
pub async fn del_redis_key(key: &str,) -> Result<()> {
let config_redis_string = get_config("redisConnectionStr");
let redis_con_string: &str = config_redis_string.as_str();
let redis_client = redis::Client::open(redis_con_string).expect("can create redis client");
let mut redis_conn = get_con(redis_client);
let mut redis_conn_unwrap = redis_conn.unwrap();
let del_result = redis_conn_unwrap.del(key).map_err(RedisCMDError)?;
FromRedisValue::from_redis_value(&del_result).map_err(|e| RedisTypeError(e).into())
}
now I want to output the delete result del_result as a string, I have tried to convert the result as a json string like this:
let result_json = serde_json::to_string(&del_result).unwrap();
println!("{}",result_json);
it seems did not work because the third party redis lib Value did not implement the serialize trait, the Value code looks like:
#[derive(PartialEq, Eq, Clone)]
pub enum Value {
/// A nil response from the server.
Nil,
/// An integer response. Note that there are a few situations
/// in which redis actually returns a string for an integer which
/// is why this library generally treats integers and strings
/// the same for all numeric responses.
Int(i64),
/// An arbitary binary data.
Data(Vec<u8>),
/// A bulk response of more data. This is generally used by redis
/// to express nested structures.
Bulk(Vec<Value>),
/// A status response.
Status(String),
/// A status response which represents the string "OK".
Okay,
}
is it possible to output the delete result as a string so that I could output into log? I am using the redis lib redis = "0.21.3".
Serde can actually derive for remote structures. Check this link for details: https://serde.rs/remote-derive.html
But in your particular example, things get a bit more tricky. Because the Value type is recursive (Bulk(Vec<Value>)) and currently you cannot use #[serde(with = ...)] inside a Vec.
A way out is to define your own serializing function for the whole Vec<Value> type. Here is an example: https://github.com/serde-rs/serde/issues/1211
I've implemented this method for you. It is not quite straightforward though.
use redis::Value;
use serde::Serialize;
#[derive(Serialize)]
#[serde(remote = "Value")]
pub enum ValueRef {
Nil,
Int(i64),
Data(Vec<u8>),
#[serde(with = "redis_value_vec")]
Bulk(Vec<Value>),
Status(String),
Okay,
}
mod redis_value_vec {
use super::ValueRef;
use redis::Value;
use serde::{Serialize, Serializer};
pub fn serialize<S: Serializer>(array: &[Value], serializer: S) -> Result<S::Ok, S::Error> {
#[derive(Serialize)]
struct W<'a>(#[serde(with = "ValueRef")] &'a Value);
serializer.collect_seq(array.iter().map(W))
}
}
fn main() {
#[derive(Serialize)]
struct W<'a>(#[serde(with = "ValueRef")] &'a Value);
let val = Value::Nil;
println!("{}", serde_json::to_string(&W(&val)).unwrap());
}
I suggest you submit a PR to that crate, adding a serde feature, optionally enabling this derive. This is very common.
If all you want to do is print out the result for logging purposes, Value implements Debug. So you can print it out with the debug format specifier:
println!("{:?}", del_result);
I am trying to deserialize a JSON structure which has an array of strings, which are basically aliases to the enum I want to retrieve.
The JSON structure is:
{
"some_field": "abc",
"species": ["homosapiens", "hs", "human"]
}
And my target structure is:
#[derive(Deserialize, Debug)]
enum Species {
#[serde(alias = "homosapiens", alias = "hs")]
HomoSapiens,
#[serde(alias = "musmusculus", alias = "mouse")]
MusMusculus
}
#[derive(Deserialize, Debug)]
struct MyStruct {
species: Species,
}
My goal is, that in deserializing, it sees that the "species" field in the JSON contains at least one matching alias. So for example, if the JSON data has "hs" in the "species" array, it is seen as the enum variant "HomoSapiens", and if it were "mouse" it would be the enum variant "MusMusculus".
I tried to add a custom deserialization method but I can not find out how to check if one of the string values matches any of the aliases.
fn deserialize_species<'de, D>(deserializer: D) -> Result<Species, D::Error>
where
D: Deserializer<'de>,
{
let species_names: Vec<String> = Deserialize::deserialize(deserializer)?;
// TODO: Check if any of these strings are an alias
Err(D::Error::custom("Could not deserialize species"))
}
Note: I did not forget to add the #[serde(deserialize_with)] tag.
Is there a method on the "Species" enum I can call that tries to serialize it from one of these strings which keeps the aliases in mind? I can not find it in the documentation but I might have looked over it.
This deserializes every string in the vector and returns the first species, which can be deserialized.
fn deserialize_species<'de, D>(deserializer: D) -> Result<Species, D::Error>
where
D: Deserializer<'de>,
{
let species_names: Vec<String> = Deserialize::deserialize(deserializer)?;
for sn in species_names {
if let Ok(species) = serde_plain::from_str(&sn) {
return Ok(species);
}
}
Err(D::Error::custom("Could not deserialize species"))
}
Is there a method on the "Species" enum I can call that tries to serialize it from one of these strings which keeps the aliases in mind?
The method is the deserialize method. serde does not offer any way to inspect the generated code. But you can use a simple deserializer like serde_plain to use it.
I am receiving a string like below (Not in JSON or HashMap neither) as kind of key value pair from implicit JSONWebkey crate:
{ "kid":"kid-value",
"kty":"RSA",
"use":"sig",
"n":"n-value",
"e":"e-value" }
Now how can I convert to proper HashMap to extract key and value of "e" and "n"? Or is there a simpler way to extract exact value of "e" and "n"?
The string is JSON, so you should just parse it. By default serde_json ignores all unknown fields, so declaring a struct with only the needed fields is enough:
#[derive(serde::Deserialize)]
struct Values {
n: String,
e: String,
}
fn main() -> Result<()> {
let s = r#"{ "kid":"kid-value",
"kty":"RSA",
"use":"sig",
"n":"n-value",
"e":"e-value" }"#;
let value = serde_json::from_str::<Values>(s)?;
println!("{}", value.e);
println!("{}", value.n);
Ok(())
}
I'm given a data-format that includes a sequence of objects with exactly one named field value each. Can I remove this layer of indirection while deserializing?
When deserializing, the natural representation would be
/// Each record has it's own `{ value: ... }` object
#[derive(serde::Deserialize)]
struct Foobar<T> {
value: T,
}
/// The naive representation, via `Foobar`...
#[derive(serde::Deserialize)]
struct FoobarContainer {
values: Vec<Foobar<T>>,
}
While Foobar adds no extra cost beyond T, I'd like to remove this layer of indirection at the type-level:
#[derive(serde::Deserialize)]
struct FoobarContainer {
values: Vec<T>,
}
Can Foobar be removed from FoobarContainer, while still using it using deserialization?
In the general case, there's no trivial way to make this transformation. For that, review these existing answers:
How do I write a Serde Visitor to convert an array of arrays of strings to a Vec<Vec<f64>>?
How to transform fields during deserialization using Serde?
The first is my normal go-to solution and looks like this in this example.
However, in your specific case, you say:
objects with exactly one named field value
And you've identified a key requirement:
While Foobar adds no extra cost beyond T
This means that you can make Foobar have a transparent representation and use unsafe Rust to transmute between the types (although not actually with mem::transmute):
struct FoobarContainer<T> {
values: Vec<T>,
}
#[derive(serde::Deserialize)]
#[repr(transparent)]
struct Foobar<T> {
value: T,
}
impl<'de, T> serde::Deserialize<'de> for FoobarContainer<T>
where
T: serde::Deserialize<'de>,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let mut v: Vec<Foobar<T>> = serde::Deserialize::deserialize(deserializer)?;
// I copied this from Stack Overflow without reading the surrounding
// text that describes why this is actually safe.
let values = unsafe {
let data = v.as_mut_ptr() as *mut T;
let len = v.len();
let cap = v.capacity();
std::mem::forget(v);
Vec::from_raw_parts(data, len, cap)
};
Ok(FoobarContainer { values })
}
}
See also:
How do I convert a Vec<T> to a Vec<U> without copying the vector?
I'm parsing data into:
struct Data {
field1: Option<f32>,
field2: Option<u64>,
// more ...
}
The problem is that my input data format formats what would be a None in Rust as "n/a".
How do tell Serde that an Option<T> should be None for the specific string n/a, as opposed to an error? We can assume that this doesn't apply to a String.
This isn't the same question as How to deserialize "NaN" as `nan` with serde_json? because that's creating an f32 from a special value whereas my question is creating an Option<Anything> from a special value. It's also not How to transform fields during deserialization using Serde? as that still concerns a specific type.
You can write your own deserialization function that handles this case:
use serde::de::Deserializer;
use serde::Deserialize;
// custom deserializer function
fn deserialize_maybe_nan<'de, D, T: Deserialize<'de>>(
deserializer: D,
) -> Result<Option<T>, D::Error>
where
D: Deserializer<'de>,
{
// we define a local enum type inside of the function
// because it is untagged, serde will deserialize as the first variant
// that it can
#[derive(Deserialize)]
#[serde(untagged)]
enum MaybeNA<U> {
// if it can be parsed as Option<T>, it will be
Value(Option<U>),
// otherwise try parsing as a string
NAString(String),
}
// deserialize into local enum
let value: MaybeNA<T> = Deserialize::deserialize(deserializer)?;
match value {
// if parsed as T or None, return that
MaybeNA::Value(value) => Ok(value),
// otherwise, if value is string an "n/a", return None
// (and fail if it is any other string)
MaybeNA::NAString(string) => {
if string == "n/a" {
Ok(None)
} else {
Err(serde::de::Error::custom("Unexpected string"))
}
}
}
}
Then you can mark your fields with #[serde(default, deserialize_with = "deserialize_maybe_nan")] to use this function instead of the default function:
#[derive(Deserialize)]
struct Data {
#[serde(default, deserialize_with = "deserialize_maybe_nan")]
field1: Option<f32>,
#[serde(default, deserialize_with = "deserialize_maybe_nan")]
field2: Option<u64>,
// more ...
}
Working playground example
More information in the documentation:
deserialize_with serde attribute
Untagged enum representation