I have a struct that contains children of its own type. These children are wrapped in Arcs, and I'm getting issues when calling serde_json::to_string on it. My struct is:
#[derive(Serialize, Deserialize)]
pub struct Category {
pub id: i32,
pub name: String,
pub parent_id: i32,
pub children: Vec<Arc<Category>>,
}
This produces the error the trait 'serde::Serialize' is not implemented for 'std::sync::Arc<db::queries::categories::Category>' I've tried a few different approaches to get serialization working, such as:
#[serde(serialize_with = "arc_category_vec")]
pub children: Vec<Arc<Category>>
fn arc_category_vec<S>(value: &Vec<Arc<Category>>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut seq = serializer.serialize_seq(Some(value.len()))?;
for e in value {
seq.serialize_element(e.as_ref())?;
}
seq.end()
}
This doesn't help as I get the same error. I also tried:
impl Serialize for Arc<Category> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut state = serializer.serialize_struct("Category", 4)?;
state.serialize_field("id", &self.id)?;
state.serialize_field("name", &self.name)?;
state.serialize_field("parent_id", &self.parent_id)?;
state.serialize_field("children", &self.children)?;
state.end();
}
}
but that gives the error impl doesn't use types inside crate
I could probably live without deserialization, since serialization is more important at this point.
Serde provides implementations of Serialize and Deserialize for Arc<T> and Rc<T>, but only if the rc feature is enabled.
There's a comment on Serde's reference website explaining why:
Opt into impls for Rc<T> and Arc<T>. Serializing and deserializing these types does not preserve identity and may result in multiple copies of the same data. Be sure that this is what you want before enabling this feature.
To enable the rc feature, you need to ask for it in your own Cargo.toml:
[dependencies]
serde = { version = "1.0", features = ["rc"] }
Related
I have a trait DataSet, for which I've implemented Serialize like so:
impl Serialize for dyn DataSet {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut seq = serializer.serialize_map(Some(2))?;
seq.serialize_entry("fields", "Hello")?;
seq.serialize_entry("measures", "There")?;
seq.end()
}
}
Now, in program/application I am sharing pointer to the trait object:
let x: Arc<Mutex<dyn DataSet>> = Arc::new(Mutex::new(data));
Where data is any object which implements DataSet.
Now, I want to turn this object into json (ie serialize it):
serde_json::to_string(&x)
It works for Box instead of Arc<Mutex<>>. But with Arc<Mutex<>> compiler complains:
the size for values of type `dyn base_engine::DataSet` cannot be known at compilation time
the trait `Sized` is not implemented for `dyn base_engine::DataSet`
the trait `Serialize` is implemented for `Arc<T>`
required because of the requirements on the impl of `Serialize` for `Mutex<dyn base_engine::DataSet>`
I've tried adding feature ["rc"] to serde, but this didn't help.
Update: Since serde 1.0.145 this code works.
Original answer:
Serde doesn't support serializing a Mutex with unsized type. I sent a PR to relax this, but in the meantime you can use a newtype:
pub struct MutexWrapper<T: ?Sized>(pub Mutex<T>);
impl<T: ?Sized + Serialize> Serialize for MutexWrapper<T> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
self.0
.lock()
.expect("mutex is poisoned")
.serialize(serializer)
}
}
let x: Arc<MutexWrapper<dyn DataSet>> = Arc::new(MutexWrapper(Mutex::new(data)));
You still need to enable the rc feature of serde, of course.
I'm trying to deserialize data in a simple non-human readable and non-self describing format to Rust structs. I've implemented a custom Deserializer for this format and it works great when I'm deserializing the data into a struct like this for example:
#[derive(Serialize, Deserialize)]
pub struct Position {
x: f32,
z: f32,
y: f32,
}
However, let's say this Position struct had a new field added (could have been removed too) in a new version:
#[derive(Serialize, Deserialize)]
pub struct Position {
x: f32,
z: f32,
y: f32,
is_visible: bool, // This field was added in a new version
}
But I still need to support both data from both versions of Position. The version of the data (known at runtime) can be given to the Deserializer but how can the Deserializer know the version of a field (known at compile time)?
I've looked at #[serde(deserialize_with)] but it didn't work because I cannot get the needed version information.
I 've also looked at implementing Deserialize manually for Position and I can receive the versions of the fields of Position by implementing something like Position::get_version(field_name: &str).
However, I cannot figure how to get the version of the data currently being deserialized because Deserialize::deserialize only has a trait bound Deserializer<'de> and I cannot make that bound stricter by adding another bound (so it doesn't know about my custom Deserializer).
At this point, I'm thinking about giving the version data of each field when instantiating the Deserializer but I'm not sure if that will work or if there is a better way to go.
Multiple structs implementing a shared trait
If you have several different versions with several different types of struct, and you want a more robust way of handling different variants, it might be a better idea to write structs for each possible format. You can then define and implement a trait for shared behavior.
trait Position {
fn x(&self) -> f32;
fn y(&self) -> f32;
fn z(&self) -> f32;
fn version_number(&self) -> usize;
}
struct PositionV0 {
x: f32,
y: f32,
z: f32
}
impl Position for PositionV0 {
fn x(&self) -> f32 {
self.x
}
// You get the idea for the fn y, fn z implementations
fn version_number(&self) -> usize {
0
}
}
struct PositionV1 {
x: f32,
y: f32,
z: f32,
is_visible: bool,
}
impl Position for PositionV1 {
fn x(&self) -> f32 {
self.x
}
// You get the idea for the fn y, fn z implementations
fn version_number(&self) -> usize {
1
}
}
Carson's answer is great when you do not have a lot of versions but for me I am working with data structures that range over 20 different versions.
I went with a solution that while I don't think is the most idiomatic, is capable of handling an arbitrary number of versions.
In short:
we implement a Version trait which gives the necessary version info to the Deserializer
Deserializer has VersionedSeqAccess (implements serde::de::SeqAccess) that sets a flag
When flag is set, we put None for that field and immediately unset the flag
The idea is to implement the following trait for the struct:
pub trait Version {
/// We must specify the name of the struct so that any of the fields that
/// are structs won't confuse the Deserializer
fn name() -> &'static str;
fn version() -> VersionInfo;
}
#[derive(Debug, Clone)]
pub enum VersionInfo {
/// Present in all versions
All,
/// Present in this version
Version([u16; 4]),
/// Represent Versions of structs
Struct(&'static [VersionInfo]),
// we can add other ways of expressing the version like a version range for ex.
}
Here is how it will be implemented for the example struct Position. This type of manual deriving is error prone so this can be improved with a derive macro (see end):
struct Position {
x: f32,
z: f32,
y: f32,
is_visible: Option<bool>, // With this solution versioned field must be wrapped in Option
}
impl Version for Position {
fn version() -> VersionInfo {
VersionInfo::Struct(&[
VersionInfo::All,
VersionInfo::All,
VersionInfo::All,
VersionInfo::Version([1, 13, 0, 0]),
])
}
fn name() -> &'static str {
"Position"
}
}
Now, the deserializer will be instansiated with the version of the data format we are currently parsing:
pub struct Deserializer<'de> {
input: &'de [u8],
/// The version the `Deserializer` expect the data format to be
de_version: [u16; 4],
/// Versions of each field. (only used when deserialzing to a struct)
version_info: VersionInfo,
/// Whether to skip deserialzing current item. This flag is set by `VersionedSeqAccess`.
/// When set, the current item is deserialized to `None`
skip: bool,
/// Name of struct we are deserialzing into. We use this to make sure we call the correct
/// visitor for children of this struct who are also structs
name: &'static str,
}
pub fn from_slice<'a, T>(input: &'a [u8], de_version: [u16; 4]) -> Result<T, Error>
where
T: Deserialize<'a> + Version,
{
let mut deserializer = Deserializer::from_slice(input, de_version, T::version(), T::name());
let t = T::deserialize(&mut deserializer)?;
Ok(t)
}
Now that the deserializer has the all the information it needs, this is how we define deserialize_struct:
fn deserialize_struct<V>(
self, name: &'static str, fields: &'static [&'static str], visitor: V,
) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
if name == self.name {
if let VersionInfo::Struct(version_info) = self.version_info {
assert!(version_info.len() == fields.len()); // Make sure the caller implemented version info somewhat correctly. I use a derive macro to implement version so this is not a problem
visitor.visit_seq(VersionedSeqAccess::new(self, fields.len(), &version_info))
} else {
panic!("Struct must always have version info of `Struct` variant")
}
} else {
// This is for children structs of the main struct. We do not support versioning for those
visitor.visit_seq(SequenceAccess::new(self, fields.len()))
}
}
Here is how serde::de::SeqAccess will be implemented for VersionedSeqAccess:
struct VersionedSeqAccess<'a, 'de: 'a> {
de: &'a mut Deserializer<'de>,
version_info: &'static [VersionInfo],
len: usize,
curr: usize,
}
impl<'de, 'a> SeqAccess<'de> for VersionedSeqAccess<'a, 'de> {
type Error = Error;
fn next_element_seed<T>(&mut self, seed: T) -> Result<Option<T::Value>, Error>
where
T: DeserializeSeed<'de>,
{
if self.curr == self.len {
// We iterated through all fields
Ok(None)
} else {
// Get version of the current field
let version = &self.version_info[self.curr as usize];
self.de.version_info = version.clone();
// Set the flag if the version does not match
if !is_correct_version(&self.de.de_version, &version) {
self.de.skip = true;
}
self.curr += 1;
seed.deserialize(&mut *self.de).map(Some)
}
}
}
The final part of the puzzle is inside deserialize_option. If we are at a field not found in current data format the skip flag will be set here and we will produce None:
fn deserialize_option<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
if self.skip == true {
self.skip = false;
visitor.visit_none()
} else {
visitor.visit_some(self)
}
}
A lengthy solution but it works great for my usecase dealing with a lot of structs with lots of fields from different versions. Please do let me know how I can make this less verbose/better. I also implemented a derive macro (not shown here) for the Version trait to be able to do this:
#[derive(Debug, Clone, EventPrinter, Version)]
pub struct Position {
x: f32,
z: f32,
y: f32,
#[version([1, 13, 0, 0])]
is_visible: Option<bool>,
}
With this derive macro, I find that this solution tends to scale well for my usecase.
I'm getting the error:
expected enum `std::result::Result<_, wasm_bindgen::JsValue>`
found enum `std::result::Result<_, serde_wasm_bindgen::error::Error>`
when I serialize a struct by implementing Serialize then passing it to serde_wasm_bindgen, which uses the example code from here
here is the min reproducable code:
use wasm_bindgen::prelude::*;
use serde::ser::{Serialize, SerializeStruct, Serializer};
struct Person {
name: String,
age: u8,
phones: Vec<String>,
}
impl Serialize for Person {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut s = serializer.serialize_struct("Person", 3)?;
s.serialize_field("name", &self.name)?;
s.serialize_field("age", &self.age)?;
s.serialize_field("phones", &self.phones)?;
s.end()
}
}
#[wasm_bindgen]
pub fn pass_value_to_js() -> Result<JsValue, JsValue> {
let p = Person {
name: String::from("Hello"),
age: 56,
phones: vec![String::from("phone")],
};
serde_wasm_bindgen::to_value(p) // error here
}
Cargo.toml
serde-wasm-bindgen = "0.1.3"
serde = "1.0.114"
I followed this issue to solve the problem.
#[wasm_bindgen]
pub fn pass_value_to_js() -> Result<JsValue, JsValue> {
serde_wasm_bindgen::to_value(&value).map_err(|err| err.into())
}
I don't know anything about these packages, but looking at the code for to_value here it appears that the result is not compatible with Result<JsValue, JsValue>.
// The Error in scope here is serde_wasm_bindgen::error::Error
type Result<T> = std::result::Result<T, Error>;
...SNIP...
/// Converts a Rust value into a [`JsValue`].
pub fn to_value<T: serde::ser::Serialize>(value: &T) -> Result<JsValue> {
value.serialize(&Serializer::new())
}
So it looks like the function you have returning this value must change to be compatible with this new Result type.
I'm guessing you are referring to the example in the README here. I very often find that code examples in github READMEs are incorrect or out of date. Unfortunately I can't find an example or test in that repo that uses to_value() without immediately unwrap()ing it.
I'm serializing a HashMap with serde, like so:
#[derive(Serialize, Deserialize)]
struct MyStruct {
map: HashMap<String, String>
}
HashMap's key order is unspecified, and since the hashing is randomized (see documentation), the keys actually end up coming out in different order between identical runs.
I'd like my HashMap to be serialized in sorted (e.g. alphabetical) key order, so that the serialization is deterministic.
I could use a BTreeMap instead of a HashMap to achieve this, as BTreeMap::keys() returns its keys in sorted order, but I'd rather not change my data structure just to accommodate the serialization logic.
How do I tell serde to sort the HashMap keys before serializing?
Use the serialize_with field attribute:
use serde::{Deserialize, Serialize, Serializer}; // 1.0.106
use serde_json; // 1.0.52
use std::collections::{BTreeMap, HashMap};
#[derive(Serialize, Deserialize, Default)]
struct MyStruct {
#[serde(serialize_with = "ordered_map")]
map: HashMap<String, String>,
}
fn ordered_map<S>(value: &HashMap<String, String>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let ordered: BTreeMap<_, _> = value.iter().collect();
ordered.serialize(serializer)
}
fn main() {
let mut m = MyStruct::default();
m.map.insert("gamma".into(), "3".into());
m.map.insert("alpha".into(), "1".into());
m.map.insert("beta".into(), "2".into());
println!("{}", serde_json::to_string_pretty(&m).unwrap());
}
Here, I've chosen to just rebuild an entire BTreeMap from the HashMap and then reuse the existing serialization implementation.
{
"map": {
"alpha": "1",
"beta": "2",
"gamma": "3"
}
}
A slightly more generic way with automatic sorting, one that uses itertools, and one that only relies on the std lib. Try it on playground
// This requires itertools crate
pub fn sorted_map<S: Serializer, K: Serialize + Ord, V: Serialize>(
value: &HashMap<K, V>,
serializer: S,
) -> Result<S::Ok, S::Error> {
value
.iter()
.sorted_by_key(|v| v.0)
.collect::<BTreeMap<_, _>>()
.serialize(serializer)
}
// This only uses std
pub fn sorted_map<S: Serializer, K: Serialize + Ord, V: Serialize>(
value: &HashMap<K, V>,
serializer: S,
) -> Result<S::Ok, S::Error> {
let mut items: Vec<(_, _)> = value.iter().collect();
items.sort_by(|a, b| a.0.cmp(&b.0));
BTreeMap::from_iter(items).serialize(serializer)
}
Both of the above functions can be used with these structs:
#[derive(Serialize)]
pub struct Obj1 {
#[serde(serialize_with = "sorted_map")]
pub table: HashMap<&'static str, i32>,
}
#[derive(Serialize)]
pub struct Obj2 {
#[serde(serialize_with = "sorted_map")]
pub table: HashMap<String, i32>,
}
I have a struct like:
struct S {
data: i32,
fun: Box<Fn()>,
}
and would like to serialize the data part using an encoder. To do that I use rustc_serialize and derive its traits like
#[derive(RustcEncodable, RustcDecodable)]
struct S {
data: i32,
fun: Box<Fn()>,
}
The problem is that fun can't be serialized as it's a function. This is fine as I only want to serialize the plain data field. Is there a way to do that?
The data field in my real use case is also a struct which can also have a Fn so I can't simply split the struct into two.
The short answer is "no". The rustc-serialize crate 1 does not provide that level of control on the automated implementations of traits like Decodable or Encodable.
To do this, you would need to implement them yourself:
extern crate rustc_serialize;
use rustc_serialize::{Encodable, Encoder};
use rustc_serialize::json;
struct S {
data: i32,
fun: Box<Fn()>,
}
impl Encodable for S {
fn encode<S: Encoder>(&self, s: &mut S) -> Result<(), S::Error> {
s.emit_struct("S", 1, |s| {
s.emit_struct_field("data", 0, |s| {
s.emit_i32(self.data)
})
})
}
}
fn main() {
let thing = S { data: 42, fun: Box::new(|| println!("I am a function")) };
let json = json::encode(&thing).expect("Couldn't encode");
println!("{}", json);
(thing.fun)();
}
If you aren't tied to rustc-serialize, you may be interested in serde which offers the #[serde(skip_serializing)] and #[serde(skip_deserializing)] annotations.
1: Technically, #[derive(RustcEncodable, RustcDecodable)] is provided by the compiler. This is why it has the Rustc prefix. It's also an ugly wart that wants to be removed but is waiting for stable support of compiler plugins.
Shepmaster's answer gives the RustcEncodable solution. For completeness, here is the equivalent Serde code:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde_json;
#[derive(Serialize, Deserialize)]
struct S {
data: i32,
#[serde(skip_serializing, skip_deserializing, default="empty_fun")]
fun: Box<Fn()>,
}
fn empty_fun() -> Box<Fn()> {
Box::new(|| {})
}
fn main() {
let s = S { data: 42, fun: Box::new(|| println!("I am a function")) };
// prints {"data":42}
println!("{}", serde_json::to_string(&s).unwrap());
}
The skip_serializing attribute means ignore the field when serializing the struct. Similarly, the skip_deserializing attribute means ignore the field when deserializing.
The default attribute gives the value to use for the field when deserializing. Fields that implement std::default::Default do not need this as Serde is able to use the value of Default::default().