I have a struct like:
struct S {
data: i32,
fun: Box<Fn()>,
}
and would like to serialize the data part using an encoder. To do that I use rustc_serialize and derive its traits like
#[derive(RustcEncodable, RustcDecodable)]
struct S {
data: i32,
fun: Box<Fn()>,
}
The problem is that fun can't be serialized as it's a function. This is fine as I only want to serialize the plain data field. Is there a way to do that?
The data field in my real use case is also a struct which can also have a Fn so I can't simply split the struct into two.
The short answer is "no". The rustc-serialize crate 1 does not provide that level of control on the automated implementations of traits like Decodable or Encodable.
To do this, you would need to implement them yourself:
extern crate rustc_serialize;
use rustc_serialize::{Encodable, Encoder};
use rustc_serialize::json;
struct S {
data: i32,
fun: Box<Fn()>,
}
impl Encodable for S {
fn encode<S: Encoder>(&self, s: &mut S) -> Result<(), S::Error> {
s.emit_struct("S", 1, |s| {
s.emit_struct_field("data", 0, |s| {
s.emit_i32(self.data)
})
})
}
}
fn main() {
let thing = S { data: 42, fun: Box::new(|| println!("I am a function")) };
let json = json::encode(&thing).expect("Couldn't encode");
println!("{}", json);
(thing.fun)();
}
If you aren't tied to rustc-serialize, you may be interested in serde which offers the #[serde(skip_serializing)] and #[serde(skip_deserializing)] annotations.
1: Technically, #[derive(RustcEncodable, RustcDecodable)] is provided by the compiler. This is why it has the Rustc prefix. It's also an ugly wart that wants to be removed but is waiting for stable support of compiler plugins.
Shepmaster's answer gives the RustcEncodable solution. For completeness, here is the equivalent Serde code:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde_json;
#[derive(Serialize, Deserialize)]
struct S {
data: i32,
#[serde(skip_serializing, skip_deserializing, default="empty_fun")]
fun: Box<Fn()>,
}
fn empty_fun() -> Box<Fn()> {
Box::new(|| {})
}
fn main() {
let s = S { data: 42, fun: Box::new(|| println!("I am a function")) };
// prints {"data":42}
println!("{}", serde_json::to_string(&s).unwrap());
}
The skip_serializing attribute means ignore the field when serializing the struct. Similarly, the skip_deserializing attribute means ignore the field when deserializing.
The default attribute gives the value to use for the field when deserializing. Fields that implement std::default::Default do not need this as Serde is able to use the value of Default::default().
Related
I'm trying to use the AES crate, which offers three algorithms: AES128, AES192 and AES256. I'm trying to create a struct that can create the correct algorithm by detecting the key size, and save it to later use.
I see they all implement the BlockEncrypt (I only need encryption) trait, but when I try to make a field in the struct with this type, even when supplying the size, i get an "the trait BlockEncrypt cannot be made into an object. the trait cannot be made into an object because it requires Self: Sized " error.
pub struct MyStruct<'a, T: Sized> {
ciph: Box< dyn BlockEncrypt<BlockSize = T>>,
}
Traits that have Sized as a supertype aren't "object safe" which means dynamic dispatch using dyn isn't possible on these kinds of traits.
You can accomplish this relatively tersely by using an enum for each possible type you want to handle, combined with a macro that will neatly generate all the match arms required to dynamically dispatch on each enum possibility.
To make things simple for the sake of this explanation, I'm going to define a new trait (with the Sized supertype) so that this answer doesn't depend on a particular crate:
trait SampleTrait: Sized {
fn method_a(&self, foo: &str);
fn method_b(&self, foo: i32, bar: String) -> String;
}
Now we need at least two implementations of this trait to demonstrate the solution:
struct ImplA;
struct ImplB;
impl SampleTrait for ImplA {
fn method_a(&self, foo: &str) {
println!("<ImplA as SampleTrait>::method_a(): {:?}", foo);
}
fn method_b(&self, foo: i32, bar: String) -> String {
println!("<ImplA as SampleTrait>::method_b(): {:?} {:?}", foo, bar);
format!("from ImplA: {}", bar)
}
}
impl SampleTrait for ImplB {
fn method_a(&self, foo: &str) {
println!("<ImplB as SampleTrait>::method_a(): {:?}", foo);
}
fn method_b(&self, foo: i32, bar: String) -> String {
println!("<ImplB as SampleTrait>::method_b(): {:?} {:?}", foo, bar);
format!("from ImplB: {}", bar)
}
}
The enum just needs to least each possible type that you need to handle:
enum DynSampleTrait {
ImplA(ImplA),
ImplB(ImplB),
}
Now to dispatch on this, you'd have to do something like:
match value {
DynSampleTrait::ImplA(v) => v.method_a("something"),
DynSampleTrait::ImplB(v) => v.method_a("something"),
}
To get around having to repeat ourselves, let's declare a macro that will generate the whole match for us:
macro_rules! dyn_sample_trait_call {
(($m:expr) $v:ident => $code:block) => {
match $m {
DynSampleTrait::ImplA($v) => $code,
DynSampleTrait::ImplB($v) => $code,
}
}
}
Now we can use the macro like so:
dyn_sample_trait_call!((value) v => { v.method_a("something") })
value goes in the match expression, and v is the identifier that unwraps the inner value held by the enum possibility. These are separate so that you can match over a reference, should you not want to consume an owned DynSampleTrait:
dyn_sample_trait_call!((&value) v => { v.method_a("something") })
(Playground)
As #cdhowie mentioned, you can't create a trait object from a trait with a Sized bound. Instead, you can create an enum:
enum MyAlgorithm {
AES128(AES128),
AES192(AES192),
AES256(AES256),
}
I'd like to have a field in struct like this:
struct Foo<T> {
bar: Smart<T>
}
where bar could be either Rc<T or Weak<T>, depending on the "ownership relationships" between different instances of Foo. Is there any idiomatic way in Rust how to do this, other than to create a custom enum?
Is there any idiomatic way in Rust how to do this, other than to create a custom enum?
Just like most other "either this or that" choices in Rust, an enum is the idiomatic way.
An Peter's answer suggested, there is is no such abstraction in the stdlib. You can define a small enum to handle both cases:
use std::rc::{Rc, Weak};
enum MaybeStrong<T> {
Strong(Rc<T>),
Weak(Weak<T>),
}
impl<T> MaybeStrong<T> {
fn get(&self) -> Option<Rc<T>> {
match self {
MaybeStrong::Strong(t) => Some(Rc::clone(t)),
MaybeStrong::Weak(w) => w.upgrade(),
}
}
}
struct Foo<T> {
bar: MaybeStrong<T>
}
impl<T> Foo<T> {
fn from_weak(inner: Weak<T>) -> Self {
Self { bar: MaybeStrong::Weak(inner) }
}
fn from_strong(inner: Rc<T>) -> Self {
Self { bar: MaybeStrong::Strong(inner) }
}
fn say(&self) where T: std::fmt::Debug {
println!("{:?}", self.bar.get())
}
}
fn main() {
let inner = Rc::new("foo!");
Foo::from_weak(Rc::downgrade(&inner)).say();
Foo::from_strong(inner).say();
}
self.bar() will always return a Some if it was created from a strong pointer and return None in case it's a Weak and it's dangling. Notice that due to the fact that get() needs to create an owned Rc first, the method can't return a &T (including Option<&T>) because that &T could be dangling. This also means that all users of bar() will own one strong count on the inner value while processing, making it safe to use in any case.
This kind of construct is often called "Either" and there's a crate that looks like it can address some of the usual use-cases: https://docs.rs/either/1.5.3/either/
Then you could write
struct Foo<T> {
bar: Either<Weak<T>, Rc<T>>
}
Then an example function to get an Option<Rc<T>> might be:
impl <T> Foo<T> {
fn get_rc(self) -> Option<Rc<T>> {
self.bar
.map_left( |weak| weak.upgrade() )
.map_right( |v| Some(v) )
.into_inner()
}
}
Then it can be used like this:
fn main() {
let x = Rc::new(1);
let f_direct = Foo{ bar:Either::Right(x.clone()) };
println!("f_direct.get_rc() = {:?}", f_direct.get_rc());
let f_weak = Foo{ bar:Either::Left(Rc::downgrade(&x)) };
println!("f_weak.get_rc() = {:?}", f_weak.get_rc());
}
Link to complete example in the playground:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c20faaa46277550e16a3d3b24f3d1750
I'm using futures, tokio, hyper, and serde_json to request and deserialize some data that I need to hold until my next request. My initial thought was to make a struct containing the hyper::Chunk and the deserialized data that borrows from the Chunk, but couldn't get the lifetimes right. I tried using the rental crate, but I can't get this to work either. Perhaps I'm using the 'buffer lifetime before declaring the buffer Vec, but maybe I've messed something else up:
#[rental]
pub struct ChunkJson<T: serde::de::Deserialize<'buffer>> {
buffer: Vec<u8>,
json: T
}
Is there some way to make the lifetimes right or should I just use DeserializeOwned and give up on zero-copy?
For more context, the following code works (periodically deserializing JSON from two URLs, retaining the results so we can do something with them both). I'd like to change my X and Y types to use Cow<'a, str> for their fields, changing from DeserializeOwned to Deserialize<'a>. For this to work, I need to store the slice that has been deserialized for each, but I don't know how to do this. I'm looking for examples that use Serde's zero-copy deserialization and retain the result, or some idea for restructuring my code that would work.
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;
extern crate futures;
extern crate tokio_core;
extern crate tokio_periodic;
extern crate hyper;
use std::collections::HashMap;
use std::error::Error;
use futures::future;
use futures::Future;
use futures::stream::Stream;
use hyper::Client;
fn stream_json<'a, T: serde::de::DeserializeOwned + Send + 'a>
(handle: &tokio_core::reactor::Handle,
url: String,
period: u64)
-> Box<Stream<Item = T, Error = Box<Error>> + 'a> {
let client = Client::new(handle);
let timer = tokio_periodic::PeriodicTimer::new(handle).unwrap();
timer
.reset(::std::time::Duration::new(period, 0))
.unwrap();
Box::new(futures::Stream::zip(timer.from_err::<Box<Error>>(), futures::stream::unfold( (), move |_| {
let uri = url.parse::<hyper::Uri>().unwrap();
let get = client.get(uri).from_err::<Box<Error>>().and_then(|res| {
res.body().concat().from_err::<Box<Error>>().and_then(|chunks| {
let p: Result<T, Box<Error>> = serde_json::from_slice::<T>(chunks.as_ref()).map_err(|e| Box::new(e) as Box<Error>);
match p {
Ok(json) => future::ok((json, ())),
Err(err) => future::err(err)
}
})
});
Some(get)
})).map(|x| { x.1 }))
}
#[derive(Serialize, Deserialize, Debug)]
pub struct X {
foo: String,
}
#[derive(Serialize, Deserialize, Debug)]
pub struct Y {
bar: String,
}
fn main() {
let mut core = tokio_core::reactor::Core::new().unwrap();
let handle = core.handle();
let x_stream = stream_json::<HashMap<String, X>>(&handle, "http://localhost/X".to_string(), 2);
let y_stream = stream_json::<HashMap<String, Y>>(&handle, "http://localhost/Y".to_string(), 5);
let mut xy_stream = x_stream.merge(y_stream);
let mut last_x = HashMap::new();
let mut last_y = HashMap::new();
loop {
match core.run(futures::Stream::into_future(xy_stream)) {
Ok((Some(item), stream)) => {
match item {
futures::stream::MergedItem::First(x) => last_x = x,
futures::stream::MergedItem::Second(y) => last_y = y,
futures::stream::MergedItem::Both(x, y) => {
last_x = x;
last_y = y;
}
}
println!("\nx = {:?}", &last_x);
println!("y = {:?}", &last_y);
// Do more stuff with &last_x and &last_y
xy_stream = stream;
}
Ok((None, stream)) => xy_stream = stream,
Err(_) => {
panic!("error");
}
}
}
}
When trying to solve a complicated programming problem, it's very useful to remove as much as you can. Take your code and remove what you can until the problem goes away. Tweak your code a bit and keep removing until you can't any more. Then, turn the problem around and build from the smallest piece and work back to the error. Doing both of these will show you where the problem lies.
First, let's make sure we deserialize correctly:
extern crate serde;
extern crate serde_json;
#[macro_use]
extern crate serde_derive;
use std::borrow::Cow;
#[derive(Debug, Deserialize)]
pub struct Example<'a> {
#[serde(borrow)]
name: Cow<'a, str>,
key: bool,
}
impl<'a> Example<'a> {
fn info(&self) {
println!("{:?}", self);
match self.name {
Cow::Borrowed(_) => println!("Is borrowed"),
Cow::Owned(_) => println!("Is owned"),
}
}
}
fn main() {
let data: Vec<_> = br#"{"key": true, "name": "alice"}"#.to_vec();
let decoded: Example = serde_json::from_slice(&data).expect("Couldn't deserialize");
decoded.info();
}
Here, I forgot to add the #[serde(borrow)] attribute, so I'm glad I did this test!
Next, we can introduce the rental crate:
#[macro_use]
extern crate rental;
rental! {
mod holding {
use super::*;
#[rental]
pub struct VecHolder {
data: Vec<u8>,
parsed: Example<'data>,
}
}
}
fn main() {
let data: Vec<_> = br#"{"key": true, "name": "alice"}"#.to_vec();
let holder = holding::VecHolder::try_new(data, |data| {
serde_json::from_slice(data)
});
let holder = match holder {
Ok(holder) => holder,
Err(_) => panic!("Unable to construct rental"),
};
holder.rent(|example| example.info());
// Make sure we can move the data and it's still valid
let holder2 = { holder };
holder2.rent(|example| example.info());
}
Next we try to create a rental of Chunk:
#[rental]
pub struct ChunkHolder {
data: Chunk,
parsed: Example<'data>,
}
Unfortunately, this fails:
--> src/main.rs:29:1
|
29 | rental! {
| ^
|
= help: message: Field `data` must have an angle-bracketed type parameter or be `String`.
Oops! Checking the docs for rental, we can add #[target_ty_hack="[u8]"] to the data field. This leads to:
error[E0277]: the trait bound `hyper::Chunk: rental::__rental_prelude::StableDeref` is not satisfied
--> src/main.rs:29:1
|
29 | rental! {
| ^ the trait `rental::__rental_prelude::StableDeref` is not implemented for `hyper::Chunk`
|
= note: required by `rental::__rental_prelude::static_assert_stable_deref`
That's annoying; since we can't implement that trait for Chunk, we just need to box Chunk, proving that it has a stable address:
#[rental]
pub struct ChunkHolder {
data: Box<Chunk>,
parsed: Example<'data>,
}
I also looked to see if there is a way to get a Vec<u8> back out of Chunk, but it doesn't appear to exist. That would have been another solution with less allocation and indirection.
At this point, "all" that's left is to integrate this back into the futures code. It's a lot of work for anyone but you to recreate that, but I don't foresee any obvious problems in doing so.
Given a struct:
#[repr(C)]
pub struct User {
pub name: *const c_char,
pub age: u8,
pub ctx: ??,
}
the field ctx would only be manipulated by C code; it's a pointer to a C struct UserAttr.
According to the Rust FFI documentation, the choice would be defined as an opaque type pub enum UserAttr {}. However, I found that Rust is unable to copy its value, e.g. why does the address of an object change across methods.
What's the right way in Rust to define such an opaque pointer, so that its value (as a pointer) gets copied across methods?
The future
RFC 1861 introduced the concept of an extern type. While implemented, it is not yet stabilized. Once it is, it will become the preferred implementation:
#![feature(extern_types)]
extern "C" {
type Foo;
}
type FooPtr = *mut Foo;
Today
The documentation states:
To do this in Rust, let’s create our own opaque types:
#[repr(C)] pub struct Foo { private: [u8; 0] }
#[repr(C)] pub struct Bar { private: [u8; 0] }
extern "C" {
pub fn foo(arg: *mut Foo);
pub fn bar(arg: *mut Bar);
}
By including a private field and no constructor, we create an opaque
type that we can’t instantiate outside of this module. An empty array
is both zero-size and compatible with #[repr(C)]. But because our
Foo and Bar types are different, we’ll get type safety between the
two of them, so we cannot accidentally pass a pointer to Foo to
bar().
An opaque pointer is created such that there's no normal way of creating such a type; you can only create pointers to it.
mod ffi {
use std::ptr;
pub struct MyTypeFromC { _private: [u8; 0] }
pub fn constructor() -> *mut MyTypeFromC {
ptr::null_mut()
}
pub fn something(_thing: *mut MyTypeFromC) {
println!("Doing a thing");
}
}
use ffi::*;
struct MyRustType {
score: u8,
the_c_thing: *mut MyTypeFromC,
}
impl MyRustType {
fn new() -> MyRustType {
MyRustType {
score: 42,
the_c_thing: constructor(),
}
}
fn something(&mut self) {
println!("My score is {}", self.score);
ffi::something(self.the_c_thing);
self.score += 1;
}
}
fn main() {
let mut my_thing = MyRustType::new();
my_thing.something();
}
Breaking it down a bit:
// opaque -----V~~~~~~~~~V
*mut MyTypeFromC
// ^~~^ ------------ pointer
Thus it's an opaque pointer. Moving the struct MyRustType will not change the value of the pointer.
The past
Previous iterations of this answer and the documentation suggested using an empty enum (enum MyTypeFromC {}). An enum with no variants is semantically equivalent to the never type (!), which is a type that cannot exist. There were concerns that using such a construct could lead to undefined behavior, so moving to an empty array was deemed safer.
I am trying to print an Option<Box<MyStruct>>, but I get a compile error when trying to implement Display for Option<Box<MyStruct>>.
use std::fmt;
fn main() {
let maybe_my_struct: Option<Box<MyStruct>> = Some(Box::new(MyStruct{foo:42}));
println!("{}", maybe_my_struct);
}
struct MyStruct {
foo: i32,
}
impl fmt::Display for Option<Box<MyStruct>> {
fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
match self {
Some(MyStruct) => write!(formatter, "{}", self.foo),
None => write!(formatter, "No struct"),
}
}
}
The error I get is :
error: the impl does not reference any types defined in this crate;
only traits defined in the current crate can be implemented for arbitrary types [E0117]
I have tried aliasing the Option type, and instead implementing Display for MyOption<Box<MyStruct>>, but that gives the same result. What am I doing wrong?
As you can see, you can't implement a trait you didn't write for a type you didn't write. This is part of what's known as "coherence" and exists to prevent really weird things like linking against a library suddenly causing unrelated parts of your program to change behaviour.
Aliasing Option to MyOption doesn't work either because, as you say, it's an alias. That is, it's just another name for the same thing, it's not an actual, different type.
Now, if you write a wrapper around Option like so:
struct MyOption<T>(Option<T>);
then MyOption will be a new, distinct type that you can implement a trait for. Of course, you'll want to write methods to wrap and unwrap the actual Option you're storing.
... But this is all rather irrelevant since you could also just derive Debug for your struct and use that.
fn main() {
let maybe_my_struct: Option<Box<MyStruct>> = Some(Box::new(MyStruct{foo:42}));
println!("{:?}", Some(maybe_my_struct));
}
#[derive(Debug)]
struct MyStruct {
foo: i32,
}
Or, if you really want the custom display logic for the Option<Box<MyStruct>> combination, you can use a marker value (this same approach is used by Path in the standard library, incidentally). Like so:
use std::fmt;
fn main() {
let maybe_my_struct: Option<Box<MyStruct>> = Some(Box::new(MyStruct{foo:42}));
println!("{:?}", maybe_my_struct);
// Instead of displaying directly, display via a custom marker.
println!("{}", maybe_my_struct.display());
println!("{}", None::<Box<MyStruct>>.display());
}
#[derive(Debug)]
struct MyStruct {
foo: i32,
}
// This is the marker we'll use to define our custom Display impl.
struct MmsDisplay<'a>(&'a Option<Box<MyStruct>>);
// This trait lets us extend Option<Box<MyStruct>> with a new method.
trait CustomMmsDisplay {
fn display<'a>(&'a self) -> MmsDisplay<'a>;
}
impl CustomMmsDisplay for Option<Box<MyStruct>> {
fn display<'a>(&'a self) -> MmsDisplay<'a> {
MmsDisplay(self)
}
}
// And here's the display logic.
impl<'a> fmt::Display for MmsDisplay<'a> {
fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
match *self.0 {
Some(ref ms) => write!(formatter, "{}", ms.foo),
None => write!(formatter, "No struct"),
}
}
}