How do I serialize/deserialize &[f64] using serde in Rust? - rust

This compiles successfully:
#[derive(Serialize, Deserialize)]
struct Person {
name: String,
value: f64
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let perspn = Person { name: "me".into(), value: 30_f64};
let serialized = serde_json::to_string(&perspn).unwrap();
println!("{:?}", serialized);
Ok(())
}
But on changing a member of the struct to &[f64], then compilation fails
#[derive(Serialize, Deserialize)]
struct Person<'a> {
name: String,
value: &'a[f64]
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let perspn = Person { name: "me".into(), value: &[30_f64]};
let serialized = serde_json::to_string(&perspn).unwrap();
println!("{:?}", serialized);
Ok(())
}
fails with error:
error[E0277]: the trait bound `&[f64]: Deserialize<'_>` is not satisfied
--> src/main.rs:168:5
|
168 | value: &'a[f64]
| ^^^^^ the trait `Deserialize<'_>` is not implemented for `&[f64]`
|
= help: the following other types implement trait `Deserialize<'de>`:
&'a [u8]
[T; 0]
[T; 10]
[T; 11]
[T; 12]
[T; 13]
[T; 14]
[T; 15]
and 26 others
Why is this? Why does serde have a default deserialiser for f64 but not for &[f64]?
Looking at the error it looks like it is only [u8] that serde can handle by default. Why is this?
Now the question is, how do I deserialize &[f64]? I know I will need to write a custom deserializer for it, but given that a custom one was not given, I am wondering if there are edge case in the implementation that made that the case and which I need to be aware of?

Related

the method `fold` exists for reference `&[T]`, but its trait bounds were not satisfied, I don't understand the given bounds in question

So I'm building a chip-8 CPU emulator and to make copying data into the memory easier I created the following two methods:
pub struct CPU {
// ... other unrelated fields
pub memory: [u8; 0x1000],
}
impl CPU {
pub fn raw_copy_to_mem(&mut self, loc: usize, data: &[u8]) {
data.chunks(2).fold(loc, |loc, bytes| {
self.raw_add_to_mem(loc, *bytes.first().unwrap(), *bytes.last().unwrap())
});
}
pub fn raw_add_to_mem(&mut self, loc: usize, high: u8, low: u8) -> usize {
self.memory[loc] = high; self.memory[loc + 1] = low;
loc + 2
}
}
Now this works for this for example:
fn make_cpu() -> processor::CPU {
processor::CPU {
// ... other unrelated fields
memory: [0; 0x1000],
}
}
#[test]
fn test_raw_memory_copy() {
let mut cpu = make_cpu();
let add_twice: [u8; 6] = [
0x80, 0x14,
0x80, 0x14,
0x00, 0xEE,
];
cpu.raw_copy_to_mem(0x100 , &add_twice);
assert_eq!(cpu.memory[0x100..0x106], add_twice);
}
Now to make doing OpCodes easier I have the following struct and functions:
#[derive(Debug, Clone, Copy)]
pub struct OpCode(u8, u8);
impl OpCode {
pub fn high_byte(&self) -> u8 {
self.0
}
pub fn low_byte(&self) -> u8 {
self.1
}
}
impl From<OpCode> for u16 {
fn from(c: OpCode) -> Self {
(c.0 as u16) << 8 | c.1 as u16
}
}
impl From<&OpCode> for u16 {
fn from(c: &OpCode) -> u16 {
c.into()
}
}
impl OpCode {
pub fn add(h: u8, l: u8) -> Self {
Self (0x8 << 4 | h, (l << 4) | 4)
}
}
And the following CPU function also works great:
impl CPU {
pub fn add_to_mem(&mut self, loc: usize, oc: &OpCode) -> usize {
self.memory[loc] = oc.high_byte(); self.memory[loc + 1] = oc.low_byte();
loc + 2
}
}
The problem occurs when I try to add a copy_to_mem() function for the OpCode struct:
impl CPU {
pub fn copy_to_mem(&mut self, loc: usize, data: &[OpCode]) {
data.fold(loc, |loc, opcode| {
self.add_to_mem(loc, opcode)
});
}
}
I get the following error that I only vaguely understand:
error[E0599]: the method `fold` exists for reference `&[OpCode]`, but its trait bounds were not satisfied
--> src/processor.rs:21:14
|
21 | data.fold(loc, |loc, bytes| {
| ^^^^ method cannot be called on `&[OpCode]` due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
`&[OpCode]: Iterator`
which is required by `&mut &[OpCode]: Iterator`
`[OpCode]: Iterator`
which is required by `&mut [OpCode]: Iterator`
I get that it's saying that whilst there is a fold() function & that it can't call it because some trait bounds weren't satisfied so it can't call it but it doesn't state what those bounds are in a way that I can understand.
So, could someone explain what is going on, and how to fix?
The error message is misleading: the problem is that you're trying to use a slice as an iterator, but it isn't. To obtain an iterator for the slice, call it's iter() method (or iter_mut() for mutable iterator).
pub fn copy_to_mem(&mut self, loc: usize, data: &[OpCode]) {
data.iter().fold(loc, |loc, bytes| {
self.add_to_mem(loc, bytes.high_byte())
});
}
Note that it'll still error with:
error[E0308]: mismatched types
--> src/lib.rs:60:34
|
60 | self.add_to_mem(loc, bytes.high_byte())
| ^^^^^^^^^^^^^^^^^ expected `&OpCode`, found `u8`
Because add_to_mem expects &OpCode but while bytes is indeed &OpCode bytes.high_byte() is u8.

Borrowed value does not live long enough requiring static lifetime

I'm getting this error with a sample code at this rust playground
Compiling playground v0.0.1 (/playground)
error[E0597]: `text` does not live long enough
--> src/main.rs:34:38
|
34 | let item = NotLongEnough { text: &text };
| ^^^^^ borrowed value does not live long enough
35 | let mut wrapper = Container { buf: Vec::new() };
36 | wrapper.add(Box::new(item));
| -------------- cast requires that `text` is borrowed for `'static`
...
40 | }
| - `text` dropped here while still borrowed
error: aborting due to previous error
For more information about this error, try `rustc --explain E0597`.
error: could not compile `playground`
To learn more, run the command again with --verbose.
The contents are:
trait TestTrait {
fn get_text(&self) -> &str;
}
#[derive(Copy, Clone)]
struct NotLongEnough<'a> {
text: &'a str,
}
impl<'a> TestTrait for NotLongEnough<'a> {
fn get_text(&self) -> &str {
self.text
}
}
struct Container {
buf: Vec<Box<dyn TestTrait>>,
}
impl Container {
pub fn add(&mut self, item: Box<dyn TestTrait>) {
self.buf.push(item);
}
pub fn output(&self) {
for item in &self.buf {
println!("{}", item.get_text());
}
}
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let text = "test".to_owned();
let item = NotLongEnough { text: &text };
let mut wrapper = Container { buf: Vec::new() };
wrapper.add(Box::new(item));
wrapper.output();
Ok(())
}
I have no clue why cast requires that text is borrowed for 'static
Could someone please help me with this? I've no idea what I've done wrong.
TLDR: Fixed version
The problem is in your Container definition:
struct Container {
buf: Vec<Box<dyn TestTrait>>,
}
The statement dyn TestTrait is equivalent to dyn TestTrait + 'static, meaning that your trait objects must not contain any references with lifetime less than 'static.
In order to fix the problem, you have to replace that trait bound with a less strict one:
struct Container<'a> {
buf: Vec<Box<dyn TestTrait + 'a>>,
}
Now instead of 'static, the container requires 'a. And you have to apply that change to the implementation as well:
pub fn add(&mut self, item: Box<dyn TestTrait + 'a>) { // notice the new trait-bound
self.buf.push(item);
}
Relevant resources:
Trait bounds

Trying to apply polymorphism with Box<_> has the error "cannot move a value ... the size cannot be statically determined"

I am applying the polymorphism solution in Rust to my problem. I would like to use this solution with Box<_> as it seems the most straight and simple, but it doesn't work.
#[derive(Clone, Copy)]
pub struct NewPost;
#[derive(Clone, Copy)]
pub struct Post;
#[derive(Clone, Copy)]
pub struct PgConnection;
#[derive(Clone, Copy)]
pub struct DBPost;
pub trait DBAdapter {
fn create(self, post: NewPost) -> Post;
fn read(self) -> Vec<Post>;
}
impl DBPost {
// DATABASE classes
pub fn establish_connection(self) -> PgConnection {
unimplemented!()
}
}
impl DBAdapter for DBPost {
fn create(self, _post: NewPost) -> Post {
unimplemented!()
}
fn read(self) -> Vec<Post> {
unimplemented!()
}
}
struct GetPostsCase {
db: Box<dyn DBAdapter>,
}
impl GetPostsCase {
pub fn new(db: Box<dyn DBAdapter>) -> GetPostsCase {
GetPostsCase { db: db }
}
pub fn run(&self) -> Vec<Post> {
let result = self.db.read();
result
}
}
The error is:
error[E0161]: cannot move a value of type dyn DBAdapter: the size of dyn DBAdapter cannot be statically determined
--> src/lib.rs:45:22
|
45 | let result = self.db.read();
| ^^^^^^^
error[E0507]: cannot move out of `*self.db` which is behind a shared reference
--> src/lib.rs:45:22
|
45 | let result = self.db.read();
| ^^^^^^^ move occurs because `*self.db` has type `dyn DBAdapter`, which does not implement the `Copy` trait
Your read method takes the (unsized) value instead of taking a reference (whose size is always the same).
You can solve the problem by changing the contract of DBAdapter
from
fn read(self) -> Vec<Post> {
to
fn read(&self) -> Vec<Post> {
// ^--- added the ampersand
(depending on your implementation you might need &mut)

Why does SmallVec have different behaviour when storing types with lifetimes compared to Vec?

I've got three examples, one using Vec one using SmallVec, and one with my own implementation of SmallVec. The ones using Vec and my own SmallVec compile but the one using the real SmallVec does not.
Working example using Vec
use std::borrow::Cow;
use std::collections::HashMap;
pub trait MyTrait {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns;
}
/// IMPORTANT PART IS HERE: `Vec<Cow<'a, str>>`
pub struct ItemTraitReturns<'a>(Vec<Cow<'a, str>>);
/// this implementation only takes items with static lifetime (but other implementations also might have different lifetimes)
pub struct MyTraitStruct {
map: HashMap<usize, ItemTraitReturns<'static>>,
}
impl MyTrait for MyTraitStruct {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns {
let temp: &ItemTraitReturns<'static> = self.map.get(&id).unwrap();
// Works as expected: I expect that I can return `&ItemTraitReturns<'_>`
// when I have `&ItemTraitReturns<'static>` (since 'static outlives everything).
temp
// Will return `&ItemTraitReturns<'_>`
}
}
Failing example with SmallVec
Uses SmallVec instead of Vec with no other changes.
use smallvec::SmallVec;
use std::borrow::Cow;
use std::collections::HashMap;
pub trait MyTrait {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns;
}
/// IMPORTANT PART IS HERE: Uses SmallVec instead of Vec
pub struct ItemTraitReturns<'a>(SmallVec<[Cow<'a, str>; 2]>);
/// this implementation only takes items with static lifetime (but other implementations also might have different lifetimes)
pub struct MyTraitStruct {
map: HashMap<usize, ItemTraitReturns<'static>>,
}
impl MyTrait for MyTraitStruct {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns {
let temp: &ItemTraitReturns<'static> = self.map.get(&id).unwrap();
temp
}
}
error[E0308]: mismatched types
--> src/lib.rs:23:9
|
23 | temp
| ^^^^ lifetime mismatch
|
= note: expected type `&ItemTraitReturns<'_>`
found type `&ItemTraitReturns<'static>`
note: the anonymous lifetime #1 defined on the method body at 18:5...
--> src/lib.rs:18:5
|
18 | / fn get_by_id(&self, id: usize) -> &ItemTraitReturns {
19 | | let temp: &ItemTraitReturns<'static> = self.map.get(&id).unwrap();
20 | | // Error:
21 | | // = note: expected type `&demo2::ItemTraitReturns<'_>`
22 | | // found type `&demo2::ItemTraitReturns<'static>`
23 | | temp
24 | | }
| |_____^
= note: ...does not necessarily outlive the static lifetime
Working example with my own SmallVec
When I implement my own (very naive) SmallVec<[T; 2]> (called NaiveSmallVec2<T>) the code also compiles... Very strange!
use std::borrow::Cow;
use std::collections::HashMap;
/// This is a very naive implementation of a SmallVec<[T; 2]>
pub struct NaiveSmallVec2<T> {
item1: Option<T>,
item2: Option<T>,
more: Vec<T>,
}
impl<T> NaiveSmallVec2<T> {
pub fn push(&mut self, item: T) {
if self.item1.is_none() {
self.item1 = Some(item);
} else if self.item2.is_none() {
self.item2 = Some(item);
} else {
self.more.push(item);
}
}
pub fn element_by_index(&self, index: usize) -> Option<&T> {
match index {
0 => self.item1.as_ref(),
1 => self.item2.as_ref(),
_ => self.more.get(index - 2),
}
}
}
pub trait MyTrait {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns;
}
/// IMPORTANT PART IS HERE: Uses NaiveSmallVec2
pub struct ItemTraitReturns<'a>(NaiveSmallVec2<Cow<'a, str>>);
/// only takes items with static lifetime
pub struct MyTraitStruct {
map: HashMap<usize, ItemTraitReturns<'static>>,
}
impl MyTrait for MyTraitStruct {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns {
let temp: &ItemTraitReturns<'static> = self.map.get(&id).unwrap();
// astonishingly this works!
temp
}
}
I expect the SmallVec version to compile like the Vec version does. I don't understand why in some cases (in the case of Vec) &ItemTraitReturns<'static> can be converted to &ItemTraitReturns<'_> and in some cases (SmallVec) it's not possible (I don't see the influence of Vec / SmallVec).
I don't want to change lifetimes of this trait:
pub trait MyTrait {
fn get_by_id(&self, id: usize) -> &ItemTraitReturns;
}
... since when using the trait I don't care about lifetimes (this should be an implementation detail) ... but still would like to use SmallVec.
This difference appears to be caused by a difference in variance between Vec and SmallVec. While Vec<T> is covariant in T, SmallVec appears to be invariant. As a result, SmallVec<[&'a T; 1]> can't be converted to SmallVec<[&'b T; 1]> even when lifetime 'a outlives 'b and the conversion should be safe. Here is a minimal example triggering the compiler error:
fn foo<'a, T>(x: SmallVec<[&'static T; 1]>) -> SmallVec<[&'a T; 1]> {
x // Compiler error here: lifetime mismatch
}
fn bar<'a, T>(x: Vec<&'static T>) -> Vec<&'a T> {
x // Compiles fine
}
This appears to be a shortcoming of SmallVec. Variance is automatically determined by the compiler, and it is sometimes cumbersome to convince the compiler that it is safe to assume that a type is covariant.
There is an open Github issue for this problem.
Unfortunately, I don't know a way of figuring out the variance of a type based on its public interface. Variance is not included in the documentation, and depends on the private implementation details of a type. You can read more on variance in the language reference or in the Nomicon.

Lifetime parameter for `Self` in trait signature

Consider this simple protocol implementation:
#[derive(PartialEq, Debug)]
enum Req<'a> {
InputData(&'a [u8]),
Stop,
}
impl<'a> Req<'a> {
fn decode(packet: &'a [u8]) -> Result<Req<'a>, String> {
match packet.first() {
Some(&0x01) => Ok(Req::InputData(&packet[1..])),
Some(&0x02) => Ok(Req::Stop),
_ => Err(format!("invalid request: {:?}", packet)),
}
}
}
#[derive(PartialEq, Debug)]
enum Rep<'a> {
OutputData(&'a [u8]),
StopAck,
}
impl<'a> Rep<'a> {
fn decode(packet: &'a [u8]) -> Result<Rep<'a>, String> {
match packet.first() {
Some(&0x01) => Ok(Rep::OutputData(&packet[1..])),
Some(&0x02) => Ok(Rep::StopAck),
_ => Err(format!("invalid reply: {:?}", packet)),
}
}
}
fn assert_req(packet: Vec<u8>, sample: Req) {
assert_eq!(Req::decode(&packet), Ok(sample));
}
fn assert_rep(packet: Vec<u8>, sample: Rep) {
assert_eq!(Rep::decode(&packet), Ok(sample));
}
fn main() {
assert_req(vec![1, 2, 3], Req::InputData(&[2, 3]));
assert_req(vec![2], Req::Stop);
assert_rep(vec![1, 2, 3], Rep::OutputData(&[2, 3]));
assert_rep(vec![2], Rep::StopAck);
}
playground
This works, but the two functions assert_req and assert_rep have identical code with only a difference in types. It is a good idea to write one generic assert_packet:
trait Decode<'a>: Sized {
fn decode(packet: &'a [u8]) -> Result<Self, String>;
}
#[derive(PartialEq, Debug)]
enum Req<'a> {
InputData(&'a [u8]),
Stop,
}
impl<'a> Decode<'a> for Req<'a> {
fn decode(packet: &'a [u8]) -> Result<Req<'a>, String> {
match packet.first() {
Some(&0x01) => Ok(Req::InputData(&packet[1..])),
Some(&0x02) => Ok(Req::Stop),
_ => Err(format!("invalid request: {:?}", packet)),
}
}
}
#[derive(PartialEq, Debug)]
enum Rep<'a> {
OutputData(&'a [u8]),
StopAck,
}
impl<'a> Decode<'a> for Rep<'a> {
fn decode(packet: &'a [u8]) -> Result<Rep<'a>, String> {
match packet.first() {
Some(&0x01) => Ok(Rep::OutputData(&packet[1..])),
Some(&0x02) => Ok(Rep::StopAck),
_ => Err(format!("invalid reply: {:?}", packet)),
}
}
}
fn assert_packet<'a, T>(packet: Vec<u8>, sample: T)
where
T: Decode<'a> + PartialEq + std::fmt::Debug,
{
assert_eq!(T::decode(&packet), Ok(sample));
}
fn main() {
assert_packet(vec![1, 2, 3], Req::InputData(&[2, 3]));
assert_packet(vec![2], Req::Stop);
assert_packet(vec![1, 2, 3], Rep::OutputData(&[2, 3]));
assert_packet(vec![2], Rep::StopAck);
}
playground
However, this triggers a "does not live long enough" error:
error[E0597]: `packet` does not live long enough
--> src/main.rs:41:27
|
41 | assert_eq!(T::decode(&packet), Ok(sample));
| ^^^^^^ does not live long enough
42 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the lifetime 'a as defined on the function body at 37:1...
--> src/main.rs:37:1
|
37 | / fn assert_packet<'a, T>(packet: Vec<u8>, sample: T)
38 | | where
39 | | T: Decode<'a> + PartialEq + std::fmt::Debug,
40 | | {
41 | | assert_eq!(T::decode(&packet), Ok(sample));
42 | | }
| |_^
If I understand correctly, the problem is in the function signature:
fn assert_packet<'a, T>(packet: Vec<u8>, sample: T)
where
T: Decode<'a>
Here, packet is destroyed when the function returns, but the user-provided 'a lifetime parameter says that the lifetime should end somewhere outside the assert_packet function. Is there any right solution? How should the signature look? Maybe higher rank trait bounds could help here?
Why does this compile:
fn assert_req(packet: Vec<u8>, sample: Req) {
assert_eq!(Req::decode(&packet), Ok(sample));
}
while this doesn't?
fn assert_packet<'a, T>(packet: Vec<u8>, sample: T) where T: Decode<'a> + PartialEq + std::fmt::Debug {
assert_eq!(T::decode(&packet), Ok(sample));
}
The difference is that in the first version, the two textual occurrences of Req name two different instances of the Req<'a> struct, with two different lifetimes. The first occurrence, on the sample parameter, is specialized with a lifetime parameter received by the assert_req function. The second occurrence, used to invoke decode, is specialized with the lifetime of the packet parameter itself (which ceases to exist as soon as the function returns). This means that the two arguments to assert_eq! don't have the same type; yet, it compiles because Req<'a> can be coerced to a Req<'b> where 'b is a shorter lifetime than 'a (Req<'a> is a subtype of Req<'b>).
On the other hand, in the second version, both occurrences of T must represent the exact same type. Lifetime parameters are always assumed to represent lifetimes that are longer than the function call, so it's an error to invoke T::decode with a lifetime that is shorter.
Here's a shorter function that exhibits the same problem:
fn decode_packet<'a, T>(packet: Vec<u8>) where T: Decode<'a> {
T::decode(&packet);
}
This function fails to compile:
<anon>:38:16: 38:22 error: `packet` does not live long enough
<anon>:38 T::decode(&packet);
^~~~~~
It's possible to use higher rank trait bounds on this function to make it compile:
fn decode_packet<T>(packet: Vec<u8>) where for<'a> T: Decode<'a> {
T::decode(&packet);
}
However, now we have another problem: we can't invoke this function! If we try to invoke it like this:
fn main() {
decode_packet(vec![1, 2, 3]);
}
We get this error:
<anon>:55:5: 55:18 error: unable to infer enough type information about `_`; type annotations or generic parameter binding required [E0282]
<anon>:55 decode_packet(vec![1, 2, 3]);
^~~~~~~~~~~~~
That's because we didn't specify which implementation of Decode we want to use, and there are no parameters the compiler can use to infer this information.
What if we specify an implementation?
fn main() {
decode_packet::<Req>(vec![1, 2, 3]);
}
We get this error:
<anon>:55:5: 55:25 error: the trait `for<'a> Decode<'a>` is not implemented for the type `Req<'_>` [E0277]
<anon>:55 decode_packet::<Req>(vec![1, 2, 3]);
^~~~~~~~~~~~~~~~~~~~
That doesn't work, because the Req we wrote is actually interpreted as Req<'_>, where '_ is a lifetime inferred by the compiler. This doesn't implement Decode for all possible lifetimes, only for one particular lifetime.
In order to be able to implement your desired function, Rust would have to support higher kinded types. It would then be possible to define a type constructor parameter (instead of a type parameter) on the function. For instance, you'd be able to pass Req or Rep as a type constructor requiring a lifetime parameter to produce a specific Req<'a> type.
It works correctly if you link the lifetimes involved to the input which you'll be referencing, e.g.
fn assert_packet<'a, T>(packet: &'a [u8], sample: T) where T: Decode<'a> ..
(You'll also need to update the tests to pass borrows rather than owned Vecs.)

Resources