I have a type Foo which contains a big array. With the help of serde_big_array, serde_derive, and serde_with, I can derive serialization and deserializalization.
use serde_big_array;
use serde_big_array::BigArray;
use serde_derive::{Deserialize, Serialize};
pub const SIZE: usize = 30000;
#[derive(Clone, Debug, Serialize, Deserialize)]
struct Foo {
#[serde(with = "BigArray")]
pub vals: [bool; SIZE],
}
This works fine.
However, when use this type in another structure I run into trouble.
#[derive(Clone, Debug, Serialize, Deserialize)]
struct Bar {
field0: Foo,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn deserialization_attempt_test() {
let foo = Foo {
vals: [false; SIZE],
};
let instance: Bar = Bar { field0: foo };
let json = serde_json::to_string(&instance).unwrap();
println!("PMD0");
let s_back = serde_json::from_str::<Bar>(&json).unwrap();
println!("PMD1");
}
}
Running this tests with cargo t deserialization_attempt_test -- --nocapture prints PMD0 but not PMD1. The test fails with the message
thread 'tests::deserialization_attempt' has overflowed its stack
fatal runtime error: stack overflow
How do I implement Serialize and Deserialize for Bar?
The problem seems to be that the stack (when running through cargo test) is too small to handle your test. Note that if you move deserialization_attempt_test into a main function and run it with cargo run it will work. Similarly if you reduce the array size to e.g. 2000 it will also work.
A workaround, although not that satsifying, is to manually set RUST_MIN_STACK when running cargo test. For instance, this will work:
RUST_MIN_STACK=8388608 cargo test
More information about the error can be found here and here. Here is the documentation on RUST_MIN_STACK.
If you wrap your JSON deserializer in a serde_stacker instance, then it works without setting the environment variable that #munksgaard suggests.
#[test]
fn serialization_bug_fix_attempt_test() {
let foo = Foo {
vals: [false; SIZE],
};
let instance: Bar = Bar { field0: foo };
let json = serde_json::to_string(&instance).unwrap();
println!("PMD0");
let mut deserializer: serde_json::Deserializer<serde_json::de::StrRead> =
serde_json::Deserializer::from_str(&json);
let deserializer = serde_stacker::Deserializer::new(&mut deserializer);
let value: Value = Value::deserialize(deserializer).unwrap();
let target: Bar = from_value(value).unwrap();
println!("PMD1");
}
Related
I'm confused why I am getting an Err(missing field "web3_node_provider") error when I cargo run using config-rs. Appears to fail at s.try_deserialize():
use config::{Config, ConfigError, Environment, File};
use serde::Deserialize;
#[derive(Debug, Deserialize)]
#[allow(unused)]
struct Web3NodeProvider {
ethereum_mainnet_node_url_http: String,
alchemy_api_key: String,
}
#[derive(Debug, Deserialize)]
#[allow(unused)]
pub struct Settings {
web3_node_provider: Web3NodeProvider,
}
impl Settings {
pub fn new() -> Result<Self, ConfigError> {
let s = Config::builder()
.add_source(File::with_name("config/default"))
.add_source(File::with_name("config/local").required(false))
.add_source(Environment::with_prefix("app"))
.build()?;
s.try_deserialize()
}
}
fn main() {
let settings = Settings::new();
println!("{:?}", settings);
}
I've pretty much followed the hierarchy example in config-rs, so I'm sure I'm just misunderstanding something basic or missing something. I am able to use "Web3NodeProvider.url" but not "web3_node_provider.ethereum_mainnet_node_url_http".
default.toml
[Web3NodeProvider]
ethereum_mainnet_node_url_http = "https://eth-mainnet.g.alchemy.com/v2/"
alchemy_api_key = "alchemy-api-key"
local.toml
[Web3NodeProvider]
alchemy_api_key = "randomapikey"
As per example, you have to name your field in config as name of attribute not name of structure (type). Like so:
[web3_node_provider]
ethereum_mainnet_node_url_http = "https://eth-mainnet.g.alchemy.com/v2/"
alchemy_api_key = "alchemy-api-key"
Wondering if there's a "proper" way of converting an Enum to a &str and back.
The problem I'm trying to solve:
In the clap crate, args/subcommands are defined and identified by &strs. (Which I'm assuming don't fully take advantage of the type checker.) I'd like to pass a Command Enum to my application instead of a &str which would be verified by the type-checker and also save me from typing (typo-ing?) strings all over the place.
This is what I came up with from searching StackOverflow and std:
use std::str::FromStr;
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum Command {
EatCake,
MakeCake,
}
impl FromStr for Command {
type Err = ();
fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {
match s.to_ascii_lowercase().as_str() {
"eat-cake" => Ok(Self::EatCake),
"make-cake" => Ok(Self::MakeCake),
_ => Err(()),
}
}
}
impl<'a> From<Command> for &'a str {
fn from(c: Command) -> Self {
match c {
Command::EatCake => "eat-cake",
Command::MakeCake => "make-cake",
}
}
}
fn main() {
let command_from_str: Command = "eat-cake".to_owned().parse().unwrap();
let str_from_command: &str = command_from_str.into();
assert_eq!(command_from_str, Command::EatCake);
assert_eq!(str_from_command, "eat-cake");
}
And here's a working playground:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=b5e9ac450fd6a79b855306e96d4707fa
Here's an abridged version of what I'm running in clap.
let matches = App::new("cake")
.setting(AppSettings::SubcommandRequiredElseHelp)
// ...
.subcommand(
SubCommand::with_name(Command::MakeCake.into())
// ...
)
.subcommand(
SubCommand::with_name(Command::EatCake.into())
// ...
)
.get_matches();
It seems to work, but I'm not sure if I'm missing something / a bigger picture.
Related:
How to use an internal library Enum for Clap Args
How do I return an error within match statement while implementing from_str in rust?
Thanks!
The strum crate may save you some work. Using strum I was able to get the simple main() you have to work without any additional From implementations.
use strum_macros::{Display, EnumString, IntoStaticStr};
#[derive(Debug, Clone, Copy, PartialEq)]
#[derive(Display, EnumString, IntoStaticStr)] // strum macros.
pub enum Command {
#[strum(serialize = "eat-cake")]
EatCake,
#[strum(serialize = "make-cake")]
MakeCake,
}
fn main() {
let command_from_str: Command = "eat-cake".to_owned().parse().unwrap();
let str_from_command: &str = command_from_str.into();
assert_eq!(command_from_str, Command::EatCake);
assert_eq!(str_from_command, "eat-cake");
}
I'm looking for a way to initialize a structopt Vec field with multiple items by default. I can do it for a single item with:
use structopt::StructOpt;
#[derive(Debug, StructOpt)]
pub struct Cli {
#[structopt(default_value = "foo")]
foo: Vec<String>,
}
fn main() {
let cli = Cli::from_iter(Vec::<String>::new());
assert_eq!(cli.foo, vec!["foo"]);
}
But how to make cli.foo to be equal let's say vec!["foo", "bar"] by default?
I've followed the L. Riemer advice, and it seems it's enough to implement FromStr only:
use structopt::StructOpt;
#[derive(Debug, PartialEq)]
struct Foo(Vec<String>);
impl std::str::FromStr for Foo {
type Err = Box<dyn std::error::Error>;
fn from_str(s: &str) -> Result<Self, Self::Err> {
Ok(Foo(s.split(",").map(|x| x.trim().to_owned()).collect()))
}
}
#[derive(StructOpt)]
pub struct Cli {
#[structopt(long, default_value = "foo, bar")]
foo: Foo,
}
fn main() {
let cli = Cli::from_iter(Vec::<String>::new());
assert_eq!(cli.foo, Foo(vec!["foo".into(), "bar".into()]));
let cli = Cli::from_iter(vec!["", "--foo", "foo"]);
assert_eq!(cli.foo, Foo(vec!["foo".into()]));
let cli = Cli::from_iter(vec!["", "--foo", "foo,bar,baz"]);
assert_eq!(cli.foo, Foo(vec!["foo".into(), "bar".into(), "baz".into()]));
}
I don't think you can do that: while StructOpt has some tricks around default values, I expect this still ends with the default value being injected in the CLI parsing as if it had been provided explicitly, which means there is likely no way to provide multiple default values (though I could certainly be wrong).
You probably want to handle this at the application-level e.g. right after having parsed the CLI, check for the arity of foo and update it if it's empty.
Take the following TOML data:
[[items]]
foo = 10
bar = 100
[[items]]
foo = 12
bar = 144
And the following rust code:
use serde_derive::Deserialize;
use toml::from_str;
use toml::value::Table;
#[derive(Deserialize)]
struct Item {
foo: String,
bar: String
}
fn main() {
let items_string: &str = "[[items]]\nfoo = 10\nbar = 100\n\n[[items]]\nfoo = 12\nbar = 144\n";
let items_table: Table = from_str(items_string).unwrap();
let items: Vec<Item> = items_table["items"].as_array().unwrap().to_vec();
// Uncomment this line to print the table
// println!("{:?}", items_table);
}
As you can see by yourself, the program does not compile, giving this error in return:
expected struct Item, found enum toml::value::Value
I understand its meaning, but I don't know how I could solve this and achieve what I wanted to do in the first place: cast a child array of a parent table into an array of structs and NOT into an array of tables.
You can parse into the pre-defined TOML types such as Table, but these types don't know about types outside of the pre-defined ones. Those types are mostly used when the actual type of the data is unknown, or unimportant.
In your case that means that the TOML Table type doesn't know about your Item type and cannot be made to know about it.
However you can easily parse into a different type:
use serde_derive::Deserialize;
use std::collections::HashMap;
use toml::from_str;
#[derive(Deserialize, Debug)]
struct Item {
foo: u64,
bar: u64,
}
fn main() {
let items_string: &str = "[[items]]\nfoo = 10\nbar = 100\n\n[[items]]\nfoo = 12\nbar = 144\n";
let items_table: HashMap<String, Vec<Item>> = from_str(items_string).unwrap();
let items: &[Item] = &items_table["items"];
println!("{:?}", items_table);
println!("{:?}", items);
}
(Permalink to the playground)
I have my types defined and then I keep them in something like ArrayVec<[MyType, 16]> (from arrayvec crate) variables (members of a structure). Restson has the RestPath trait, allow us to define a path used to form a URI when performing a REST query.
However, due to the restriction that only local traits can be implemented for arbitrary types (AKA the orphan rule) I can't use it straightforwardly for ArrayVec<[MyType, 16]>.
I somehow overcame the problem by implementing the trait for ~specialization~ instantiation of the following enum:
enum ModelArray<T> {
Array(T)
}
and then decorticate the instance of T using:
type MyArray = ModelArray<ArrayVec<[MyType; 16]>>;
let encapsulated_array: MyArray = client.get(()).unwrap();
let ModelArray::<ArrayVec<[MyType; 16]>>::Array(myarray) = encapsulated_array;
This works as minimal example, but I suffer from the fact I cannot call client.get(()).unwrap() directly to the member of other structure.
I'm surprised full specialization of a generic type isn't treated by Rust as local type and orphan rule still applies. Why?
Are there other nice ways to overcome the limitation and would let me nicely assign result of Restson's get() into members of a structure?
Working code:
extern crate restson;
extern crate arrayvec;
extern crate serde_derive;
use restson::RestPath;
use arrayvec::ArrayVec;
#[derive(Deserialize, Debug)]
struct MyType {
id: u16,
data: u32,
}
struct DataModel {
my_data: ArrayVec<[MyType; 16]>
}
#[derive(Deserialize, Debug)]
#[serde(untagged)]
enum ModelArray<T> {
Array(T)
}
impl RestPath<u16> for MyType {
fn get_path(id: u16) -> Result<String, Error> {
Ok(format!("data/MyType/{}", id))
}
}
impl RestPath<()> for ModelArray<ArrayVec<[MyType; 16]>> {
fn get_path(_: ()) -> Result<String, Error> {
Ok(String::from("data/MyType"))
}
}
use restson::RestClient;
pub fn load_data() {
let mut client = RestClient::new(&format!("http://{}", "localhost:8080")).unwrap();
let element: Type = client.get(24).unwrap();
println!("Room: {:?}", elementh);
type ModelArray = ModelArray<ArrayVec<[MyType; 16]>>;
let encapsulated: ModelArray = client.get(()).unwrap();
let ModelArray::<ArrayVec<[MyType; 16]>>::Array(elements) = encapsulated;
println!("Room: {:?}", elements[0]);
}
On Rust Playground (lack of restson crate wouldn't allow you to build)
Respective complete code: on GitHubGist