Rust TinyTemplate with dynamic values (HashMap)? - rust

I serve an Actix Web Rust website using the TinyTemplate crate, where they keys and values are defined in a TOML file that is serialized using the config crate.
This works great with predefined keys (known at compile time) like "title" but I want to add a dynamic "see also" section and I can't figure out how to implement this.
config.toml
title= "Example Title" # this works great
[seealso] # dynamic values, is this possible?
"Project Homepage" = "https://my-project-page.eu"
"Repository" = "https://github.com/myorg/myrepo"
template.html
{title}
{{ for x in seealso }}
...
{{ endfor }}
main.rs
[...]
#[derive(Serialize, Deserialize, Debug)]
pub struct Config {
pub title: String,
pub seealso: HashMap<String, String>,
}
impl Config {
pub fn new() -> Result<Self, ConfigError> {
config::Config::builder()
.add_source(File::from_str(DEFAULT, FileFormat::Toml))
.build()?
.try_deserialize()
}
}
[...]
lazy_static! {
pub static ref CONFIG: Config = Config::new().expect("Error reading configuration.");
}
[...]
let mut template = TinyTemplate::new();
template.add_template("index", INDEX).expect("Could not parse default template");
let body = template().render("index", &*CONFIG).unwrap();
Output
thread 'actix-rt|system:0|arbiter:0' panicked at 'called `Result::unwrap()` on an `Err` value: RenderError { msg: "Expected an array for path 'seealso' but found a non-iterable value.", line: 32, column: 17 }', src/main.rs:80:53
I assume that serde deserializes the HashMap into a JSON object, with the HashMap keys as object keys, which is why I assume Rust won't let me iterate over them. The problem is that TinyTemplate is quite basic in functionality by design, and won't let me put arbitrary code between {{ }} so I can't transform the object (struct?) into something that TinyTemplate can iterate over. I'm quite new to Rust so I may be missing something obvious here, but is there a way to use dynamic values like this with TinyTemplates or is there no way to handle HashMaps? The result doesn't have to use HashMaps though.
The goal is to achieve something like the following pseudocode:
{{ if ! seealso.empty() }}
See also:
<ul>
{{ for (key, value) in seealso }}
<li>{key}</li>
{{ endfor }}
</ul>
{{ endif }}

I have took a look on the code on tinytemplate. It converts the given object into a json object with serde and works with the json representation for further processing. Given a Hashmap, it will create an plain object node which isn't iterable according to the code of the crate.
You cloud create a pull request for the crate; iterating over object fields should be straight forward.
Another possible solution could be to create a second representation of your config.
use config::{ConfigError, File};
use lazy_static::lazy_static;
use serde::Deserialize;
use serde::Serialize;
use std::collections::HashMap;
use std::path::Path;
use tinytemplate::TinyTemplate;
#[derive(Serialize, Deserialize, Debug)]
pub struct Config {
pub title: String,
pub seealso: HashMap<String, String>,
}
#[derive(Serialize, Deserialize, Debug)]
pub struct IterableConfig {
pub title: String,
pub seealso: Vec<(String, String)>,
}
impl Config {
pub fn new() -> Result<Self, ConfigError> {
config::Config::builder()
.add_source(File::from(Path::new("config.toml")))
.build()?
.try_deserialize()
}
}
lazy_static! {
pub static ref CONFIG: Config = Config::new().expect("Error reading configuration.");
}
fn main() {
let mut template = TinyTemplate::new();
template
.add_template(
"index",
"
{title}
{{ for x in seealso }}
{x.0}
{x.1}
{{ endfor }}
",
)
.expect("Could not parse default template");
let body = template
.render(
"index",
&IterableConfig {
title: CONFIG.title.clone(),
seealso: CONFIG
.seealso
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect(),
},
)
.unwrap();
eprintln!("{}", body);
}

Related

Why am I getting a "missing field" error using config-rs?

I'm confused why I am getting an Err(missing field "web3_node_provider") error when I cargo run using config-rs. Appears to fail at s.try_deserialize():
use config::{Config, ConfigError, Environment, File};
use serde::Deserialize;
#[derive(Debug, Deserialize)]
#[allow(unused)]
struct Web3NodeProvider {
ethereum_mainnet_node_url_http: String,
alchemy_api_key: String,
}
#[derive(Debug, Deserialize)]
#[allow(unused)]
pub struct Settings {
web3_node_provider: Web3NodeProvider,
}
impl Settings {
pub fn new() -> Result<Self, ConfigError> {
let s = Config::builder()
.add_source(File::with_name("config/default"))
.add_source(File::with_name("config/local").required(false))
.add_source(Environment::with_prefix("app"))
.build()?;
s.try_deserialize()
}
}
fn main() {
let settings = Settings::new();
println!("{:?}", settings);
}
I've pretty much followed the hierarchy example in config-rs, so I'm sure I'm just misunderstanding something basic or missing something. I am able to use "Web3NodeProvider.url" but not "web3_node_provider.ethereum_mainnet_node_url_http".
default.toml
[Web3NodeProvider]
ethereum_mainnet_node_url_http = "https://eth-mainnet.g.alchemy.com/v2/"
alchemy_api_key = "alchemy-api-key"
local.toml
[Web3NodeProvider]
alchemy_api_key = "randomapikey"
As per example, you have to name your field in config as name of attribute not name of structure (type). Like so:
[web3_node_provider]
ethereum_mainnet_node_url_http = "https://eth-mainnet.g.alchemy.com/v2/"
alchemy_api_key = "alchemy-api-key"

Deserialize hcl with labels

I'm attempting to use hcl-rs = 0.7.0 to parse some HCL. I'm just experimenting with arbitrary HCL, so I'm not looking to parse terraform specific code.
I'd like to be able to parse a block like this and get it's label as part of result
nested_block "nested_block_label" {
foo = 123
}
This currently doesn't work, but hopefully it shows my intention. Is something like this possible?
#[test]
fn deserialize_struct_with_label() {
#[derive(Deserialize, PartialEq, Debug)]
struct TestRoot {
nested_block: TestNested,
}
#[derive(Deserialize, PartialEq, Debug)]
struct TestNested {
label: String,
foo: u32,
}
let input = r#"
nested_block "nested_block_label" {
foo = 123
}"#;
let expected = TestRoot{ nested_block: TestNested { label: String::from("nested_block_label"), foo: 123 } };
assert_eq!(expected, from_str::<TestRoot>(input).unwrap());
}
Your problem is that hcl by default seems to interpret
nested_block "nested_block_label" {
foo = 123
}
as the following "serde structure":
"nested_block" -> {
"nested_block_label" -> {
"foo" -> 123
}
}
but for your Rust structs, it would have to be
"nested_block" -> {
"label" -> "nested_block_label"
"foo" -> 123
}
I'm not aware of any attributes that would allow you to bend the former into the latter.
As usual, when faced with this kind of situation, it is often easiest to first deserialize to a generic structure like hcl::Block and then convert to whatever struct you want manually. Disadvantage is that you'd have to do that for every struct separately.
You could, in theory, implement a generic deserialization function that wraps the Deserializer it receives and flattens the two-level structure you get into the one-level structure you want. But implementing Deserializers requires massive boilerplate. Possibly, somebody has done that before, but I'm not aware of any crate that would help you here, either.
As a sort of medium effort solution, you could have a special Labelled wrapper structure that always catches and flattens this intermediate level:
#[derive(Debug, PartialEq)]
struct Labelled<T> {
label: String,
t: T,
}
impl<'de, T: Deserialize<'de>> Deserialize<'de> for Labelled<T> {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
struct V<T>(std::marker::PhantomData<T>);
impl<'de, T: Deserialize<'de>> serde::de::Visitor<'de> for V<T> {
type Value = Labelled<T>;
fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>
where
A: serde::de::MapAccess<'de>,
{
if let (Some((label, t)), None) =
(map.next_entry()?, map.next_entry::<String, ()>()?)
{
Ok(Labelled { label, t })
} else {
Err(serde::de::Error::invalid_type(
serde::de::Unexpected::Other("Singleton map"),
&self,
))
}
}
}
deserializer.deserialize_map(V::<T>(Default::default()))
}
}
would be used like this:
#[derive(Deserialize, PartialEq, Debug)]
struct TestRoot {
nested_block: Labelled<TestNested>,
}
#[derive(Deserialize, PartialEq, Debug)]
struct TestNested {
foo: u32,
}
Lastly, there may be some trick like adding #[serde(rename = "$hcl::label")]. Other serialization libraries (e.g. quick-xml) have similar and allow marking fields as something special this way. hcl-rs does the same internally, but it's undocumented and I can't figure out from the source whether what you need is possible.

Parse list of tuples/maps into structs with serde-yaml

I would like to parse the following part of a .yaml-file:
networks:
foo1: 192.168.1.0/24
bar1: 192.168.2.0/24
foo2: 2001:CAFE:1::/64
bar2: 2001:CAFE:2::/64
... where the key of each property is the labels the name of a network and the value indicates the assigned subnet to it.
The deserialized structs should look like the following:
#[derive(Debug, PartialEq, Serialize, Deserialize)]
struct Networks {
networks: Vec<Network>
}
#[derive(Debug, PartialEq, Serialize, Deserialize)]
struct Network {
name: String,
subnet: String,
}
I would prefer this notation over
networks:
- name: foo1
subnet: 192.168.1.0/24
- ...
since this would add unnecessary boilerplate. The problem is that I cannot properly parse this list of networks into structs, since to my current knowledge, the property of a struct has to have an equivalent key to it - and since the key-names here are "random" I cannot do that.
The only other workaround I've found is parsing the entries as HashMaps and automatically convert those into tuples of (String, String) (networks would then be a Vec<(String, String)>) via the serde-tuple-vec-map crate (which loses the named parameters).
This seems like something that would be easy to configure, yet I haven't found any solutions elsewhere.
Implementing a custom deserializer and using it with #[serde(deserialize_with = "network_tuples")] networks: Vec<Network> is one way of doing it:
fn network_tuples<'de, D>(des: D) -> Result<Vec<Network>, D::Error>
where
D: serde::Deserializer<'de>,
{
struct Vis(Vec<Network>);
impl<'de> serde::de::Visitor<'de> for Vis {
type Value = Vec<Network>;
fn expecting(&self, _formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
todo!("return nice descriptive error")
}
fn visit_map<A>(mut self, mut map: A) -> Result<Self::Value, A::Error>
where
A: serde::de::MapAccess<'de>,
{
while let Some((name, subnet)) = map.next_entry()? {
self.0.push(Network { name, subnet });
}
Ok(self.0)
}
}
des.deserialize_map(Vis(vec![]))
}
You could do without the visitor by first deserializing to a HashMap<String, String> and then converting, but that has a performance penalty and you'd lose the ability to have two networks with the same name (not sure if you want that though).
Playground
Of course, you could also combine my favorite serde trick #[serde(from = "SomeStructWithVecStringString")] with the serde-tuple-vec-map crate. But I've already written many answers using from, so I'll refer to an example.

Render Template Rust Rocket Handlebars

I am trying to render a template with rust rocket and handlebars, this works fine but I can't get any arguments to the html file to work, the page loads, but the argument is not there. I am familiar with flask and jinja for python but not as much with rocket and handlebars. I have tried countless things but I can't find any documentation that works.
main.rs
#![feature(decl_macro)]
extern crate rocket;
extern crate rocket_contrib;
use rocket::{get, routes};
use rocket_contrib::templates::Template;
#[get("/one")]
fn one() -> String {
format!("One")
}
#[get("/page")]
fn page() -> Template {
let context = "string";
Template::render("page", &context)
}
fn main() {
rocket::ignite()
.mount("/", routes![one])
.attach(Template::fairing())
.launch();
}
page.html.hbs
<!DOCTYPE html>
<head></head>
<html>
<body>
<h1>Argument is: {{context}}</h1>
</body>
</html>
The template system doesn't know that the local variable is called context. It just knows that your context is a string, which doesn't have an attribute called context. You need to provide a type implementing Serialize, with a member called context for the template expression {{context}} to expand to anything. You could do this with a HashMap or a custom type.
With a HashMap:
#[get("/page")]
fn page() -> Template {
let mut context = HashMap::new();
context.insert("context", "string");
Template::render("page", &context)
}
With a custom type:
#[get("/page")]
fn page() -> Template {
#[derive(serde_derive::Serialize)]
struct PageContext {
pub context: String,
}
let context = PageContext { context: "string" };
Template::render("page", &context)
}
Custom types will generally perform better, and are self-documenting.

Deserialize two different types into the same one

I have a simple structure that contains a resources field. I would like my resources to always be Urls.
In the actual file that I am trying to deserialize, resources can either be URLs or Path.
Here is my structure:
pub struct Myfile {
pub resources: Vec<Resource>,
}
pub type Resource = Url;
I would like to use serde to:
Try to deserialize each Resource using the implementation from the url crate.
If it fails, try to deserialize each one of them into a Path and then use url::from_*_path() to get a Url.
I am trying to adapt the string or map,map and structure examples but I am struggling to understand where to even start.
Since my end result will by a Url, the examples seem to show that I should be implementing Deserialize for Url. But I still need to current implementation. My Resource is an alias so I can't implement Deserialize for it.
Is there any simple way to deserialize both Paths and Urls into Urls?
PS: I will probably go for the answer proposed but after reading through this post, I tried the following which seems to work too:
#[derive(Clone, Debug)]
pub struct Resource(Url);
impl<'de> Deserialize<'de> for Resource {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where D: Deserializer<'de>
{
let s = String::deserialize(deserializer)?;
if let path = Path::new(&s) {
if path.is_file() {
Ok(Resource(Url::from_file_path(path).unwrap()))
} else {
Ok(Resource(Url::from_directory_path(path).unwrap()))
}
} else {
Url::deserialize(s.into_deserializer()).map(|x| Resource(x))
}
}
}
But is less convenient to work with than regular Urls.
I think the key to doing this with reasonable effort is to realize that serde::Deserialize for Url is also just cooking with water, i.e. just expecting a string and calling Url::parse on it.
So it's time to deploy my favourite serde trick: deserialize to a struct that serde can handle easily:
#[derive(Deserialize)]
pub struct MyfileDeserialize {
pub resources: Vec<String>,
}
Tell serde that it should get the struct you finally want from that easily handlable struct:
#[derive(Deserialize, Debug)]
#[serde(try_from = "MyfileDeserialize")]
pub struct Myfile {
pub resources: Vec<Resource>,
}
Finally, you just need to define how to turn MyfileDeserialize into Myfile.
impl TryFrom<MyfileDeserialize> for Myfile {
type Error = &'static str; // TODO: replace with anyhow or a proper error type
fn try_from(t: MyfileDeserialize) -> Result<Self, &'static str> {
Ok(Self {
resources: t
.resources
.into_iter()
.map(|url| {
if let Ok(url) = Url::parse(&url) {
return Ok(url);
};
if let Ok(url) = Url::from_file_path(url) {
return Ok(url);
};
// try more from_*_path here.
Err("Can't as url or path")
})
.collect::<Result<Vec<_>, _>>()?,
})
}
}
Playground
Edit regarding your PS:
If you are willing to mess around with manually implementing deserializer traits and functions, I suppose you do have the option of completely getting rid of any wrapper structs or mediating TryFrom types: add a #[serde(deserialize_with = …)] to your resources, and in there, first do a <Vec<String>>::deserialize(de)?, and then turn that into a Vec<Url> as usual.
Playground

Resources