Derive macro generation - rust

I'm making my own Serializable trait, in the context of a client / server system.
My idea was that the messages sent by the system is an enum made by the user of this system, so it can be customize as needed.
Too ease implementing the trait on the enum, I would like to use the #[derive(Serializable)] method, as implementing it is always the same thing.
Here is the trait :
pub trait NetworkSerializable {
fn id(&self) -> usize;
fn size(&self) -> usize;
fn serialize(self) -> Vec<u8>;
fn deserialize(id: usize, data: Vec<u8>) -> Self;
}
Now, I've tried to look at the book (this one too) and this example to try to wrap my head around derive macros, but I'm really struggling to understand them and how to implement them. I've read about token streams and abstract trees, and I think I understand the basics.
Let's take the example of the id() method : it should gives a unique id for each variant of the enum, to allow headers of messages to tell which message is incoming.
let's say I have this enum as a message system :
enum NetworkMessages {
ErrorMessage,
SpawnPlayer(usize, bool, Transform), // player id, is_mine, position
MovePlayer(usize, Transform), // player id, new_position
DestroyPlayer(usize) // player_id
}
Then, the id() function should look like this :
fn id(&self) -> usize {
match &self {
&ErrorMessage => 0,
&SpawnPlayer => 1,
&MovePlayer => 2,
&DestroyPlayer => 3,
}
}
Here was my go with writting this using a derive macro :
#[proc_macro_derive(NetworkSerializable)]
pub fn network_serializable_derive(input: TokenStream) -> TokenStream {
// Construct a representation of Rust code as a syntax tree
// that we can manipulate
let ast = syn::parse(input).unwrap();
// Build the trait implementation
impl_network_serializable_macro(&ast)
}
fn impl_network_serializable_macro(ast: &syn::DeriveInput) -> TokenStream {
// get enum name
let ref name = ast.ident;
let ref data = ast.data;
let (id_func, size_func, serialize_func, deserialize_func) = match data {
// Only if data is an enum, we do parsing
Data::Enum(data_enum) => {
// Iterate over enum variants
let mut id_func_internal = TokenStream2::new();
let mut variant_id: usize = 0;
for variant in &data_enum.variants {
// add the branch for the variant
id_func_internal.extend(quote_spanned!{
variant.span() => &variant_id,
});
variant_id += 1;
}
(id_func_internal, (), (), ())
}
_ => {(TokenStream2::new(), (), (), ())},
};
let expanded = quote! {
impl NetworkSerializable for #name {
// variant_checker_functions gets replaced by all the functions
// that were constructed above
fn size(&self) -> usize {
match &self {
#id_func
}
}
/*
#size_func
#serialize_func
#deserialize_func
*/
}
};
expanded.into()
}
So this is generating quite a lot of errors, with the "proc macro NetworkSerializable not expanded: no proc macro dylib present" being first. So I'm guessing there a lot of misunderstaning from my part in here.

Related

Cannot get wrapping of filter to compile

I have the goal of wrapping an Iterator<Item = rusb::Device<_> to Iterator<Item = LitraDevice>. The latter contains specific implementation.
To make this work I tried the following code:
use std::iter::Filter;
use rusb;
const VENDOR: u16 = 0x046d;
const PRODUCT: u16 = 0xc900;
struct LitraDevice {
dev: rusb::Device<rusb::GlobalContext>,
}
pub struct LitraDevices {
unfiltered: rusb::DeviceList<rusb::GlobalContext>,
}
struct LitraDeviceIterator<'a> {
it: Filter<rusb::Devices<'a, rusb::GlobalContext>, for<'r> fn(&'r rusb::Device<rusb::GlobalContext>) -> bool>,
}
impl LitraDevices {
pub fn new() -> Self {
let unfiltered = rusb::devices().unwrap();
LitraDevices { unfiltered }
}
fn can_not_handle<'r>(dev: &'r rusb::Device<rusb::GlobalContext>) -> bool {
let desc = dev.device_descriptor().unwrap();
match (desc.vendor_id(), desc.product_id()) {
(VENDOR, PRODUCT) => (),
_ => return true,
}
match desc.class_code() {
LIBUSB_CLASS_HID => return true, // Skip HID devices, they are handled directly by OS libraries
_ => return false,
}
}
pub fn iter<'a>(self) -> LitraDeviceIterator<'a> {
let it = self.unfiltered.iter().filter(Self::can_not_handle);
LitraDeviceIterator{
it,
}
}
}
impl <'a> Iterator for LitraDeviceIterator<'a> {
type Item = LitraDevice;
fn next(&mut self) -> Option<Self::Item> {
let n = self.it.next();
match n {
Some(Device) => return Some(LitraDevice{dev: n.unwrap()}),
None => return None,
}
}
}
Now I really cannot figure out how to code LitraDeviceIterator so that it wraps the filtered iterator.
All code iterations I have tried so far turn into a generic nightmare very quickly.
I rewrote your iter() to yield LitraDevice, you can surely take it wherever you wanted to go from there.
The first underlying issue is that filter() yields references, but in cases like these, you actually mean to move yielded items while filtering. That's what filter_map() is capable of. That way, you can scrap the references, greatly simplifying your code.
(This code does not work yet, read on)
pub fn iter(self) -> impl Iterator<Item = LitraDevice> {
self.unfiltered.iter().filter_map(|dev| {
(!Self::can_not_handle(&dev))
.then_some(dev)
.map(|dev| LitraDevice { dev })
})
}
Now, there's a second little issue at play her: rusb::DeviceList<T : UsbContext>>::iter(&self) returns rusb::Devices<'_, T>, '_ being the anonymous lifetime inferred from &self. Meaning, while you can drive rusb::Devices<'_, T> to yield Device<T>s, you can not actually keep it around longer than self.unfiltered. More specifically, as you consume self in iter(), you can not return an iterator referencing that rusb::Devices<'_, T> from iter(). One solution is to immediately collect, then again moving into an iterator.
pub fn iter(self) -> impl Iterator<Item = LitraDevice> {
let devices = self.unfiltered.iter().collect::<Vec<_>>();
devices.into_iter().filter_map(|dev| {
(!Self::can_not_handle(&dev))
.then_some(dev)
.map(|dev| LitraDevice { dev })
})
}

How do I extract information about the type in a derive macro?

I am implementing a derive macro to reduce the amount of boilerplate I have to write for similar types.
I want the macro to operate on structs which have the following format:
#[derive(MyTrait)]
struct SomeStruct {
records: HashMap<Id, Record>
}
Calling the macro should generate an implementation like so:
impl MyTrait for SomeStruct {
fn foo(&self, id: Id) -> Record { ... }
}
So I understand how to generate the code using quote:
#[proc_macro_derive(MyTrait)]
pub fn derive_answer_fn(item: TokenStream) -> TokenStream {
...
let generated = quote!{
impl MyTrait for #struct_name {
fn foo(&self, id: #id_type) -> #record_type { ... }
}
}
...
}
But what is the best way to get #struct_name, #id_type and #record_type from the input token stream?
One way is to use the venial crate to parse the TokenStream.
use proc_macro2;
use quote::quote;
use venial;
#[proc_macro_derive(MyTrait)]
pub fn derive_answer_fn(item: proc_macro::TokenStream) -> proc_macro::TokenStream {
// Ensure it's deriving for a struct.
let s = match venial::parse_declaration(proc_macro2::TokenStream::from(item)) {
Ok(venial::Declaration::Struct(s)) => s,
Ok(_) => panic!("Can only derive this trait on a struct"),
Err(_) => panic!("Error parsing into valid Rust"),
};
let struct_name = s.name;
// Get the struct's first field.
let fields = s.fields;
let named_fields = match fields {
venial::StructFields::Named(named_fields) => named_fields,
_ => panic!("Expected a named field"),
};
let inners: Vec<(venial::NamedField, proc_macro2::Punct)> = named_fields.fields.inner;
if inners.len() != 1 {
panic!("Expected exactly one named field");
}
// Get the name and type of the first field.
let first_field_name = &inners[0].0.name;
let first_field_type = &inners[0].0.ty;
// Extract Id and Record from the type HashMap<Id, Record>
if first_field_type.tokens.len() != 6 {
panic!("Expected type T<R, S> for first named field");
}
let id = first_field_type.tokens[2].clone();
let record = first_field_type.tokens[4].clone();
// Implement MyTrait.
let generated = quote! {
impl MyTrait for #struct_name {
fn foo(&self, id: #id) -> #record { *self.#first_field_name.get(&id).unwrap() }
}
};
proc_macro::TokenStream::from(generated)
}

Function that generates a HashMap of Enum variants

I'm working with apollo_parser to parse a GraphQL query. It defines an enum, apollo_parser::ast::Definition, that has several variants including apollo_parser::ast::OperationDefintion and apollo_parser::ast::FragmentDefinition. I'd like to have a single Trait I can apply to apollo_parser::ast::Definition that provides a function definition_map that returns a HashMap mapping the operation name to the variant instance.
I've got as far as the trait, but I don't know how to implement it. Also, I don't know how to constrain T to be a variant of Definition.
trait Mappable {
fn definition_map<T>(&self) -> HashMap<String, T>;
}
EDIT:
Here's a Rust-ish pseudocode implementation.
impl Mappable for Document {
fn definition_map<T>(&self) -> HashMap<String, T> {
let defs = Vec<T> = self.definitions
.filter_map(|def: Definition| match def {
T(foo) => Some(foo),
_ => None
}).collect();
let map = HashMap::new();
for def: T in definitions {
map.insert(def.name(), def);
}
map
}
}
and it would output
// From a document consisting of OperationDefinitions "operation1" and "operation2"
// and FragmentDefinitons "fragment1" and "fragment2"
{
"operation1": OperationDefinition(...),
"operation2": OperationDefinition(...),
}
{
"fragment1": FragmentDefinition(...),
"fragment2": FragmentDefinition(...)
}
I don't know how to constrain T to be a variant of Definition.
There is no such thing in Rust. There's the name of the variant and the name of the type contained within that variant, there is no relationship between the two. The variants can be named whatever they want, and multiple variant can contain the same type. So there's no shorthand for pulling a T out of an enum which has a variant with a T.
You need to make your own trait that says how to get a T from a Definition:
trait TryFromDefinition {
fn try_from_def(definition: Definition) -> Option<Self> where Self: Sized;
fn name(&self) -> String;
}
And using that, your implementation is simple:
impl Mappable for Document {
fn definition_map<T: TryFromDefinition>(&self) -> HashMap<String, T> {
self.definitions()
.filter_map(T::try_from_def)
.map(|t| (t.name(), t))
.collect()
}
}
You just have to define TryFromDefinition for all the types you want to use:
impl TryFromDefinition for OperationDefinition {
fn try_from_def(definition: Definition) -> Option<Self> {
match definition {
Definition::OperationDefinition(operation) => Some(operation),
_ => None,
}
}
fn name(&self) -> String {
self.name().unwrap().ident_token().unwrap().text().into()
}
}
impl TryFromDefinition for FragmentDefinition {
fn try_from_def(definition: Definition) -> Option<Self> {
match definition {
Definition::FragmentDefinition(operation) => Some(operation),
_ => None,
}
}
fn name(&self) -> String {
self.fragment_name().unwrap().name().unwrap().ident_token().unwrap().text().into()
}
}
...
Some of this could probably be condensed using macros, but there's no normalized way that I can tell to get a name from a definition, so that would still have to be custom per type.
You should also decide how you want to handle definitions that don't have a name; you'd probably want to return Option<String> to avoid all those .unwrap()s, but I don't know how you'd want to put that in your HashMap.
Without knowing your whole workflow, I might suggest a different route instead:
struct Definitions {
operations: HashMap<String, OperationDefinition>,
fragments: HashMap<String, FragmentDefinition>,
...
}
impl Definitions {
fn from_document(document: &Document) -> Self {
let mut operations = HashMap::new();
let mut fragments = HashMap::new();
...
for definition in document.definitions() {
match definition {
Definition::OperationDefinition(operation) => {
let name: String = operation.name().unwrap().ident_token().unwrap().text().into();
operations.insert(name, operation);
},
Definition::FragmentDefinition(fragment) => {
let name: String = fragment.fragment_name().unwrap().name().unwrap().ident_token().unwrap().text().into();
fragments.insert(name, fragment);
},
...
}
}
Definitions {
operations,
fragments,
...
}
}
}

Implementing Strategy pattern in rust without knowing which strategy are we using at compile time

I've been trying to implement a Strategy pattern in rust, but I'm having trouble understanding how to make it work.
So let's imagine we have a trait Adder and Element:
pub trait Element {
fn to_string(&self) -> String;
}
pub trait Adder {
type E: Element;
fn add (&self, a: &Self::E, b: &Self::E) -> Self::E;
}
And we have two implementations StringAdder with StringElements and UsizeAdder with UsizeElements:
// usize
pub struct UsizeElement {
pub value: usize
}
impl Element for UsizeElement {
fn to_string(&self) -> String {
self.value.to_string()
}
}
pub struct UsizeAdder {
}
impl Adder for UsizeAdder{
type E = UsizeElement;
fn add(&self, a: &UsizeElement, b: &UsizeElement) -> UsizeElement{
UsizeElement { value: a.value + b.value }
}
}
// String
pub struct StringElement {
pub value: String
}
impl Element for StringElement {
fn to_string(&self) -> String {
self.value.to_string()
}
}
pub struct StringAdder {
}
impl Adder for StringAdder {
type E = StringElement;
fn add(&self, a: &StringElement, b: &StringElement) -> StringElement {
let a: usize = a.value.parse().unwrap();
let b: usize = b.value.parse().unwrap();
StringElement {
value: (a + b).to_string()
}
}
}
And I want to write a code that uses trait methods from Adder trait and it's corresponding elements without knowing at compile time which strategy is going to be used.
fn main() {
let policy = "usize";
let element = "1";
let adder = get_adder(&policy);
let element_a = get_element(&policy, element);
let result = adder.add(element_a, element_a);
}
To simplify I'm going to assign a string to policy and element but normally that would be read from a file.
Is the only way to implement get_adder and get_element using dynamic dispatch? And by extension should I define Adder and Element traits to use trait objects and or the Any trait?
Edit: Here is what I managed to figure out so far.
An example of possible implementation is using match to help define concrete types for the compiler.
fn main() {
let policy = "string";
let element = "1";
let secret_key = "5";
let result = cesar(policy, element, secret_key);
dbg!(result.to_string());
}
fn cesar(policy: &str, element: &str, secret_key: &str) -> Box<dyn Element>{
match policy {
"usize" => {
let adder = UsizeAdder{};
let element = UsizeElement{ value: element.parse().unwrap() };
let secret_key = UsizeElement{ value: secret_key.parse().unwrap() };
Box::new(cesar_impl(&adder, &element, &secret_key))
}
"string" => {
let adder = StringAdder{};
let element = StringElement{ value: element.to_string() };
let secret_key = StringElement{ value: secret_key.to_string() };
Box::new(cesar_impl(&adder, &element, &secret_key))
}
_ => {
panic!("Policy not supported!")
}
}
}
fn cesar_impl<A>(adder: &A, element: &A::E, secret_key: &A::E) -> A::E where A: Adder, A::E : Element {
adder.add(&element, &secret_key)
}
However the issue is that I have to wrap every function I want to implement using a match function to determine the concrete type, and also case for every policy available.
It does not seem like the proper way of implementing it as it will bloat the code, make it more error prone and less maintainable unless I end up using macros.
Edit 2: Here you can find an example using dynamic dispatch. However I'm not convinced it's the proper way to implement the solution.
Example using dynamic dispatch
Thank you for your help :)

How do I make my custom derive macro accept trait generic parameters?

I am trying to implement custom derive macros for my traits, and they actually work!
However I have a slight problem. I can't seem to find a way to include generic parameters to the trait.
Specifically, I want to do something like this : #[derive(MyCustomDerive<'a, B, C>)]
Instead, right now I am hard-coding the generics, like so :
let gen = quote! {
impl #impl_generics Graph<'a, V, E> for #name #ty_generics #where_clause {
fn Map(&self) -> &MAP<V, E> {
&self.map
}
...
}
As you can see, I am including 'a, V and E fixed within the quote block, instead of something I want to achieve, which is being able to flexibly derive the trait with the generic types I want.
What I would like is something akin to this :
#[derive(MyCustomDerive<'a, B, C>)]
to result in something equivalent to this
let gen = quote! {
impl #impl_generics Graph<'a, B, C> for #name #ty_generics #where_clause {
fn Map(&self) -> &MAP<B, C> {
&self.map
}
...
}
This would allow me to reserve (of course if necessary) V and E for other things and in my opinion make code more controllable.
Thank you for your help!
Update 1 :
This is how my derive function looks
pub fn derive(ast: &syn::DeriveInput) -> TokenStream {
let name = &ast.ident;
let generics = &ast.generics;
let (impl_generics, ty_generics, where_clause) = generics.split_for_impl();
let gen = quote! {
impl #impl_generics Graph<'a, V, E> for #name #ty_generics #where_clause {
fn Map(&self) -> &MAP<V, E> {
&self.map
} ...
I don't believe it's possible to use exactly the syntax you describe in your post (#[derive(MyCustomDerive<'a, B, C>)]). However, consider the following syntax that used an additional custom attribute instead:
#[derive(MyTrait)]
#[my_trait('a, B, C)]
struct MyStruct {
// ...
}
To allow the my_trait attribute to be used, you'll have to add an attributes section to your proc_macro_derive attribute.
#[proc_macro_derive(MyTrait, attributes(my_trait))]
pub fn derive_my_trait(input: TokenStream) -> TokenStream {
// ...
}
For help with parsing the attribute itself, take a look at syn::Attribute. The tokens field is a TokenStream from which you could extract the necessary parameters. For example, if your trait has one lifetime and two type parameters, your parsing logic might look something like this:
struct MyParams(syn::Lifetime, syn::Ident, syn::Ident);
impl syn::Parse for MyParams {
fn parse(input: syn::ParseStream) -> Result<Self> {
let content;
syn::parenthesized!(content in input);
let lifetime = content.parse()?;
content.parse::<Token![,]>()?;
let type1 = content.parse()?;
content.parse::<Token![,]>()?;
let type2 = content.parse()?;
Ok(MyParams(lifetime, type1, type2))
}
}
pub fn derive(ast: &syn::DeriveInput) -> TokenStream {
let attribute = ast.attrs.iter().filter(
|a| a.path.segments.len() == 1 && a.path.segments[0].ident == "my_trait"
).nth(0).expect("my_trait attribute required for deriving MyTrait!");
let parameters: MyParams = syn::parse2(attribute.tokens.clone()).expect("Invalid my_trait attribute!");
// ... do stuff with `parameters`
}

Resources