I'd like to create a macro that operates on a given list of types, but I need to be able to store which other types are being processed.
A simple example of something I want to do:
struct Foo;
struct Bar {
foo: Foo,
data: u32,
}
baz!(Foo, Bar);
// outputs
struct OptFoo;
struct OptBar {
foo: OptFoo,
data: u32
}
The problem is that it doesn't seem like macro_rules allows me to store a temporary state (i.e a HashSet where I would tag which types are part of the macro invocation). The only workaround I have in mind is to write what I want as a proc_macro_derive and manually adding a custom attribute for each type I need but that's obviously far from perfect...
Edit:
The question is similar to this one, but here I'mm trying to save a state locally and temporarily (basically doing two passes over the arguments while storing data about those) in a single macro call. However it seems it's also not possible.
As pointed by #trentcl, what I want to achieve is indeed possible with proc macro (I thought proc macros were limited to Derive and attributes...)
#[proc_macro]
pub fn generate(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let input = proc_macro2::TokenStream::from(input);
println!("{:?}", input);
proc_macro::TokenStream::from(input)
}
generate!(struct Foo;);
// outputs its argument without changing anything, i.e:
// struct Foo ;
The previous example demonstrates a trivial macro that prints to sdout the parsed input: TokenStream [Ident { ident: "struct", span: #0 bytes(330..336) }, Ident { ident: "Foo", span: #0 bytes(337..340) }, Punct { ch: ';', spacing: Alone, span: #0 bytes(340..341) }]
Note that it parses the tokens but does not create an AST; we'll have to use syn for that.
This repo has examples many of what can be done with proc macros, pretty helpful!
Related
I'm working on a DSL implemented as a proc macro, and would like to be able to get to the original source, as a string, of the contents of the proc macro.
So for instance, if my proc macro is invoked like so:
my_macro! {
one
two
three
}
I would like to get access to this string inside the proc macro implementation:
let source: String = get_source(input) // this is what I want to be able to do
println!("{}", source);
// prints: "one\ntwo\nthree"
I can't figure out how to do this. When I try to convert the token stream to a string, the whitespace information is lost.
So for instance if I try to convert the token stream directly to a string, each token is simply separated by a space:
#[proc_macro]
pub fn my_macro(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
println!("{}", input);
// prints "one two three"
...
How can I get the input source?
I created a procedural macro that implements a trait, but in order for this to work I need to get the raw bytes for every field. The problem is how to get the bytes of a field differs depending on the type of field.
Is there some way of testing if a function exists on a field and if it does not tries another function?
E.g. something like this:
if item.field::function_exist {
//do code
} else {
//do other code
}
Currently I am looking at creating another trait/member function that I just have to create for all primitives and create a procedural macro for larger fields such as structs. For example:
if item.field::as_bytes().exists {
(&self.#index).as_bytes()
} else {
let bytes = (&self.#index).to_bytes();
&bytes
}
With a string, it has a as_bytes member function, while i32 does not. This means I need extra code, when the member field of the struct is not a string. I might need a match rather than an if, but the if will suffice for the example.
Is it possible to tell if a field is a certain type or implements a certain method in a procedural macro?
No, it is not.
Macros operate on the abstract syntax tree (AST) of the Rust code. This means that you basically just get the characters that the user typed in.
If user code has something like type Foo = Option<Result<i32, MyError>>, and you process some code that uses Foo, the macro will not know that it's "really" an Option.
Even if it did know the type, knowing what methods are available would be even harder. Future crates can create traits which add methods to existing types. At the point in time that the procedural macro is running, these crates may not have even been compiled yet.
I am looking at creating another trait/member function that I just have to create for all primitives and create a procedural macro for larger fields such as structs.
This is the correct solution. If you look at any existing well-used procedural macro, that's exactly what it does. This allows the compiler to do what the compiler is intended to do.
This is also way better for maintainability — now these primitive implementations live in a standard Rust file, as opposed to embedded inside of a macro. Much easier to read and debug.
Your crate will have something like this:
// No real design put into this trait
trait ToBytes {
fn encode(&self, buf: &mut Vec<u8>);
}
impl ToBytes for str {
fn encode(&self, buf: &mut Vec<u8>) {
buf.extend(self.as_bytes())
}
}
// Other base implementations
And your procedural macro will implement this in the straightforward way:
#[derive(ToBytes)]
struct Foo {
a: A,
b: B,
}
becomes
impl ToBytes for Foo {
fn encode(&self, buf: &mut Vec<u8>) {
ToBytes::encode(&self.a, buf);
ToBytes::encode(&self.b, buf);
}
}
As a concrete example, Serde does the same thing, with multiple ways of serializing to and from binary data:
Bincode
CBOR
MessagePack
etc.
I am trying to write a macro that generates a struct in Rust. This macro will add different Serde attributes to struct fields based on the type of field. This is the final goal.
For now, I'm simply trying to write a macro that uses another macro for generating a recursive code.
This is what the code looks like:
macro_rules! f_list {
($fname: ident, $ftype: ty) => {
pub $fname: $ftype,
}
}
macro_rules! mk_str {
($sname: ident; $($fname: ident: $ftype: ty,)+) => {
#[derive(Debug, Clone)]
pub struct $sname {
$(
f_list!($fname, $ftype)
)+
}
}
}
mk_str! {
Yo;
name: String,
}
fn main() {
println!("{:?}", Yo { name: "yo".to_string() })
}
This code on running gives below error, which I'm not able to understand.
error: expected `:`, found `!`
--> src/main.rs:12:23
|
12 | f_list!($fname, $ftype);
| ^ expected `:`
What's wrong here?
Here's a playground link
When writing declarative macros (macro_rules!), it's important to understand that the output of a macro must be a pattern, a statement, an expression, an item or an impl. Effectively, you should think of the output as something that can stand alone, syntactically speaking.
In your code, the macro f_list, if it worked, would output code like
name1: type1,
name2: type2,
name3: type3,
While this could be part of a declaration for a struct, it is not itself something that can stand alone.
Why is the error in the other macro then? mk_str successfully expands to
#[derive(Debug, Clone)]
pub struct Yo {
f_list!(name, String)
}
However, the parser doesn't expect a macro inside of a struct declaration. The inside of a struct isn't a pattern, a statement, an expression, an item or an impl. Thus, when it sees the !, it gives up and reports an error.
How can you fix this? In this particular example, f_list is fairly redundant. You could simply replace f_list!($fname, $ftype) with pub $fname: $ftype, in mk_str and it'll work as written. If this doesn't work for your goal, take a look at The Little Book of Rust Macros. It has some patterns for doing very complicated things with macros. Most of the information in this answer came from the section "Macros in the AST".
From the std::default::Default docs:
#[derive(Default)]
struct SomeOptions {
foo: i32,
bar: f32,
}
fn main() {
let options = SomeOptions { foo: 42, ..Default::default() };
}
What is the .. prefix doing to the returned value of Default::default() and why is it necessary here? It almost seems like it's acting as a spread operator, but I'm not sure. I understand what ..Default::default() is doing -- filling in the remaining struct parameters with the default values of SomeOptions, but not how .. works. What is the name of this operator?
This is the struct update syntax. It is "needed" only to have a succinct way of moving / copying all of the members of a struct to a new one, potentially with some small modifications.
The "long" way of writing this would be:
let a = SomeOptions::default();
let options = SomeOptions { foo: 42, bar: a.bar };
You could indeed think of it similar to the JavaScript "spread" operator, but Rust's nuances of ownership and strong typing still come into play, so it's not as widely used. For example, you can't use this syntax to go between values of different types.
Is there a way to print out a complete list of available members of a type or instance in Rust?
For example:
In Python, I can use print(dir(object))
In C, Clang has a Python API that can parse C code and introspect it.
Being unfamiliar with Rust tools, I'm interested to know if there is some way to do this, either at run-time or compile-time, either with compiler features (macros for example), or using external tools.
This question is intentionally broad because the exact method isn't important. It is common in any language to want to find all of a variable's methods/functions. Not knowing Rust well, I'm not limiting the question to specific methods for discovery.
The reason I don't define the exact method is that I assume IDEs will need this information, so there will need to be some kinds of introspection available to support this (eventually). For all I know, Rust has something similar.
I don't think this is a duplicate of Get fields of a struct type in a macro since this answer could include use of external tools (not necessarily macros).
Is there a way to print out a complete list of available members of a type or instance in Rust?
Currently, there is no such built-in API that you can get the fields at runtime. However you can retrieve fields by using two different ways.
Declarative Macros
Procedural Macros
Solution By Using Declarative Macro
macro_rules! generate_struct {
($name:ident {$($field_name:ident : $field_type:ty),+}) => {
struct $name { $($field_name: $field_type),+ }
impl $name {
fn introspect() {
$(
let field_name = stringify!($field_name);
let field_type = stringify!($field_type);
println!("Field Name: {:?} , Field Type: {:?}",field_name,field_type);
)*
}
}
};
}
generate_struct! { MyStruct { num: i32, s: String } }
fn main() {
MyStruct::introspect();
}
This will give you the output:
Field Name: "num" , Field Type: "i32"
Field Name: "s" , Field Type: "String"
Playground
Solution Using Procedural Macro
Since procedural macros are more complicated from the declarative macros, you better to read some references(ref1, ref2, ref3) before starting.
We are going to write a custom derive which is named "Instrospect". To create this custom derive, we need to parse our struct as a TokenStream with the help of syn crate.
#[proc_macro_derive(Introspect)]
pub fn derive_introspect(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as ItemStruct);
// ...
}
Since our input can be parsed as ItemStruct and ItemStruct has the fields() method in it, we can use this to get fields of our struct.
After we get these fields, we can parse them as named and we can print their field name and field type accordingly.
input
.fields
.iter()
.for_each(|field| match field.parse_named() {
Ok(field) => println!("{:?}", field),
Err(_) => println!("Field can not be parsed successfully"),
});
If you want to attach this behavior to your custom derive you can use the following with the help of the quote crate:
let name = &input.ident;
let output = quote! {
impl #name {
pub fn introspect(){
input
.fields
.iter()
.for_each(|field| match field.parse_named() {
Ok(field) => println!("{:?}", field),
Err(_) => println!("Field can not be parsed successfully"),
});
}
}
};
// Return output TokenStream so your custom derive behavior will be attached.
TokenStream::from(output)
Since the behaviour injected to your struct as introspect function, you can call it in your application like following:
#[derive(Introspect)]
struct MyStruct {
num: i32,
text: String
}
MyStruct::introspect();
Note: Since the example you are looking for similar to this question. This Proc Macro Answer and Declarative Macro Answer should give you insight as well
To expand on my comment, you can use rustdoc, the Rust documentation generator, to view almost everything you're asking for (at compile time). rustdoc will show:
Structs (including public members and their impl blocks)
Enums
Traits
Functions
Any documentation comments written by the crate author with /// or //!.
rustdoc also automatically links to the source of each file in the [src] link.
Here is an example of the output of rustdoc.
Standard Library
The standard library API reference is available here and is available for anything in the std namespace.
Crates
You can get documentation for any crate available on crates.io on docs.rs. This automatically generates documentation for each crate every time it is released on crates.io.
Your Project
You can generate documentation for your project with Cargo, like so:
cargo doc
This will also automatically generate documentation for your dependencies (but not the standard library).
I have written a very simple crate which uses procedural macro. It gives you access to members information plus some simple information about struct/enum you use. Information about methods can not be given because procedural macros simply can't get this information, and as far as I know, there are no any methods which may give such information.
I don't think there is anything that will do this out of the box.
It may be possible to write a compiler plugin which can do that by examining the AST.
If you need the field names inside your program then you probably need to use macros. Either wrap your struct definition in macro and pattern match to create some function to get their names, or use procedural macro to derive structs for traits with such functions.
See examples in syn for derived traits. In particular, see syn::Data::Struct which has fields.
According to question of #AlexandreMahdhaoui, I would say: at least on latest Rust versions the proc_macro from accepted answer will not work, because you will need to pass tokens into quote! using "#". So you could try smth like next:
use proc_macro::{TokenStream};
use quote::{quote, ToTokens};
use syn::{parse_macro_input, ItemStruct};
#[proc_macro_derive(Introspect)]
pub fn derive(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as ItemStruct);
let ident = input.ident;
let field_data = input.fields.iter().map(|f| {
let field_type = f.ty.clone();
format!(
"Name={}, Type={}",
f.ident.clone().unwrap().to_string(),
quote!(#field_type).to_string()
)
}).collect::<Vec<_>>();
let output = quote! {
impl #ident {
pub fn introspect() {
println!("{:#?}", vec![#(#field_data),*]);
}
}
};
TokenStream::from(output)
}
#[derive(Introspect)]
struct Test {
size: u8,
test_field: u8,
}
fn main() {
Test::introspect();
}
Regarding methods, defined in impl I didn't find any info in output, so not sure if it possible. Probably somebody could share in comments ?
I use something like this:
println!("{:?}", variable); // struct, enum whatever
If it's a large type, use the # version:
println!("{:#?}", variable); // struct, enum whatever