I'm stuck trying to "rename" a identifier inside a declarative macro, I have the renaming function already, but can't find a way to "execute" it inside the macro. Is this possible with declarative macros?
Example:
fn to_pascal_case(s: &str) -> String { ... }
macro_rules! test_gl_enum {
($n: ident) => { struct to_pascal_case(stringify!($n)); }
}
test_gl_enum!(VERTEX_SHADER);
expands to:
struct to_pascal_case(());
desired result would be:
struct VertexShader;
You can't execute arbitrary code in a declarative macro. You have a couple of options:
Proc Macro
Proc macros are basically just functions that take a TokenStream and return a TokenStream (with some variations between function-like, attribute and derive macros). They can contain arbitrary code, so you could absolutely parse the identifier and apply some transformation to it.
If you're going to take this route, you will likely want to use syn and quote, and I'd also very strongly recommend going through the proc macro workshop, which is a series of exercises to teach you the basics.
Or use a crate
As is often the case, someone else has also had this problem, and has written a crate for it: https://crates.io/crates/casey
With that, you can write:
use casey::*;
macro_rules! test_gl_enum {
($n:ident) => {
struct pascal!($n);
};
}
Here, pascal!() is a proc macro that does the transformation, and since it's a macro (rather than a function) it is also evaluated at compile time, so the expansion won't look like struct pascal!(my_enum_name);
Related
In C, you can define macros using #define to have something globally replaced by the C preprocessor. How can I achieve the same plain-text replacement in rust?
My goal is to define macros that simplifies Rust syntax and reduce boilerplate, such as replacing !!. with .unwrap(). to simulate the "non-null assertion operator" syntax in Kotlin, Swift, or TypeScript, and replacing (\w+)\?\? with Option<$1> to simplify optional type syntax.
For example, the following struct
struct ReturnPath {
name: Option<String>,
file_type: Option<String>,
mtime: Option<i64>
}
would be much shorter in the processed syntax:
struct ReturnPath {
name: String??,
file_type: String??,
mtime: i64??
}
How can I achieve the same plain-text replacement in rust?
You can't do this without a DSL macro, essentially.
C macros are token-replacement and happen before the syntax-parsing step -- Rust macros are more akin to Lisp macros, which happen during the parsing step. See: https://softwareengineering.stackexchange.com/a/28387
replacing (\w+)?? with Option<$1> to simplify optional type syntax.
This also won't do what you want it to do in more complicated scenarios, such as if you have Result<T, R>?? or Vec<&'static str>?? or something like that. This is because Rust's grammar is more powerful than regexes -- it is mostly context-free with some context-sensitive parts (link).
What you're probably looking for is some sort of declarative and syntactical replacement from ($t:ty) \? \? => Option<$t>.
In addition, this regex can also probably match on non-types like a?? if a: Option<Option<T>>.
If you wanted to create a special syntax for this, you'd probably do something like:
macro_rules! shorthand_struct_fields {
/* rows of idents to types and then adjust struct definition but with T?? substituted for Option<T> */
}
struct Foo {
shorthand_struct_fields! {
x: i32??,
}
}
I'm not sure if this is such a good idea, but it you do want to create this large macro, you would be best suited by writing a declarative macros with sub-macros that use # rules, like so: http://danielkeep.github.io/tlborm/book/pat-internal-rules.html
I would like to generate a variable type and initializer from a (procedural) macro. Since macros consume and generate streams (and not trees) of tokens, morally, it should be possible to do something like:
extern crate quote;
use quote::quote;
#[proc_macro]
pub fn foo(tokens: TokenStream) -> TokenStream {
TokenStream::from(quote! {
: u8 = 42
})
}
pub static FOO foo!();
But alas, this fails with
|
28 | pub static FOO foo!();
| ^^^ expected one of `:`, `;`, or `=`
Now, of course I could turn this inside out and require writing something like
foo!(FOO)
which would expand to
pub static FOO: u8 = 42;
but the reason I would like to do things piecewise is to be able to use the same macro to generate types-and-initializers in different contexts (e.g. for local let bindings, not just statics).
You can't do what you are trying to do, not because : u8 = 42 is not correctly converted to a TokenStream, but simply because macro invocations (independently of their definition) cannot occur anywhere. In particular, they can only occur where a handle of the TokenStream would be reduced to a node, which is not the case in your example. This is because Rust has to parse the file once, before it knows anything about what macros do, exactly to find out all macros. In this first step, it only knows about its own syntax. It is smart enough to say "ok, this whole node of my AST is just this macro invocation" without knowing what the macro invocation will actually do, but it can't say "this macro invocation will feed me these tokens, so the next token I should read is ;" (which would be the case in your example), because it doesn't know what the macro does yet!
To convince yourself, image that the macro foo!() produces the token mod A {, and bar!() the token }. Then, the following example auto-destroys itself
foo!()
#[proc_macro]
fn foo(_: TokenStream) -> TokenStream { ... }
#[proc_macro]
fn bar(_: TokenStream) -> TokenStream { ... }
bar!()
Because if you expand correctly the macros, then they hide themselves by putting themselves into a module, which prevents the macros from being expanded, which makes them visible again, ... Even worse examples (where the issue occurs before visibility checks, at the parsing step) are easy to provide as soon as you allow what you are trying to do.
I want to write a macro_rules based macro that will be used to wrap a series of type aliases and struct definitions. I can match on "items" with $e:item, but this will match both aliases and structs. I would like to treat the two separately (I need to add some #[derive(...)] just on the structs). Do I have to imitate their syntax directly by matching on something like type $name:ident = $type:ty; or is there a better way? This route seems annoying for structs because of regular vs tuple like. If I also wanted to distinguish functions that would be really painful because they have a lot of syntactical variation.
I believe for that problem somewhat simple cases can be solved with macro_rules!, but that probably would be limited (you can't lookahead) and super error-prone. I only cover an example for types, I hope that would be convincing enough to avoid macro_rules!. Consider this simple macro:
macro_rules! traverse_types {
($(type $tp:ident = $alias:ident;)*) => {
$(type $tp = $alias;)*
}
}
traverse_types!(
type T = i32;
type Y = Result<i32, u64>;
);
That works fine for the trivial aliases. At some point you probably also would like to handle generic type aliases (i.e. in the form type R<Y> = ...). Ok, your might still rewrite the macro to the recursive form (and that already a non-trivial task) to handle all of cases. Then you figure out that generics can be complex (type-bounds, lifetime parameters, where-clause, default types, etc):
type W<A: Send + Sync> = Option<A>;
type X<A: Iterator<Item = usize>> where A: 'static = Option<A>;
type Y<'a, X, Y: for<'t> Trait<'t>> = Result<&'a X, Y>;
type Z<A, B = u64> = Result<A, B>;
Probably all of these cases still can be handled with a barely readable macro_rules!. Nevertheless it would be really hard to understand (even to the person who wrote it) what's going on. Besides, it is hard to support new syntax (e.g. impl-trait alias type T = impl K), you may even need to have a complete rewrite of the macro. And I only cover the type aliases part, there's more to handle for the structs.
My point is: one better avoid macro_rules! for that (and similar) problem(-s), procedural macros is a way much a better tool for that. It easier to read (and thus extend) and handles new syntax for free (if syn and quote crates are maintained). For the type alias this can be done as simple as:
extern crate proc_macro;
use proc_macro::TokenStream;
use syn::parse::{Parse, ParseStream};
struct TypeAliases {
aliases: Vec<syn::ItemType>,
}
impl Parse for TypeAliases {
fn parse(input: ParseStream) -> syn::Result<Self> {
let mut aliases = vec![];
while !input.is_empty() {
aliases.push(input.parse()?);
}
Ok(Self { aliases })
}
}
#[proc_macro]
pub fn traverse_types(token: TokenStream) -> TokenStream {
let input = syn::parse_macro_input!(token as TypeAliases);
// do smth with input here
// You may remove this binding by implementing a `Deref` or `quote::ToTokens` for `TypeAliases`
let aliases = input.aliases;
let gen = quote::quote! {
#(#aliases)*
};
TokenStream::from(gen)
}
For the struct parsing code is the same using ItemStruct type and also you need a lookahead to determine wether it's an type-alias or a struct, there's very similar example at syn for that.
I have created a function-like procedural macro. I call it with a declarative macro as parameter. I was hoping to find a way that the declarative macro is expanded before processing the procedural macro (or expand it inside the procedural macro). Example (code is also available at https://github.com/Jasperav/rec_macro):
Call site:
macro_rules! t {
($e:expr) => {
$e.to_string() + " I am a macro"
}
}
fn main() {
re!(t!("hi"));
}
Proc macro crate:
#[proc_macro_hack]
pub fn re(input: TokenStream) -> TokenStream {
panic!("{:#?}", input) // I can see "hi" and something like t!, but not the result of the macro expansion
}
Inside the panic message, I see see that the declarative macro (t) is inside. I just want the raw value of hi I am a macro in this case (or some syn type). In my case, the declarative macro should always expand to a &str or String.
Is it possible to expand t before invoking re, or to expand t manually inside the proc macro? I want to see the result of the macro (in this case: "hi I am a macro") inside my proc functional like macro.
Thanks.
I have the type Foo:
pub struct Foo { ... }
Now I want to create a procedural macro that creates an instance of this struct. This might involve heavy computation, file access, or other stuff only procedural macros can do, but the exact details of how to create that instance are not important here.
I defined my procedural macro like this:
#[proc_macro]
pub fn create_foo(_: TokenStream) -> TokenStream {
let foo_value: Foo = /* some complex computation */;
// TODO: return `foo_value`
}
The users of my procedural macros should be able to write this:
fn main() {
let a: Foo = create_foo!();
}
Please note that Foo could contain a lot of data, like many megabytes of Vec data.
How can I return the Foo value from my procedural macro?
While this seems like an easy request, there is actually a lot to unroll here.
Most importantly, it is crucial to understand that procedural macros only return tokens (i.e. Rust code). To put it bluntly: the Rust compiler executes your procedural macro, takes the resulting tokens and pastes them in the users code where your procedural macro invocation was. You can think of procedural macros as a pre-processing step that takes your Rust code, transforms it and spits out another .rs file. That file is then fed to the compiler.
In order to "return a value of Foo" you have to return a TokenStream that represents an expression which evaluates to Foo. For example:
#[proc_macro]
pub fn create_foo(_: TokenStream) -> TokenStream {
quote! { Foo { data: vec![1, 2, 3] } }
}
In the user's crate:
let a: Foo = create_foo!();
Which would expand to:
let a: Foo = Foo { data: vec![1, 2, 3] };
The data: vec![1, 2, 3] part could be generated dynamically by the procedural macro. If your Foo instance is very large, the code creating that instance is probably very large as well. This means that compile times might increase because the Rust compiler has to parse and check this huge expression.
So you can't return the value directly? No. You might think that you could do it with unsafe code. For example, emit a big const DATA: &[u8] = ...; and mem::transmute it to Foo, but you can't for several reasons:
The procedural macro and the user's crate might not run on the same platform (CPU, OS, ...) which all might influence how Foo is represented in memory. The same Foo instance might be represented differently in memory for your procedural macro and your user crate, so you can't transmute.
If Foo contains heap allocated structures (Vec), you can't do it anyway.
If you must generate the value in your procedural macro, then there is only one solution to get it to the user, but this is not optimal. Alternatively, maybe calculating it at runtime once isn't that bad.