Basically, there are two parts to this question:
Can you pass an unknown identifier to a macro in Rust?
Can you combine strings to generate new variable names in a Rust macro?
For example, something like:
macro_rules! expand(
($x:ident) => (
let mut x_$x = 0;
)
)
Calling expand!(hi) obvious fails because hi is an unknown identifier; but can you somehow do this?
ie. The equivalent in C of something like:
#include <stdio.h>
#define FN(Name, base) \
int x1_##Name = 0 + base; \
int x2_##Name = 2 + base; \
int x3_##Name = 4 + base; \
int x4_##Name = 8 + base; \
int x5_##Name = 16 + base;
int main() {
FN(hello, 10)
printf("%d %d %d %d %d\n", x1_hello, x2_hello, x3_hello, x4_hello, x5_hello);
return 0;
}
Why you say, what a terrible idea. Why would you ever want to do that?
I'm glad you asked!
Consider this rust block:
{
let marker = 0;
let borrowed = borrow_with_block_lifetime(data, &marker);
unsafe {
perform_ffi_call(borrowed);
}
}
You now have a borrowed value with an explicitly bounded lifetime (marker) that isn't using a structure lifetime, but that we can guarantee exists for the entire scope of the ffi call; at the same time we don't run into obscure errors where a * is de-referenced unsafely inside an unsafe block and so the compiler doesn't catch it as an error, despite the error being made inside a safe block.
(see also Why are all my pointers pointing to the same place with to_c_str() in rust?)
The use a macro that can declare temporary variables for this purpose would considerably ease the troubles I have fighting with the compiler. That's why I want to do this.
Yes however this is only available as a nightly-only experimental API which may be removed.
You can pass arbitrary identifier into a macro and yes, you can concatenate identifiers into a new identifier using concat_idents!() macro:
#![feature(concat_idents)]
macro_rules! test {
($x:ident) => ({
let z = concat_idents!(hello_, $x);
z();
})
}
fn hello_world() { }
fn main() {
test!(world);
}
However, as far as I know, because concat_idents!() itself is a macro, you can't use this concatenated identifier everywhere you could use plain identifier, only in certain places like in example above, and this, in my opinion, is a HUGE drawback. Just yesterday I tried to write a macro which could remove a lot of boilerplate in my code, but eventually I was not able to do it because macros do not support arbitrary placement of concatenated identifiers.
BTW, if I understand your idea correctly, you don't really need concatenating identifiers to obtain unique names. Rust macros, contrary to the C ones, are hygienic. This means that all names of local variables introduced inside a macro won't leak to the scope where this macro is called. For example, you could assume that this code would work:
macro_rules! test {
($body:expr) => ({ let x = 10; $body })
}
fn main() {
let y = test!(x + 10);
println!("{}", y);
}
That is, we create a variable x and put an expression after its declaration. It is then natural to think that x in test!(x + 10) refers to that variable declared by the macro, and everything should be fine, but in fact this code won't compile:
main3.rs:8:19: 8:20 error: unresolved name `x`.
main3.rs:8 let y = test!(x + 10);
^
main3.rs:3:1: 5:2 note: in expansion of test!
main3.rs:8:13: 8:27 note: expansion site
error: aborting due to previous error
So if all you need is uniqueness of locals, then you can safely do nothing and use any names you want, they will be unique automatically. This is explained in macro tutorial, though I find the example there somewhat confusing.
There is also https://github.com/dtolnay/paste, which works well in cases where concat_idents is underpowered or in cases where you can't target the nightly compiler.
macro_rules! foo_macro {
( $( $name:ident ),+ ) => {
paste! {
#[test]
fn [<test_ $name>]() {
assert! false
}
}
};
}
In cases where concat_idents doesn't work (which is most cases I'd like to use it) changing the problem from concatenated identifiers to using namespaces does work.
That is, instead of the non-working code:
macro_rules! test {
($x:ident) => ({
struct concat_idents!(hello_, $x) {}
enum contact_idents!(hello_, $x) {}
})
}
The user can name the namespace, and then have preset names as shown below:
macro_rules! test {
($x:ident) => ({
mod $x {
struct HelloStruct {}
enum HelloEnum {}
}
})
}
Now you have a name based on the macro's argument. This technique is only helpful in specific cases.
You can collect your identifiers into a struct if you don't want to use nightly and external crates and your identifiers are types.
use std::fmt::Debug;
fn print_f<T: Debug>(v: &T){
println!("{:?}", v);
}
macro_rules! print_all {
($($name:ident),+) => {
struct Values{
$($name: $name),+
}
let values = Values{
$(
$name: $name::default()
),+
};
$(
print_f(&values.$name);
)+
};
}
fn main(){
print_all!(String, i32, usize);
}
This code prints
""
0
0
If you fear that Value will conflict with some type name, you can use some long UUID as part of the name:
struct Values_110cf51d7a694c808e6fe79bf1485d5b{
$($name:$name),+
}
Related
lets say I wan't to generate this with just syntactic macros:
struct Foo {
bar: [Bar; 100],
gee: [Gee; 100],
}
with just this macro call:
make_struct!(100; Bar, Gee);
The first and obvious problem is is that lowercase bar/gee which as far as I know is impossible to create on syntactic macros so I made a small utility proc macro.
use proc_macro::{Ident, Span, TokenTree, TokenStream};
/// Take identifier and return it lowercase
#[proc_macro]
pub fn lower(stream: TokenStream) -> TokenStream {
let tt = stream.into_iter().nth(0).unwrap();
if let TokenTree::Ident(ident) = tt {
dbg!(&ident);
let new_ident = ident.to_string().to_lowercase();
dbg!(&new_ident);
TokenTree::Ident(
Ident::new(&new_ident[..], Span::mixed_site())).into()
} else {
panic!("Invalid input to `lower` macro, just accepts one literal");
}
}
Then how do I use this macro inside a syntactic macro? Because this works:
macro_rules! test {
($x:ident) => {
let lower!($x) = 10;
}
}
test!(Foo); // -> let foo = 10;
But this fails:
macro_rules! test {
($x:ident) => {
struct Test { lower!($x) : [$x; 100] }
}
}
test!(Foo); // -> (); ERROR
Is this a compiler error? or I'm doing something wrong?
The problem you are encountering is that macro invocations are not allowed in arbitrary locations in the source code, but only certain contexts (like expressions and items).
There is a workaround, however, and it is already implemented by the paste library which does what your proc macro does as part of its main job of concatenating identifiers. The macro can be invoked from a larger context and process the tokens within it however it likes. With paste:
macro_rules! test {
($x:ident) => {
paste::paste! {
struct Test { [<$x:lower>] : [$x; 100] }
}
}
}
In use, the paste! macro by itself can be placed anywhere a macro invocation can, and it does nothing until it sees the token sequence [< ... >] telling it to process the tokens within those regions in particular. Thus paste bypasses the restriction on macro invocation positions.
I'm trying to achieve something like this (simplified):
macro_rules! atest {
($closure:tt) => {
let x = 5;
println!("Result is {}", $closure())
};
}
fn main() {
//let x = 50;
atest!((|| 5 + x));
}
It does not work because the argument to the atest macro is considered by the compiler before macro evaluation:
error[E0425]: cannot find value `x` in this scope
--> src/main.rs:10:20
|
10 | atest!((|| 5 + x));
| ^ not found in this scope
Is it possible to make this work? My understanding was that macros are expanded before compilation.
Peter's answer explains why what you're doing won't work. This is part of what's referred to as "macro hygiene": things declared inside macros can't "leak" into the surrounding scope.
A common workaround for the problem you're facing is to pass the name of the identifier into the macro as another argument:
macro_rules! atest {
($x:ident, $closure:tt) => {
let $x = 5;
println!("Result is {}", $closure())
};
}
fn main() {
atest!(x, (|| 5 + x));
}
This will work because naming x puts it in the caller's scope, even though the declaration is inside the macro.
You might notice that the closure is kind of unnecessary, at least in this example -- you could just pass 5 + x as an expression to the macro and have it expanded inline.
macro_rules! atest {
($x:ident, $value:expr) => {
let $x = 5;
println!("Result is {}", $value)
};
}
You call this macro like atest!(x, 5 + x), which looks a little bit like a closure of its own. That might give you the idea of writing atest!(|x| 5 + x) instead. And that will also work, with a variable scoped to the closure:
macro_rules! atest {
($closure:expr) => {
let x = 5;
println!("Result is {}", $closure(x))
};
}
References
Macros in Rust pt1
Is it possible to make this work? My understanding was that macros are expanded before compilation?
Macros are expanded before compilation, but not before parsing. The raw input code has already been parsed and macros operate on an abstract syntax tree, not on text. For example, the closure is already understood to be a closure, and its free variables are already bound to variables in its lexical scope.
This is different to some other languages macros, for example C/C++, which operate on raw text, and let you screw things up pretty badly if you're not careful.
macro_rules! log {
($($x:expr),*) => {
{
$(
//how to detect $x in the Macro repetition?
)*
}
};
}
I don't want to use ($($x:ident),*). The log! macro can log the expression and then I want to match the variable type.
Macros only can access token streams, and cannot say anything more about those tokens, apart from what is explicitly given to the macro.
For example, consider this code:
fn main() {
let mut foo: i32 = 1;
my_macro!(foo);
}
The only information available to the macro my_macro is the input token foo. It can match that foo is is an identifier but it can't say anything more about it. The fact that there is mutable binding called foo, or that this binding is an i32 are just not available inside the macro.
I'm trying to achieve something like this (simplified):
macro_rules! atest {
($closure:tt) => {
let x = 5;
println!("Result is {}", $closure())
};
}
fn main() {
//let x = 50;
atest!((|| 5 + x));
}
It does not work because the argument to the atest macro is considered by the compiler before macro evaluation:
error[E0425]: cannot find value `x` in this scope
--> src/main.rs:10:20
|
10 | atest!((|| 5 + x));
| ^ not found in this scope
Is it possible to make this work? My understanding was that macros are expanded before compilation.
Peter's answer explains why what you're doing won't work. This is part of what's referred to as "macro hygiene": things declared inside macros can't "leak" into the surrounding scope.
A common workaround for the problem you're facing is to pass the name of the identifier into the macro as another argument:
macro_rules! atest {
($x:ident, $closure:tt) => {
let $x = 5;
println!("Result is {}", $closure())
};
}
fn main() {
atest!(x, (|| 5 + x));
}
This will work because naming x puts it in the caller's scope, even though the declaration is inside the macro.
You might notice that the closure is kind of unnecessary, at least in this example -- you could just pass 5 + x as an expression to the macro and have it expanded inline.
macro_rules! atest {
($x:ident, $value:expr) => {
let $x = 5;
println!("Result is {}", $value)
};
}
You call this macro like atest!(x, 5 + x), which looks a little bit like a closure of its own. That might give you the idea of writing atest!(|x| 5 + x) instead. And that will also work, with a variable scoped to the closure:
macro_rules! atest {
($closure:expr) => {
let x = 5;
println!("Result is {}", $closure(x))
};
}
References
Macros in Rust pt1
Is it possible to make this work? My understanding was that macros are expanded before compilation?
Macros are expanded before compilation, but not before parsing. The raw input code has already been parsed and macros operate on an abstract syntax tree, not on text. For example, the closure is already understood to be a closure, and its free variables are already bound to variables in its lexical scope.
This is different to some other languages macros, for example C/C++, which operate on raw text, and let you screw things up pretty badly if you're not careful.
This works, but test_macro accepts only one argument:
macro_rules! test_define (
($name:ident) => (
macro_rules! $name (
( $x:expr ) => (
// something
)
);
)
);
test_define!(test_macro);
If I try and do this:
macro_rules! test_define2 (
($name:ident) => (
macro_rules! $name (
( $($x:expr),* ) => (
// something
)
);
)
);
test_define2!(test_macro2);
Compilation fails with:
error: attempted to repeat an expression containing no syntax variables matched as repeating at this depth
--> src/main.rs:4:16
|
4 | ( $($x:expr),* ) => (
| ^^^^^^^^^
It is a known bug that nested macros don't allow repetitions in binding patterns (issue #35853).
Unfortunately, there is no workaround. The only solution is to change your API to not rely on repetition inside nested macros.
While it is not possible to do this directly, as kennytm described in this answer, you could do this using procedural macros(in case you really need to, otherwise I would not recommend it).
While it's probably a lot simpler when using nightly, it is also possible in stable.
Limitations
Please tell me in case I missed anything or in case there might be problems when using custom derive like this in the future.
You need to create a struct, which causes one of the following problems
there is either a maximum of 1 macro call per module
the user is required to add another string of characters when calling the outer macro
you create a private module which contains the required struct and could be accessed
So let's just copy the example from here, and just look at this part of the code:
fn impl_hello_world(ast: &syn::DeriveInput) -> quote::Tokens {
let name = &ast.ident;
quote! {
impl HelloWorld for #name {
fn hello_world() {
println!("Hello, World! My name is {}", stringify!(#name));
}
}
}
}
inside of the quote! macro, you are not limited to the implementation of the struct. you could change this to
quote! {
macro_rules! #name {
($($expr:expr),*) => {
// something
}
}
}
now you have a macro with the same name as a struct which takes an infinite amount of arguments.
To do this in another macro, the outer macro just has to look like this:
macro_rules! test_define {
($name:ident) => {
#[allow(non_camel_case_types)] // because macro names are lower case
#[allow(dead_code)] // because this struct should never be used
#[derive(HelloWorld)]
struct $name { }
}
};
And now you can call test_define and then the inner macro:
test_define!(foo);
fn main() {
foo!()
}
However, there is still one problem: people could accidentally access your struct. So there are ways to circumvent this(each solution is a directly linked the problem with the same number):
name the struct in a way which prevents accidental access by pure chance:
macro_rules! test_define {
($name:ident) => {
#[allow(dead_code)]
#[derive(HelloWorld)]
struct Dgfggsdfgujksdsdsdfsdfsdg {
$name: u8,
}
}
};
You have to change your proc macro to use the struct field instead of name inside of quote! and in case the test_define! is called more than once in the same crate you have 2 structs with identical names which causes a compile time error.
to prevent two identical struct names you could also change test_define! to take an additional argument:
macro_rules! test_define {
($name:ident, $rep_guard:ident) => {
#[allow(non_camel_case_types)]
#[allow(dead_code]
#[derive(HelloWorld)]
struct $rep_guard {
$name: u8,
}
}
};
You are using a struct field instead of the name with this method. To use you now have to write test_define!(foo,fvgfdgdfgdfgd), which is really awkward, so I would not recommend this.
This is probably the best option, now you can keep your strange struct name from solution 1 and just put the whole thing in a module. Meaning that no one can accidentally access the created struct and you can have an infinite amount of calls to test_define!.
macro_rules! test_define {
($name:ident) => {
#[macro_use] // to use the macro in the current module
mod $name {
#[allow(dead_code)]
#[derive(HelloWorld)]
struct Dgfggsdfgujksdsdsdfsdfsdg {
$name: u8,
}
}
}
};
The compiler should simply remove all of those structs, as they are dead_code
(at least when building with the --release flag). You could adapt the quote! by adding #[macro_export] if you need it.
Another advantage is that proc macros use your source code in the same way as a String or as Tokens which can be cast to a String, this means that you could create multiple macros, for example:
test_derive!(foo) => foo!(), foo_with_var!(75)
In case there is anything you don't understand, just ask.
You can define arbitrary meta macros if you get the parent macro to write the outer $ signs. A way to do it is to pass a $ in your parent macro and store it in a variable so you can use it later.
For you can define test_define2 as follow:
macro_rules! test_define2 (
(($dollar:tt) $name:ident) => (
macro_rules! $name (
( $dollar($x:expr),* ) => (
// something
)
);
);
);
Calling it like this:
test_define2(($) test_macro2);
will expand to:
macro_rules! test_macro2 (
( $($x:expr),* ) => (
($($x),*) // or something
)
);
Which works as expected.
To avoid having to add the extra ($) when calling meta macros you can write a second macro that call the meta macro with the added symbol or if it is possible, add this final match arm:
macro_rules! test_define2 (
(($d:tt) $name:ident) => (
macro_rules! $name (
( $d($x:expr),* ) => (
($d($x),*) // or something
)
);
);
($($t:tt)+) => (
// parenthesis or curly braces are necessary to properly "escape" the $
test_define2!(($) $($t)+);
);
);
// test_define2!(test_macro2);
// expands to:
// test_define2!(($)test_macro2);
// expands to:
// macro_rules!test_macro2(
// ($($x:expr),*) => (
// ($($x),*) // or something
// )
// );