lets say I wan't to generate this with just syntactic macros:
struct Foo {
bar: [Bar; 100],
gee: [Gee; 100],
}
with just this macro call:
make_struct!(100; Bar, Gee);
The first and obvious problem is is that lowercase bar/gee which as far as I know is impossible to create on syntactic macros so I made a small utility proc macro.
use proc_macro::{Ident, Span, TokenTree, TokenStream};
/// Take identifier and return it lowercase
#[proc_macro]
pub fn lower(stream: TokenStream) -> TokenStream {
let tt = stream.into_iter().nth(0).unwrap();
if let TokenTree::Ident(ident) = tt {
dbg!(&ident);
let new_ident = ident.to_string().to_lowercase();
dbg!(&new_ident);
TokenTree::Ident(
Ident::new(&new_ident[..], Span::mixed_site())).into()
} else {
panic!("Invalid input to `lower` macro, just accepts one literal");
}
}
Then how do I use this macro inside a syntactic macro? Because this works:
macro_rules! test {
($x:ident) => {
let lower!($x) = 10;
}
}
test!(Foo); // -> let foo = 10;
But this fails:
macro_rules! test {
($x:ident) => {
struct Test { lower!($x) : [$x; 100] }
}
}
test!(Foo); // -> (); ERROR
Is this a compiler error? or I'm doing something wrong?
The problem you are encountering is that macro invocations are not allowed in arbitrary locations in the source code, but only certain contexts (like expressions and items).
There is a workaround, however, and it is already implemented by the paste library which does what your proc macro does as part of its main job of concatenating identifiers. The macro can be invoked from a larger context and process the tokens within it however it likes. With paste:
macro_rules! test {
($x:ident) => {
paste::paste! {
struct Test { [<$x:lower>] : [$x; 100] }
}
}
}
In use, the paste! macro by itself can be placed anywhere a macro invocation can, and it does nothing until it sees the token sequence [< ... >] telling it to process the tokens within those regions in particular. Thus paste bypasses the restriction on macro invocation positions.
Related
As the title says, I want to convert a string literal into code. So, naturally, I come up with macro:
// main.rs
use quote::quote;
use syn::ExprClosure;
// declare a macro to expand the entire input TokenStream as a token tree.
macro_rules! foo {
($t:tt) => {
$t
};
}
fn main() {
let code = "|x:&i32|x+1";
// parse string literal to syn::ExprClosure type.
let expr = syn::parse_str::<ExprClosure>(code).unwrap();
// return a value has a TokenStream type.
let expr_ts = quote! {
let slice = vec![1,2,3];
let slice2: Vec<i32> = slice.iter()
.map(#expr)
.collect();
println!("{:#?}",slice);
};
// call foo! to expand `expr_ts`
foo!(expr_ts);
}
which can be compiled perfectly, printing, however, nothing at all.
As far as I could understand, the reason why it prints nothing is in that $t in foo! is possibly not expanded as what I want it to. So I turned to this to try figure out if the type of $t went wrong:
macro_rules! foo {
($t:tt) => {
println!("{:#?}", $t);
};
}
which successfully printed the syntax tree of $t(which has a syn::ExprClosure type).
Update:
After doing some research on macro, I found that the problem was likely a misuse (or a misunderstanding) of different macro types: procedural macro vs. declarative macro.
So I refactor the code, which turns out to be a success:
// another_crate::lib
#[proc_macro]
pub fn foo(item : TokenStream) -> TokenStream {
let code = "|x:&i32|x+1";
let expr = syn::parse_str::<ExprClosure>(code).unwrap();
let expr_ts = quote! {
let slice = vec![1,2,3];
let slice2: Vec<i32> = slice.iter()
.map(#expr)
.collect();
println!("{:?}",slice2);
};
TokenStream::from(expr_ts)
}
// main.rs
use another_crate::foo;
fn main() {
foo!();
}
// output
// [2, 3, 4]
The quote! macro evaluates to an expression of type proc_macro2::TokenStream. Meanwhile Rust procedural macros are expected to return the type proc_macro::TokenStream.
The difference between the two types is that proc_macro types are entirely specific to procedural macros and cannot ever exist in code outside of a procedural macro, while proc_macro2 types may exist anywhere including tests and non-macro code like main.rs and build.rs.
As noted above (quote's doc), we can't use proc_macro::TokenStream in main.rs，which was exactly what I did in the first place: I used quote! in main.rs which returns a proc_macro2::TokenStream and expected that it would be parsed and expanded by compiler, which is not possible because the compiler accepts TokenStream input only if it is of proc_macro from another_crate(which is a procedural macro lib),not of proc_macro2 from main.rs.
Now the question here is that if I can implement the convertion just through declarative macro? It seems that procedural macros don't accept any variable as arguments, which means you can only hardcode string literal.
Really appreciate any replies regarding my concern.
Thanks.
macro_rules! log {
($($x:expr),*) => {
{
$(
//how to detect $x in the Macro repetition?
)*
}
};
}
I don't want to use ($($x:ident),*). The log! macro can log the expression and then I want to match the variable type.
Macros only can access token streams, and cannot say anything more about those tokens, apart from what is explicitly given to the macro.
For example, consider this code:
fn main() {
let mut foo: i32 = 1;
my_macro!(foo);
}
The only information available to the macro my_macro is the input token foo. It can match that foo is is an identifier but it can't say anything more about it. The fact that there is mutable binding called foo, or that this binding is an i32 are just not available inside the macro.
Using match (like in bar) seems to be a common approach..
#[derive(Debug)]
pub enum MyErrors {
SomeError,
}
fn foo(x: Option<u64>) -> Result<u64, MyErrors> {
if x.is_none() {
return Err(MyErrors::SomeError);
}
// .. some long code where more options
// are checked and matched
// The ok here is just so the code is simple and compiles
Ok(x.unwrap() * 2)
}
fn bar(x: Option<u64>) -> Result<u64, MyErrors> {
match x {
None => {
return Err(MyErrors::SomeError)?;
}
Some(v) => {
// .. some long code where more options
// are checked and matched
// The ok here is just so the code is simple and compiles
Ok(x.unwrap() * 2)
}
}
}
fn main() {
foo(Some(1));
bar(Some(2));
}
However, early returns (such as in foo) significantly reduce how nested the code looks like. If there are multiple times when an option has to be unwrapped or an error returned, code like bar gets very nested...
What is the recommended practice for early returning an error in the case of empty options?
If a longer method chain is undesirable due to complex logic inside, there are still a few readable, low-indent options.
ok_or and ?
We can convert an Option to a Result with a desired error, and immediately unwrap it with the ? operator. This solution probably provides the least indent possible, and can be easily used to "unwrap" multiple Options.
fn bar1(x: Option<u64>) -> Result<u64, MyErrors> {
let x = x.ok_or(MyErrors::SomeError)?;
// A lot of stuff going on.
Ok(x * 2)
}
This will evaluate the error inside ok_or regardless of whether or not it will actually be used. If this computation is expensive, ok_or_else, which produces the error lazily, will be more efficient (related question).
if let
This solution can still lead to a staircase of code if nested, but may be more appropriate if the else branch logic is more involved.
fn bar2(x: Option<u64>) -> Result<u64, MyErrors> {
if let Some(x) = x {
// Lot of stuff here as well.
Ok(x * 2)
} else {
Err(MyErrors::SomeError)
}
}
Basically, there are two parts to this question:
Can you pass an unknown identifier to a macro in Rust?
Can you combine strings to generate new variable names in a Rust macro?
For example, something like:
macro_rules! expand(
($x:ident) => (
let mut x_$x = 0;
)
)
Calling expand!(hi) obvious fails because hi is an unknown identifier; but can you somehow do this?
ie. The equivalent in C of something like:
#include <stdio.h>
#define FN(Name, base) \
int x1_##Name = 0 + base; \
int x2_##Name = 2 + base; \
int x3_##Name = 4 + base; \
int x4_##Name = 8 + base; \
int x5_##Name = 16 + base;
int main() {
FN(hello, 10)
printf("%d %d %d %d %d\n", x1_hello, x2_hello, x3_hello, x4_hello, x5_hello);
return 0;
}
Why you say, what a terrible idea. Why would you ever want to do that?
I'm glad you asked!
Consider this rust block:
{
let marker = 0;
let borrowed = borrow_with_block_lifetime(data, &marker);
unsafe {
perform_ffi_call(borrowed);
}
}
You now have a borrowed value with an explicitly bounded lifetime (marker) that isn't using a structure lifetime, but that we can guarantee exists for the entire scope of the ffi call; at the same time we don't run into obscure errors where a * is de-referenced unsafely inside an unsafe block and so the compiler doesn't catch it as an error, despite the error being made inside a safe block.
(see also Why are all my pointers pointing to the same place with to_c_str() in rust?)
The use a macro that can declare temporary variables for this purpose would considerably ease the troubles I have fighting with the compiler. That's why I want to do this.
Yes however this is only available as a nightly-only experimental API which may be removed.
You can pass arbitrary identifier into a macro and yes, you can concatenate identifiers into a new identifier using concat_idents!() macro:
#![feature(concat_idents)]
macro_rules! test {
($x:ident) => ({
let z = concat_idents!(hello_, $x);
z();
})
}
fn hello_world() { }
fn main() {
test!(world);
}
However, as far as I know, because concat_idents!() itself is a macro, you can't use this concatenated identifier everywhere you could use plain identifier, only in certain places like in example above, and this, in my opinion, is a HUGE drawback. Just yesterday I tried to write a macro which could remove a lot of boilerplate in my code, but eventually I was not able to do it because macros do not support arbitrary placement of concatenated identifiers.
BTW, if I understand your idea correctly, you don't really need concatenating identifiers to obtain unique names. Rust macros, contrary to the C ones, are hygienic. This means that all names of local variables introduced inside a macro won't leak to the scope where this macro is called. For example, you could assume that this code would work:
macro_rules! test {
($body:expr) => ({ let x = 10; $body })
}
fn main() {
let y = test!(x + 10);
println!("{}", y);
}
That is, we create a variable x and put an expression after its declaration. It is then natural to think that x in test!(x + 10) refers to that variable declared by the macro, and everything should be fine, but in fact this code won't compile:
main3.rs:8:19: 8:20 error: unresolved name `x`.
main3.rs:8 let y = test!(x + 10);
^
main3.rs:3:1: 5:2 note: in expansion of test!
main3.rs:8:13: 8:27 note: expansion site
error: aborting due to previous error
So if all you need is uniqueness of locals, then you can safely do nothing and use any names you want, they will be unique automatically. This is explained in macro tutorial, though I find the example there somewhat confusing.
There is also https://github.com/dtolnay/paste, which works well in cases where concat_idents is underpowered or in cases where you can't target the nightly compiler.
macro_rules! foo_macro {
( $( $name:ident ),+ ) => {
paste! {
#[test]
fn [<test_ $name>]() {
assert! false
}
}
};
}
In cases where concat_idents doesn't work (which is most cases I'd like to use it) changing the problem from concatenated identifiers to using namespaces does work.
That is, instead of the non-working code:
macro_rules! test {
($x:ident) => ({
struct concat_idents!(hello_, $x) {}
enum contact_idents!(hello_, $x) {}
})
}
The user can name the namespace, and then have preset names as shown below:
macro_rules! test {
($x:ident) => ({
mod $x {
struct HelloStruct {}
enum HelloEnum {}
}
})
}
Now you have a name based on the macro's argument. This technique is only helpful in specific cases.
You can collect your identifiers into a struct if you don't want to use nightly and external crates and your identifiers are types.
use std::fmt::Debug;
fn print_f<T: Debug>(v: &T){
println!("{:?}", v);
}
macro_rules! print_all {
($($name:ident),+) => {
struct Values{
$($name: $name),+
}
let values = Values{
$(
$name: $name::default()
),+
};
$(
print_f(&values.$name);
)+
};
}
fn main(){
print_all!(String, i32, usize);
}
This code prints
""
0
0
If you fear that Value will conflict with some type name, you can use some long UUID as part of the name:
struct Values_110cf51d7a694c808e6fe79bf1485d5b{
$($name:$name),+
}
I'm trying to create a macro that generates a struct that provides a set of methods that are passed into the macro. For example, calling:
create_impl!(StructName, fn foo() -> u32 { return 432 })
should generate an empty struct StructName that provides the method foo().
My initial attempt at this uses the item macro arg type. However, when I try and use an item in the rule, I get the following compiler error:
error: expected one of `const`, `default`, `extern`, `fn`, `pub`, `type`, `unsafe`, or `}`, found `fn foo() -> u32 { return 42; }`
--> src/lib.rs:40:13
|
40 | $($function)*
| ^^^^^^^^^
Is it possible to use item arguments to define methods in generated structs this way? Is there something I'm missing?
Here's the full macro I've defined:
macro_rules! create_impl {
($struct_name:ident, $($function:item),*) => {
struct $struct_name {
}
impl $struct_name {
// This is the part that fails.
$($function)*
}
};
}
The short answer is "no, you can't use the item matcher for a method".
According to the reference, items are the top level things in a crate or module, so functions, types, and so on. While a struct or impl block is an item, the things inside them aren't. Even though syntactically, a method definition looks identical to a top level function, that doesn't make it an item.
The way Rust's macro system works is that once a fragment has been parsed as an item, e.g. using $foo:item, it's then forever an item; it's split back into tokens for reparsing once the macro is expanded.
The result of this is that $foo:item can only be in the macro's output in item position, which generally means top-level.
There are a couple of alternatives.
The simplest is to use the good old tt (token tree) matcher. A token tree is either a non-bracket token or a sequence of tokens surrounded by balanced brackets; so $(foo:tt)* matches anything. However, that means it will gobble up commas too, so it's easier to just add braces around each item:
macro_rules! create_impl {
($struct_name:ident, $({ $($function:tt)* }),*) => {
struct $struct_name {
}
impl $struct_name {
$($($function)*)*
}
};
}
Then you have to use it with the extra braces:
create_impl!(StructName, { fn foo() -> u32 { return 432 } }, { fn bar() -> u32 { return 765 } });
You can also just match the syntax you want directly, rather than delegating to the item matcher:
macro_rules! create_impl2 {
($struct_name:ident, $(fn $fname:ident($($arg:tt)*) -> $t:ty $body:block),*) => {
struct $struct_name {
}
impl $struct_name {
$(fn $fname($($arg)*) -> $t $body)*
}
}
}
Of course since it's explicit, that means that if you want to support functions without a return type, you need to add another case to your macro.