Question on invoking another macro_rules in macro_rules definition - rust

Question

I'm implementing writing TLV packet to somewhat impl std::io::Write.
First I implement WriteBE<T> trait, whose write_be(&mut self, data: T) method can write data with type T to Self. (implementation details omitted)
And I'm trying to use macro_rules! to implement calculation of total packet length in compile time (because most packets have fixed length in my case). macros are as follows:
macro_rules! len_in_expr {
(
self.write_be( $data: expr $(,)? ) $(?)? $(;)*
) => {
std::mem::size_of_val(&$data)
};
(
write_be(self, $data: expr $(,)? ) $(?)? $(;)*
) => {
std::mem::size_of_val(&$data)
};
(
$other: expr
) => {
0
};
}
/// calculate total write size in block
macro_rules! tlv_len_in_block {
({
$( $e: expr );* $(;)?
}) => {
0 $(
+ ( len_in_expr!($e) )
)*
};
}
But when I calculating total length like this:
fn main() {
let y = tlv_len_in_block!({
write_be(self, 0u32,)?;
});
println!("y={}", y);
}
I get a result 0.
If I comment the $other: expr match arm, I get a compile error:
6 | macro_rules! len_in_expr {
| ------------------------ when calling this macro
...
30 | + ( len_in_expr!($e) )
| ^^ no rules expected this token in macro call
...
39 | let y = tlv_len_in_block!({
| _____________-
40 | | write_be(self, 0u32,)?;
41 | | });
| |______- in this macro invocation
What's the problem with my code? And how can I fix it?

Answer 1

Once metavariables inside macro_rules! are captured into some fragment specifier (e.g. expr), they cannot be decomposed anymore. Quoting the reference:
When forwarding a matched fragment to another macro-by-example, matchers in the second macro will see an opaque AST of the fragment type. The second macro can't use literal tokens to match the fragments in the matcher, only a fragment specifier of the same type. The ident, lifetime, and tt fragment types are an exception, and can be matched by literal tokens. The following illustrates this restriction:
macro_rules! foo {
($l:expr) => { bar!($l); }
// ERROR: ^^ no rules expected this token in macro call
}
macro_rules! bar {
(3) => {}
}
foo!(3);
The following illustrates how tokens can be directly matched after matching a tt fragment:
// compiles OK
macro_rules! foo {
($l:tt) => { bar!($l); }
}
macro_rules! bar {
(3) => {}
}
foo!(3);
Once tlv_len_in_block!() captured write_be(self, 0u32,)? inside $e, it cannot be decomposed into write_be(self, $data:expr $(,)? ) and thus, cannot be matched by the second case of the len_in_expr!()` macro, as it should have been.
There are generally two solutions to this problem:
The first is, if possible, decomposing them from the beginning. The problem is that this is not always possible. In this case, for example, I don't see a way for that to work.
The second way is much more complicated and it is using the Push-down Accumulation technique together with tt Munching.
The idea is as follows: instead of parsing the input as whole, we parse each piece one at a time. Then, recursively, we forward the parsed bits and the yet-to-parse bit to ourselves. We also should have a stop condition on an empty input.
Here is how it will look like in your example:
macro_rules! tlv_len_in_block_impl {
// Stop condition - no input left to parse.
(
parsed = [ $($parsed:tt)* ]
rest = [ ]
) => {
$($parsed)*
};
(
parsed = [ $($parsed:tt)* ]
rest = [
self.write_be( $data:expr $(,)? ) $(?)? ;
$($rest:tt)*
]
) => {
tlv_len_in_block_impl!(
parsed = [
$($parsed)*
+ std::mem::size_of_val(&$data)
]
rest = [ $($rest)* ]
)
};
(
parsed = [ $($parsed:tt)* ]
rest = [
write_be(self, $data:expr $(,)? ) $(?)? ;
$($rest:tt)*
]
) => {
tlv_len_in_block_impl!(
parsed = [
$($parsed)*
+ std::mem::size_of_val(&$data)
]
rest = [ $($rest)* ]
)
};
}
/// calculate total write size in block
macro_rules! tlv_len_in_block {
({
$($input:tt)*
}) => {
tlv_len_in_block_impl!(
parsed = [ 0 ]
rest = [ $($input)* ]
)
};
}
(Note that this is not exactly the same as your macro - mine requires a trailing semicolon, while in yours it's optional. It's possible to make it optional here, too, but it will be much more complicated.
Playground.