I have the next function, that parses a body of a proc-macro.
pub fn user_body_builder(func: syn::ItemFn, macro_tokens: &mut Vec<TokenStream>) -> Result<(), TokenStream> {
// Gets a Vec<Stmt> with all the staments in the body of the fn
let function_statements = func.clone().block.stmts;
for stmt in function_statements {
let quote = quote! {#stmt};
let quoterino = quote
.to_string()
.parse();
if quoterino.is_err() {
return Err(
if quoterino.is_err() {
return Err(
syn::Error::new(stmt.span(), "Macro error")
.to_compile_error()
)
} else {
macro_tokens.push(quoterino.unwrap());
}
)
} else {
macro_tokens.push(quoterino.unwrap());
}
}
Ok(())
}
When I introduce a syntax error on purpose on main, sometimes reports the error correctly, but sometimes not.
And even when the report of the error it's done, I always face this:
error[E0601]: main function not found in crate 'some_crate'
What w'd be the correct way of handling and report code errors when a macro wraps a function body when faced?
EDIT 1:
Changed quote_spanned! for syn::Error::new. Error it's the same.
EDIT 2:
I've already found why when there's an error on main the compiler does not reach my function to handling the errors.
#[proc_macro_attribute]
pub fn macro(_meta: CompilerTokenStream, input: CompilerTokenStream) -> CompilerTokenStream {
// Get the function that this attribute is attached to
let func = parse_macro_input!(input as ItemFn);
The parse_macro_input!(...) macro throws the errors on main, so the compiler never reaches the code that handles the errors detected on the stmts.
Related
I'm trying to parse a file with syn, and add a line to the single function in it. However, it seems to not modify the file at all when writing it back out. I'm fairly sure that I don't understand fully proc-macro and am using it wrong.
In my Cargo.toml I define a lib and bin like so:
[lib]
name = "gen"
path = "src/gen.rs"
proc-macro = true
[[bin]]
name = "main"
path = "src/main.rs"
In my gen.rs file, I define a macro to take in the input, get the function and modify it like so:
use proc_macro::TokenStream;
use quote::quote;
#[proc_macro]
pub fn gen(input: TokenStream) -> TokenStream {
let item = syn::parse(input.clone());
match item {
Ok(mut v) => {
let fn_item = match &mut v {
syn::Item::Fn(fn_item) => fn_item,
_ => panic!("expected fn"),
};
fn_item.block.stmts.insert(
0,
syn::parse(quote!(println!("count me in");).into()).unwrap(),
);
use quote::ToTokens;
return v.into_token_stream().into();
}
Err(error) => {
println!("{:?}", error);
return input;
}
};
}
Now in my main.rs file, I read the file, convert it to a TokenStream, and use my macro on it and write out the output to a file:
fn main() {
if let Err(error) = try_main() {
let _ = writeln!(io::stderr(), "{}", error);
process::exit(1);
}
}
fn try_main() -> Result<(), Error> {
let mut args = env::args_os();
let _ = args.next(); // executable name
let filepath = PathBuf::from("./src/file-to-parse.rs");
let code = fs::read_to_string(&filepath).map_err(Error::ReadFile)?;
let syntax = syn::parse_file(&code).map_err({
|error| Error::ParseFile {
error,
filepath,
source_code: code,
}
})?;
let mut token_stream = TokenStream::new();
syntax.to_tokens(&mut token_stream);
let file_contents_updated = gen::gen!(&token_stream);
std::fs::write("./src/file-updated.rs", file_contents_updated.to_string());
Ok(())
}
Running this, my output file looks the same as the input. For reference, my input file looks like:
fn init() {
println!("Hello, world!");
}
Yes, you've been misunderstood what proc macros do.
gen::gen!(&token_stream) will invoke gen!() at compile time with the literal tokens & token_stream. Since that doesn't look very much like a function, syn will fail to parse this, and your code will println!("{:?}", error); return input; (which by the way, is a bad idea for proc macro: parsing failure should abort compilation. Use return err.into_compile_error().into()). So it will return its input, meaning the output will be the same as the input.
You can use syn and quote for general purpose code generation, but you should not use proc macros for that - rather, use them as libraries. That is, gen::gen(token_stream) instead of gen::gen!(&token_stream). You can also not mark it proc_macro and put it in the same crate.
I'm implementing a simple attribute proc macro that's supposed to throw a warning if an invalid option is provided as an argument to it. Assume the following code:
#[proc_macro_error::proc_macro_error]
#[proc_macro_attribute]
pub fn my_attribute_with_warning(
args: proc_macro::TokenStream,
in_func: proc_macro::TokenStream,
) -> proc_macro::TokenStream {
let mut in_func_parsed = syn::parse_macro_input!(in_func as syn::ItemFn);
let func_name = in_func_parsed.sig.ident.to_string();
let args_parsed = syn::punctuated::Punctuated::<syn::Path, syn::Token![,]>::parse_terminated
.parse(args)
.unwrap();
for a in args_parsed {
let arg_string = a.into_token_stream().to_string();
let arg_valid = check_validity(&arg_string);
if !arg_valid {
proc_macro_error::emit_warning!(
proc_macro::Span::mixed_site(),
format!(
"Invalid option '{}' provided for my_attribute_with_warning on {}",
arg_string, func_name,
)
);
}
}
proc_macro::TokenStream::from(in_func_parsed.into_token_stream())
}
What I would expect to happen is that a warning is emitted at compile time when an argument is provided that results in check_validity returning false. This is however not the case. Am I wrong in that assumption? Is there a mistake in the above code that can be fixed to achieve the desired behaviour?
Note that if proc_macro_error::emit_error is used an error is generated at compile time as expected.
I have requirement to get the source location of the caller of every method. I am trying to create a proc_macro_attribute to capture the location and print it.
#[proc_macro_attribute]
pub fn get_location(attr: TokenStream, item: TokenStream) -> TokenStream {
// Get and print file!(), line!() of source
// Should print line no. 11
item
}
#[get_location]
fn add(x: u32, y: u32) -> u32 {
x + y
}
fn main() {
add(1, 5); // Line No. 11
}
TL;DR
Here is a procedural macro that uses syn and quote to do what you've described:
// print_caller_location/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
use syn::spanned::Spanned;
// Create a procedural attribute macro
//
// Notably, this must be placed alone in its own crate
#[proc_macro_attribute]
pub fn print_caller_location(_attr: TokenStream, item: TokenStream) -> TokenStream {
// Parse the passed item as a function
let func = syn::parse_macro_input!(item as syn::ItemFn);
// Break the function down into its parts
let syn::ItemFn {
attrs,
vis,
sig,
block,
} = func;
// Ensure that it isn't an `async fn`
if let Some(async_token) = sig.asyncness {
// Error out if so
let error = syn::Error::new(
async_token.span(),
"async functions do not support caller tracking functionality
help: consider returning `impl Future` instead",
);
return TokenStream::from(error.to_compile_error());
}
// Wrap body in a closure only if function doesn't already have #[track_caller]
let block = if attrs.iter().any(|attr| attr.path.is_ident("track_caller")) {
quote! { #block }
} else {
quote! {
(move || #block)()
}
};
// Extract function name for prettier output
let name = format!("{}", sig.ident);
// Generate the output, adding `#[track_caller]` as well as a `println!`
let output = quote! {
#[track_caller]
#(#attrs)*
#vis #sig {
println!(
"entering `fn {}`: called from `{}`",
#name,
::core::panic::Location::caller()
);
#block
}
};
// Convert the output from a `proc_macro2::TokenStream` to a `proc_macro::TokenStream`
TokenStream::from(output)
}
Make sure to put it in its on crate and add these lines to its Cargo.toml:
# print_caller_location/Cargo.toml
[lib]
proc-macro = true
[dependencies]
syn = {version = "1.0.16", features = ["full"]}
quote = "1.0.3"
proc-macro2 = "1.0.9"
In-depth explanation
A macro can only expand to code that's possible to write by hand to begin with. Knowing this, I see two questions here:
How can I write a function that tracks the location of its caller?
See How can I access a function's calling location each time it's called?
Short answer: to obtain the location in which your function gets called, mark it with #[track_caller] and use std::panic::Location::caller in its body.
How can I write a procedural macro that creates such functions?
Initial attempt
We want a procedural macro that
takes a function,
marks it #[track_caller],
and adds a line that prints Location::caller.
For example, it would transform a function like this:
fn foo() {
// body of foo
}
into
#[track_caller]
fn foo() {
println!("{}", std::panic::Location::caller());
// body of foo
}
Below, I present a procedural macro that executes that transformation exactly — although, as you'll see in later versions, you probably want something different. To try this code, like before in the TL;DR section, put it into its own crate and add its dependencies to the Cargo.toml.
// print_caller_location/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
// Create a procedural attribute macro
//
// Notably, this must be placed alone in its own crate
#[proc_macro_attribute]
pub fn print_caller_location(_attr: TokenStream, item: TokenStream) -> TokenStream {
// Parse the passed item as a function
let func = syn::parse_macro_input!(item as syn::ItemFn);
// Break the function down into its parts
let syn::ItemFn {
attrs,
vis,
sig,
block,
} = func;
// Extract function name for prettier output
let name = format!("{}", sig.ident);
// Generate the output, adding `#[track_caller]` as well as a `println!`
let output = quote! {
#[track_caller]
#(#attrs)*
#vis #sig {
println!(
"entering `fn {}`: called from `{}`",
#name,
::core::panic::Location::caller()
);
#block
}
};
// Convert the output from a `proc_macro2::TokenStream` to a `proc_macro::TokenStream`
TokenStream::from(output)
}
Example usage:
// example1/src/main.rs
#![feature(track_caller)]
#[print_caller_location::print_caller_location]
fn add(x: u32, y: u32) -> u32 {
x + y
}
fn main() {
add(1, 5); // entering `fn add`: called from `example1/src/main.rs:11:5`
add(1, 5); // entering `fn add`: called from `example1/src/main.rs:12:5`
}
Unfortunately, we won't be able to get away with that simple version. There are at least two problems with that version:
How it composes with async fns:
Instead of printing the caller location, it prints the location in which our macro (#[print_caller_location]) is invoked. For example:
// example2/src/main.rs
#![feature(track_caller)]
#[print_caller_location::print_caller_location]
async fn foo() {}
fn main() {
let future = foo();
// ^ oops! prints nothing
futures::executor::block_on(future);
// ^ oops! prints "entering `fn foo`: called from `example2/src/main.rs:5:1`"
let future = foo();
// ^ oops! prints nothing
futures::executor::block_on(future);
// ^ oops! prints "entering `fn foo`: called from `example2/src/main.rs:5:1`"
}
How it works with other invocations of itself, or generally, of #[track_caller]:
Nested functions with #[print_caller_location] will print the location of the root caller, rather than the direct caller of a given function. For example:
// example3/src/main.rs
#![feature(track_caller)]
#[print_caller_location::print_caller_location]
fn add(x: u32, y: u32) -> u32 {
x + y
}
#[print_caller_location::print_caller_location]
fn add_outer(x: u32, y: u32) -> u32 {
add(x, y)
// ^ we would expect "entering `fn add`: called from `example3/src/main.rs:12:5`"
}
fn main() {
add(1, 5);
// ^ "entering `fn add`: called from `example3/src/main.rs:17:5`"
add(1, 5);
// ^ "entering `fn add`: called from `example3/src/main.rs:19:5`"
add_outer(1, 5);
// ^ "entering `fn add_outer`: called from `example3/src/main.rs:21:5`"
// ^ oops! "entering `fn add`: called from `example3/src/main.rs:21:5`"
//
// In reality, `add` was called on line 12, from within the body of `add_outer`
add_outer(1, 5);
// ^ "entering `fn add_outer`: called from `example3/src/main.rs:26:5`"
// oops! ^ entering `fn add`: called from `example3/src/main.rs:26:5`
//
// In reality, `add` was called on line 12, from within the body of `add_outer`
}
Addressing async fns
It is possible to work around the problem with async fns using -> impl Future, for example, if we wanted our async fn counter-example to work correctly, we could instead write:
// example4/src/main.rs
#![feature(track_caller)]
use std::future::Future;
#[print_caller_location::print_caller_location]
fn foo() -> impl Future<Output = ()> {
async move {
// body of foo
}
}
fn main() {
let future = foo();
// ^ prints "entering `fn foo`: called from `example4/src/main.rs:15:18`"
futures::executor::block_on(future);
// ^ prints nothing
let future = foo();
// ^ prints "entering `fn foo`: called from `example4/src/main.rs:19:18`"
futures::executor::block_on(future);
// ^ prints nothing
}
We could add a special case that applies this transformation to our macro. However, that transformation changes the public API of the function from async fn foo() to fn foo() -> impl Future<Output = ()> in addition to affecting the auto traits that the returned future can have.
Therefore I recommend that we allow users to use that workaround if they desire, and simply emit an error if our macro is used on an async fn. We can do this by adding these lines to our macro code:
// Ensure that it isn't an `async fn`
if let Some(async_token) = sig.asyncness {
// Error out if so
let error = syn::Error::new(
async_token.span(),
"async functions do not support caller tracking functionality
help: consider returning `impl Future` instead",
);
return TokenStream::from(error.to_compile_error());
}
Fixing nested behavior of #[print_caller_location] functions
The problematic behaviour minimizes down to this fact: When a #[track_caller] function, foo, directly calls into another #[track_caller] function, bar, Location::caller will give both of them access to foo's caller. In other words, Location::caller gives access to the root caller in the case of nested #[track_caller] functions:
#![feature(track_caller)]
fn main() {
foo(); // prints `src/main.rs:4:5` instead of the line number in `foo`
}
#[track_caller]
fn foo() {
bar();
}
#[track_caller]
fn bar() {
println!("{}", std::panic::Location::caller());
}
playground link
To remedy this, we need to break the chain of #[track_caller] calls. We can break the chain by hiding the nested call to bar within a closure:
#![feature(track_caller)]
fn main() {
foo();
}
#[track_caller]
fn foo() {
(move || {
bar(); // prints `src/main.rs:10:9`
})()
}
#[track_caller]
fn bar() {
println!("{}", std::panic::Location::caller());
}
playground link
Now that we know how to break the chain of #[track_caller] functions, we can address this problem. We just need to make sure that if the user actually marks their function with #[track_caller] on purpose, we refrain from inserting the closure and breaking the chain.
We can add these lines to our solution:
// Wrap body in a closure only if function doesn't already have #[track_caller]
let block = if attrs.iter().any(|attr| attr.path.is_ident("track_caller")) {
quote! { #block }
} else {
quote! {
(move || #block)()
}
};
Final solution
After those two changes, we've ended up with this code:
// print_caller_location/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
use syn::spanned::Spanned;
// Create a procedural attribute macro
//
// Notably, this must be placed alone in its own crate
#[proc_macro_attribute]
pub fn print_caller_location(_attr: TokenStream, item: TokenStream) -> TokenStream {
// Parse the passed item as a function
let func = syn::parse_macro_input!(item as syn::ItemFn);
// Break the function down into its parts
let syn::ItemFn {
attrs,
vis,
sig,
block,
} = func;
// Ensure that it isn't an `async fn`
if let Some(async_token) = sig.asyncness {
// Error out if so
let error = syn::Error::new(
async_token.span(),
"async functions do not support caller tracking functionality
help: consider returning `impl Future` instead",
);
return TokenStream::from(error.to_compile_error());
}
// Wrap body in a closure only if function doesn't already have #[track_caller]
let block = if attrs.iter().any(|attr| attr.path.is_ident("track_caller")) {
quote! { #block }
} else {
quote! {
(move || #block)()
}
};
// Extract function name for prettier output
let name = format!("{}", sig.ident);
// Generate the output, adding `#[track_caller]` as well as a `println!`
let output = quote! {
#[track_caller]
#(#attrs)*
#vis #sig {
println!(
"entering `fn {}`: called from `{}`",
#name,
::core::panic::Location::caller()
);
#block
}
};
// Convert the output from a `proc_macro2::TokenStream` to a `proc_macro::TokenStream`
TokenStream::from(output)
}
Ready to use solutions are available (see #timotree 's comment). If you want to do this yourself, have more flexibility or learn, you can write a procedural macro that will parse a backtrace (obtained from inside the function that is called) and print the information that you need. Here is a procedural macro inside a lib.rs:
extern crate proc_macro;
use proc_macro::{TokenStream, TokenTree};
#[proc_macro_attribute]
pub fn get_location(_attr: TokenStream, item: TokenStream) -> TokenStream {
// prefix code to be added to the function's body
let mut prefix: TokenStream = "
// find earliest symbol in source file using backtrace
let ps = Backtrace::new().frames().iter()
.flat_map(BacktraceFrame::symbols)
.skip_while(|s| s.filename()
.map(|p|!p.ends_with(file!())).unwrap_or(true))
.nth(1 as usize).unwrap();
println!(\"Called from {:?} at line {:?}\",
ps.filename().unwrap(), ps.lineno().unwrap());
".parse().unwrap(); // parse string into TokenStream
item.into_iter().map(|tt| { // edit input TokenStream
match tt {
TokenTree::Group(ref g) // match the function's body
if g.delimiter() == proc_macro::Delimiter::Brace => {
prefix.extend(g.stream()); // add parsed string
TokenTree::Group(proc_macro::Group::new(
proc_macro::Delimiter::Brace, prefix.clone()))
},
other => other, // else just forward TokenTree
}
}).collect()
}
The backtrace is parsed to find the earliest symbol inside the source file (retrieved using file!(), another macro). The code we need to add to the function is defined in a string, that is then parsed as a TokenStream and added at the beginning of the function's body. We could have added this logic at the end, but then returning a value without a semicolon wouldn't work anymore. You can then use the procedural macro in your main.rs as follow:
extern crate backtrace;
use backtrace::{Backtrace, BacktraceFrame};
use mylib::get_location;
#[get_location]
fn add(x: u32, y: u32) -> u32 { x + y }
fn main() {
add(1, 41);
add(41, 1);
}
The output is:
> Called from "src/main.rs" at line 10
> Called from "src/main.rs" at line 11
Don't forget to specify that your lib crate is providing procedural macros by adding these two lines to your Cargo.toml:
[lib]
proc-macro = true
I'm writing a function that will be called in an infinite loop and only execute something when getting well-formed data from a web-service. If the service is down, returns non-json, or returns json we do not understand, the function should just log the error and return (to be called again after a pause).
I found myself copying and pasting something like this:
let v = match v {
Ok(data) => data,
Err(error) => {
println!("Error decoding json: {:?}", error);
return;
}
};
The body of the error matcher would be different each time. Sometimes it's panic, sometimes it has different messages, and sometimes elements of error could be broken down further to form a better message, but the rest of the construct would be the same.
Is there a shorthand for this? I'm aware of the ? syntax, but that's for propagation. I don't feel that propagation will help with the scenario when you need slightly different processing in case of the error like in the scenario described above. This is because the particular differences in handling belong right here, not up the stack.
I have not written a lot of code in Rust yet so it is very likely that I'm missing something obvious.
In C#, the above would look something like this:
if (v == null)
{
Console.WriteLine("Error decoding json!");
return;
}
or
if (error != null)
{
Console.WriteLine($"Error decoding json: {error}");
return;
}
both of which is much less verbose than in Rust.
If I understood the comments below, one way of shortening would be something like this:
if let Err(error) = v {
println!("Error decoding json: {:?}", error);
return;
}
let v = v.unwrap();
This looks more compact, thank you. Is this idiomatic? Would you write it this way?
I don't feel that propagation will help with the scenario when you need slightly different processing in case of the error like in the scenario described above. This is because the particular differences in handling belong right here, not up the stack.
This is something a custom error type can help with. In this case you have a common behavior ("log an error") and you want to do that in slightly different ways for different values. It makes sense to move the "log an error" part up to the caller (let's call the function try_poll):
loop {
if let Err(e) = try_poll() {
println!("{}", e);
}
sleep(100);
}
And create a type that implements Display, and From<E> for each error type E:
enum PollError {
NetworkError(NetworkError),
JsonParseError(JsonParseError),
}
impl fmt::Display for PollError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match *self {
PollError::NetworkError(ref e) => write!(f, "Error downloading file: {:?}", e),
PollError::JsonParseError(ref e) => write!(f, "Error parsing JSON: {:?}", e),
}
}
}
impl From<NetworkError> for PollError {
fn from(e: NetworkError) -> Self {
PollError::NetworkError(e)
}
}
impl From<JsonParseError> for PollError {
fn from(e: JsonParseError) -> Self {
PollError::JsonParseError(e)
}
}
Now you can use ? to propagate the error, but the caller still doesn't have to be concerned with which error specifically it is.
fn try_poll() -> Result<(), PollError> {
let data = try_fetch_content()?;
let json = try_parse_json(data)?;
println!("Parsed {:?}", json);
Ok(())
}
(playground)
Ok, I want that, but without all the From implementations.
The tedious part about this is all the impl Froms, which are necessary because of the custom error type. If the only thing that will ever be done with an error is log and ignore it, a custom error type is not particularly useful -- the only thing that really needs to be returned is the error message itself.
In that case, have try_poll instead return Result<(), String>, and use Result::map_err to turn each individual error immediately into an error message, before using ? to propagate it:
fn try_poll() -> Result<(), String> {
let data = try_fetch_content()
.map_err(|e| format!("Error downloading file: {:?}", e))?;
let json = try_parse_json(data)
.map_err(|e| format!("Error parsing JSON: {:?}", e))?;
println!("Parsed {:?}", json);
Ok(())
}
(playground)
The first edition of The Rust Programming Language has this to say about String as an error type:
A rule of thumb is to define your own error type, but a String error type will do in a pinch, particularly if you're writing an application. If you're writing a library, defining your own error type should be strongly preferred so that you don't remove choices from the caller unnecessarily.
As an alternative to a custom macro_rule you could also use ? with Option<T> and a trait extension for Result to print errors and convert successful values.
Playground
pub trait ResultOkPrintErrExt<T> {
fn ok_or_print_err(self, msg: &str) -> Option<T>;
}
impl<T, E> ResultOkPrintErrExt<T> for Result<T, E>
where
E: ::std::fmt::Debug,
{
fn ok_or_print_err(self, msg: &str) -> Option<T> {
match self {
Ok(v) => Some(v),
Err(e) => {
eprintln!("{}: {:?}", msg, e);
None
}
}
}
}
fn read_input() -> Result<u32, ()> {
// Ok(5)
Err(())
}
fn run() -> Option<()> {
let v: u32 = read_input().ok_or_print_err("invalid input")?;
println!("got input: {}", v);
Some(())
}
fn main() {
run();
}
There are tons of articles on error handling in Rust, but they all seem to result in returning an error object causing a panic or something like that.
I want to check user input for an error, so I can't just pass it down the stack and I can't just panic (It's hardly user friendly)
Currently I'm using an empty if let with an else block to handle this, but it seems ugly and awkward. In the mountain of convenience methods in Option and Result, is there nothing that allows me to call a final method?
unwrap_or_else just complains that the closure return value is the wrong type.
use std::io::{stderr, Write};
use std::env;
use std::process::exit;
fn error_out(e: &str) {
writeln!(&mut stderr(), "{}", e).expect(format!("Failed to print error: {}", e).as_str());
exit(1);
}
fn main() {
if let Some(filename) = env::args().nth(1) {} else {
error_out("No filename provided");
};
}
You can add the return type ! to error_out to indicate that it doesn't return. Then its type doesn't matter and it is OK to call it in unwrap_or_else:
use std::process::exit;
fn error_out(e: &str) -> ! {
writeln!(&mut stderr(), "{}", e).expect(format!("Failed to print error: {}", e).as_str());
exit(1);
}
fn main() {
let filename: String = env::args().nth(1).unwrap_or_else(|| error_out("No filename provided"));
println!("{}", filename);
}