I'm currently using syn following an example to create an AST that can be mutated. I understand that I can modify the node I'm travesing (as shown below in my current code) but
I'm curious if I can add some code in between the current node and the next node. Is the syn crate capable of this?
use syn::visit_mut::{self, VisitMut};
use syn::Expr;
#[derive(Debug)]
struct MyStruct;
impl VisitMut for MyStruct {
fn visit_expr_mut(&mut self, node: &mut Expr) {
if let Expr::MethodCall(expr) = &node.to_owned() {
// I can modify the existing node like so:
*node = parse_quote!("// Hello World");
// How could I add something after this node and before the next?
}
}
}
pub fn create() {
let current_dir = std::env::current_dir().expect("Unable to get current directory");
let rust_file = std::fs::read_to_string(current_dir.join("src").join("lib.rs")).expect("Unable to read rust file");
let ast = syn::parse_file(&rust_file).expect("Unable to create AST from rust file");
MyStruct.visit_file_mut(&mut ast);
}
Edit to show use case:
The file I'm currently parsing looks like:
#[macro_use]
extern crate foo;
mod test;
fn init(handle: foo::InitHandle) {
handle.add_class::<Test::test>();
}
Let's say that when I read the AST, I want to add another mod and another handle for it like so:
#[macro_use]
extern crate foo;
mod test;
mod store;
fn init(handle: foo::InitHandle) {
handle.add_class::<Test::test>();
handle.add_class::<Store::store>();
}
As I commented, it highly depends on what you want to insert. Because you can't just insert anything before or after node easily.
For your specific case, you could use parse_quote! to produce an ExprBlock.
*node = parse_quote!(
{
#expr;
handle.add_class::<Store::store>();
}
);
Which with the following input:
fn init(handle: foo::InitHandle) {
handle.add_class::<Test::test>();
}
Would produce this output:
fn init(handle: foo::InitHandle) {
{
handle.add_class::<Test::test>();
handle.add_class::<Store::store>();
};
}
(Note I have reformatted the output, to be prettier)
Alternatively, you could override visit_block_mut() instead. That way you'd have access to stmts: Vec<Stmt>, and would be able to insert before and after a Stmt. The downside is that by doing it that way, you wouldn't be able to easily visit all Exprs, as by using visit_expr_mut().
Related
How can I expose a struct generated from the quote macro in my derive macro without having to introduce a struct name out of the blue in my usage file (due to macro expansion)?
To illustrate the point, currently, my code looks something like this:
// "/my_derive/lib.rs"
// inside a derive macro function
let tokens = quote! {
struct MyDeriveMacroInternalStruct {
variant: #ident_name,
// other stuff ...
}
impl #ident_name {
pub fn something() -> Vec<MyDeriveMacroInternalStruct> {
vec![MyDeriveMacroInternalStruct { variant: #ident_name::#variant_name, /*...*/ }, /*...*/]
}
}
};
tokens.into()
The usage of my code would look something like this:
use my_derive::MyDerive;
#[derive(MyDerive)]
enum Something {
A,
B,
C,
}
fn process_data() -> Vec<MyDeriveMacroInternalStruct> { // having to write that struct name that came out of nowhere bothers me
Something::something()
}
fn main() {
let result = process_data();
// do stuff...
}
This is a condensed version of my actual code (process_data is in another file). To reiterate my question in light of the example, how can I access the struct without having it randomly appear out of nowhere (due to macro expansion)? To me the code unchanged is hard to understand, read, and change.
I would like to be able to do something like this:
use my_derive::{MyDerive, MyDeriveStruct};
#[derive(MyDerive)]
enum Something {
A,
B,
C,
}
fn process_data() -> Vec<MyDeriveStruct> { // importing the struct instead of magically appearing
Something::something()
}
fn main() {
let result = process_data();
// do stuff...
}
Obviously the idea seems quite stupid, but there has to be a way around it (an arbitrary struct definition). If what I imagined isn't possible, is there some way to be more clear about where the random struct came from?
Actually I thought of something better. Your derive should probably be associated with a trait of the same name.
Add an associated type to your trait:
trait MyDerive {
type Output;
...
}
Then set the associated type when you impl the trait:
struct MyDeriveMacroInternalStruct {
variant: #ident_name,
// other stuff ...
}
impl MyDerive for #ident_name {
type Output = MyDeriveMacroInternalStruct;
pub fn something() -> Vec<MyDeriveMacroInternalStruct> {
vec![MyDeriveMacroInternalStruct { variant: #ident_name::#variant_name, /*...*/ }, /*...*/]
}
}
Then you can refer to that associated type in return position or wherever:
use my_derive::MyDerive;
#[derive(MyDerive)]
enum Something {
A,
B,
C,
}
fn process_data() -> Vec<<Something as MyDerive>::Output> {
Something::something()
}
fn main() {
let result = process_data();
// do stuff...
}
Note: the convention is for #[derive(Trait)] to correspond to an impl for the given Trait, but your proc macro crate can't export a trait directly for importing in your library code.
So generally the solution is to have two crates:
my-trait is the "library" crate which contains the MyTrait trait definition
my-trait-derive is the proc-macro crate which contains the derive macro code
my-trait has my-trait-derive as a direct dependency, and re-exports the proc macro from it:
// my-trait lib.rs
pub use my_trait_derive::MyTrait;
// macro and trait names can overlap as they're
// treated as different item kinds
pub trait MyTrait {
type Output;
fn something();
}
see how clap does it here (they also re-export the whole clap_derive)
Then a user can use your proc macro + trait like this:
use my_trait::MyTrait;
#[derive(MyTrait)]
enum Something {}
fn process_data() -> Vec<<Something as MyTrait>::Output> {
Something::something()
}
Older Answer
What I would do is create a trait MyDeriveOutput or something with whatever stuff you want exposed from MyDeriveMacroInternalStruct:
trait MyDeriveOutput {
fn variant() ...
}
And then generate an impl for each internal struct you create:
struct MyDeriveMacroInternalStruct {
variant: #ident_name,
// other stuff ...
}
impl MyDeriveOutput for MyDeriveMacroInternalStruct {
// whatever
}
Then you can expose the trait and require it to be imported and used with impl Trait in return position:
use my_derive::{MyDerive, MyDeriveOutput};
#[derive(MyDerive)]
enum Something {
A,
B,
C,
}
fn process_data() -> Vec<impl MyDeriveOutput> {
Something::something()
}
fn main() {
let result = process_data();
// do stuff...
}
I have defined a few structs in my code and if a certain feature is enabled on the crate, I would like to generate Python bindings for those structs as well. Right now I am not able to get it correctly. Let's say I have a struct MyStruct for which I want to optionally generate Python Bindings.
I have tried something like the following
cfg_if! {
if #[cfg(feature = "python-bindings")] {
#[pyclass]
}
else {
}
}
struct MyStruct{
value: i32
}
I would like to only add #[pyclass] if feature python-bindings is enabled and not otherwise.
This works fine if python-bindings is not enabled. But if I compile with --features python-bindings, I get the following error.
error: expected item after attributes
As far as possible I do not want to duplicate the code. like
cfg_if! {
if #[cfg(feature = "python-bindings")] {
#[pyclass]
struct MyStruct{
value: i32
}
}
else {
struct MyStruct{
value: i32
}
}
}
Is there a way of doing it without duplicating the code?
Yes, with #[cfg_attr]:
#[cfg_attr(feature = "python-bindings", pyclass)]
struct MyStruct {
value: i32
}
I have the following structure:
Inside queues.rs I have a #[macro_export],
#[macro_export]
macro_rules! queue {...}
but I am not sure how to import it in lib.rs.
What I have looks like this:
use crate::utils::queues::*;
// use it here
let mut service_queue: Queue<isize> = queue![];
This throws an error
error: cannot find macro `queue` in this scope
--> src/broker.rs:54:51
|
54 | let mut service_queue: Queue<isize> = queue![];
| ^^^^^
|
= note: consider importing this macro:
crate::queue
= help: have you added the `#[macro_use]` on the module/import?
What is the correct way to type #[macro_use] here, and import my custom macros from utils?
You can use the queue! macro from lib.rs using the #[macro_use] attribute (playground):
#[macro_use]
mod utils {
#[macro_use]
mod queues {
macro_rules! queue {
() => { vec![] };
}
}
}
pub fn foo() {
let _service_queue: Vec<isize> = queue![];
}
In your existing code you are using #[macro_export] which means it is declared in the crate root scope and not in the utils::queues module. In your error message you can see the compile is suggesting to consider importing this macro: crate::queue.
To access it in this case (playground):
mod utils {
mod queues {
#[macro_export]
macro_rules! queue {
() => { vec![] };
}
}
}
pub fn foo() {
let _service_queue: Vec<isize> = queue![];
}
Note that in either case the use crate::utils::queues::*; statement does not help, in fact you'll get an unused_imports warning (unless you happen to have other items declared in that module).
There is also this trick (playground):
pub mod utils {
pub mod queues {
macro_rules! queue {
() => { vec![] };
}
pub(crate) use queue;
}
}
pub fn foo() {
let _service_queue: Vec<isize> = crate::utils::queues::queue![];
}
Aside: if you plan to use the macro in other crates and you wish to retain the module hierarchy (i.e. access from my_crate::utils::queues::queue! from my_other_crate) there is a (convoluted) way to do that.
Here's the code:
use crate::rooms::room::RoomInterface;
pub mod dogroom {
pub struct R;
impl RoomInterface for R {
}
}
Here's /rooms/mod.rs:
pub mod room {
// Irrelevant stuff
pub trait RoomInterface {
// stuff
}
// stuff
}
Here's what it tells me:
I find it quite arcane that I import the very same thing it wants me to import, and yet it doesn't work.
I've tried pretty much all the permutations of the use keyword, and I can't make it work. What's going on?
uses are scoped to the module that imports them, not the file they are in.
Move the import into the dogroom module:
pub mod dogroom {
use crate::rooms::room::RoomInterface;
pub struct R;
impl RoomInterface for R {
}
}
Alternatively, you might want the dogroom module to reuse everything from the parent module:
use crate::rooms::room::RoomInterface;
pub mod dogroom {
use super::*;
pub struct R;
impl RoomInterface for R {
}
}
I'm trying to use syn to create an AST from a Rust file and then using quote to write it to another. However, when I write it, it puts extra spaces between everything.
Note that the example below is just to demonstrate the minimum reproducible problem I'm having. I realize that if I just wanted to copy the code over I could copy the file but it doesn't fit my case and I need to use an AST.
pub fn build_file() {
let current_dir = std::env::current_dir().expect("Unable to get current directory");
let rust_file = std::fs::read_to_string(current_dir.join("src").join("lib.rs")).expect("Unable to read rust file");
let ast = syn::parse_file(&rust_file).expect("Unable to create AST from rust file");
match std::fs::write("src/utils.rs", quote::quote!(#ast).to_string());
}
The file that it creates an AST of is this:
#[macro_use]
extern crate foo;
mod test;
fn init(handle: foo::InitHandle) {
handle.add_class::<Test::test>();
}
What it outputs is this:
# [macro_use] extern crate foo ; mod test ; fn init (handle : foo :: InitHandle) { handle . add_class :: < Test :: test > () ; }
I've even tried running it through rustfmt after writing it to the file like so:
utils::write_file("src/utils.rs", quote::quote!(#ast).to_string());
match std::process::Command::new("cargo").arg("fmt").output() {
Ok(_v) => (),
Err(e) => std::process::exit(1),
}
But it doesn't seem to make any difference.
The quote crate is not really concerned with pretty printing the generated code. You can run it through rustfmt, you just have to execute rustfmt src/utils.rs or cargo fmt -- src/utils.rs.
use std::fs;
use std::io;
use std::path::Path;
use std::process::Command;
fn write_and_fmt<P: AsRef<Path>, S: ToString>(path: P, code: S) -> io::Result<()> {
fs::write(&path, code.to_string())?;
Command::new("rustfmt")
.arg(path.as_ref())
.spawn()?
.wait()?;
Ok(())
}
Now you can just execute:
write_and_fmt("src/utils.rs", quote::quote!(#ast)).expect("unable to save or format");
See also "Any interest in a pretty-printing crate for Syn?" on the Rust forum.
As Martin mentioned in his answer, prettyplease can be used to format code fragments, which can be quite useful when testing proc macro where the standard to_string() on proc_macro2::TokenStream is rather hard to read.
Here a code sample to pretty print a proc_macro2::TokenStream parsable as a syn::Item:
fn pretty_print_item(item: proc_macro2::TokenStream) -> String {
let item = syn::parse2(item).unwrap();
let file = syn::File {
attrs: vec![],
items: vec![item],
shebang: None,
};
prettyplease::unparse(&file)
}
I used this in my tests to help me understand where is the wrong generated code:
assert_eq!(
expected.to_string(),
generate_event().to_string(),
"\n\nActual:\n {}",
pretty_print_item(generate_event())
);
Please see the new prettyplease crate. Advantages:
It can be used directly as a library.
It can handle code fragments while rustfmt only handles full files.
It is fast because it uses a simpler algorithm.
Similar to other answers, I also use prettyplease.
I use this little trick to pretty-print a proc_macro2::TokenStream (e.g. what you get from calling quote::quote!):
fn pretty_print(ts: &proc_macro2::TokenStream) -> String {
let file = syn::parse_file(&ts.to_string()).unwrap();
prettyplease::unparse(&file)
}
Basically, I convert the token stream to an unformatted String, then parse that String into a syn::File, and then pass that to prettyplease package.
Usage:
#[test]
fn it_works() {
let tokens = quote::quote! {
struct Foo {
bar: String,
baz: u64,
}
};
let formatted = pretty_print(&tokens);
let expected = "struct Foo {\n bar: String,\n baz: u64,\n}\n";
assert_eq!(formatted, expected);
}