Why do proc-macros have to be defined in proc-macro crate? - rust

I was trying to create a derive macro for my trait, to simplify some stuff.
I've encountered some problems:
the #[proc_macro_derive] attribute is only usable with crates of the proc-macro crate type
and, after the small fix proc-macro=true:
proc-macro crate types cannot export any items other than functions tagged with #[proc_macro_derive] currently
functions tagged with #[proc_macro_derive] must currently reside in the root of the crate`
What is the reason for this behavior?

Procedural macros are fundamentally different from normal dependencies in your code. A normal library is just linked into your code, but a procedural macro is actually a compiler plugin.
Consider the case of cross-compiling: you are working on a Linux machine, but building a WASM project.
A normal crate will be cross-compiled, generate WASM code and linked with the rest of the crates.
A proc-macro crate must be compiled natively, in this case to Linux code, linked with the current compiler runtime (stable, beta, nightly) and be loaded by the compiler itself when compiling the crates where it is actually used. It will not be linked to the rest of the crates (different architecture!).
And since the compilation flow is different, the crate type must also be different, that is why the proc_macro=true is needed.
About this restriction:
proc-macro crate types cannot export any items other than functions tagged with #[proc_macro_derive]
Well, since the proc-macro crate is loaded by the compiler, not linked to the rest of your crates, any non-proc-macro code you export from this crate would be useless.
Note that the error message is inexact, as you can also export functions tagget with #[proc_macro].
And about this other restriction:
functions tagged with #[proc_macro_derive] must currently reside in the root of the crate
Adding proc_macro or proc_macro_derive items in nested modules is not currently supported, and does not seem to be particularly useful, IMHO.

Related

Rust - can I ask/force compiler to do monomorphization code generation while compiling a crate (instead of postponing it to the caller crate )

I have project that build with cargo workspace with including lot of crates.
One of the lower level crates contains generic data-structure with a lot of serde-code involved.
In order to reduce the compile time , I tried to crate objects with monomorphized instances on the the data-struct in a crate that is lower in the compilation hierarchy and use those in the higher-level crates. My goal to compile the lower-level crate only once, and then work on the higher level crate - without generating the monomorphized instances every time.
example:
lower-level crate
-----------------
pub struct MyCache<T> {
//generic implementation of cache
}
pub struct MyCacheString {
cache: MyCache<String>
}
higher-level crate
------------------
use MyCacheString;
but the problem is that the compiler generated that monomorphized in the higer-level crate (according to "cargo llvm-lines")
Is there is a way to ask/force the compiler to generate the monorphized code while it's compile the lower level crate?
ok, according to rustc_monomorphize::collector docstring:
Lazy mode means that items will only be instantiated when actually referenced
So in order to tigger the compiler to monomorphized the code in the low-level crate I need to call the object functions from public non-generic function lower-level crate. (not sure why - but it's not working from async function).
(If someone know about macro that tell the compiler to use the 'Eager Mode' it's will be better )

Is it OK to treat the proc-macro crate in a two-crate library as implementation detail and not follow semantic versioning?

Many libraries that contain procedural macros consist of two crates: a proc-macro crate implementing the actual macro and a normal "main" crate that reexports or wraps the proc macro. This is done because proc-macro crates cannot publicly export anything other than proc macros. Assuming the main crate is called foo, the macro crate is usually called foo-derive or foo-macros.
This brings up a couple of questions about how to version the proc-macro crate. Of course, the main crate follows semantic versioning. But should the macro crate follow it as well? I don't want people to use the macro crate directly, but only through the main crate. I clearly stated that in the proc-macro crate's description. I want to treat the macro crate as an implementation detail.
In that case, can I shouldn't need to follow semantic versioning, right? The main crate would then just require one exact version via foo-macro = "=0.0.4".
Is this fine? Or can something break with this approach? Are there some established best practices in the community?

Linking against binary crate

There's a crate I want to use as a library for some of my own code (speedtest-rs specifically, but it doesn't really matter). However, whenever I try to use this crate, the compiler doesn't want to play nice with it.
$ cargo build
Compiling my-project v0.1.0 (/home/nick/Documents/code/my-project)
error[E0432]: unresolved import `speedtest_rs`
--> src/main.rs:1:5
|
1 | use speedtest_rs::*;
| ^^^^^^^^^^^^ use of undeclared type or module `speedtest_rs`
Looking at the Rust book, it seems like there's a distinction between a binary and library crae
The rand crate is a library crate which contains code intended to be used in other programs
Some googling has shown me that binary crates just have an extra link step, so I should be able to link against them, right? I know a lot of Rust packages have both a library and a binary in them, but what do you do when an author does not seem to follow this pattern?
Some googling has shown me that binary crates just have an extra link step, so I should be able to link against them, right?
No. It's not that simple. Plus that extra step creates an executable file rather than a library file. An executable cannot be used as a library.
I know a lot of Rust packages have both a library and a binary in them, but what do you do when an author does not seem to follow this pattern?
You can:
Ask them on GitHub to publish a library.
Fork the crate and make your own library (which you can do since it is published with the usual dual “Apache License, Version 2.0” + “MIT” license).
There isn't an automated way to use a binary crate as a library because in particular:
Rust won't generate a library.
Since the crate is missing a src/lib.rs file, nothing is exported. This is akin to have all items in that crate private. You wouldn't be able to use anything.

Why do I need to declare "extern crate core" to use libcore?

With the core library stabilized in Rust 1.6, the following becomes possible, and I do not need to replace libcore with libstd any more:
//extern crate core; //won't work without this line
extern crate num;
use core::ops::Add;
use num::bigint::{BigInt};
fn main() {
let mut big = "8705702225074732811211966512111".parse::<BigInt>().unwrap();
let one = "1".parse::<BigInt>().unwrap();
big = big.add(&one);
println!("{:?}", big);
}
But there is one thing puzzles me - why do I need to declare "extern crate core;"? As far as I know, libstd is meant to be built on top of libcore. libcore is meant to be OS independent, while the implementation of libstd can be OS specific. I never had the need to specify "extern crate std". What also puzzles me is that I do not need to add libcore as a dependency in Cargo.toml in the above case, although it is an extern crate.
Is libcore the only such case? Is this a temporary thing while the implementation of the language is getting stabilized?
In fact everything works in a pretty logical way.
First of all, libstd crate is indeed special. Rust compiler knows about it and it injects extern crate std; implicitly unless #![no_std] attribute is present on the crate root. Additionally it also imports standard prelude in every module of your crate (again, unless #![no_std] is present).
Now you can probably see why you have to specify extern crate core; while at the same time you don't need to specify extern crate std;. You also don't need to specify core in Cargo.toml because libcore, as well as several other libraries (libcollections, liballoc, liblibc, etc.; you can find an up-to-date list in Rust source directory) are present in the Rust compiler distribution. There is in fact a desire to allow these libraries to be available through Cargo as well (expressed in form of RFCs), but as of now these libraries are provided with the compiler distribution only.
And finally, remember that Rust crates are independent. Rust ABI is designed in such a way so you can have different versions of the same crate built into the final executable. While it is invalid for one crate to directly depend on multiple versions of the same crate, it is possible that its dependencies transitively depend on different versions of another crate. This is one of the reasons why you always have to explicitly specify dependencies of your crate: if you don't specify that your crate depends on libcore, even though libstd does depend on libcore, libcore won't be pulled by your crate automatically if it uses libstd.

What's the difference between use and extern crate?

I think that use is used to import identifiers into the current scope and extern crate is used to declare an external module. But this understanding (maybe wrong) doesn't make any sense to me. Can someone explain why Rust has these two concepts and what are the suitable cases to use them?
extern crate foo indicates that you want to link against an external library and brings the top-level crate name into scope (equivalent to use foo). As of Rust 2018, in most cases you won't need to use extern crate anymore because Cargo informs the compiler about what crates are present. (There are one or two exceptions)
use bar is a shorthand for referencing fully-qualified symbols.
Theoretically, the language doesn't need use — you could always just fully-qualify the names, but typing std::collections::HashMap.new(...) would get very tedious! Instead, you can just type use std::collections::HashMap once and then HashMap will refer to that.
Since Rust 2018 you're only required to add external dependencies to your Cargo.toml, so you no longer need to use extern crate foo.
use works the same as before.
Read more in the official documentation.

Resources