Basic setup
A simple C function that is called from Rust via FFI, and a static library (Windows) that is linked in build.rs.
// api.c
int increment(int value) {
return value + 1;
}
// main.rs
extern {
fn increment(value: i32) -> i32;
}
fn main() {
let num = unsafe { increment(4) };
println!("The incremented number is {}", num);
}
// build.rs
fn main() {
println!("cargo:rustc-link-search=src/ffi");
println!("cargo:rustc-link-lib=static=api");
}
With this directory structure:
Cargo.toml
Cargo.lock
build.rs
src
+-- main.rs
+-- ffi
+-- api.c
+-- api.lib
Cargo.toml has nothing but a crate name, version, author and Rust edition.
So far, this works perfectly, and "The incremented number is 5" is printed as expected.
The problem
Now I add an empty lib.rs file, so I can use my crate as both a library and a binary:
Cargo.toml
Cargo.lock
build.rs
src
+-- lib.rs <-- NEW
+-- main.rs
+-- ffi
+-- api.c
+-- api.lib
I do not change Cargo.toml.
This alone makes the link step fail (MSVC linker):
error LNK2019: Unresolved external symbol "increment" referenced in function "_ZN16my_crate4main17h887bd80180495b7eE"
The error persists even if I explicitly specify the presence of both a library and binary in Cargo.toml and run using cargo run --bin my_main:
[lib]
name = "my_lib"
path = "src/lib.rs"
[[bin]]
name = "my_main"
path = "src/main.rs"
I also made sure that build.rs is still executed in the binary case, by letting it panic to abort the build.
I'm aware that I could solve it by splitting the crate, but I'd like to understand what exactly happens. So:
Why does the presence of an empty lib.rs cause the linker to fail?
And is there a way to make it succeed, in a single crate?
In the presence of both a binary and a Rust library in a single crate, Rust statically links the binary against the Rust library itself, as if the library were any other external crate (e.g. from crates.io). I assume to prevent errors due to duplicate symbols, or possibly just to avoid code bloat or extra work, it appears to avoid linking any external artifacts directly against the binary, and does not make an effort to determine if the code in question is even available in lib.rs before making this judgment. Normally this happens in the background without you noticing (i.e. it's doing this to the Rust standard library, your system standard libraries, and external crates as well for everything you compile), but when you introduce custom FFI dependencies it becomes obvious.
If you change your lib.rs to
extern {
pub fn increment(value: i32) -> i32;
}
And your main.rs to
fn main() {
let num = unsafe { libname::increment(4) };
println!("The incremented number is {}", num);
}
Where libname is either the name of your library as listed in Cargo.toml or the project name if that's not specificed, it will compile and run properly. Indeed, this follows no matter how complex you get. You can bury the FFI function a million modules deep and include it from wherever, but it will fail with the same error if you reference a definition of an extern "C" function without going through the master library crate first.
I don't know if there's any way (in build.rs or otherwise) to insist that Rust statically links a library either twice against the two different targets, or else only against the binary itself, but if there is it's not well documented or obvious.
That said, in most cases this is unlikely to be a problem, since if you're making your functionality usable as a library, it's likely you'll have the common code, including FFI, in there anyway. It could conceivably be an issue if you have a C dependency (e.g. for device access) that only makes sense in the binary, but I think that case is rare enough it's reasonable to force you to split the crates using workspaces if that's your specific situation.
Related
So I want to have some modules defined in lib.rs but not make them public but only available for use within the project.
in my lib.rs, if I have the definition as this:
pub mod args;
in my main.rs, I can use the args modules this way:
use my_lib::args::Cli;
where my_lib is defined in Cargo.tml as
[lib]
name = "my_lib"
path = "src/lib.rs"
but I don't want pub mod args;. I tried changing to pub(crate) mod args; but this leads to compilation error that the args module cannot be found.
How do I make a module like args defined in lib.rs available without have to give it the most permissive visibility?
Since Rust separates the library lib.rs and binary main.rs into separate crates there is no simple way to include things that are in lib.rs and not pub from main.rs.
I suggest you follow the way of big crates (serde comes to mind) and add a pub mod __private; which conveys the meaning. You can additionaly annotate it with #[doc(hidden)] to hide it from the documentation making it even more obvious.
this is my src folder structure
src/
|-- main.rs
|-- problems
| |-- mod.rs
| |-- p1.rs
| `-- p2.rs
`-- utilities.rs
I have my utilities.rs file containing 2 functions I use in p1.rs and p2.rs. In both p1.rs and p2.rs are the lines
#[path = "../utilities.rs"]
mod utilities;
that, as far as I understand, allow me to use the functions defined in the utilities.rs
The two functions defined in utilities.rs are
pub fn get_lines(num: &str) -> Vec<String> {...}
pub fn split(s: &String, separator: &str) -> Vec<String> {...}
In p2.rs I use them both, but in p1.rs I only use get_lines. This causes cargo to warn me of a "never used" function, it being split, when I run or build. But the function is used, and not only in functions that are never called, so I don't understand why cargo is warning me. It would seem I have to use every single module function in every single file where I include a self-made module, because if I do call split in p1.rs then the warning disappears. What do I not understand?
To be clearer, the problem is that when I cargo run I get the following
warning: function is never used: `split`
--> src/problems/../utilities.rs:23:8
|
23 | pub fn split(s: &String, separator: &str) -> Vec<String> {
| ^^^^^
|
= note: `#[warn(dead_code)]` on by default
You should generally never use the #[path] attribute in ordinary situations, because it lets you make this mistake.
The key to understanding Rust modules is that mod defines a module and its contents. Every mod item is a different module from any others. Therefore, if you write mod twice in such a way that both refer to the same file, that file gets compiled twice, producing two copies of what's defined in that file. That's why you're getting dead_code warnings — you've defined two different functions, problems::p1::utilities::split() and problems::p2::utilities::split(), and one of them isn't used.
As a general rule, in projects with simple structure, there should be only one mod item for any module source file in your project. In this case, the natural place to locate that mod item is in main.rs, and it should be:
mod utilities;
Then, you can refer to this module in your p1.rs or p2.rs as:
use crate::utilities;
On the other hand, maybe the utilities are more associated with the problems module only. In that case, you would
place the file at src/problems/utilities.rs,
write mod utilities; in src/problems/mod.rs, and
refer to the utilities in p1 and p2 as either use super::utilities; (relative path) or use crate::problems::utilities; (absolute path).
In general, the procedure for introducing a new module is:
Pick a module to be the parent module of your module. (Don't think in directories and files, think in modules.)
Write mod my_module_name; in that parent module.
Put the file where it needs to be, given that. (If you try to compile the program at this point, the compiler will tell you what file is missing; or if you use rust-analyzer in an IDE you can ask it to create the file.)
Everywhere you want to use that module, refer to it by the appropriate absolute or relative path according to where its mod is; and use use if you want to make a convenient short alias for its path.
You could use a crate import instead of the path modifier (which I cannot prove, but is probably the guilty one), in both p1 and p2:
use crate::utilities;
This question already has answers here:
How do I do a basic import/include of a function from one module to another in Rust 2015?
(3 answers)
Split a module across several files
(7 answers)
How can I include a module from another file from the same project?
(6 answers)
Closed 5 years ago.
How do I include a file with the full path my_project/src/include_me.rs in main.rs?
I've checked the dependencies guide, and all of them appear to be including a binary. I've also checked "The Book", but none of the examples there end in ".rs" either.
How do I make include_me.rs compile with the rest of the project?
There are basically two (main) ways in Rust to include code from somewhere else:
1. "Including" internal code
If your include_me.rs belongs to your project, you should move it to the same folder main.rs lies in:
└── src
├── include_me.rs
└── main.rs
Then you can write this in your main.rs:
mod include_me;
fn main() {
// Call a function defined in the other file (module)
include_me::some_function();
}
A mod declaration makes the Rust compiler look for the corresponding .rs files automatically!
So everything that belongs to your project, belongs in the same folder (or a subfolder thereof) as the folder where main.rs (or lib.rs) is lying. The files are then "included" via the module system. To read a good introduction into modules, please read the chapter on modules in the Rust book. You could also check out Rust by Example on that topic. The module system is pretty central and thus important to learning Rust.
2. "Including" external code
If your include_me.rs is something that doesn't belong to your actual project, but is rather a collection of useful things you are using in multiple projects, it should be seen as a library. To include code of such external libraries, you have to include it as an extern crate. And to make your life easier, you really want to use Cargo!
So let's prepare your include_me.rs as Cargo library project. You need the following file structure (which is automatically generated by cargo new my_library --lib):
. my_library
├── Cargo.toml
└── src
└── lib.rs
Now copy all the contents from include_me.rs into lib.rs (it is just convention to call the root file of a library project lib.rs). Let's say that the my_library folder's full path is ~/code/my_library.
Now let's prepare your main project's Cargo project. It has a similar file
structure (but a main.rs instead of lib.rs, because it's a executable-project, not a library-project):
. my_project
├── Cargo.toml
└── src
└── main.rs
To declare your dependency on my_library, you have to put this into your Cargo.toml:
[package]
name = "my_project"
version = "0.1.0"
authors = ["you"]
edition = "2018"
[dependencies]
my_library = { path = "~/code/my_library" }
You can also use relative paths ("../my_library"), but it only makes sense if you know that the two projects always stay where they are, relative to one another (like if they are both managed in the same repository).
Now you can, in your main.rs, write:
use my_library::some_function;
fn main() {
// Call a function defined in the other file (extern crate)
some_function();
}
If you want to upload any of those two projects, you have to interact with crates.io (or another crates registry, if your company has one), but this is another topic.
(Note: some time ago, it was necessary to write extern crate my_library; inside main.rs. This is not necessary anymore, but you might find old code with extern crate declarations.)
Any other ways?
Yes, but you shouldn't use those. There is the include!() macro which allows you to verbatim include other files, just like the #include from C-land. However, it is strongly discouraged to use this in situations where the module system would be able to solve your problem. include!() is only useful in very special situations, often linked to a more complex build system which generates code.
I have read Managing Growing Projects with Packages, Crates, and Modules, but I still don't really understand the hierarchy. I know what is crate and that there should be at least one crate in package: max 1 library crate and 0 or more binary crates.
First: Assume I want to have both lib.rs and main.rs crates in package. How do I access/call functions from lib.rs in main.rs?
Second: when I create new library with cargo new --lib library-name it creates directory with that name and bunch of files there, the only way I figured out to call functions from that library in src/main.rs is:
mod some_library;
use crate::library_name::library_name::foo;
fn main() {
foo();
}
// Filename: src/library_name.rs
pub mod library_name; // I don't really understand this
// Filename: src/library_name/library_name.rs
pub fn foo() {
// ...
}
where I have the following hierarchy:
- package_name
- src
- library_name
- src
- lib.rs
- Cargo.toml
- library_name.rs
- library_name.rs
- main.rs
- Cargo.toml
Is it necessary for src/library_name.rs to have the same name as src/library_name library? I'm really confused.
It's a hierarchy of three or four levels.
Workspace (optional): At the top there's the workspace. A workspace consists of one or more packages. The Cargo.toml file is special and more or less only lists the workspace members.
Workspaces are optional and used for big projects. Smaller projects with only one package don't need them. In such cases we can ignore workspaces. We have a package at the top and the hierarchy is only three levels deep.
Package: Then there's the package. A package is what haves a real Cargo.toml file. A package can compile to crates.
Crate: A crate is a library or an executable compiled from the package. A library crate has a lib.rs file as a starting point. An executable crate has a main function as a starting point. A package can compile to at most one library and several executables.
Module: Then there are modules. Modules are the hierarchical way how Rust organizes items like struct, enum, functions and others in the source code.
And how to you use this to give items unique names?
Answer: A fully qualified name starts with the package, then a series of module names then finally the item's name.
An example:
serde::de::value::StringDeserializer (see here) has as package serde, as module path de then value then the struct is called StringDeserializer.
And where's the crate?
The package serde has only one crate, the library. If you look at the package's Cargo.toml there are no binaries listed.
(People tend to be confused about packages and crates. Even sometimes I am not precise and say "the serde crate" when I mean the package.)
In short: A package is code under a name (for example serde). A crate is a library or an executable.
I'm trying to read the code of servo. As an example, I'm looking at this code in layout_task.rs:
use url::Url;
..and I want to know which code this refers to (the answer is rust-url).
Per the Rust reference §6.1.2.2 Use declarations,
the paths contained in use items are relative to the crate root [...]
It is also possible to use self and super at the beginning of a use item to refer to the current and direct parent modules respectively.
All rules regarding accessing declared modules in use declarations apply to both module declarations and extern crate declarations.
The reference (§5 Crates and source files) does not explicitly say what a "crate root" is, but it does share that:
A crate contains a tree of nested module scopes. The top level of this tree is a module that is anonymous [...] The Rust compiler is always invoked with a single source file as input, and always produces a single output crate. The processing of that source file may result in other source files being loaded as modules.
So it seems that to find the crate root that the current file (layout_task.rs) belongs to, we need to figure out what source file rustc is invoked with when building the crate. With Cargo this is specified in Cargo.toml and defaults to src/lib.rs:
[lib]
path = "src/lib.rs"
In my case, here's Cargo.toml and the lib.rs has:
extern crate url;
pub mod layout_task;
So far so good. To find out what the extern crate refers to, we need to look at Cargo.toml again:
[dependencies.url]
version = "0.2"
The cargo docs claim that "Dependencies from crates.io are not declared with separate sections", but apparently they can be... So we look the package up on crates.io: https://crates.io/crates/url