So basically my problem is very similar to the problem presented in this question:
Correct place for private module to be used by multiple executables
My project structure is as follows:
crate
├── Cargo.toml
└── src
├── bin
│ ├── my-bin-1
│ │ ├── cli.rs
│ │ ├── main.rs
│ └── my-bin-2
│ ├── cli.rs
│ ├── main.rs
├── common.rs
└── lib.rs
lib.rs contains public code, and common.rs contains private code.
I want to have both of my binaries re-use code from common.rs, but I need this code to remain private.
That's why the aformentioned question's solution which is to define a public function in common.rs is not good for my case.
// common.rs
pub fn cant_leak_this_function() {} // uh-oh.. `pub` identifier should NOT be used here
I tried using the pub(crate) identifier, but that made the functions in common.rs private to the binaries.
What I need is some sort of a pub(crate-and-also-the-binaries-in-this-crate-but-not-for-external-crates) specifier (with a shorter name obviously..) which would allow me to use shared private code in my-bin-1/main.rs and in my-bin-2/main.rs, but will also keep that code hidden from external crates.
So far, the only solution that worked for me was creating
a mod.rs file inside a common directory within src/bin, (resulting in an awkward src/bin/common/mod.rs path) and then explicitly using the #[path = ..] feature to consume the shared code from the binaries. However, this method caused rust-analyzer to mark the functions from common/mod.rs as unused even though they clearly were used, and even worse, from some reading online I've come to understand that using the #[path = ..] feature is considered bad practice.
Is there a better way to solve this problem? What am I missing here?
Thanks in advance!
All items are either public or private outside of the crate. There is no mechanism to give other crates special permission. The best you can do is make it clear that while the function is technically public, it should not be used as such.
Take a trick out of the serde crate, which has some exports that must be visible to macros, but shouldn't be used directly (source):
// Used by generated code and doc tests. Not public API.
#[doc(hidden)]
#[path = "private/mod.rs"]
pub mod __private;
The #[doc(hidden)] means it won't sure up in https://docs.rs or your typical cargo doc invocations. And the __private module should hopefully make it obvious to anyone who does stumble across it that it shouldn't be used like other public items.
If this is still unsavory to you, then the only way to truly make it private is indeed to use #[path = ...] to import the file as modules in your separate binaries as the linked Q&A mentions. Then you don't need a library crate at all. The reason that #[path = ...] is bad practice, is because its easy to get wrong and will deceive those unaware of how it actually works.
The reason you would get an "unused" warning from your common functions is because it is essential duplicating the file. Within the specific binary that it is imported, the function may truly not be used. Even if it were used in the other binary, that wouldn't be considered when emitting that warning.
Related
This question already has answers here:
How do I do a basic import/include of a function from one module to another in Rust 2015?
(3 answers)
Split a module across several files
(7 answers)
How can I include a module from another file from the same project?
(6 answers)
Closed 5 years ago.
How do I include a file with the full path my_project/src/include_me.rs in main.rs?
I've checked the dependencies guide, and all of them appear to be including a binary. I've also checked "The Book", but none of the examples there end in ".rs" either.
How do I make include_me.rs compile with the rest of the project?
There are basically two (main) ways in Rust to include code from somewhere else:
1. "Including" internal code
If your include_me.rs belongs to your project, you should move it to the same folder main.rs lies in:
└── src
├── include_me.rs
└── main.rs
Then you can write this in your main.rs:
mod include_me;
fn main() {
// Call a function defined in the other file (module)
include_me::some_function();
}
A mod declaration makes the Rust compiler look for the corresponding .rs files automatically!
So everything that belongs to your project, belongs in the same folder (or a subfolder thereof) as the folder where main.rs (or lib.rs) is lying. The files are then "included" via the module system. To read a good introduction into modules, please read the chapter on modules in the Rust book. You could also check out Rust by Example on that topic. The module system is pretty central and thus important to learning Rust.
2. "Including" external code
If your include_me.rs is something that doesn't belong to your actual project, but is rather a collection of useful things you are using in multiple projects, it should be seen as a library. To include code of such external libraries, you have to include it as an extern crate. And to make your life easier, you really want to use Cargo!
So let's prepare your include_me.rs as Cargo library project. You need the following file structure (which is automatically generated by cargo new my_library --lib):
. my_library
├── Cargo.toml
└── src
└── lib.rs
Now copy all the contents from include_me.rs into lib.rs (it is just convention to call the root file of a library project lib.rs). Let's say that the my_library folder's full path is ~/code/my_library.
Now let's prepare your main project's Cargo project. It has a similar file
structure (but a main.rs instead of lib.rs, because it's a executable-project, not a library-project):
. my_project
├── Cargo.toml
└── src
└── main.rs
To declare your dependency on my_library, you have to put this into your Cargo.toml:
[package]
name = "my_project"
version = "0.1.0"
authors = ["you"]
edition = "2018"
[dependencies]
my_library = { path = "~/code/my_library" }
You can also use relative paths ("../my_library"), but it only makes sense if you know that the two projects always stay where they are, relative to one another (like if they are both managed in the same repository).
Now you can, in your main.rs, write:
use my_library::some_function;
fn main() {
// Call a function defined in the other file (extern crate)
some_function();
}
If you want to upload any of those two projects, you have to interact with crates.io (or another crates registry, if your company has one), but this is another topic.
(Note: some time ago, it was necessary to write extern crate my_library; inside main.rs. This is not necessary anymore, but you might find old code with extern crate declarations.)
Any other ways?
Yes, but you shouldn't use those. There is the include!() macro which allows you to verbatim include other files, just like the #include from C-land. However, it is strongly discouraged to use this in situations where the module system would be able to solve your problem. include!() is only useful in very special situations, often linked to a more complex build system which generates code.
Basic setup
A simple C function that is called from Rust via FFI, and a static library (Windows) that is linked in build.rs.
// api.c
int increment(int value) {
return value + 1;
}
// main.rs
extern {
fn increment(value: i32) -> i32;
}
fn main() {
let num = unsafe { increment(4) };
println!("The incremented number is {}", num);
}
// build.rs
fn main() {
println!("cargo:rustc-link-search=src/ffi");
println!("cargo:rustc-link-lib=static=api");
}
With this directory structure:
Cargo.toml
Cargo.lock
build.rs
src
+-- main.rs
+-- ffi
+-- api.c
+-- api.lib
Cargo.toml has nothing but a crate name, version, author and Rust edition.
So far, this works perfectly, and "The incremented number is 5" is printed as expected.
The problem
Now I add an empty lib.rs file, so I can use my crate as both a library and a binary:
Cargo.toml
Cargo.lock
build.rs
src
+-- lib.rs <-- NEW
+-- main.rs
+-- ffi
+-- api.c
+-- api.lib
I do not change Cargo.toml.
This alone makes the link step fail (MSVC linker):
error LNK2019: Unresolved external symbol "increment" referenced in function "_ZN16my_crate4main17h887bd80180495b7eE"
The error persists even if I explicitly specify the presence of both a library and binary in Cargo.toml and run using cargo run --bin my_main:
[lib]
name = "my_lib"
path = "src/lib.rs"
[[bin]]
name = "my_main"
path = "src/main.rs"
I also made sure that build.rs is still executed in the binary case, by letting it panic to abort the build.
I'm aware that I could solve it by splitting the crate, but I'd like to understand what exactly happens. So:
Why does the presence of an empty lib.rs cause the linker to fail?
And is there a way to make it succeed, in a single crate?
In the presence of both a binary and a Rust library in a single crate, Rust statically links the binary against the Rust library itself, as if the library were any other external crate (e.g. from crates.io). I assume to prevent errors due to duplicate symbols, or possibly just to avoid code bloat or extra work, it appears to avoid linking any external artifacts directly against the binary, and does not make an effort to determine if the code in question is even available in lib.rs before making this judgment. Normally this happens in the background without you noticing (i.e. it's doing this to the Rust standard library, your system standard libraries, and external crates as well for everything you compile), but when you introduce custom FFI dependencies it becomes obvious.
If you change your lib.rs to
extern {
pub fn increment(value: i32) -> i32;
}
And your main.rs to
fn main() {
let num = unsafe { libname::increment(4) };
println!("The incremented number is {}", num);
}
Where libname is either the name of your library as listed in Cargo.toml or the project name if that's not specificed, it will compile and run properly. Indeed, this follows no matter how complex you get. You can bury the FFI function a million modules deep and include it from wherever, but it will fail with the same error if you reference a definition of an extern "C" function without going through the master library crate first.
I don't know if there's any way (in build.rs or otherwise) to insist that Rust statically links a library either twice against the two different targets, or else only against the binary itself, but if there is it's not well documented or obvious.
That said, in most cases this is unlikely to be a problem, since if you're making your functionality usable as a library, it's likely you'll have the common code, including FFI, in there anyway. It could conceivably be an issue if you have a C dependency (e.g. for device access) that only makes sense in the binary, but I think that case is rare enough it's reasonable to force you to split the crates using workspaces if that's your specific situation.
I am creating pypi python package and for that i tired many solution avilable in different websites.
First of all I followed this Approach1, in this approach author has created class and functions but when tried this one my package gave me an error that no module and class name is not define
I tired many other Approaches like this one Approach2 in this approach author created only functions without class and __init__.py
and then tried this one also which is very common Approach3
and here is important files structure provided by pypi officials
├── LICENSE
├── README.md
├── example_pkg
│ └── __init__.py
├── setup.py
└── tests
I am ready with my package but after spending whole night finding which one is Best coding practice? and now got confused
so my questions are as follows
what is the use of file __init__.py?
best practice for giving directory, class and functions names so that it will help during importing.because according to these answers some people said import directory name, some said filename followed by class etc..
Can anyone provide me proper steps so that i can follow in upcoming packages also
I'd recommend following the Python Packaging Authorities guides and tutorials at https://packaging.python.org.
In addition, the PyPA publishes an example project skeleton that you can use as a starting point: https://github.com/pypa/sampleproject/
I'm am trying to re-export a submodule from another file from a parent module.
Give this file structure,
src/
- lib.rs
- module/
-- mod.rs
-- submodule/
--- mod.rs
...can this be accomplished in a single line?
// mod.rs
mod submodule;
pub use submodule::*;
// something like pub use mod submodule::*;
If I understand correctly, you're asking if the mod and the use can be combined into a single line.
No, there is no allowance for combining a mod with a use. The right way to think about mod is that it defines where the module is located in the module hierarchy. The fact that it also brings the module name into the namespace of the current module, as if you had written use some_module;, is essentially a part of that, since it would be pointless to ever write mod some_module without making it visible in its parent.
If you want to also make additional names available, there is no special syntax for combining it with a mod; you have to use use.
The compile time functions slurp/gorge/staticRead/staticExec seem to use the directory of the source file as working directory. In most cases that is the desired behavior, because the relation of the source code and compile time resources is fixed. But how can I use these functions in a library so that they refer to resources provided by users?
Example structure:
.
├── client
│ ├── client.nim
│ └── resource.data
└── library
└── library.nim
I want to provide a bundle function in the library, which allows the client to call something like bundle("resource.data"). Internally the library may use e.g. slurp(givenResourcePath). However this will fail, because slurp looks up resource.data relative to library.nim. Is there a way to use these functions and refer to files relative to the callsite?
Note: I tried to generate the AST performing the slurp with a template/macro, but even than the lookup is relative to library.nim.
The easiest way to handle this is to rely on a helper template that uses instantiationInfo to obtain the source path of the caller of the macro.
You can create a module called bundles.nim:
import os
macro bundleImpl(userPath, resource: static string): untyped =
let resourcePath = splitFile(userPath).dir / resource
echo "FULL RESOURCE PATH ", resourcePath
echo "FILE CONTENTS:"
echo staticRead(resourcePath)
template bundle*(resource: static string) =
bundleImpl(instantiationInfo(-1, fullPaths = true).filename, resource)
Then, you can use it from any module in the expected way:
import
bundles
bundle "test.txt"
The result on my system is something like:
FULL RESOURCE PATH /Users/zahary/nim/scratch/test.txt
FILE CONTENTS:
<test.txt contents>
This can be solved with a macro using a small trick: Looking up the implementation of slurp shows that it uses the lineinfo of the slurp AST node to determine its working directory. By default, constructing an AST with a macro attaches a lineinfo which refers to library.nim, and thus, slurp uses the library path. To modify the behavior, we can read the lineinfo from the callsite and attach it to the slurp node:
macro bundle*(resource: string): untyped =
# create slurp call node
var slurpCall = newCall(ident "slurp", newStrLitNode resource.strVal)
# forward callsite lineinfo to affect working directory behavior
slurpCall.copyLineInfo(resource)
# embed slurpCall somewhere in output AST
# ...