Use dependency of a dependency? [duplicate] - rust

I want to use the dijkstra function from the pathfinding crate:
pub fn dijkstra<N, C, FN, IN, FS>(
start: &N,
neighbours: FN,
success: FS
) -> Option<(Vec<N>, C)>
where
N: Eq + Hash + Clone,
C: Zero + Ord + Copy,
FN: Fn(&N) -> IN,
IN: IntoIterator<Item = (N, C)>,
FS: Fn(&N) -> bool,
To use it I need to implement the Zero trait from the num_traits crate. But how can I import Zero? An obvious way is to add extern crate num_traits; into my crate and fix my Cargo.toml appropriately. But in doing so, I have to watch a dependency of a dependency, which is not good.
Can I somehow implement Zero without explicit dependency on the num_traits crate, like below?
use pathfinding::num_traits::Zero;

Given the original intent of importing non-exposed dependencies from a crate (such as pathfinding) into a dependent project, that is currently not allowed. If a dependency is not re-exported by the crate, that makes it more of an implementation detail than part of the API. Allowing a dependent to access any "sub-dependency" would therefore be catastrophic.
In this case however, since num_traits is clearly used in the crate's public API, it also makes sense for the dependent to have access to it. As it is, you are expected to add the dependency in your own project, while taking care to keep a compatible version. Otherwise, cargo might end up building duplicate dependencies.
[dependencies]
num_traits = "0.1"
In order to avoid this, pathfinding would benefit from exporting its own num_traits, as below. PR #6 was created for this purpose, and has been merged into version 0.1.12 (thanks, #SamuelTardieu).
pub extern crate num_traits;
With that done, you can now do exactly as written at the end of your question:
use pathfinding::num_traits::Zero;

Related

Why can a no_std crate depend on a crate that uses std?

In the example, hs reexport HashSet from std. But it compiles without error or warning. Why?
#![no_std]
pub use hs::HashSet;
pub fn new() -> HashSet<usize> {
HashSet::new()
}
pub fn insert(a: &mut HashSet<usize>, v: usize) {
a.insert(v);
}
Well #![no_std] just means that you don't include std by default. It doesn't mean you can't explicitly or implicitly (i.e. through other crates) still include std. In other words #![no_std] does not prohibit using std but it disables using it by default.
For example this works, too:
#![no_std]
extern crate std;
fn main() {
std::println!("hello!");
}
Every crate in Rust is opt-in, that is, you need to explicitly ask it as a dependency to have it as a dependency of your code (usually through Cargo.toml). All, but one: the standard library (hence the name), which is opt-out. That is to say, if you do nothing, Rust will implicitly understand you require it as a dependency, and you explicitly need to tell it that you don't (with #![no_std]) to prevent this. Besides this, std behaves like any other crate, that is, even if it's not one of your direct dependencies, you can still access to reexports of your actual dependencies.
It should be noted, however, that there is no reason to not want to standard library as a dependency except in situations where you just don't have it (for instance, if you are writing embedded code, or a kernel, or ...). In this case, having std as a (transitive) dependency will just fail at compile time, which means that not only you have to use #[no_std], but so should all the dependencies you have, and their dependencies, and so on... In your case, you have annotated your code with #[no_std], but there was no hard requirement for you to do so, so there is no problem in having a dependency that has access to std.
Finally, note that some parts of the standard library are accessible even if the whole standard library is not. For instance, core is not OS-dependent, so you can use it even on embedded, and usually obtaining alloc one way or another (that is, implementing what is required yourself for instance) also provides you quite a lot of the standard library. In particular, I think HashSet only requires core and alloc to be available, not the whole standard library.

Substrate - cannot find type `Vec` in this scope

I'm trying to play around with the substrate node template example and add some custom structs around.
However, I get the following compilation error:
"cannot find type Vec in this scope"
And example of the struct being used (+ associated traits and where I'm importing them from) is:
use codec::{Encode, Decode};
use scale_info::TypeInfo;
// , WrapperTypeDecode, WrapperTypeEncode, TypeInfo
#[derive(Encode, Decode, Default, Clone, PartialEq, TypeInfo)]
#[cfg_attr(feature = "std", derive(Debug))]
pub struct Image {
// Note: Consider changing to JPG compression in the future
pixels: Vec<Vec<(u8, u8, u8)>>,
height: u32,
width: u32
}
But given that substrate requires quite a lot of bootstrapping I can't provide a working example.
The one thing I tried is adding #[cfg(feature = "std")] which afacit should have enabled either the rust std or the std defined in the Cargo.toml, I'm honestly unsure which. But it didn't help.
My main question is... how do I fix this? What imports am I missing where?
My secondary more important question is why would this be happening? Inside the https://docs.substrate.io/tutorials/v3/kitties/pt1/ tutorial vec! and Vec seem to be used without any issues or further additions... Or rather, the one addition I see is #[cfg(feature = "std")] being used before all impl blocks that use vec! but not before using Vec. Why is this the case? Is this simply allowing a block to import anything from the rust std? Or does it refer to the std defined in the Cargo.toml ? Why would it be necessary for vec! but not Vec ?
Have a look at CRUST parachain, https://github.com/paritytech/substrate-lfs or other web3 / IPFS off-chain storage mechanisms.
Generally you want to store as little state on-chain as possible. Storing the image data directly (even as a highly compressed jpeg) would probably not be a great design decision as it could get very expensive (unless your image sizes are very small).
First of all you don't want to enable std when you are working with substrate, as non-std is required by SCALE codec, this also enables wasm compilation. More on SCALE here
About Vec, you might be just lacking using some frame_support crate, not much that I can tell just for checking the struct you provided. But I will let you here the storage entry about runtime storage of the substrate docs.
And would recommend to compare your code with some pallet on the substrate repo so you can see from other pallets using Vec what you might be missing.
For instance, nicks pallet stores the nick itself as a Vec so that could be a good place to start looking.
Hope this is helpful enough.
What #[cfg(feature = "std")] does is "if the Cargo feature of my crate called std is enabled, include the code block below. This is a form of conditional compilation. Apparently substrate needs to build the code in no-std contexts too. The parts that need to run in no-std won't be able to use Vec.
I solved it by adding default-features = false, in my dependencies in my runtime/Cargo.toml
pallet-evm = { git = "https://github.com/paritytech/frontier", default-features = false, branch = "polkadot-v0.9.25" }

How can I use a trait implementation from another crate in the original crate?

I have a Rust project which divided among several crates in a workspace. One of these crates is a test crate which holds utilities for use in unit and integration tests within the other crates.
In one crate I define a trait that is implemented by a struct in the test crate. However, when I try to use the struct from the test crate in the original crate which defined the trait, I encounter errors when trying to use the trait's member functions.
Here is a short example:
In the project-trait crate:
trait Trait {
fn do_something(&self)
}
In the project-test crate:
use project_trait::Trait;
pub struct TestObject;
impl Trait for TestObject {
fn do_something(&self) {
// ...
}
}
Finally, back in the project-trait crate:
#[cfg(test)]
mod test {
use crate::Trait;
use project_test::TestObject;
#[test]
fn use_test_object() {
let object = TestObject;
object.do_something();
}
}
When running cargo test in the project-trait crate, I get error E0599 which says I should import project_trait::Trait to use the do_something method. It seems Rust doesn't see that crate::Trait and project_trait::Trait are the same trait.
Is there any workaround for this?
There can't be cycles in the crate dependency graph. Therefore, whatever you've actually done in your project configuration, it can't be that the #[cfg(test)] code that is depending on project-test is using the same crate::Trait as what project-test sees as project_trait::Trait.
You have probably done something that causes some of the code to be compiled as part of a different crate (such as misuse of mod so as to compile the same source file in two crates). The result of this is that you have two different traits that happen to both be named Trait. Thus, you get an error because the one that TestObject implements is not the same as the one you have imported.
(In order to get a more precise explanation of what's going wrong, you'll need to provide a more complete example — preferably reproducible, i.e. "if I copy these files into a local folder they will be sufficient to demonstrate the problem".)
In general, whatever code is needed to run project-trait's tests must be part of the project-trait crate or its dependencies — you can't split out test helpers into an independent library crate.

When to use crate:: vs proj_name:: in imports?

I understand that one can import functionality within the project in 3 different ways:
Using project name
use proj_name::mod1::mod2::fn_name;
Using crate
use crate::mod1::mod2::fn_name;
Using a relative path
use mod1::mod2::fn_name;
What I'm confused about is that it seems that sometimes the compiler wants me to use (1) and sometimes (2) / (3). I can't figure out the ruleset for which should be used when. Can someone help?
Only the crate itself is allowed to refer to itself as The crate. Every single dependant uses its name prefix. That includes binary targets within the same crate. They might be coupled together in the project, but they are NOT part of library crate.
Binary targets in fact are all separate crates for all purposes of the build system that work the same as libraries that have lib.rs at their root and writing crate in them refers to root of that specific binary target.
Demo:
#[derive(Debug)]
pub struct A;
mod b {
#[derive(Debug)]
pub struct B(pub crate::A);
}
fn main() {
println!("{:?}", b::B(A));
}
Playground

Consistent TypeIds for dynamically loaded crates

I have a plugin system where I pass &dyn Any to a dynamically loaded rust function, but downcasting the reference fails because the TypeIds differ (for the same type), although I added rustflags = ["-Cmetadata=12345678"] to both crates' cargo config. Also it seems as if only types from external crates are affected (I tried () and it yielded the same TypeId in both crates). I am currently casting the raw pointers (unsafe { &*(v as *const dyn Any as *const Type) }) to work around this issue, but I would prefer a solution without unsafe code.
For example the following code:
println!("CRATE 1: TypeId of `()`: `{:?}`, TypeId of `toml::Value`: `{:?}`",
TypeId::of::<()>(), TypeId::of::<toml::Value>());
produces this output:
CRATE 1: TypeId of `()`: `TypeId { t: 7549865886324542212 }`, TypeId of `toml::Value`: `TypeId { t: 9270396907601429078 }`
CRATE 2: TypeId of `()`: `TypeId { t: 7549865886324542212 }`, TypeId of `toml::Value`: `TypeId { t: 5704635987193303200 }`
EDIT:
This does not seem to be a problem with different dependency versions, as crate 2 (which is dynamically loaded) depends on crate 3 (which is also dynamically loaded) and the problem still occurs, although both, crate 2 and crate 3, are local dependencies (so there is only one version). Crate 1 is btw. the crate that loads crate 2 & 3.
EDIT:
I removed the -Cmetadata option from all 3 crates and now I get the same TypeId for toml::Value, however the TypeId of a type in crate 1 that I want to downcast in crate 2 still differs.
After some testing I found out that the TypeIds are different because the loader crate (crate 1) is used as a library in the other 2 crates but executed as a binary.
To work around the issue I extracted all of crate 1's types I wanted to use in the loaded crates to a new crate and added it to each crate's dependencies. This new crate is only ever used as a library and not a binary, thus the TypeIds should be consistent.
A summary of everything I had to do to get it working:
use the same toolchain version for all crates
use the same dependency versions in all crates
do not use -Cmetadata, this doesn't work anymore and has in fact the opposite effect
extract all types that are used in the loader crate and the loaded crates into a new crate and add it to all crates' dependencies
whenever you change something in this 'common types crate' you have to recompile all the other crates, so the TypeIdS are up to date
If you still have a problem you can fall back to unsafe rust (e.g. unsafe { &*(value as *const dyn Any as *const Type) }). Note that there are no checks at all, so you might encounter segmentation faults if the types don't match.
Special thanks to #trentcl for pointing me in the right direction.

Resources