I am creating a library, that is nearly close to its first release, so I would like to upload it to crates.io. Library has a multi-crate design, so I ended with something like:
- CrateA
- CrateProcMacros
- CrateC
- CrateD
- CrateE
- CrateF
- Cargo.toml (handles the workspace)
- Cargo.lock
...
where CrateA is the parent of the other crates, and has dependencies on another of those local crates, and some of those crates also depends on another ones. I mean, it's the primary crate of the library, the one responsible for exposing the public API of the project, and the unique one that I would like to be published in crates.io.
Reading the cargo docs I am seeing that I won't be able to publish a unique crate to the registry. All will be uploaded and published.
So, what alternatives I have to only publish my CrateA to the registry? Should I change my project's structure, and move to CrateA all the other packages and then try to publish it? Or there's some way to achieve this?
EDIT
CrateA have direct dependencies on another crates. An those others also depends on another one inside my workspace.
The way Cargo packaging works is that you are publishing your source code nearly unchanged. There is no pre-compilation step. There is no step where multiple library crates are gathered into one package. The only way to publish your CrateA is to publish all of its dependencies too.
There is interest in making a multi-crate project easier to publish, but for now, you've got to do it all explicitly.
Make sure each package in your project declares a [package] name that makes sense in public. (The name of the directory you keep it in doesn't matter.) It's common to have names like myproject-partoftheproject, where the package people actually use normally would be named myproject.
Make sure that each dependency declaration has a version number (not just a path) matching what you're going to publish. (You don't have to remove the path; that will be done for you within publication.)
Publish each package. You must do this in reverse dependency order — that is, CrateA last.
No one will mind that you've published extra packages that aren't meant for direct use — for example, lots of libraries necessarily have separate proc-macro packages. Though, if you have any crates that are really just for code organization and don't have any particular benefit, you could consider making them into modules inside fewer crates.
master Cargo.toml should be like this
[workspace]
members = [
"CrateA",
"CrateB",
...
]
And CrateA/Cargo.toml should be like this
[package]
name = "Foo"
version = "0.0.0"
edition = "2021"
authors = ["Foo <Foo#gmail.com>"]
license = "Bar"
description = "Baz"
[dependencies]
CrateB = { path = "../CrateB", version = "0.0.0" }
CrateC = "0.0.0"
Related
Let's say we want to add jsonwebtoken 8.2.0 to our Cargo.toml. That crate is dependent on ring 0.16.20, which is dependent on the web-sys 0.3.60 crate and that dependency is declared like this:
[target.'cfg(all(target_arch = "wasm32", target_vendor = "unknown", target_os = "unknown", target_env = ""))'.dependencies]
web-sys = { version = "0.3.37", default-features = false, features = ["Crypto", "Window"] }
For my project I don't need any JS/Wasm-related stuff whatsoever. But I noticed in my Cargo.lock several such dependencies were added as a result of that web-sys crate (such as wasm-bindgen for example). Is there a way to avoid that "noise"?
The Cargo.lock decides versions to be used for compilation on all possible platforms and to my knowledge, there is no way to override that.
Consider: Not doing so would be a headache. If someone on a new platform cloned your repo and built your crate, they'd have to have their Cargo.lock modified (and send you a patch?). If you updated a crate dependency on your system it would only update the locked versions for your platform, other people might end up with an inconsistent lock. To avoid the headache, just let cargo do with the Cargo.lock as it wants and ignore the contents.
If you want to reduce the noise a bit and hide the lock file diffs from git output, you could mark Cargo.lock as a binary file like shown here:
echo >>.gitattributes Cargo.lock binary
(though I don't know whether that won't have weird effects on windows line endings.)
Background
I'm currently in the process of writing a crate that's bindings for a C library and I need the user to specify where the built library is. Previously, I've seen this handles by the llvm-sys crate using environment variables. However, having used this a lot it can become quite cumbersome to have to write this every time I want to run a project.
similar questions
I found this post that asked a similar thing, but seemed to only be able to use specified tags, so not able to specify a path.
What I want to do
I would like to know if there's any way I can have a specific custom value in the Cargo.toml file. The idea being the crate I'm writing would require a specific value (I'll call it example here) that can take any string value. so when this crate were to be used in another project, that projects Cargo.toml file would be similar to this::
[package]
name = "some-other-project"
version = "0.1.0"
edition = "2021"
[dependencies.my_crate]
version = "0.1.0"
example = "something goes here"
This value would then be accessible by the crate I'm writing (the dependency in regards to the above manifest file) and usable in some way.
You can set environment variables using a Cargo configration file (.cargo/config.toml) then you can retrieve them in downstream crates by using e.g. env!()/option_env!()/build scripts.
I primarily want to use Debian's Rust packages, rather than fetching some random code from the wider Internet (I'm old-fashioned, I know, let's not get into that part). To this end, my ~/.cargo/config.toml looks like
[net]
offline = true
[source]
[source.crates-io]
replace-with = "debian"
[source.debian]
directory = "/usr/share/cargo/registry"
This works great after I install the librust-*-dev packages that I desire. However, in some specific projects, I'd like to lift this rule and tell Cargo "hey, you can in fact go wild and get whatever you want from crates.io". According to the Cargo book, a project-specific /project/.cargo/config.toml should take precedence over my user one. Assume this project-specific .cargo/config.toml:
[net]
offline = false
[source]
[source.crates-io]
I'm still not able to cargo build a project with dependencies from outside of my replacement source. If for example, I make a Cargo.toml that depends on yew (a randomly chosen crate that I know isn't available in my replacement source) I get
$ cargo build
error: no matching package found
searched package name: `yew`
What am I misunderstanding about Cargo's sources, replacement and per-project overrides?
The answer suggested by #blackgreen is one possible workaround for the underlying problem until issues 10045 and 10057 (or a combination thereof) are solved. Another, perhaps slightly less ugly, workaround follows below for those who need it.
I ended up working around the problem using UnionFS (I guess the more modern OverlayFS should work well too).
I simply add
[source.crates-io]
replace-with = "union"
[source.union]
directory = "/home/gspr/.cargo-overlay/union-registry"
to my ~/.cargo/config.toml and then do
unionfs -o ro /usr/share/cargo/registry:/home/gspr/.cargo-overlay/local-registry /home/gspr/.cargo-overlay/union-registry
Now /home/gspr/.cargo-overlay/union-registry reflects the union of /usr/share/cargo/registry and /home/gspr/.cargo-overlay/local-registry, with priority to the former in case of conflicts.
So what goes in ~/.cargo-overlay/local-registry? Individual extra crates, in the same way as in Debian's /usr/share/cargo/registry. That is to say, directories named cratename-version as they are distributed by upstream – but with a single extra file, namely .cargo-checksum.json added to them. The content of that extra file can be extracted from the crates.io index as follows.
Suppose we have cloned the crates.io index into ~/.cargo-overlay/crates.io-index, i.e.
git clone https://github.com/rust-lang/crates.io-index.git ~/.cargo-overlay/crates.io-index
Then suppose we've extracted a crate foo at version 0.1.2 into ~/.cargo-overlay/local-registry/foo-0.1.2. We can generate the missing .cargo-checksum.json like so:
cd ~/.cargo-overlay
index_file=$(find crates.io-index -type f -name foo)
cksum=$(jq -r "select(.name == \"foo\" and .vers == \"0.1.2\" ) | .cksum" ${index_file})
jo package="${cksum}" files="{}" > local-registry/foo-0.1.2/.cargo-checksum.json
It looks as if you are suffering from this issue: https://github.com/rust-lang/cargo/issues/8687
You would like to unset a config key on a upper-level config.toml but this is not supported.
I've played a bit with the config, and the only way I got it to work was to overwrite in the project-local config.toml the properties that were set in the upper-level config.toml.
In your case your upper-level config.toml specifies replace-with, so you have to overwrite that. But you can't overwrite it with crates-io, which is the registry you want to use, because that is exactly the registry with the replace-with key.
So until the above issue gets acted upon, we have to, essentially, use a mirror, both in the config and as an actual registry to download from:
[net]
offline = false
[source]
[source.crates-io]
replace-with = "crates-io-mirror"
[source.crates-io-mirror]
registry = "https://gitlab.com/integer32llc/crates.io-index"
As we both tested, it seems it's not possible to reuse the normal crates.io registry url because that is already defined and will fail with:
error: source crates-io-mirror defines source registry https://github.com/rust-lang/crates.io-index, but that source is already defined by crates-io note: Sources are not allowed to be defined multiple times.
So instead the URL above is an actual mirror server of crates.io. Then you can run cargo build successfully in the local project.
The recently released Cargo 1.56 adds a feature that should let one do what my question asks for: patch tables can now be specified in a project-specific .cargo/config.toml, which means that [patch] stanzas can now be introduced outside of Cargo.toml. That should do the trick! I haven't yet verified this, as I am stuck with an older Cargo for a little while still.
Could there be any future problem in naming a dependency .rs for example,
[dependencies]
gccjit.rs = { git = "https://github.com/swgillespie/gccjit.rs.git" }
In the above code, I use .rs for something which is not a Rust source code file. Is that not ideal or is it okay because it would be easier to default to the same name as the repository?
If an object is named .rs it could maybe be automatically recognized as Rust source code but in this case it is not.
Yes, there is a problem. Do not do this.
How you could have determined this for yourself
Try to use the code that you've proposed, you'll see:
$ cargo build
error: failed to parse manifest at `.../Cargo.toml`
Caused by:
could not parse input as TOML
Caused by:
expected an equals, found a period at line 9
You could have also attempted to create a package with the same name. You would have then seen:
$ cargo new 'gccjit.rs.git'
error: Invalid character `.` in crate name: `gccjit.rs.git`
use --name to override crate name
Who controls crate names
You don't get to control the name of an imported crate that way; the crate determines it and it's already picked one:
[package]
name = "gccjit"
If you want to rename an existing package on import, you have to use the package key to match the real name:
some_name = { package = "gccjit", git = "https://github.com/swgillespie/gccjit.rs.git" }
See How to idiomatically alias a crate in Rust 2018? for more.
To use a period in the name, you can seemingly use a string key (although I think this is a bug):
"gcc.jit" = { package = "gccjit", git = "https://github.com/swgillespie/gccjit.rs.git" }
However, a package name has to be a valid Rust identifier, which periods are not. If you do this, there's no way to use the package.
It's redundant anyway
More opinion based, such a name is completely pointless. You don't need to say "rs" or "rust" in the name because of course it's Rust code. If it wasn't, you couldn't use it as a dependency in the first place.
We don't call our packages "computer-source-code-awesome-tool" for the same reason — that much is implied.
Package names are different from source control
Cargo and Rust don't care what the name of your source control repository is. It's separate from the package name. While it's better to have them be somewhat similar, there's no real reason they have to be whatsoever related.
Package names are different from library names
This is a feature with a very small number of uses, but the name of your package (a.k.a. name of the crate on Crates.io) can be different from the name of your library (what is imported into the code).
Piston is the biggest "offender" of this that I know of:
[package]
name = "piston2d-graphics"
version = "0.30.0"
[lib]
name = "graphics"
Please do not use this as it's simply maddening to attempt to debug.
A few days ago, cross-compiling to JavaScript via Emscripten has finally hit nightly. I wanted to compile a project using glium in that manner. However, there are still many Emscripten-related bugs in many crates. While maintainers usually fix those bugs quickly, they don't necessarily release those bug fixes to crates.io immediately.
In my case, glium depends on glutin. glutin had a bug which is fixed now, but only in the git repository, not on crates.io. Note: glutin is not a direct dependency of my project; only an indirect one through glium!
How do I tell Cargo to use the glutin repository as source for glutin instead of crates.io?
You can use the [replace] section in your project's Cargo.toml. You can find the documentation about that feature here in the Cargo documentation.
In your case, glium depends on glutin 0.6.1. The version 0.6.1 on crates.io still contains the bug. So just add this to your Cargo.toml:
[replace]
"glutin:0.6.1" = { git = 'https://github.com/tomaka/glutin' }
Note however,
[...] that the replaced crate must not only have the same name but also the same version.
But even in the case of a version-mismatch (the repository already contains a newer version), you could still be in luck if the crate's maintainer creates git tags for every version (many in the Rust community do). In that case you can just specify the tag:
[replace]
"glutin:0.6.1" = {
git = 'https://github.com/tomaka/glutin'
tag = 'v0.6.1'
}
Sadly, this won't work with glutin, because the maintainer did not create tags for every version. In that case you can simply find the last commit before the version was bumped and specify it with rev = 'b4a3d0...' or specify a specific branch with the branch = '...' key.