What is the intended/planned way of configuring/installing software that uses Rust Cargo as build system? - rust

Existing build systems usually have some kind of install targets, that is used either manually (for installing in /usr/local or other location that user can access) or automatically (by package build systems of binary based distros or by package managers of source based ones).
What is the intended way of installing software that uses Cargo? How an analog of make install should look like?
Cargo itself uses additional configure/make stuff that handles configuration, detection of system dependencies, running cargo build and installation.
Is this the right way for any other software built with Cargo? It means are there plans to cover this tasks by Cargo itself or is Cargo intended only as a tool for dependency fetching and compilation without any configuration/detection of installed deps/installation?
Or are any plans to add this functionality?

cargo install
As of Rust 1.5 you can use cargo install to install binary crates onto your system. You can install crates from:
crates.io (the default), using cargo install crate_name
any git repository, using cargo install --git repository_url
any directory, using cargo install --path /path/to/crate
The first two have additional options you can specify:
With crates.io, you can use --vers to specify the crate version.
With git repositories, you can use --branch to set the branch to install from, --tag to specify the tagged release to use, and --rev to build from a specific commit.
Installation location:
cargo install can be configured to install in a custom directory through the following methods, in order of precedence (highest first):
By passing --root /path/to/directory (this path can be relative)
By setting the $CARGO_INSTALL_ROOT environment variable
By setting the install.root configuration key
By setting the $CARGO_HOME environment variable (which will affect more than the installation directory of cargo install)
If none of the above are present, cargo will install the crates in ~/.cargo/bin.
In all of the above cases, the output files will actually be placed in the bin subdirectory (e.g. --root /path/to/directory will actually place ouput in /path/to/directory/bin).
Uninstallation
cargo uninstall can be used to remove previously-installed crates. If you have multiple crates with the same name installed, you can specify --root to only remove the version in that directory.
Example: I want to use rustfmt:
I can use the version on crates.io:
cargo install rustfmt
I like using original releases:
cargo install rustfmt --vers 0.0.1
I want it installed in /opt/rust_crates:
cargo install rustfmt --root /opt/rust_crates
I really need to use the bleeding-edge version:
cargo install --git https://github.com/rust-lang-nursery/rustfmt.git
The latest commit has a bug in it!
cargo install --git https://github.com/rust-lang-nursery/rustfmt.git --rev f5bd7b76e0185e8dd37ae6b1b5fb5e11187f0b8c
I truly desire the version that uses git submodules for its dependencies:
cargo install --git https://github.com/rust-lang-nursery/rustfmt.git --branch submods
I've cloned it and made some edits:
cargo install --path ~/my_rustfmt
Actually, I insist on doing my formatting entirely manually:
cargo uninstall rustfmt

(This answer is intended for developers who want to distribute their own programs, not users who have received a Cargo project and need to install it on their own system; That's the domain of Toby's answer).
In addition to Toby's Answer:
Cargo's install feature is not meant to be the primary way to distribute Rust programs. It's designed to be used only for distribution to other Rust developers. There's several drawbacks to using this feature:
Cargo requires end-users to install the entire Rust toolchain first.
Users will have to build the program locally, which can be slow, especially in Rust.
There's no support for upgrading programs once they're installed (without an additional tool).
There's (currently) no way to include assets, such as documentation.
The packages will be installed only for the current user unless flags are passed to cargo install.
In other words, cargo install is the make && sudo make install of Cargo programs; It's not the ideal way to distribute a Rust program, unless it's intended primarily for Rust programmers.
So what is the correct way?
Let's look at the alternatives.
Manually distribute a tarball/zip
You can replicate the effects of cargo install by simply using cargo build --release. This will place a (mostly, see the drawbacks below) statically linked crate binary in target/release/crate_name, and this can be repackaged into a .tar.gz or .zip and given out to other users.
Pros:
Doesn't require users to install Rust or build the program themselves.
Allows developers to copy assets into the tarball/zip and distribute them along with the program itself.
Cons:
Installing a .tar.gz/.zip is nonstandard and generally not considered ideal for most users.
If the crate needs any system dependencies beyond libc, it will fail to load them with a difficult to understand error.
This requires a developer to manually build a package to release for each version and platform combination.
Use a CI service to build releases
It's possible to recreate any of these methods using a cloud-based CI service. For example, using Travis CI, you can use the Trust project to automatically deploy in much the same way that you would from a tarball, but automatically with only a tag being required.
Pros:
(All of the advantages of a tarball, plus)
The developers don't have to manually release the program, they just need to tag a release.
As a side effect, it's possible to build for every package the program supports at once.
Cons:
The process can be frustrating to debug if it doesn't work correctly, because there's limited control over the server.
The build process is tied to a service, which means releases can be missed if the service is down when they are released.
With Trust or similar tools, you're still ultimately distributing a .tar.gz/.zip, which means there's still inconvenience for users and a lack of system dependency management.
In addition to Travis, see Appveyor and GitHub Actions as possible build platforms.
Provide a package
This is considered the ideal method for many end users, and is the standard way to distribute any program, not just Cargo programs. This alleviates almost every issue with the tarball approach, though not without some problems of its own.
Pros:
Included in the system like any other program.
Can be submitted to Linux distribution repositories to allow programs to be installed in only one command.
Allows updating, removal, and asset inclusion.
Tracks system dependencies, which is especially helpful for GUI apps.
Cons:
By far the most complex of these options.
Requires building a package separately for every supported platform (this can be alleviated with CI, but it will be even more complex to setup this way.)
This approach is best handled with additional tools:
cargo-deb: Build a package for Debian and Ubuntu.
cargo-rpm: Build a package for Fedora, Red Hat, and CentOS.
cargo-aur: Build a package for Arch Linux.
cargo-wix: Make a Windows Installer package.
These usually are tools that are meant to be run by developers, to create the files that are used to generate packages. See their own documentation for more information.
Source: https://rust-cli.github.io/book/tutorial/packaging.html

Related

Why do cabal configure?

In the Cabal User Guide it says that Cabal is often compared with autoconf and automake since the command line interface for actually configuring and building packages follows the same steps steps:
./configure --prefix=...
make
make install
compared to
cabal configure --prefix=...
cabal build
cabal install
My understanding is that ./configure uses a config file (produced by autoconf) to adapt the make process to the environment in which it will run and also to check dependencies. So ./configure therefore always have an "input" to conform to. But if cabal configure is not given any arguments what does it do, and why is it necessary before running cabal build?
The cabal configure step does at least two things I know of:
Check that the package description parses OK.
Check that all required dependencies are already installed (and report an error if not).
Basically it's running the constraint solver to decide exactly which packages you're going to build against. (E.g., if you have several versions of ByteString installed, which version are you going to use? Well it might depend on which version the packages you depend on are expecting...)
Also I believe it's possible to supply options at configure time which change exactly which features of the package get built (but I don't have experience with this).
I think originally you had to call configure before you could call build, but I believe now the cabal command-line tool does that step for you automatically in many cases. (E.g., cabal run now seems to automatically reconfigure if the package description file is newer than the configuration DB.)

Should end user utilities/applications be registered on crates.io?

Is it acceptable to register generally useful (utilities / applications) on crates.io?
The FAQ doesn't address this and from browsing, there are examples of end-user applications (mostly command line tools).
Or is crates.io? meant only for libraries?
I'm asking this because the documentation hints at library use, semantic versioning for API's etc. but doesn't reference the case for packaging applications explicitly.
Yes, because you can use cargo install to install and manage those applications system-wide. If this use were discouraged, I would suspect that command to not exist at all, or at least have a very limited applicability.
Snippet from cargo install --help:
Usage:
cargo install [options] [<crate>]
cargo install [options] --list
[...]
This command manages Cargo's local set of installed binary crates.
Only packages which have [[bin]] targets can be installed, and all
binaries are installed into the installation root's bin folder. The
installation root is determined, in order of precedence, by --root,
$CARGO_INSTALL_ROOT, the install.root configuration key, and
finally the home directory (which is either $CARGO_HOME if set or
$HOME/.cargo by default).
There are multiple sources from which a crate can be installed. The
default location is crates.io but the --git and --path flags can
change this source. If the source contains more than one package (such
as crates.io or a git repository with multiple crates) the <crate>
argument is required to indicate which crate should be installed.
This should not be the primary reason to publish an application to crates.io, but I'm listing it here because it's still a good reason. :)
The Rust team will occasionally use a tool called crater to check for regressions on all crates published on crates.io, usually before merging a pull request that has uncertain consequences. If you wrote some code that happens to compile today but would stop compiling1 due to a bug fix in the compiler, then they may even submit a pull request to your project that fixes your code!
1 Usually, when such breaking changes occur, there's at least one prior release in which a warning will be reported before the warning is turned into an error.

How to build a project using Cargo in an offline environment?

I have a laptop with an OS X system which can access the internet, and a Linux server which can not access internet for some security reason.
I want to build a Rust project with some dependency on the Linux server. Because the Linux server can not access internet, I run the cargo build command on the OSX laptop and download the dependency source file to the .cargo directory and then copy these files to the Linux server and put them into /root/.cargo directory.
I made the file structures the same, but when I run cargo build on the Linux server, it still tries to connect to this website and the build fails. The cargo build command always tries to connect internet although the dep source files are already in the .cargo directory.
How can I build a Rust project with dependencies in an offline environment? Why does copying the source file of the dependencies not work?
Good News! As of Rust 1.37, you can use Cargo's own vendor command to download and bundle your crate's dependencies in the crate itself:
First, run cargo vendor. This will setup a new directory named vendor in the root of your crate. It will then download dependencies from crates.io and git, and store them in this new directory.
When cargo vendor is done downloading all the required dependencies, it will print a set of instructions that you'll need to follow. At the time of this writing, you only need to copy a few lines to .cargo/config.toml. Note that config.toml is relative to the root of your crate and is not the one in your home directory.
Once you're done, your crate will be completely self-contained as far as dependencies are concerned. You can couple this approach with Rust's offline installers to build Rust programs completely offline.
For Rust 1.37+ see: https://stackoverflow.com/a/65254702/147192
The short answer is: up to 1.37 (excluded), it's complicated.
The long answer is that cargo will attempt to connect to github in order to check the index of the crates that the Cargo.toml file references.
I recommend you to check the cargo-vendor prototype to solve this issue (by aturon, a member of the Rust tooling subteam), and otherwise you could look at how some people created a mirror for crates.io in order to avoid the dependency on Internet.
There is a demand for Rust builds not to require Internet, and people working on it, however there is no blessed solution for now.
Check out romt - Rust Offline Mirror Tool.
Romt (Rust Offline Mirror Tool) aids in using the Rust programming language in an offline context. Instructions and tooling are provided for:
Mirroring of Rust ecosystem artifacts:
Toolchains (Rustc, Cargo, libraries, etc.)
Rustup (toolchain multiplexer)
Crates.io (community-supplied Crates)
Incremental artifact downloading.
Incremental artifact transfer to offline network.
Artifact serving in offline context (offline computer, disconnected network).
I could replace official crates.io registry use by adding this
.cargo/config file in my project (under windows %USERPROFILE%\.cargo\config seems ignored) :
[source]
[source.mirror]
registry = "http://localhost:8185/auser/crates.io-index.git"
[source.crates-io]
replace-with = "mirror"
Also works using a file based git registry clone:
registry = "file://c:/github/crates.io-index.git"
downloaded either using git clone --bare or --mirror
cargo build now print
Updating 'c:\github\crates.io-index.git' index
instead of Updating crates.io index

How to to build src from a CygPort?

I have a question about the structure of the source code from a cygport package.
Here is the contents of a Cygports source file:
the actual source bundle for the project (tar.gz, tar.bz2, etc.)
the any number of *.patch files.
a .cygport file
I am trying to build gedit-3.4.2 from cygports repository.
How does the .cygport file help me run the proper options in the ./configure ?
For instance, in gedit if i don't specify --disable-spell it won't proceed due to error. How do I get the list of ./configure options that were used to build the project when the cygport was built?
Is there some way we can use the cygport executable to build the cygport and change the prefix too?
Here is the contents of gedit-3.4.2-1.cygport:
inherit python gnome2
DESCRIPTION="GNOME text editor"
PATCH_URI="3.4.2-cygwin.patch"
DEPEND="gnome-common gtk-doc
girepository(Gtk-3.0)
pkgconfig(enchant)
pkgconfig(gtksourceview-3.0)
pkgconfig(libpeas-gtk-1.0)"
PKG_NAMES="${PN} ${PN}-devel"
PKG_HINTS="setup devel"
gedit_CONTENTS="--exclude=gtk-doc --exclude=libgedit* etc/ usr/bin/ usr/lib/gedit/ ${PYTHON_SITELIB#/} usr/share/"
gedit_devel_CONTENTS="usr/include/ usr/lib/gedit/libgedit* usr/lib/pkgconfig/ usr/share/gtk-doc/"
DIFF_EXCLUDES="*.desktop.in *.schemas.in *-marshal.h"
CYGCONF_ARGS="--libexecdir=/usr/lib --enable-python"
KEEP_LA_FILES="none"
EDIT Someone from Cygwin Ports mailing list said:
"The configure options are
--libexecdir=/usr/lib --enable-python
Which is from CYGCONF_ARGS."
Here is the contents of a Cygports source file:
You'd do better to think of it as a Cygwin package source file.
cygport is simply a tool for automating the creation of Cygwin binary and source packages. It is the primary tool available, but unlike with some other packaging systems, there's really nothing forcing you to use it. It is quite possible to build a Cygwin package entirely by hand, since it is really nothing more than a tarball that Cygwin's setup.exe can blindly unpack into the Cygwin root directory (typically c:\cygwin) with the expectation that this will put the package's files in sensible locations.
Before cygport existed, people did build their own ad hoc packaging systems. Many Cygwin package maintainers still use these tools they created. (Yours truly included; two of my three packages use cygport, but the third still uses a custom build system.)
Ultimately, you want to read the cygport manual, in /usr/share/doc/cygport/manual.html.
(Yes, I know, "RTFM" answers are frowned on here. But, as one who currently maintains two cygport based packages in the official Cygwin package repository, please believe me when I tell you that the manual is still the single best resource available on this topic.)
How does the .cygport file help me run the proper options in the ./configure ?
As you found out through other resources, you'd first need to edit the CYGCONF_ARGS value in the .cygport file.
The simplest possible step after that is cygport gedit-3.4.2-1.cygport all. That attempts to rebuild all the binary packages in a single step. It also builds a new source package containing updated .cygport and patch files.
If something breaks in the all build process, it is usually faster to switch to using the sub-commands contained by all instead of completely restarting the process. The all step just runs prep, compile, install, package, and finish for you, in that order. For instance, if all fails during the compilation step, there's probably no need to repeat the prep step.
(It is exceptionally uncommon for cygport or a sane build system to wreck the build tree, forcing you to re-run prep. Far more commonly, you end up needing to re-do prep when you manually wreck the build tree while trying to get a new package to build for the first time and need to start over.)
For instance, in gedit if i don't specify --disable-spell it won't proceed due to error.
You can probably fix that by installing the libaspell-devel package from the official Cygwin package repository with setup.exe.
Personally, I wouldn't disable any feature unless it meant installing unofficial packages, such as those from the Cygwin Ports project.[*] It is nice to have Cygwin Ports repository, but because it contains so many packages, installing one can end up creating an "install the world" situation: package A depends on packages B, C and D, and C depends on E, F, G, H, and G depends on I, J, K, and... Dependency hierarchies within the Cygwin package repo tend to be flatter and narrower than those in the Cygports repo.
Is there some way we can use the cygport executable to build the cygport and change the prefix too?
You have guessed that you just add --prefix=/my/private/program/tree to CYGCONF_ARGS, I trust.
[*] If you are feeling confused about "Cygwin Ports" and cygport, the naming similarity is no coincidence. cygport is a tool created by Yaakov Selkowitz for himself when creating the Cygwin Ports package repository. Later, it became popular enough among other Cygwin package maintainers that it pushed out most of the competing build systems.

Building rpm, overriding _topdir, but getting BuildRequires deps?

I have a libfoo-devel rpm that I can create, using the trick to override _topdir. Now I want to build a package "bar" which has a BuildRequires 'libfoo-devel". I can't seem to find the Right Way to get access to the contents of libfoo-devel without having to install it on the build host. How should I be doing it?
EDIT:
My build and target distros are both SuSE.
I prefer solutions that don't require mock, since I believe SuSE does not include it in its stock repo.
Subsequent EDIT:
I believe that the answer I seek is in the build package. Perhaps it's SuSE's answer to mock? Or it's the distributed version of the oBS service?
DESCRIPTION
build is a tool to build SuSE Linux
RPMs in a safe and clean way. build
will install a minimal SuSE Linux as
build system into some directory and
will chroot to this system to compile
the package. This way you don't risk
to corrupt your working system (due to
a broken spec file for example), even
if the package does not use BuildRoot.
build searches the spec file for a
BuildRequires: line; if such a line is
found, all the specified rpms are
installed. Otherwise a selection of
default packages are used. Note that
build doesn't automatically resolve
missing dependencies, so the specified
rpms have to be sufficient for the
build.
Note that if you really don't need libfoo-devel installed to build package bar the most sensible alternative would be to remove libfoo-devel from the BuildRequires directive (and maybe put the requirement where it belongs).
However, if you cannot do that for some reason, create a "development" rpm database. Basically it involves using rpm --initdb --root /path/to/fake/root. Then populate it with all of the "target packages" of your standard distro installation.
That's a lot of rpm --install --root /path/to/fake/root --justdb package-name.rpm commands, but maybe you can figure out a way to copy over your /var/lib/rpm/* database files and use those as a starting point. Once you have the alternative rpm database, you can fake the installation of the libfoo-devel package with a --justdb option. Then you'll be home free on the actual rpm build.
If neither mock nor the openSUSE Build Service are a viable choice then you will have to buckle down and install the package, either directly or in a chroot; the package provides files that the SRPM packager has decided are required to build, and hence is in the BuildRequires tag.

Resources