Can Cargo download and build dependencies without also building the application? - rust

Question

Is there a way to tell Cargo to install and build all my dependencies, but not attempt to build my application?
I thought cargo install would do that, but it actually goes all the way to building my app too. I want to get to a state where cargo build would find all dependencies ready to use, but without touching the /src directory.
What I'm really trying to accomplish:
I'm trying to build a Docker image for a Rust application, where I'd like to do the following steps:
Build time (docker build .):
import a docker image with rust tooling installed
add my Cargo.toml and Cargo.lock files
download and build all dependencies
add my source directory to the image
build my source code
Run time (docker run ...):
run the application
I've tried the following Dockerfile, but the indicated step builds my application as well (which of course fails since the source directory isn't there yet):
FROM jimmycuadra/rust
ADD Cargo.toml /source
ADD Cargo.lock /source
RUN cargo install # <-- failure here
ADD src /source/src
RUN cargo build
ENTRYPOINT cargo run
The reason I want to separate the install dependencies step from actually building my application, is that if I don't change the dependencies, I want Docker to be able use a cached image with all dependencies already installed and built. Thus, I can't ADD /src /source/src until after installing the dependecies, as that would invalidate the cached image when I change my own code.

Answer 1

There is no native support for building just the dependencies in Cargo, as far as I know. There is an open issue for it. I wouldn't be surprised if you could submit something to Cargo to accomplish it though, or perhaps create a third-party Cargo addon. I've wanted this functionality for cargo doc as well, when my own code is too broken to compile ;-)
However, the Rust playground that I maintain does accomplish your end goal. There's a base Docker container that installs Rustup and copies in a Cargo.toml with all of the crates available for the playground. The build steps create a blank project (with a dummy src/lib.rs), then calls cargo build and cargo build --release to compile the crates:
RUN cd / && \
cargo new playground
WORKDIR /playground
ADD Cargo.toml /playground/Cargo.toml
RUN cargo build
RUN cargo build --release
RUN rm src/*.rs
All of the downloaded crates are stored in the Docker image's $HOME/.cargo directory and all of the built crates are stored in the applications target/{debug,release} directories.
Later on, the real source files are copied into the container and cargo build / cargo run can be executed again, using the now-compiled crates.
If you were building an executable project, you'd want to copy in the Cargo.lock as well.

Answer 2

If you add a dummy main or lib file, you can use cargo build to just pull down the dependencies. I'm currently using this solution for my Docker based project:
COPY Cargo.toml .
RUN mkdir src \
&& echo "// dummy file" > src/lib.rs \
&& cargo build
I'm using --volumes, so I'm done at this point. The host volumes come in and blow away the dummy file, and cargo uses the cached dependencies when I go to build the source later. This solution will work just as well if you want to add a COPY (or ADD) later and use the cached dependencies though.

Answer 3

Based on a GitHub comment
FROM rust:1.37
WORKDIR /usr/src
# Create blank project
RUN USER=root cargo new PROJ
# We want dependencies cached, so copy those first.
COPY Cargo.toml /usr/src/PROJ/
COPY Cargo.lock /usr/src/PROJ/
WORKDIR /usr/src/PROJ
# This is a dummy build to get the dependencies cached.
RUN cargo build --release
# Now copy in the rest of the sources
COPY MyPROJECT/src /usr/src/PROJ/src/
# This is the actual build.
RUN cargo build --release \
&& mv target/release/appname /bin \
&& rm -rf /usr/src/PROJ
WORKDIR /
EXPOSE 8888
CMD ["/bin/appname"]

Answer 4

The cargo-chef tool is designed to solve this problem. Here's an example from the README on how you can use it in the Dockerfile:
FROM lukemathwalker/cargo-chef as planner
WORKDIR app
COPY . .
RUN cargo chef prepare --recipe-path recipe.json
FROM lukemathwalker/cargo-chef as cacher
WORKDIR app
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
FROM rust as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
COPY --from=cacher $CARGO_HOME $CARGO_HOME
RUN cargo build --release --bin app
FROM rust as runtime
WORKDIR app
COPY --from=builder /app/target/release/app /usr/local/bin
ENTRYPOINT ["/usr/local/bin/app"]

Answer 5

I just wanted to post this here so others will see it going forward. There's an experimental tool for Docker I've just started using called cargo-wharf (https://github.com/denzp/cargo-wharf/tree/master/cargo-wharf-frontend). It's a Docker BuildKit frontend that caches built cargo dependencies for you. If you only change one of your source files, that's the only thing that gets rebuilt when you call docker build. You use it by annotating your Cargo.toml file, then directing Docker to your Cargo.toml instead of a Dockerfile. Go check it out, it's exactly what I wanted. (I am in no way affiliated with the project.)

Answer 6

It can be done via cargo init, cargo build, and cargo install. For example, for a project called foo, define the following Dockerfile:
FROM rust:slim-bullseye
# Build dependencies only.
RUN cargo init foo
COPY Cargo.toml foo/
RUN cargo build --release; \
rm -rf foo
# Install `foo`.
COPY . .
RUN echo "// force Cargo cache invalidation" >> foo/src/main.rs; \
cargo install --path foo
CMD ["foo"]
Here, cargo init creates the placeholder files that Cargo expects, cargo build builds the dependencies which were specified in the Cargo.toml, and cargo install creates the foo binary. For some reason, the Docker kept building the default project created by cargo init foo. This problem is resolved above by forcing an update for main.rs by appending // force Cargo cache invalidation.
To avoid slow builds due to large build contexts and large layers, make sure that unimportant folders such as target are ignored via .dockerignore. For example, define the following .dockerignore:
**/*.lock
LICENSE
README.md
target