Dependancy builds not caching

[frasermarch]

Hi all,

I'm trying to use cargo-chef to speed up my google cloud run build times. The first stage of my docker file is hitting the cache, but the stage the builds the dependencies isn't. Any help would be really appreciated!

FROM rust AS chef
RUN cargo install cargo-chef
WORKDIR app

FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder 
COPY --from=planner /app/recipe.json recipe.json
# Build dependencies - this is the caching Docker layer!
RUN cargo chef cook --release --recipe-path recipe.json
# Build application
COPY . .
RUN cargo build --release --bin app

# We do not need the Rust toolchain to run the binary!
FROM gcr.io/distroless/cc-debian11 AS runtime
WORKDIR app
COPY --from=builder /app/target/release/mail-ai /usr/local/app
ENTRYPOINT ["/usr/local/app"]

when I run gcloud config list

[builds]
kaniko_cache_ttl = 72
use_kaniko = True

gcloud build command:

gcloud builds submit --tag gcro.io/${PROJECT}/${CLOUDRUN_NAME}

[LukeMathWalker]

It's impossible for me to help you with this little information. As far as I can see, the Dockerfile is using cargo-chef correctly.

[klefevre]

Hi, I'm running into the same issue. We're using Kaniko on GitLab and the

RUN cargo chef cook --release --recipe-path recipe.json

step is replayed with each build whereas it works perfectly locally.

@LukeMathWalker I deeply think the issue comes from Kaniko rather than cargo-chef. What kind of information you might need to help us debug this?

[detly]

I just had a similar problem, and it was because the CI system was "helpfully" caching the target dir, which meant that whatever cargo chef prepared was blown away and replaced upon the COPY or ADD command. I solved this by (a) not doing that, and (b) adding a .dockerignore file to the repo so it wouldn't happen locally too. Check if your CI is doing this.

[athoscouto]

I'm seeing a similar problem when trying to build sqld.
cargo build step runs from scratch, not using the builds generated by cargo chef cook.

Dockerfile for reference:

# install dependencies
FROM rust:slim-bullseye AS compiler
RUN apt update \
    && apt install -y libclang-dev clang \
        build-essential tcl protobuf-compiler file \
        libssl-dev pkg-config \
    && apt clean

RUN cargo install cargo-chef
WORKDIR /sqld

# prepare recipe
FROM compiler AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

# build sqld
FROM compiler AS builder
COPY --from=planner sqld/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
COPY . .
RUN cargo build -p sqld --release

# runtime
FROM debian:bullseye-slim
COPY --from=builder /sqld/target/release/sqld /bin/sqld
RUN adduser --system --home /var/lib/sqld --uid 666 sqld 
RUN apt-get update && apt-get install -y ca-certificates
COPY docker-entrypoint.sh /usr/local/bin
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
VOLUME [ "/var/lib/sqld" ]
WORKDIR /var/lib/sqld
USER sqld
EXPOSE 5001 5432 8080
CMD ["/bin/sqld"]

I think I managed to track down what causes it.

If I run (outside of Docker):

cargo clean
cargo chef cook --release --recipe-path recipe.json
cargo build -p sqld --release

I can see that cargo build reuses the dependencies from cargo chef cook.

But, as warned, cargo chef cook changes a lot of my local files.

And if I do:

cargo clean
cargo chef cook --release --recipe-path recipe.json
git checkout .
cargo build -p sqld --release

Then cargo build will build from scratch.

The thing is that doing COPY . . after cargo chef cook has the same effect of git checkout ..
It removes the changes that cargo chef cook would have made to the code that is going to be built.

Is there a way to run cargo chef cook without changing the repository?
Any other way around this?
It may be the case that I'm just using it the wrong way.

[haaawk]

I think you might be hitting #231
Try replacing:

FROM rust AS chef
RUN cargo install cargo-chef
WORKDIR app

with:

FROM rust AS chef
RUN cargo install cargo-chef
WORKDIR app
COPY rust-toolchain.toml rust-toolchain.toml

If it helps then it's #231

It solved the problem for me in sqld @athoscouto mentioned above. See libsql/sqld#549 for more details.

FYI @frasermarch @klefevre

[LukeMathWalker]

Closing this since #231 has been merged.