Using --mount=type=cache means the cache is not available at runtime

[sberney]

I am actually using a different language and toolchain, but here is a simple illustration of the problem.

My dockerfile:

FROM ubuntu:focal

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get --assume-yes install \
    nodejs npm

WORKDIR /app
COPY . /app

# Problem is here, with --mount
RUN --mount=type=cache,target=/app/node_modules \
    npm install

CMD ["/bin/bash"]

This can be built into an image and a container, but when I try to execute commands in the container, the cache is not there. node_modules exists, but is empty. I must reinstall it every time I wish to enter the container and execute a one-off command.

I'm actually not trying to cache node_modules nor use JavaScript at all, but what I'm doing operates largely the same way as yarn's cache. I'm trying to cache the equivalent of /home/user/.cache/yarn/v6, so that I can try to run offline installs.
I'm having this issue with haskell's cabal-install, and the process takes several minutes to complete. In any case, these build tools' caches are apparently permanently discarded after the image is built; and if I run install (even when node_modules equivalent is populated) it tries to repopulate the cache.

If I remove the --mount=type=cache directive, it speeds up the execution of commands on the container by keeping a copy of the generated files. If I keep the --mount directive, it speeds up the image build just a little bit, but I have to download and repopulate the cache over and over again after the image has been built; and I need to be able to go into a container, tweak a few things, and then run reinstall. I would really like a way to keep the cache around (read-only is fine) at runtime, or to allow the build to write to a volume at build time, so I can manage the cache myself. Like DOCKER_BUILDKIT=1 docker run --mount=type=cache,etc.

[andrewnazarov]

I was forwarded to the following thread today: #1512
Probably it'll help in your case as well:)

[tonistiigi]

--mount=type=cache does not keep files in image by design. You may be using it for a case it is not meant for. If you want some files from your cache inside your image you need to copy them out of the cache mount with cp/rsync etc.

[KevinMind]

FWIW, ran into this issue and found a semi-working solution. npm has a cache directory for installs that is different than node_modules

You can specify this location --cache parameter. Make it something different than your node_modules.

Use that as the target, and then you can cache the npm cache and not the node_modules which is your actual installed dependencies.

To make it even better you can split stages and copy the node_modules from the builder stage. Add --loglevel verbose to see what npm does and if the cache is used

This means the cache worked.

#9 1.952 npm http fetch GET 200 https://registry.npmjs.org/strip-ansi 98ms (from cache)
#9 1.953 npm http fetch GET 200 https://registry.npmjs.org/wrap-ansi 98ms (from cache)
#9 2.491 npm http fetch GET 304 https://registry.npmjs.org/string-width 646ms (from cache)
#9 2.507 npm http fetch GET 200 https://registry.npmjs.org/emoji-regex 7ms (from cache)
#9 2.547 npm http fetch GET 200 https://registry.npmjs.org/ansi-regex 31ms (from cache)`

The multi stage here is not super useful, but if you needed to do more things in each layer the benefits compound.

FROM ubuntu:focal as base

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get --assume-yes install \
    nodejs npm

FROM base as builder

WORKDIR /app
COPY package*.json /app

RUN --mount=type=cache,target=/app/npm/cache \
    npm install --cache /app/npm/cache --loglevel error

FROM base as runner

# Copy node_modules
COPY --from=builder /app/node_modules /app/node_modules
# Copy your files
COPY . /app

CMD ["/bin/bash"]