Copying content from a stage before it is defined results in an unhelpful error message

[jsternberg]

Take the following Dockerfile:

FROM scratch AS b
COPY --from=a /a.txt /a.txt

FROM scratch AS a
COPY <<EOF /a.txt
Hello, World!
EOF

When you run the following build command, you get a strange error message:

$ docker buildx build --target=b -o type=local,dest=out .
[+] Building 0.1s (3/3) FINISHED                                                                                    docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 175B                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                   0.0s
 => => transferring context: 2B                                                                                                     0.0s
 => ERROR [b 1/1] COPY --from=a /a.txt /a.txt                                                                                       0.0s
------
 > [b 1/1] COPY --from=a /a.txt /a.txt:
------
Dockerfile:2
--------------------
   1 |     FROM scratch AS b
   2 | >>> COPY --from=a /a.txt /a.txt
   3 |
   4 |     FROM scratch AS a
--------------------
ERROR: failed to solve: lstat /var/lib/docker/tmp/buildkit-mount1016613292/a.txt: no such file or directory

In contrast, when you attempt to use FROM a before it is defined, the Dockerfile attempts to pull an image from docker.io/library/a and, in this situation, it fails because that image doesn't exist.

The behavior I was expecting depends on the intended behavior of the dockerfile frontend when a stage hasn't been defined yet. If the stages are intended to be listed in any order, then I would expect FROM a to work and COPY --from=a to correctly pick up the resulting image from the multi-stage build. If the intended behavior is that stages aren't available until after they are defined, then I would expect the same behavior as FROM a where it fails on attempting to pull docker.io/library/a and for circumstances where the image exists for it to copy from that image.

[jsternberg]

I wanted to write out the bug report before diving deeper into causes.

I found this while I was investigating an unrelated issue. It seems that when the dockerfile frontend initializes stages, it's possible for us to accidentally pass a null llb.Output to the COPY command as an input.

When we use FROM a from the example above, we hit this code: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerfile2llb/convert.go#L806-L809

Since the stage isn't defined, the if statement doesn't run and ds.base stays as nil. This later gets initialized here: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerfile2llb/convert.go#L358-L359

That's why we get the resolve image behavior. This seems fine to me. This is largely because we're resolving the d.base value before we've read all stages in the dockerfile so it correctly determines we haven't defined the stage yet.

This is as opposed to COPY --from. That happens here: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerfile2llb/convert.go#L323

This is after we've already initialized every stage and each dispatchState has their state value initialized to the zero value (which contains a nil output). That's fine since this section is only resolving the reference and isn't doing anything with it yet. The problem then happens in this loop: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerfile2llb/convert.go#L535-L539

In this section, we're now building the llb.State value that will form the final output. It performs the dispatches in the order that stages were defined. Since the COPY --from references a stage that hasn't been initialized, the state value for that stage is still the zero value. The dispatchCopy method then uses an uninitialized value. It sends it to the resolver which seems to have some default behavior (maybe utilizing scratch?).

Whichever the intended behavior, I think we should probably fix this so it's not trying to copy a value from an uninitialized llb.State value.

[tonistiigi]

This was fixed in #4568