build: allow external Dockerfile on remote context

[corhere]

Support Dockerfile from stdin or a local file with remote contexts. Fixes moby/moby#38254 now that buildx is the only BuildKit client included in the Docker CLI (docker/cli#3314).

Support for external Dockerfile on remote contexts was added to BuildKit in moby/buildkit#947, which is included in the BuildKit v0.5.0 release and first vendored into Moby in moby/moby#39132, included in v19.03.0. The client side was the only missing piece.

[corhere]

@thaJeztah @tonistiigi @tiborvass PTAL

[thaJeztah]

nice!! thank you for working on this!

[crazy-max]

LGTM

[thaJeztah]

@tonistiigi PTAL

[tonistiigi]

(no specific issue, just want to make sure I review this before we merge)

[tonistiigi]

This is currently breaking:

» docker buildx build "https://github.com/moby/buildkit.git" -f ./frontend/dockerfile/cmd/dockerfile-frontend/Dockerfile
[+] Building 0.0s (0/0)
error: could not find frontend/dockerfile/cmd/dockerfile-frontend: stat frontend/dockerfile/cmd/dockerfile-frontend: no such file or directory

while the current version works fine. I think the definition of -f for such cases needs to remain as is.

What we can support are cases like:

docker buildx build -f - "git://" < ./path/to/Dockerfile
docker buildx build -f ./path/to/Docerfile - < ./path/to/context.tar

[thaJeztah]

I think for this case;

» docker buildx build "https://github.com/moby/buildkit.git" -f ./frontend/dockerfile/cmd/dockerfile-frontend/Dockerfile
[+] Building 0.0s (0/0)
error: could not find frontend/dockerfile/cmd/dockerfile-frontend: stat frontend/dockerfile/cmd/dockerfile-frontend: no such file or directory

In the classic builder we implicitly changed the path to - ("read from stdin") and then read the Dockerfile content; would a similar workaround work here?

[tonistiigi]

@thaJeztah I don't understand but the command above works in legacy builder as well. -f means file path in relation to the git repository.

[thaJeztah]

Ah, you're right; I thought it would take the local file (due to ./)

Maybe I misunderstood your "I think the definition of -f for such cases needs to remain as is." (I interpreted your comment as "won't fix")

So, the issue to fix is "remote context" but file from stdin; so my example from the issue;

docker buildx build -t statx --no-cache -f- https://github.com/whotwagner/statx-fun.git <<-EOF
FROM ubuntu:18.04
RUN apt-get update && apt-get install -y gcc make
RUN mkdir /src/
WORKDIR /src/
COPY . .
RUN make
EOF

[corhere]

If I'm understanding it correctly, the desired behavior is that the -f argument should always be interpreted as either a path relative to the root of the context or STDIN?

docker buildx build ...	Interpretation
-f - "git://context" < custom-dockerfile	Dockerfile from STDIN applied to remote context
-f ./path/to/custom-dockerfile "git://context"	Dockerfile read from git://context/path/to/custom-dockerfile
-f ./path/to/custom-dockerfile - < context.tar	Dockerfile read from file named path/to/custom-dockerfile in context.tar?

[tonistiigi]

If context is a local dir then -f is relative to the working dir. Otherwise it is relative to the context (and can't be outside context). If -f - then it is just data and path does not matter. If context comes from stdin then I guess we can't use external Dockerfile as stdin is already used, although API should support that case as well.