Tag multiple stages and/or build multiple targets with one invocation of buildkit

[brad-jones]

Wanted to re look at this moby/buildkit#609 now that we have docker buildx bake.

I am trying to determine if bake has ultimately been built to solve this issue or are it's goals slightly different?

If I use a bake file like this:

group "default" {
    targets = ["prj1", "prj2"]
}

target "prj1" {
    dockerfile = "Dockerfile"
    target = "prj1",
    tags = ["prj1"]
}

target "prj2" {
    inherits = ["prj1"]
    target = "prj2",
    tags = ["prj2"]
}

And then execute with docker buildx bake -f ./docker-bake.hcl I only get prj1 image exported. prj2 still builds just doesn't get exported.

Also can / could a bake file be used to effectively stitch multiple dockerfiles together?

Our monorepo has 20+ projects, all with their own Dockerfile "segments" - I say this because if you tried to build one of these files by it's self it would probably fail as it depends on parent build stages defined in other files.

I have written a script that concatenates all the Dockerfiles together into a single Dockerfile in the correct order so that build stages refer to each other correctly and then I execute a buildkit build on this giant Dockerfile. Using the approach outlined at moby/buildkit#609

It would be great if I could define normal standalone Dockerfiles and then use a bake file to do the stitching / build in correct order, consider the following contrived example:

node-modules.dockerfile

FROM node
RUN npm install

prj1.dockerfile

FROM node-modules
RUN npm build

prj2.dockerfile

FROM node-modules
RUN npm build

docker-bake.hcl

group "default" {
    targets = ["prj1", "prj2"]
}

target "node-modules" {
    dockerfile = "node-modules.dockerfile"
    tags = ["node-modules"]
}

target "prj1" {
    depends = ["node-modules"]
    dockerfile = "prj1.dockerfile"
    tags = ["prj1"]
}

target "prj2" {
    depends = ["node-modules"]
    dockerfile = "prj2.dockerfile"
    tags = ["prj1"]
}

Perhaps bake can already do this but I haven't figured it out yet?

[brad-jones]

Just came across https://github.com/avirshup/DockerMake I feel like bake could take some inspiration from it.

And the punch line that truly sums up what I am after:

DockerMake.yml lets you split your image build up into discrete, easy to manage steps that can be mixed together (as a dependency graph) to produce your desired container image.

Build stages allow us to do that in a single file. In my opinion there needs to be a way to achieve this across multiple Dockerfiles.

[prodrigestivill]

The same happends in this project prodrigestivill/postgres-backup-local, here the docker-bake.hcl file.

It only pushes all archictures for the first tag.
For the rest of the tags it only pushes the local architecture if it gets pushed.

[dduportal]

Same thing is happening on the Jenkins project: jenkinsci/docker-agent#323.

We had to split a given target into 2 targets with distinct subsets of architectures in jenkinsci/docker-agent#310 and now only 1 of these targets is pushed, while all are builts.

It seems the "push" behavior of buildx is not generating the expected manifest.