--export-cache option crashes buildkitd on custom frontend

[drodriguezhdez]

OS: macOS Catalina 10.15.2
Docker: Docker version 19.03.5, build 633a0ea
Buildctl: github.com/moby/buildkit 0.6.3
Buildkitd: v0.6.3

Hi,

I'm creating a PoC for a new buildkit frontend for Maven (which download the code and execute the tests).

It seems that the --export-cache option in the buildctl command is crashing the buildkitd container.

After command enters into the export cache phase, it returns the following message

#18 exporting cache
#18 preparing build cache for export
error: failed to receive status: rpc error: code = Unavailable desc = transport is closing

Killing the buildkitd container.

Command to reproduce:

buildctl --addr=docker-container://buildkitd build \
--frontend=gateway.v0 \
--opt source=codescope/mavenfile-test \
--opt context=https://github.com/drodriguezhdez/hello-world.git \
--export-cache type=local,dest=cache \
--output type=local,dest=output

Using export-cache type=registry or inline, it is obtaining the same error.

Is it needed something specific in the frontend code to support this option --export-cache?

Thank you so much.

[drodriguezhdez]

/cc @fermayo @tonyredondo

[tonistiigi]

@drodriguezhdez is there source code for codescope/mavenfile-test?

[drodriguezhdez]

Hi @tonistiigi,

I created a repository with the minimum amount of code which reproduces this issue.
Find it here: https://github.com/drodriguezhdez/buildkit-cache-sandbox

Command:

buildctl --addr=docker-container://buildkitd build \
--frontend=gateway.v0 \
--opt source=drodriguezh/cache-sandbox-2-test \
--export-cache type=local,dest=cache \
--output type=local,dest=output

Findings:

It seems that if in the file builder/build.go, in this code

buildInterm := llb.Image("alpine:latest").File(llb.Copy(git, "pom.xml", "pom.xml"))

......

buildFinal := llb.Image("alpine:latest").File(llb.Copy(buildInterm, "pom.xml", "pom.xml"))

makes the --export-cache fails.

But, if the second copy from buildInterm to buildFinal is removed, the --export-cache seems to work fine.

Logs of the failed --export-cache

[+] Building 63.4s (8/9)
 => resolve image config for docker.io/drodriguezh/cache-sandbox-2-test:latest                                                                                                                                                           1.7s
 => docker-image://docker.io/drodriguezh/cache-sandbox-2-test@sha256:a72f0051444f46996ba433fe5c6061fcc0c7c2134aab46c934c55fb03a7a8ad7                                                                                                    1.1s
 => => resolve docker.io/drodriguezh/cache-sandbox-2-test@sha256:a72f0051444f46996ba433fe5c6061fcc0c7c2134aab46c934c55fb03a7a8ad7                                                                                                        0.0s
 => => sha256:a72f0051444f46996ba433fe5c6061fcc0c7c2134aab46c934c55fb03a7a8ad7 528B / 528B                                                                                                                                               0.0s
 => => sha256:c0b3f700501b7670a7c6763239359fbd3f3ef4747d3dc09823a6c07fe538b5a7 4.52MB / 4.52MB                                                                                                                                           0.7s
 => => sha256:2c64ab7b1506ab6f77f6c2c7a8d6426c567265f518fe448e6581bc50ba759bef 1.56kB / 1.56kB                                                                                                                                           0.0s
 => => unpacking docker.io/drodriguezh/cache-sandbox-2-test@sha256:a72f0051444f46996ba433fe5c6061fcc0c7c2134aab46c934c55fb03a7a8ad7                                                                                                      0.2s
 => git://github.com/drodriguezhdez/hello-world.git#master                                                                                                                                                                               1.6s
 => docker-image://docker.io/library/alpine:latest                                                                                                                                                                                       0.6s
 => => resolve docker.io/library/alpine:latest                                                                                                                                                                                           0.6s
 => => sha256:ab00606a42621fb68f2ed6ad3c88be54397f981a7b70a79db3d1172b11c4367d 1.64kB / 1.64kB                                                                                                                                           0.0s
 => => sha256:ddba4d27a7ffc3f86dd6c2f92041af252a1f23a8e742c90e6e1297bfa1bc0c45 528B / 528B                                                                                                                                               0.0s
 => => sha256:c9b1b535fdd91a9855fb7f82348177e5f019329a58c53c47272962dd60f71fc9 2.80MB / 2.80MB                                                                                                                                           0.4s
 => => sha256:e7d92cdc71feacf90708cb59182d0df1b911f8ae022d29e8e95d75ca6a99776a 1.51kB / 1.51kB                                                                                                                                           0.0s
 => => unpacking docker.io/library/alpine:latest                                                                                                                                                                                         0.1s
 => copy /pom.xml /pom.xml                                                                                                                                                                                                               0.2s
 => CACHED copy /pom.xml /pom.xml                                                                                                                                                                                                        0.0s
 => copy /* /                                                                                                                                                                                                                            0.1s
 => exporting to client                                                                                                                                                                                                                  0.2s
 => => copying files 5.61MB                                                                                                                                                                                                              0.1s
 => exporting cache                                                                                                                                                                                                                     56.6s
 => => preparing build cache for export                                                                                                                                                                                                 56.6s
error: failed to receive status: rpc error: code = Unavailable desc = transport is closing