docker is unable to build private git repository with ssh access using buildkit

[stalb]

Problem:

If you have a private git repository you access using ssh key, you are not able to build it using buildkit.

export DOCKER_BUILDKIT=1 ; docker build [email protected]:xxxx/yyyy.git fails with permission denied, Could not read from remote repository...

Of course the normal build command is working as expected:
export DOCKER_BUILDKIT=0 ; docker build [email protected]:xxxx/yyyy.git

Expected behavior:

export DOCKER_BUILDKIT=1 ; docker build [email protected]:xxxx/yyyy.git should be able to use ssh key or ssh-agent in order to download the private project and then build the project.

Docker version:

Client: Docker Engine - Community
 Version:           19.03.0-beta5
 API version:       1.40
 Go version:        go1.12.4
 Git commit:        4a18bf42fb
 Built:             Wed May 29 22:59:42 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.0-beta5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.4
  Git commit:       4a18bf42fb
  Built:            Wed May 29 22:58:17 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[Bluesboy]

I'm also experiencing the same issue but with Concourse CI (vito/oci-build-task).
@stalb do you found out workaround?

[stalb]

@Bluesboy no I just don't use Buildkit for this repositories.

[tonistiigi]

The workaround is to do git clone .. x && docker build x. In fact this is exactly what the legacy mode does before the actual builder is invoked.

We do plan to add proper support for this #1048

[Bluesboy]

It seems to me that error happened because buildkit doesn't interpolates environment variables from host inside build environment. However if I provide variables with ARG directive in Dockerfile and then specify them in build arguments everything works fine. Hope that'll help somebody.

[tonistiigi]

@alexcb I guess this has been addressed but can you update documentation for the extra fields flags required to forward ssh.

[alexcb]

@tonistiigi I think I might be missing something here, which struct/fields needs documenting?

I'm surprised that this code isn't being triggered automatically: https://github.com/moby/buildkit/blob/master/client/llb/source.go#L292-L310

[tonistiigi]

@alexcb I'd assume at least --ssh default is needed. Although you might be correct that if the context path is already an ssh git URL we could maybe add it automatically on the cli side.

[alexcb]

I just compiled a new version of buildx, and if I specify a socket it works:

./bin/buildx build --ssh default=$(echo $SSH_AUTH_SOCK) [email protected]:xxx/yyy.git

however when I run it without any --ssh option it passes an empty slice to https://github.com/docker/buildx/blob/master/build/ssh.go#L11

I wonder if something like this would be a reasonable fix? docker/buildx#577

edit: this is a simpler fix: docker/buildx#578

[playground]

I'm having the save error. Anyone got this working with buildkit?

[tonistiigi]

docker/buildx#581