Unable to Start Workspace on Unstable (main.4187)

[jimmybrancaccio]

Bug description

This is related to #11878 I believe. When attempting to start a workspace I see:

From the image-builder-mk3 logs:

{"addr":"127.0.0.1:9500","level":"info","message":"started Prometheus metrics server","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"INFO","time":"2022-08-05T16:18:08Z"}
{"addr":"127.0.0.1:6060","level":"info","message":"serving pprof service","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"INFO","time":"2022-08-05T16:18:08Z"}
{"level":"warning","message":"no TLS configured - gRPC server will be unsecured","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"WARNING","time":"2022-08-05T16:18:08Z"}
{"interval":"6h0m0s","level":"info","message":"starting Docker ref pre-cache","refs":["docker.io/gitpod/workspace-full:latest"],"serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"INFO","time":"2022-08-05T16:18:08Z"}
{"addr":":8080","level":"info","message":"started workspace content server","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"INFO","time":"2022-08-05T16:18:08Z"}
{"level":"info","message":"👷 image-builder is up and running. Stop with SIGINT or CTRL+C","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"INFO","time":"2022-08-05T16:18:08Z"}
{"error":"rpc error: code = Unavailable desc = error reading from server: EOF","level":"info","message":"connection to ws-manager lost - retrying","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"INFO","time":"2022-08-05T16:18:10Z"}
{"a":{"All":false,"Explicit":null,"Additional":null},"level":"error","message":"registry not allowed - you may want to add this to the list of allowed registries in your installation config","ref":{},"reg":"docker.io","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"ERROR","time":"2022-08-05T16:20:51Z"}
{"a":{"All":false,"Explicit":null,"Additional":null},"level":"error","message":"registry not allowed - you may want to add this to the list of allowed registries in your installation config","ref":{},"reg":"docker.io","serviceContext":{"service":"image-builder-mk3","version":"commit-225344b78f0f006016648afb2c4b2ad859fa1080"},"severity":"ERROR","time":"2022-08-05T16:24:58Z"}

When running kubectl get configmaps -n gitpod gitpod -o jsonpath='{.data.config\.yaml}' I see:

containerRegistry:
  inCluster: true
  privateBaseImageAllowList: []

Steps to reproduce

1. Spin up an instance with main.4187.
2. Attempt to create a new workspace. I did using my test repository - https://github.com/jimmybrancaccio/gitpod-test
3. Observe an error about authentication.

Workspace affected

No response

Expected behavior

I expect the workspace to be created without issue.

Example repository

No response

Anything else?

No response

[jimmybrancaccio]

Closing this out. The instance was hitting an external Docker registry I had setup the other day 🤦🏼