Unable to find container images

[timbrd]

I am not able to install sealed-secrets anymore. Using helm chart (v2.1.5), my k8s cluster is not able to find the container image:

13s         Normal    Pulling                       pod/sealed-secrets-controller-5d8db499cd-zthst    Pulling image "quay.io/bitnami/sealed-secrets-controller:v0.17.4"
12s         Warning   Failed                        pod/sealed-secrets-controller-5d8db499cd-zthst    Failed to pull image "quay.io/bitnami/sealed-secrets-controller:v0.17.4": rpc error: code = Unknown desc = reading manifest v0.17.4 in quay.io/bitnami/sealed-secrets-controller: unauthorized: access to the requested resource is not authorized

Shouldn't there be a repository at https://quay.io/repository/bitnami/sealed-secrets-controller? Currently you are forced to login to quay.io and then it says "No repository found".

It seems that the sealed-secrets image repository has been deleted?

docker pull quay.io/bitnami/sealed-secrets-controller:v0.17.4
Error response from daemon: unauthorized: access to the requested resource is not authorized

Solution
Update the sealed-secrets-controller repository to our new oficial dockehub location:

image:
  registry: docker.io
  repository: bitnami/sealed-secrets-controller

See #822 (comment)

[mhulscher]

Looks like someone made the organization private, or purged all images 🤷🏾‍♂️

[Nicarim]

Seeing same issue (bumping for visibility). Can't proceed with deployments with that missing.

[jeanguirro]

Having the same issue:

> docker pull quay.io/bitnami/sealed-secrets-controller:v0.16.0
Error response from daemon: unauthorized: access to the requested resource is not authorized

[radoslav-tomov]

+1

[jackcaldwell]

Facing same issue here.

Failed to pull image "quay.io/bitnami/sealed-secrets-controller:v0.13.1": rpc error: code = Unknown desc = Error response from daemon: unauthorized: access to the requested resource is not authorized

[mprusinski]

+1

[RobMokkink]

Have the same issue, please fix it.

[BarrieShieh]

the images in docker hub also gone
https://hub.docker.com/r/bitnami/sealed-secrets-controller/tags

[Jc2k]

There seems to be a mirror here - https://quay.io/repository/56401deb-7290-4d8b-8731-baf6d8b5d27d/sealed-secrets-controller?tab=tags. The hash for 0.17.4 matches what I have locally (f4bc4aca1ae011b89e0e6b997f63a89c8cf464cd41c0bd4544218878200b9b69).

[pigfrown]

+1 .. anyone know how to build the docker image from this repo? the Dockerfile doesn't build (files in dist/ are missing)

[timbrd]

There seems to be a mirror here - https://quay.io/repository/56401deb-7290-4d8b-8731-baf6d8b5d27d/sealed-secrets-controller?tab=tags. The hash for 0.17.4 matches what I have locally (f4bc4aca1ae011b89e0e6b997f63a89c8cf464cd41c0bd4544218878200b9b69).

docker pull quay.io/56401deb-7290-4d8b-8731-baf6d8b5d27d/sealed-secrets-controller:v0.17.4
Error response from daemon: Head "https://quay.io/v2/56401deb-7290-4d8b-8731-baf6d8b5d27d/sealed-secrets-controller/manifests/v0.17.4": denied: Namespace 56401deb-7290-4d8b-8731-baf6d8b5d27d has been disabled. Please contact a system administrator.

This repo also doesn't work unfortunately.

[arnaud-mey]

@pigfrown you need to first run make controller which creates the controller binary

then you can copy it to the correct directory in dist/ and build the image

[LPby]

+1

$ docker pull quay.io/bitnami/sealed-secrets-controller:v0.17.4
Error response from daemon: unauthorized: access to the requested resource is not authorized

[Jc2k]

#824 - prepare for a new release on a different repo I guess.

[LPby]

any estimation when image will be available at dockerhub?

[Jc2k]

#824 - prepare for a new release on a different repo I guess.

Awesome. How about switching to a new registry, but keeping the images at the old one for some time?

That's a great idea, for someone who works on this project. I myself would like to ask them if removing the old repo was because of a security incident.

[Jc2k]

any estimation when image will be available at dockerhub?

You can watch GitHub actions - e.g. https://github.com/bitnami-labs/sealed-secrets/runs/6096838026?check_suite_focus=true

[timbrd]

The new docker image, tagged as v0.17.5 and uploaded to dockerhub, works for me.

@agarcia-oss,@alvneiayu Thanks for fixing it.

[agarcia-oss]

Hi everyone,

The new controller image is already available from the new oficial DockerHub repository:
https://hub.docker.com/r/bitnami/sealed-secrets-controller/tags

The reason for this change is to use our Dockerhub Bitnami account as a distribution platform of the controller image. Doing so we can leverage in the advantages of being a verified publisher, such as not been affected by dockerhub pull rate limits.

Unfortunately, quay.io could not provide that level of support for our images and we were forced to migrate the images.

Please accept our apologies if this action has created some disturbances during the process.
Regards

[DionJones615]

The new controller image is already available from the new oficial DockerHub repository: https://hub.docker.com/r/bitnami/sealed-secrets-controller/tags

Older images that were on quay.io are not accessible on quay or dockerhub right now, impacting our production systems. Where can we reference older versions?

[agarcia-oss]

We are investigating the best way to publish older versions of the controller in the new DockerHub repository.

However, we fully recommend upgrading your clusters to the newest release (v.0.17.5) Upgrading should be quite straightforward and older versions of sealed secrets will not have our latest improvements and bugfixes.

[rosenbjerg]

It seems bitnami has completely dropped quay.io support based on the response here:
https://community.bitnami.com/t/repositories-no-longer-available-at-quay-io/105179/3

It may be related to this change:
https://cloud.redhat.com/blog/quay.io-will-require-a-red-hat-account-after-june-30-2021...wait-what

So yeah, switching to docker hub or GCR seems to be the migration path
Just for future reference

[bit-herder]

this is super annoying. the old release should work.People's entire clusters depend on sealed secrets working, and new "functionality" in a minor release should not break literally every other release. Why not just keep the old repo AND use the new one?

[agarcia-oss]

Hi @bit-herder

Releases older that 2 years ago have been ported to the dockerhub repository, so there should not be any impact on any consuming cluster, but a minor change in the repository location for images. Do you have any specific concern? Are you using a sealed-secrets controller version not present in the new repository?

We're trying to facilite the transition to the new repo as much as we can.

[bit-herder]

hi @rufreakde

We are not going to re-release older versions of the chart. The latest version of the chart is already updated to point to the new registry, and we suggest you to upgrade and use this latest version.

If you don't want to update your chart, you can install your specific chart version using the helm flag --set to modify the registry to docker.io, like this --set image.registry=docker.io.

Thanks a lot and sorry for the inconveniences.

Álvaro

Apologies, i should specify helm releases are the issue. I had to DR a production cluster and had a reasonably recent helm release of sealed-secrets. I was pretty surprised when i restored the cluster and could not pull the image. I was having quite the day when I wrote that, so i apologize for the tone. It would be helpful if the recent helm releases were back ported to include the repository change so that this would not suprise folks that had a working installation until the repo was changed. It may seem like a minor change but when sealed secrets suddenly does not work its a bit harrowing.

[zoharaharoni]

@agarcia-oss Hi, I have cluster that until not so long ago was running 0.7.0. recently it stopped working due to the same reason mentioned in this issue.
since this version is not available also in the new repo - I was wondering if it is possible to upgrade directly to the latest chart or if there are breaking changes between 0.7 and the latest version.

Thanks!

[agarcia-oss]

Hi @zoharaharoni,
We will definitely recommend an upgrade to the latest Sealed Secrets version (0.18.0). The version you're running on your cluster is more than 5 years old and it'll be difficult for us to provide support or assistance for such version.

Out of curiosity, what K8S version are you using there?