[BUG] Installed with helm, CrashLoopBackOff

[eschbach-dd]

Note: Make sure to check out known issues (https://akv2k8s.io/troubleshooting/known-issues/) before submitting

Components and versions
Select which component(s) the bug relates to with [X].

[x] Controller, version: x.x.x (docker image tag)
[ ] Env-Injector (webhook), version: x.x.x (docker image tag)
[ ] Other

Describe the bug

To Reproduce
Steps to reproduce the behavior:
I followed the instructions to setup using help. But I applied it with terraform:

resource "helm_release" "akv2k8s" {
  name             = "akv2k8s"
  repository       = "https://charts.spvapi.no"
  chart            = "akv2k8s"
  create_namespace = true
  namespace        = "akv2k8s"
  version          = "2.3.5"
}

Logs
If applicable, add logs to help explain your problem.

I0605 14:07:45.565698       1 main.go:103] "log settings" format="text" level="2"
I0605 14:07:45.565742       1 version.go:31] "version info" version="1.4.0" commit="15d87b2" buildDate="2022-12-08T21:18:59Z" component="controller"
W0605 14:07:45.566001       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0605 14:07:45.566748       1 main.go:168] "Creating event broadcaster"
I0605 14:07:45.576709       1 controller.go:167] "setting up event handlers"
I0605 14:07:45.576791       1 controller.go:178] "starting azurekeyvaultsecret controller"
I0605 14:07:45.676969       1 controller.go:196] "starting azure key vault secret queue"
I0605 14:07:45.677018       1 controller.go:199] "starting azure key vault deleted secret queue"
I0605 14:07:45.677038       1 controller.go:202] "starting azure key vault queue"
I0605 14:07:45.677054       1 controller.go:205] "started workers"
I0605 14:08:10.874449       1 worker.go:61] Shutting down DeletedAzureKeyVaultSecrets Queue
I0605 14:08:10.874492       1 worker.go:61] Shutting down AzureKeyVaultSecrets Queue
I0605 14:08:10.874501       1 worker.go:61] Shutting down AzureKeyVault Queue
I0605 14:08:10.874514       1 controller.go:207] "Shutting down workers"

Even debug logs don't say much. Looks like it's restarting because the livenessprobe fails.

[jakub-klapka]

Hi, we've started to have exactly the same issue. I've enabled verbose logging and it seems, that with each iteration, it updates all of the secrets once again. It can be seen also at AKV resources, where LastChanged attribute is updated all the time. So maybe something changed in KV api, that now akv resolves all secret as always changed?

[Timo-Weike]

We also face the same issue. The image for the controller is spvest/azure-keyvault-controller:1.4.0

[franrebo84]

Same issue here, upgrading from helm chart 2.1.0.

[EnricoOr]

It is a problem of the helm chart 2.3.5. Probes where added but as you can see from the controller main.go code, the probe endpoint is dependant from the metric server so the endpoint is available only with the metric server enabled. The current workaround is to enable the metric server in your values.yml or downgrade your chart version.

[eschbach-dd]

Thanks for your info. Which chart version should work?

[kenny-y-dev]

As a workaround I was able to get the controller to a stay up by adding the config option inline to the helm install command with helm upgrade --install akv2k8s spv-charts/akv2k8s --set global.metrics.enabled=true --namespace akv2k8s

[iamraj007]

As a workaround I was able to get the controller to a stay up by adding the config option inline to the helm install command with helm upgrade --install akv2k8s spv-charts/akv2k8s --set global.metrics.enabled=true --namespace akv2k8s

Above given solution works fine and make akv2k8s working again, but this is a manual step after helm...
Can someone please fix this as its breaking automation and trust both, please.

@torresdal
@181192

[EnricoOr]

Thanks for your info. Which chart version should work?

any previous version should work fine since the template for the controller doesn't include the probes

[tspearconquest]

Above given solution works fine and make akv2k8s working again, but this is a manual step after helm... Can someone please fix this as its breaking automation and trust both, please.

It's not a manual step that you have to do after helm. You can add this to values.yaml:

global:
  metrics:
    enabled: true