Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TriggerAuthentication Secret is not reloaded when Secret changes #563

Closed
dustin-decker opened this issue Jan 25, 2020 · 6 comments
Closed
Labels
bug Something isn't working stale All issues that are marked as stale due to inactivity

Comments

@dustin-decker
Copy link

dustin-decker commented Jan 25, 2020

Our deployment generates a random password for Redis (used as the trigger) during every deployment. The operator does not reload the credential when the Secret changes, so we see,

{"level":"error","ts":1579935742.1253839,"logger":"redis_scaler","msg":"error","error":"ERR invalid password","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128\ngithub.com/kedacore/keda/pkg/scalers.(*redisScaler).IsActive\n\tkeda/pkg/scalers/redis_scaler.go:96\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).handleScaleDeployment\n\tkeda/pkg/handler/scale_loop.go:119\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).handleScale\n\tkeda/pkg/handler/scale_loop.go:45\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).HandleScaleLoop\n\tkeda/pkg/handler/scale_loop.go:14"}

Expected Behavior

I expect the Secret resource to be watched and reload during create/update events.

Actual Behavior

The original Secret value is used and results in an invalid password error.

Steps to Reproduce the Problem

  1. Deploy a ScaledObject + TriggerAuthentication that uses a Kubernetes Secret
  2. Change the Secret's value
  3. Observe invalid password failures

Specifications

  • Version: v1.1.0
  • Scaler(s): Redis (probably affects all TriggerAuthentication resources)
@dustin-decker dustin-decker added the bug Something isn't working label Jan 25, 2020
@tomkerkhove tomkerkhove added this to the v1.2 milestone Jan 27, 2020
@tomkerkhove
Copy link
Member

Nice find @dustin-decker, we should definately fix this! Should be fairly easy or not @ahmelsayed ?

@zroubalik
Copy link
Member

Related to this #511

@zroubalik
Copy link
Member

Could you please show me example configuraton of your TriggerAuthentication? To make it clear, the password change happens several times during the Deployment?

@tomkerkhove tomkerkhove modified the milestones: v1.2, v1.3 Feb 11, 2020
@tomkerkhove tomkerkhove removed this from the v1.4 milestone Apr 23, 2020
@zroubalik
Copy link
Member

zroubalik commented May 4, 2020

Should be fairly easy or not

Replying to your question @tomkerkhove 😄

TriggerAuthentication and relevant Secret should be updated and fetched by KEDA each pollingInterval, but if the Secret is updated in the meantime and scaling is happening, this problem occurs.

We will need to watch all TriggerAuthentication resources and referenced Secret and if there's any change, we will need to find all ScaledObjects that are using this TA and restart the scale loop logic. Because there's no direct reference between ScaledObject and TA.

@stale
Copy link

stale bot commented Oct 13, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale All issues that are marked as stale due to inactivity label Oct 13, 2021
@stale
Copy link

stale bot commented Oct 20, 2021

This issue has been automatically closed due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working stale All issues that are marked as stale due to inactivity
Projects
None yet
Development

No branches or pull requests

3 participants