Declarative scaler config

[wozniakjan]

Various scaler configurations are inconsistent in implementation as discussed in #5037. There has been a great effort in improving the situation with getParameterFromConfigV2() in #5228 (with further enhancements in #5319 and #5314).

There are likely always going to be some situations that require imperative parsing of the provided configs and the helper getParameterFromConfigV2(), but for the majority of cases, I believe we can implement an even more generic, more convenient and declarative algorithm, similar for example to the API of well known json.Unmarshal().

My desired goal is to allow each scaler to define the configuration struct with proper field tags to fine-tune the parsing algorithm, call a single helper method TypedConfig() defined on ScalerConfig struct and consume unmarshalled typed config without the need for explicit typecasting.

Example:

type MyConfigXYZ struct {
  Flag    bool           `keda:"name=flag,    order=triggerMetadata"`
  Addr    string         `keda:"name=addr,    order=triggerMetadata;resolvedEnv, optional, default=d"`
  Headers map[string]int `keda:"name=headers, order=triggerMetadata,             optional, default=d"`
}

sc := &scalerconfig.ScalerConfig{ ... }
conf := MyConfigXYZ{}
if err := sc.TypedConfig(conf); err != nil {
   ...
}

Where the MyConfigXYZ is an arbitrary structure with fields containing properly formatted keda tag. Callers can control

• where to obtain the value from - triggerMetadata, resolvedEnv, authParams, and which map has a precedence
• whether it's optional or deprecated
• what is default value
• name as a key to the map
• allowed values with enum
• mutually exclusive values with exclusive

The algorithm is based on go reflections and dynamic parsing of a structure, natively supports slices and maps and url.Values so it might eventually supersede helpers in pkg/util/parse_string.go

PR structure

• 4475519 core implementation
• d5d0b10 sample conversion of prometheus scaler
• 43b6c10 implementation of prometheus auth parsing

Checklist

• I have verified that my change is according to the deprecations & breaking changes policy
• Tests have been added
• Changelog has been updated and is aligned with our changelog requirements
• Commits are signed with Developer Certificate of Origin (DCO - learn more)

Relates to #5037

[zroubalik]

[SpiritZhou]

Could this be used to generate scalers' schema?

[wozniakjan]

Could this be used to generate scalers' schema?

we may need another set of field tags but I think it should be possible

[wozniakjan]

just fyi, I started refactoring auth params config so I will mark this as draft until it's ready for a review again

[wozniakjan]

/run-e2e
Update: You can check the progress here

[wozniakjan]

I think we can give it a try, ptal @kedacore/keda-core-contributors

after this merges, I will try to tackle the scalers one by one, would you like to divide and conquer @dttung2905?

[dttung2905]

I think we can give it a try, ptal @kedacore/keda-core-contributors

after this merges, I will try to tackle the scalers one by one, would you like to divide and conquer @dttung2905?

Absolutely, since there are many scalers, we can divide amongst ourselves and other community members who are interested as well. We can create a separate issue to track completion progress once this PR is merged 🤘

[dttung2905]

Thanks @wozniakjan for spending times on making this changes, it's not an easy feat at all.
I have 1 minor comment in my first pass going through the PR. Will probably go through it during the weekend when I have time 😄

[wozniakjan]

/run-e2e prometheus
Update: You can check the progress here

[zroubalik]

LGTM, great job!

[zroubalik]

@dttung2905 let me know once you are done with reviews, so we can merge this :)

[dttung2905]

Its all good for me now. Thanks @zroubalik