Refresh flattened references when interpolating values #8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sometimes using nested references to lookup values for parameters based on a parameter which is set differently in the hierarchy for different nodes leads to a ResolveError. The original issue was discovered in Project Syn component [backup-k8up], which supports being compiled multiple times with different configurations in multiple Kapitan targets. If the two configurations use different values for the parameter `backup_k8up.majorVersion`, the nested references which use this parameter to look up values for the appropriate major version sometimes use stale flattened refs when checking whether all nested references are resolved causing a `ResolveError`. From what I was able to determine this happens because nested references are flattened in `RefItem.assembleRefs()` which can cause interpolation to operate with stale flattened references if the same `RefItem` is part of the parameters structure for multiple nodes. We address the problem by calling `value.assembleRefs(self._base)` for each value which is processed in `Parameters._interpolate_inner()`. This fixes the issue both in the included minimal test case and our real-world case when rendering the reclass inventory for Project Syn clusters which include component [backup-k8up] as described in the [component v2 to v3 migration how-to]. I haven't been able to determine exactly how a `RefItem` object can be part of multiple nodes and therefore I'm not sure if it's expected that the same `RefItem` object can be part of multiple nodes or not. Unfortunately this means that I can't tell if my fix is just masking another bug somewhere which wrongly adds the same `RefItem` to the parameters of multiple nodes. [backup-k8up]: https://github.com/projectsyn/component-backup-k8up.git [component migration how-to]: https://hub.syn.tools/backup-k8up/how-tos/upgrade-v2-v3.html#_steps_to_run_two_instances_at_once
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sometimes using nested references to lookup values for parameters based on a parameter which is set differently in the hierarchy for different nodes leads to a ResolveError.
The original issue was discovered in Project Syn component backup-k8up, which supports being compiled multiple times with
different configurations in multiple Kapitan targets. If the two configurations use different values for the parameter
backup_k8up.majorVersion, the nested references which use this parameter to look up values for the appropriate major version sometimes use stale flattened refs when checking whether all nested references are resolved causing aResolveError.From what I was able to determine this happens because nested references are flattened in
RefItem.assembleRefs()which can cause interpolation to operate with stale flattened references if the sameRefItemis part of the parameters structure for multiple nodes.We address the problem by calling
value.assembleRefs(self._base)for each value which is processed inParameters._interpolate_inner(). This fixes the issue both in the included minimal test case and our real-world case when rendering the reclass inventory for Project Syn clusters which include component backup-k8up as described in the component v2 to v3 migration how-to.I haven't been able to determine exactly how a
RefItemobject can be part of multiple nodes and therefore I'm not sure if it's expected that the sameRefItemobject can be part of multiple nodes or not. Unfortunately this means that I can't tell if my fix is just masking another bug somewhere which wrongly adds the sameRefItemto the parameters of multiple nodes.