External variable discovery

[killianmuldoon]

External variable discovery is being proposed in #7932

This umbrella issue covers the steps proposed to implement external variable discovery in the CAPI repo. Some of these tasks, where larger effort is needed or the implementation needs clarification, may be represented by their own issues.

This list is ordered, by component and subtask, to reduce the dependencies between tasks when following it from top to bottom. Where tasks earlier in the list depend on later tasks this should be explicitly mentioned.

Webhooks

• Allow Clusters to be created when the associated ClusterClass doesn't exist 🌱 Add clusterclass validation to topology reconciler #7954
• Change variable defaulting and validation to operate on variables in ClusterClass .status.variables (Requires changes in ClusterClass) 🌱 Default and Validate Cluster variables based on ClusterClass status #8076
• Make inline a reserved name for patches defined in a ClusterClass 🌱 Make 'inline' a reserved name for patches #8025
• Update ClusterClass webhook validation to allow variable definition changes with existing Clusters. 🌱 Weaken ClusterClass webhook variable validation on update #8153

Runtime SDK

• Add API definition for DiscoverVariables Runtime Hook 🌱 Add API types for DiscoverVariables hook #7986
• Ensure DiscoverVariables is registered by the ExtensionConfig controller.

ClusterClass

• Add status observedGeneration to ClusterClass and set observedGeneration in the ClusterClass reconcile. 🌱 Add observedGeneration to ClusterClass status #7987
• Add variables to the ClusterClass status and write all inline variables to status.variables 🌱 Add ClusterClass variables to status on reconcile #7991
• Add a watch on ExtensionConfig for the ClusterClass reconciler ✨ Add discoverVariables runtime hook to ClusterClass reconcile loop #8006
• Add discoverVariablesExtension definition to ClusterClass .spec.patches[].external ✨ Add discoverVariables runtime hook to ClusterClass reconcile loop #8006
• Add call to externalVariables Extensions in the ClusterClass reconciler ✨ Add discoverVariables runtime hook to ClusterClass reconcile loop #8006
• Write all variables to ClusterClass .status.variables and dedupe variables based on their namespace. Note: ✨ Add discoverVariables runtime hook to ClusterClass reconcile loop #8006 (comment) ✨ Handle variable definition conflicts for external variables #8107

Cluster Topology

• Add ClusterClass-based Cluster defaulting and validation to the Cluster/Topology reconciler 🌱 Add clusterclass validation to topology reconciler #7954
• Add observed generation success check to ClusterClass object early in the topology reconciler 🌱 Add ClusterClass generation check to Cluster Topology reconciler #8023
• Do variable defaulting and validation based on ClusterClass .status.variables. 🌱 Default and Validate Cluster variables based on ClusterClass status #8076
• Add namespacing to variable definitions in Cluster .spec.topology.variables 🌱 Add DefinitionFrom field to ClusterVariable #8031
• Add validation for cases where variable definitions change i.e. multiple variables with the same name but conflicting definitions. ✨ Handle variable definition conflicts for external variables #8107
• Build variable list for patches from ClusterClass .status.variables
• Modify external patch call to send variables with the same origin as the patch - inline or patch name. builtin variables can be sent to all patches. ⚠️ Add filter to associate variables with specific patches #8128

Testing and reference implementation

• Use external variables from test extension for patches. 🌱 Add DiscoverVariables hook implementation for test extension #8026

Planned for v1.4 release

• Review UX and API from ClusterClass author and Cluster author perspective ref: 🌱 Add ClusterClass variables to status on reconcile #7991 (comment)
  • Make discoverVariable hook call implicit in the ClusterClass. i.e. specified by the generate-patches call. Simplify external patch definition in ClusterClass #8210
  • Review strictness of equality checking of variables.
  • Follow up on UX for variables in status when reconciliation does not happen correctly - e.g. should old variables be used or should status.variables be empty, or only include variables from successful discoverVariables and inline.
• Ensure managed fields work so that SSA does not cause inconsistencies. See 🌱 Add clusterclass validation to topology reconciler #7954 (comment) .
• Flake (https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/periodic-cluster-api-e2e-main/1629138713080500224) possibly a race condition between defaulting and validation - i.e. variable definition isn't reconciled at default time, but is reconciled at validation time.

Nice-to-have

• Create an index of ExtensionConfigs -> ClusterClass to quickly find all ClusterClasses using a particular ExtensionConfig. Follow-up: Improve ExtensionConfig reconcile performance #8495

• Ensure watch on ExtensionConfig correctly filters by extensions actually referenced in the ClusterClass. Requires full validation of the discoverVariablesExtension name in ClusterClass webhook to make naming convention hard. Improve ExtensionConfig reconcile performance #8495

• Add a warning to the webhooks when a Cluster can not be fully vaildated because the ClusterClass does not yet exist. This is dependent on webhook warning support being added in Controller Runtime. Follow-up: Add webhook warning for Clusters without updated ClusterClass #8496

• Enforce strict validation for ClusterClass-based clusters in webhook with annotation. i.e. mechanism to cause error on creation if the Cluster's associated ClusterClass does not exist. Add webhook warning for Clusters without updated ClusterClass #8496

• Add best practice and guidance for creating variables to the Topology mutation guide in the book. Improve testing for Runtime SDK variable discovery #8497

• Introduce unit tests for the test-extension to show how variable definitions can be tested. Improve testing for Runtime SDK variable discovery #8497

• Use settings to modify the variables returned in different tests. Improve testing for Runtime SDK variable discovery #8497

• Add server library helper functions that allow easy definition of variables and enablement of a discoverVariables
  hook. Continued on this PR https://github.com/kubernetes-sigs/cluster-api/pull/8013/files

/kind feature
/area runtime-sdk

[killianmuldoon]

/triage accepted
/milestone v1.4

[k8s-ci-robot]

@killianmuldoon: You must be a member of the kubernetes-sigs/cluster-api-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Cluster API Maintainers and have them propose you as an additional delegate for this responsibility.

In response to this:

/triage accepted
/milestone v1.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sbueringer]

/milestone v1.4

[furkatgofurov7]

@killianmuldoon @sbueringer can we close this already? Asking because it has v1.4 milestone, so if not fully completed it may need to be moved to next milestone

[killianmuldoon]

/close

Will open some follow up issue to cover over the additional tasks not planned for the release.

[k8s-ci-robot]

@killianmuldoon: Closing this issue.

In response to this:

/close

Will open some follow up issue to cover over the additional tasks not planned for the release.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.