E2e testing for Topology Mutation Hook and Runtime Hooks for Addon-Management

[sbueringer]

Goal of this issue is to implement e2e testing for the Topology Mutation Hook and Runtime Hooks for Addon management.
I've combined this in one (umbrella) issue as there are probably a lot of synergy effects.

The (rough) plan is the following:

• Implement a test RuntimeExtension which implements all Topology Mutation and Runtime Hooks for Addon management.
• Deploy it in our CAPD e2e tests during mgmt cluster setup.
• Test the RuntimeExtension by using it in an additional/new CAPD-specific e2e test which goes through the entire lifecycle of a Cluster (create+upgrade+delete).

Some details:

• RuntimeExtension should be a separate binary in a kubebuilder-like package below test/*
  • we probably only have go code, the config folder, Dockerfile (+ corresponding make targets in the top-level Makefile)
  • the config folder will (at least) contain the following resources: Deployment, Service, Certificate and Issuer (it shouldn't include ExtensionConfig)
  • we will deploy the resulting YAML via Apply without any clusterctl integration (the Apply should be part of the mgmt cluster setup not the cluster deployment)
  • we have to make sure the artifacts (docker image, YAML) are available for our test code
• For Topology Mutation Hook:
  • Implement some of our inline patches or additional ones (it's important that they are actually used in the e2e test to actually verify that it works)
  • We have to add the external patch config to (only) the ClusterClass used in the new e2e test
• For Runtime Hooks for Addon Management:
  • RuntimeExtension implementation should respond to lifecycle hook requests based on a ConfigMap (per cluster) (otherwise default to blocking)
• Create a new e2e test (only for CAPD) which goes through the entire Cluster lifecycle (create+update+delete) to verify all hooks are called
  • the idea is to configure the RuntimeExtension to first block the cluster lifecycle at a specific stage, then verify that the the lifecycle is blocked, reconfigure RuntimeExtension to continue, same for the next stage, ...
  • we have to deploy an ExtensionConfig as part of the test. This ExtensionConfig should only apply to the current test namespace (by leveraging namespaceSelector)

General considerations:

• Let's try to create a first unexported version of a server lib when implementing the RuntimeExtension

Later:

• would be good to be able to deploy the RuntimeExtension in our Tilt env as well
• the test RuntimeExtension should write to somewhere (e.g. ConfigMap) when it was called, this can be used for verification

/kind feature
/area runtime-sdk

[sbueringer]

@killianmuldoon When you've got some time. Can you please review the issue description if it makes sense to you.

I tried to mention all the points of previous discussions, not sure how much I missed but I guess if I got 80% of it it should be good enough :)

[killianmuldoon]

I think this captures everything 🙂

[sbueringer]

Just fyi. I'll take a look at this and maybe implement an early prototype (in the context of the topology mutation hook implementation)

[sbueringer]

Let's close this issue with #6635
We then make sure in the topology mutation / lifecycle hook issue that all hooks have sufficient coverage