Can't delete XRD due to missing "DeleteOptions" #2284
Comments
|
It's worth noting we're using |
|
Did a little digging. I can't immediately find any related issues on controller-runtime or the greater Googles. I've determined that this was a regression between Crossplane v1.1.1 and v1.2.0. I can reproduce it on both Kubernetes v1.16.15 and v.1.20.2 using Crossplane v1.2.0. |
|
Hrm, I had hoped to blame some underlying controller-runtime or API machinery changes between 1.1.1 and 1.2.0 but as far as I can tell none of that changed. 🤔 release-1.1...release-1.2 (see go.mod). |
|
Worth noting that we did go from crossplane-runtime ~v0.12 to v0.13, which bumped its version of |
I've confirmed there were negligible differences in the Kube dependencies between v1.1.1 and v1.2.0. |
|
It's also worth noting that the actual error is really weird I'm pretty sure This reads almost like some function like |
|
It seems like it was not the crossplane-runtime bump. I've tested the commit that made the bump and everything works fine. |
|
I went through a bit of a manual |
|
The above PR switches us over from adding Crossplane specific types to the default controller-manager scheme (which appears to be https://github.com/kubernetes/client-go/blob/master/kubernetes/scheme/register.go#L72) to the (recommended) approach of creating our own scheme and adding what we need. Notably Edit: Nope, I'm wrong. We're adding the same scheme controller-runtime seems to use as its default to our scheme. |
|
Well it's definitely something to do with how we're using the scheme. If I make this following changes to 4e8f359 it all works as expected: --- a/cmd/crossplane/core/core.go
+++ b/cmd/crossplane/core/core.go
@@ -22,11 +22,14 @@ import (
"github.com/pkg/errors"
"github.com/spf13/afero"
"gopkg.in/alecthomas/kingpin.v2"
+ extv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
+ extv1beta1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"github.com/crossplane/crossplane-runtime/pkg/logging"
+ "github.com/crossplane/crossplane/apis"
"github.com/crossplane/crossplane/internal/controller/apiextensions"
"github.com/crossplane/crossplane/internal/controller/pkg"
"github.com/crossplane/crossplane/internal/xpkg"
@@ -57,7 +60,7 @@ func FromKingpin(cmd *kingpin.CmdClause) (*Command, *InitCommand) {
}
// Run core Crossplane controllers.
-func (c *Command) Run(s *runtime.Scheme, log logging.Logger) error {
+func (c *Command) Run(_ *runtime.Scheme, log logging.Logger) error {
cfg, err := ctrl.GetConfig()
if err != nil {
return errors.Wrap(err, "Cannot get config")
@@ -65,7 +68,7 @@ func (c *Command) Run(s *runtime.Scheme, log logging.Logger) error {
log.Debug("Starting", "sync-period", c.Sync.String())
mgr, err := ctrl.NewManager(cfg, ctrl.Options{
- Scheme: s,
+ // Scheme: s,
LeaderElection: c.LeaderElection,
LeaderElectionID: "crossplane-leader-election-core",
SyncPeriod: &c.Sync,
@@ -74,6 +77,10 @@ func (c *Command) Run(s *runtime.Scheme, log logging.Logger) error {
return errors.Wrap(err, "Cannot create manager")
}
+ kingpin.FatalIfError(extv1.AddToScheme(mgr.GetScheme()), "cannot add client-go extensions v1 scheme")
+ kingpin.FatalIfError(extv1beta1.AddToScheme(mgr.GetScheme()), "cannot add client-go extensions v1beta1 scheme")
+ kingpin.FatalIfError(apis.AddToScheme(mgr.GetScheme()), "cannot add crossplane scheme")
+
if err := apiextensions.Setup(mgr, log); err != nil {
return errors.Wrap(err, "Cannot setup API extension controllers")
} |
|
I believe somehow, somewhere, we're missing a call to That said, it's not clear to me how the apiserver-apiextensions scheme is coming into play here at all. |
This appears to be required to avoid issues like crossplane#2284 Signed-off-by: Nic Cope <[email protected]>
This appears to be required to avoid issues like crossplane#2284 Signed-off-by: Nic Cope <[email protected]>
What happened?
When deleting an XRD I'm seeing the following event, and my XRD refuses to delete.
Warning TerminateComposite 2m37s defined/compositeresourcedefinition.apiextensions.crossplane.io cannot delete defined composite resources: no kind "DeleteOptions" is registered for version "admissionregistration.k8s.io/v1" in scheme "k8s.io/apiextensions-apiserver/pkg/apiserver/apiserver.go:52"Interestingly I never actually created any XRs using this XRD.
How can we reproduce it?
Here's the XRD I'm using:
This is the first time I'm testing composition with a Kubernetes cluster newer than 1.16. Perhaps this is due to the newer Kubernetes version I'm running?
What environment did it happen in?
Crossplane version:
The text was updated successfully, but these errors were encountered: