Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Node-Group MR status does not not reflect or update when node-group is upgrading versions #1570

Open
1 task done
whall89 opened this issue Nov 18, 2024 · 0 comments
Labels
bug Something isn't working needs:triage

Comments

@whall89
Copy link

whall89 commented Nov 18, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Affected Resource(s)

eks.aws.upbound.io/v1beta1 -- NodeGroup

Resource MRs required to reproduce the bug

- name: node-group base: apiVersion: eks.aws.upbound.io/v1beta1 kind: NodeGroup metadata: name: node-group labels: name: node-group cluster: cluster spec: forProvider: region: us-east-1 clusterNameSelector: matchLabels: name: cluster nodeRoleArnSelector: matchLabels: name: cluster-node-role subnetIdSelector: matchLabels: name: cluster-private-subnet-01 scalingConfig: - desiredSize: 2 maxSize: 10 minSize: 1 instanceTypes: - t3.medium version: - "1.30" tags: Name: node-group providerConfigRef: name: aws-provider patches: - fromFieldPath: "spec.parameters.ng-exname" toFieldPath: "metadata.annotations.crossplane.io/external-name" - fromFieldPath: "spec.parameters.kubernetesVersion" toFieldPath: "spec.forProvider.version" - fromFieldPath: "spec.parameters.region" toFieldPath: "spec.forProvider.region" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "metadata.name" transforms: - type: string string: type: Format fmt: "%s-node-group" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "metadata.labels.name" transforms: - type: string string: type: Format fmt: "%s-node-group" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "metadata.labels.cluster" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "spec.forProvider.clusterNameSelector.matchLabels.name" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "spec.forProvider.nodeRoleArnSelector.matchLabels.name" transforms: - type: string string: type: Format fmt: "%s-node-role" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "spec.forProvider.subnetIdSelector.matchLabels.name" transforms: - type: string string: type: Format fmt: "%s-private-subnet-01" - fromFieldPath: "spec.parameters.clusterName" toFieldPath: "spec.forProvider.tags.Name" transforms: - type: string string: type: Format fmt: "%s-node-group" - fromFieldPath: "spec.parameters.nodeGroupCount" toFieldPath: "spec.forProvider.scalingConfig[0].desiredSize" - fromFieldPath: "spec.parameters.nodeGroupMaxSize" toFieldPath: "spec.forProvider.scalingConfig[0].maxSize" - fromFieldPath: "spec.parameters.nodeGroupMinSize" toFieldPath: "spec.forProvider.scalingConfig[0].minSize" - fromFieldPath: "spec.parameters.nodeInstanceType" toFieldPath: "spec.forProvider.instanceTypes[0]" readinessChecks: - type: MatchString fieldPath: status.atProvider.status matchString: "ACTIVE"

Steps to Reproduce

  1. initiate upgrade of node group
  2. kubectl describe nodegroup.eks.aws.upbound.io/cluster-01-node-group
  3. confirm node-group is upgrading in AWS

What happened?

  • status of node-group MR stays ACTIVE in crossplane
  • status of node group in AWS Console changes to UPDATING

Relevant Error Output Snippet

Name:         cluster-01-node-group
Namespace:
Labels:       cluster=cluster-01
              crossplane.io/claim-name=cluster-01
              crossplane.io/claim-namespace=default
              crossplane.io/composite=cluster-01-XXXXX
              name=cluster-01-node-group
Annotations:  crossplane.io/composition-resource-name: node-group
              crossplane.io/external-create-pending: 2024-11-15T00:36:24Z
              crossplane.io/external-create-succeeded: 2024-11-15T00:36:24Z
              crossplane.io/external-name: cluster-01-node-group
API Version:  eks.aws.upbound.io/v1beta2
Kind:         NodeGroup
Metadata:
  Creation Timestamp:  2024-11-15T00:24:22Z
  Finalizers:
    finalizer.managedresource.crossplane.io
  Generate Name:  cluster-01-XXXXX-
  Generation:     6
  Owner References:
    API Version:           aws.example.org/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  XEKSCluster
    Name:                  cluster-01-XXXX
    UID:                   XXXXXX-XXXX-XXXX-XXXX
  Resource Version:        196408
  UID:                     XXXXXX-XXXX-XXXX-XXXX
Spec:
  Deletion Policy:  Delete
  For Provider:
    Ami Type:       AL2_x86_64
    Capacity Type:  ON_DEMAND
    Cluster Name:   cluster-01
    Cluster Name Ref:
      Name:  cluster-01
    Cluster Name Selector:
      Match Labels:
        Name:   cluster-01
    Disk Size:  20
    Instance Types:
      t3.micro
    Node Role Arn:  arn:aws:iam::***************:role/cluster-01-node-role
    Node Role Arn Ref:
      Name:  cluster-01-node-role
    Node Role Arn Selector:
      Match Labels:
        Name:  cluster-01-node-role
    Region:    us-east-1
    Scaling Config:
      Desired Size:  10
      Max Size:      10
      Min Size:      1
    Subnet Id Refs:
      Name:  cluster-01-private-subnet-01
    Subnet Id Selector:
      Match Labels:
        Name:  cluster-01-private-subnet-01
    Subnet Ids:
      subnet-08cce6e0ee4860a82
    Tags:
      Name:                         cluster-01-node-group
      Crossplane - Kind:            nodegroup.eks.aws.upbound.io
      Crossplane - Name:            cluster-01-node-group
      Crossplane - Providerconfig:  aws-provider
    Update Config:
      Max Unavailable:  1
    Version:            1.28
  Init Provider:
  Management Policies:
    *
  Provider Config Ref:
    Name:  aws-provider
Status:
  At Provider:
    Ami Type:       AL2_x86_64
    Arn:            arn:aws:eks:us-east-1:***************:nodegroup/cluster-01/cluster-01-node-group/XXXXXX-XXXX-XXXX-XXXX
    Capacity Type:  ON_DEMAND
    Cluster Name:   cluster-01
    Disk Size:      20
    Id:             cluster-01:cluster-01-node-group
    Instance Types:
      t3.micro
    Node Role Arn:    arn:aws:iam::***************:role/cluster-01-node-role
    Release Version:  1.28.13-20241109
    Resources:
      Autoscaling Groups:
        Name:                           eks-cluster-01-node-group-XXXXXX-XXXX-XXXX-XXXX
      Remote Access Security Group Id:
    Scaling Config:
      Desired Size:  10
      Max Size:      10
      Min Size:      1
    Status:          ACTIVE
    Subnet Ids:
      subnet-08cce6e0ee4860a82
    Tags:
      Name:                         cluster-01-node-group
      Crossplane - Kind:            nodegroup.eks.aws.upbound.io
      Crossplane - Name:            cluster-01-node-group
      Crossplane - Providerconfig:  aws-provider
    Tags All:
      Name:                         cluster-01-node-group
      Crossplane - Kind:            nodegroup.eks.aws.upbound.io
      Crossplane - Name:            cluster-01-node-group
      Crossplane - Providerconfig:  aws-provider
    Taint:
    Update Config:
      Max Unavailable:             1
      Max Unavailable Percentage:  0
    Version:                       1.28
  Conditions:
    Last Transition Time:  2024-11-15T21:17:31Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2024-11-15T00:38:45Z
    Reason:                Available
    Status:                True
    Type:                  Ready
    Last Transition Time:  2024-11-15T21:17:33Z
    Reason:                Success
    Status:                True
    Type:                  LastAsyncOperation
Events:                    <none>

Crossplane Version

1.18

Provider Version

1.16

Kubernetes Version

1.28

Kubernetes Distribution

EKS

Additional Info

having MR status correctly reflect actual status of AWS resource is important for sequencing upgrades

available status for node-group

status
The current status of the managed node group.

Type: String

Valid Values: CREATING | ACTIVE | UPDATING | DELETING | CREATE_FAILED | DELETE_FAILED | DEGRADED

@whall89 whall89 added bug Something isn't working needs:triage labels Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs:triage
Projects
None yet
Development

No branches or pull requests

1 participant