Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AzureManagedMachinePool with Spot instances endlessly reconciles node taints #3976

Closed
nojnhuh opened this issue Sep 8, 2023 · 2 comments · Fixed by #4066
Closed

AzureManagedMachinePool with Spot instances endlessly reconciles node taints #3976

nojnhuh opened this issue Sep 8, 2023 · 2 comments · Fixed by #4066
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Milestone

Comments

@nojnhuh
Copy link
Contributor

nojnhuh commented Sep 8, 2023

/kind bug
/area managedclusters

What steps did you take and what happened:
Create an AzureMachinePool that looks like this, using Spot instances and with no taints defined:

apiVersion: cluster.x-k8s.io/v1beta1
kind: MachinePool
metadata:
  name: jon-aks-$NAME
  namespace: default
spec:
  clusterName: jon-aks
  replicas: 1
  template:
    metadata: {}
    spec:
      bootstrap:
        dataSecretName: ""
      clusterName: jon-aks
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: AzureManagedMachinePool
        name: jon-aks-$NAME
      version: v1.26.3
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureManagedMachinePool
metadata:
  name: jon-aks-$NAME
  namespace: default
spec:
  mode: User
  name: $NAME
  sku: Standard_D2s_v3
  scaleSetPriority: Spot
  scaling:
    minSize: 0
    maxSize: 9
  spotMaxPrice: "123.456789"
  scaleDownMode: Deallocate

After the agent pool is created, CAPZ continuously tries to update it with this diff:

  armcontainerservice.AgentPool{
  	Properties: &armcontainerservice.ManagedClusterAgentPoolProfileProperties{
  		... // 17 identical fields
  		NodeLabels:           {"kubernetes.azure.com/scalesetpriority": &"spot"},
  		NodePublicIPPrefixID: nil,
- 		NodeTaints:           nil,
+ 		NodeTaints:           []*string{&"kubernetes.azure.com/scalesetpriority=spot:NoSchedule"},
  		OSDiskSizeGB:         nil,
  		OSDiskType:           nil,
  		... // 19 identical fields
  	},
  	ID:   nil,
  	Name: nil,
  	Type: nil,
  }

What did you expect to happen:

The AzureManagedMachinePool converges.

Anything else you would like to add:

Environment:

@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Sep 8, 2023
@nojnhuh
Copy link
Contributor Author

nojnhuh commented Sep 8, 2023

lol some keyboard shortcuts really borked this. Updating...

@nojnhuh nojnhuh changed the title AzureManagedMachinePool with AzureManagedMachinePool with Spot instances endless reconciles node taints Sep 8, 2023
@nojnhuh nojnhuh changed the title AzureManagedMachinePool with Spot instances endless reconciles node taints AzureManagedMachinePool with Spot instances endlessly reconciles node taints Sep 11, 2023
@sonasingh46 sonasingh46 added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Sep 21, 2023
@sonasingh46 sonasingh46 added this to the v1.12 milestone Sep 21, 2023
@dtzar dtzar added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Sep 23, 2023
@willie-yao
Copy link
Contributor

/assign

@willie-yao willie-yao moved this to In Progress in CAPZ Planning Oct 2, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in CAPZ Planning Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants