Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate machineset before reconciling #669

Conversation

ingvagabund
Copy link
Contributor

@ingvagabund ingvagabund commented Jan 11, 2019

Even thought the MachineSet type implements Validate() function,
it's not called by default. The validation function is responsible
for making sure every machine set matchLabels selector is matched with
machine template labels. Given the validation is not performed by default,
it is possible to create an invalid machineset that causes the machineset
controller to start creating machine object one by one without any upper
bound. Causing the machine controller to launch as many instances as
there is machine objects.

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

How to reproduce it? Deploy a machineset with at least one replica. Then run kubectl edit to change the machineset matchLabels the way it does not match machine template labels.

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:


@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 11, 2019
@frobware
Copy link

It would be great to have a unit test for this behavioural change.

@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch from 172dd77 to b886317 Compare January 11, 2019 11:51
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 11, 2019
@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch from b886317 to 0e6bdbe Compare January 11, 2019 11:55
@ingvagabund
Copy link
Contributor Author

/test pull-cluster-api-test

@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch from 0e6bdbe to 093a79d Compare January 14, 2019 11:44
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 14, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ingvagabund
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: kris-nova

If they are not already assigned, you can assign the PR to them by writing /assign @kris-nova in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch 9 times, most recently from 2b2ce48 to 35d9925 Compare January 14, 2019 13:15
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 14, 2019
@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch 2 times, most recently from b4b9985 to 8ab12f5 Compare January 14, 2019 14:00
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 14, 2019
@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch 2 times, most recently from 6fa6a53 to d529f03 Compare January 14, 2019 14:02
@ingvagabund
Copy link
Contributor Author

/test pull-cluster-api-test

@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch from d529f03 to 7ee9873 Compare January 14, 2019 14:26
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 14, 2019
@ingvagabund ingvagabund force-pushed the validate-machineset-in-reconsiliation branch from 7ee9873 to 0367fcb Compare January 14, 2019 14:32
Even thought the MachineSet type implements Validate() function,
it's not called by default. The validation function is responsible
for making sure every machine set matchLabels selector is matched with
machine template labels. Given the validation is not performed by default,
it is possible to create an invalid machineset that causes the machineset
controller to start creating machine object one by one without any upper
bound. Causing the machine controller to launch as many instances as
there is machine objects.
@ingvagabund
Copy link
Contributor Author

/retest

@@ -150,6 +150,13 @@ func (r *ReconcileMachineSet) Reconcile(request reconcile.Request) (reconcile.Re
}

klog.V(4).Infof("Reconcile machineset %v", machineSet.Name)

if errList := machineSet.Validate(); len(errList) > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could consider to run and Admission Webhook for this and similar features. This is better than we have today though

expectedRequest: reconcile.Request{NamespacedName: types.NamespacedName{Name: "invalidfoo", Namespace: "default"}},
verifyFnc: func() {
// expecting machineset validation error
if _, err := r.Reconcile(reconcile.Request{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could also validate reconcile.Result{} here

@enxebre
Copy link
Member

enxebre commented Jan 16, 2019

overall lgtm

@enxebre
Copy link
Member

enxebre commented Feb 11, 2019

This can probably be closed now since it is superseded by #739 cc @vincepri

@ingvagabund ingvagabund deleted the validate-machineset-in-reconsiliation branch February 13, 2019 13:44
jayunit100 pushed a commit to jayunit100/cluster-api that referenced this pull request Jan 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants