Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent calling controller-expand if volume in-use #86

Merged
merged 2 commits into from
Jun 10, 2020

Conversation

gnufied
Copy link
Contributor

@gnufied gnufied commented Jun 3, 2020

If volume can not be expanded when in-use we must not call ControllerExpandVolume until we know volume is not in-use.

Fixes #81

Do not call controller-expand if volume is in-use and can not be expanded while in-use

@k8s-ci-robot k8s-ci-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Jun 3, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gnufied

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested review from msau42 and saad-ali June 3, 2020 23:05
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 3, 2020
pkg/controller/controller.go Outdated Show resolved Hide resolved

func (ctrl *resizeController) deletePod(obj interface{}) {
pod := parsePod(obj)
if pod != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here.

@@ -98,17 +112,38 @@ func NewResizeController(
DeleteFunc: ctrl.deletePVC,
}, resyncPeriod)

podInformer.Informer().AddEventHandlerWithResyncPeriod(cache.ResourceEventHandlerFuncs{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that the in-tree PVC protection controller also checks the pod. What is the advantage of checking thru the pod vs checking the VolumeAttachment object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because not all volume types support attach/detach and hence there may not be an VolumeAttachment object.

if ctrl.usedPVCs.hasInUseErrors(pvc) && ctrl.usedPVCs.checkForUse(pvc) {
// Record an event to indicate that resizer is not expanding the pvc
ctrl.eventRecorder.Event(pvc, v1.EventTypeWarning, util.VolumeResizeFailed,
fmt.Sprintf("CSI resizer is not expanding %s because it is in-use", pv.Name))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a message to indicate that the CSI driver only supports offline volume expansion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

}
pod, ok := obj.(*v1.Pod)
if !ok {
tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you pick a different name than "tombstone"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to staleObj

return false
}
// if this is a failed precondition error then that means driver does not support expansion
// of in-use volumes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment linking to the CSI spec description about this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 4, 2020
pkg/controller/controller.go Outdated Show resolved Hide resolved
pkg/controller/controller.go Outdated Show resolved Hide resolved
pkg/controller/controller.go Outdated Show resolved Hide resolved
@gnufied gnufied force-pushed the add-volume-in-use-error branch from 1bc70d8 to 38e5ebf Compare June 10, 2020 16:35
@gnufied
Copy link
Contributor Author

gnufied commented Jun 10, 2020

@xing-yang PTAL. addressed last comments.

@xing-yang
Copy link
Contributor

/retest

pvcInUse: true,
},
{
Name: "Resize PVC, no FS resize, pvc-inuse but no failedprecondition error",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one should be "with FS resize" as there is no failedprecondition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should not matter. The "FS resize" only decides whether node expansion is pending or not. But since in this case, all we are trying to test is - if a pvc was in-use but has no failedprecondition from before then expansion should proceed as usual.

@xing-yang
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 10, 2020
@k8s-ci-robot k8s-ci-robot merged commit 9805ed3 into kubernetes-csi:master Jun 10, 2020
@msau42 msau42 mentioned this pull request Jul 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Respect volume-in-use error when calling ControllerExpand volume
3 participants