Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ISSUE-1018] Refine Storage Group Feature on OBS Release 1.3 branch #1044

Merged
merged 2 commits into from
Jul 31, 2023
Merged

[ISSUE-1018] Refine Storage Group Feature on OBS Release 1.3 branch #1044

merged 2 commits into from
Jul 31, 2023

Conversation

CraneShiEMC
Copy link
Collaborator

@CraneShiEMC CraneShiEMC commented Jul 30, 2023 •
edited
Loading

Purpose

Resolves #1018

Refine Storage Group Feature:

  1. Add StorageGroupStatus
  2. If we perform drive removal on the drive selected by some storagegroup with numberDrivesPerNode>0, the storagegroup's reconcilation should be triggered and it can select another drive if applicable.
  3. Block StorageGroupSpec Update
  4. Add StorageGroupController UT
  5. For the drive with existing volumes that matches the driveSelector of some storage group, the drive should be selected into its matching storage group when its exsiting volumes removed.
  6. The blocked storagegroup deletion can continue instantly after all exisitng volumes on drives of this storage group is removed.

PR checklist

  • Add link to the issue
  • Choose Project
  • Choose PR label
  • New unit tests added
  • Modified code has meaningful comments
  • All TODOs are linked with the issues
  • All comments are resolved

Testing

Manual test passed.
Custom CI passed: https://asd-ecs-jenkins.isus.emc.com/job/csi-custom-ci/1559/
Custom Acceptance Passed: https://asd-ecs-jenkins.isus.emc.com/job/csi-custom-acceptance-tar_b_ona/36/

@CraneShiEMC CraneShiEMC force-pushed the feature-refine-storage-group-release-obs-1.3 branch from 3cd7d62 to 2ba8131 Compare July 30, 2023 20:49
@CraneShiEMC
[ISSUE-1018] Refine Storage Group Feature (#1031)
b1575b8 
* add StorageGroupStatus

Signed-off-by: Shi, Crane <[email protected]>

* support StorageGroupStatus in current workflows

Signed-off-by: Shi, Crane <[email protected]>

* refine log

Signed-off-by: Shi, Crane <[email protected]>

* refine error handling

Signed-off-by: Shi, Crane <[email protected]>

* trigger storage group resync if applicable in drive removal

Signed-off-by: Shi, Crane <[email protected]>

* A drive whose Usage is REMOVED will not be selected in any storage group and its existing sg label takes no effect

Signed-off-by: Shi, Crane <[email protected]>

* sg feature will not apply to drive physically removed

Signed-off-by: Shi, Crane <[email protected]>

* handle the drive removal case of drive with manual sg label

Signed-off-by: Shi, Crane <[email protected]>

* fix go lint error

Signed-off-by: Shi, Crane <[email protected]>

* add UT case for drive-removal-triggered sg sync

Signed-off-by: Shi, Crane <[email protected]>

* improve UT coverage

Signed-off-by: Shi, Crane <[email protected]>

* refine sg annotation for drive removal

Signed-off-by: Shi, Crane <[email protected]>

* handle case of invalid sg for drive removal

Signed-off-by: Shi, Crane <[email protected]>

* also exclude removing sg for trigger sg resync in drive removal

Signed-off-by: Shi, Crane <[email protected]>

* refine sg removal status handling

Signed-off-by: Shi, Crane <[email protected]>

* Revert "refine error handling"

This reverts commit 06607e7.

Signed-off-by: Shi, Crane <[email protected]>

* refine log and some code logic

Signed-off-by: Shi, Crane <[email protected]>

* try to add immutability validation rule to storagegroup spec

Signed-off-by: Shi, Crane <[email protected]>

* upgrade controller-gen version to v0.9.2

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT initial

Signed-off-by: Shi, Crane <[email protected]>

* Revert "add storagegroupcontroller UT initial"

This reverts commit 1ea8660.

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* fix

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* refactor and add UT of storagegroupcontroller

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* fix storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* refine the logic of sg deletion

Signed-off-by: Shi, Crane <[email protected]>

* refine

Signed-off-by: Shi, Crane <[email protected]>

* fix bug

Signed-off-by: Shi, Crane <[email protected]>

* fix go-lint err

Signed-off-by: Shi, Crane <[email protected]>

* fix go-lint error

Signed-off-by: Shi, Crane <[email protected]>

* add drive IsClean support, decrease k8s api call, remove manual sg labeling support

Signed-off-by: Shi, Crane <[email protected]>

* fix

Signed-off-by: Shi, Crane <[email protected]>

* fix

Signed-off-by: Shi, Crane <[email protected]>

* fix UT

Signed-off-by: Shi, Crane <[email protected]>

* refine corner case handling

Signed-off-by: Shi, Crane <[email protected]>

* fix

Signed-off-by: Shi, Crane <[email protected]>

* refine and add UT to storagegroupcontroller

Signed-off-by: Shi, Crane <[email protected]>

* refine storagegroupcontroller and add UT

Signed-off-by: Shi, Crane <[email protected]>

* make controller svc's k8scache also sync sg and lvg objs'

Signed-off-by: Shi, Crane <[email protected]>

* use k8s cache, re-support sg label manual change and refine in sg ctrl

Signed-off-by: Shi, Crane <[email protected]>

* fix lint err

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* add storagegroupcontroller UT

Signed-off-by: Shi, Crane <[email protected]>

* storagegroup controller will not reconcile on drive delete event

Signed-off-by: Shi, Crane <[email protected]>

* not support Health, Status, Usage and IsClean as DriveSelector's MatchFields

Signed-off-by: Shi, Crane <[email protected]>

* in storagegroupcontroller's reconcile, only sync drive when reqName is uuid

Signed-off-by: Shi, Crane <[email protected]>

* refine the logic to avoid nil pointer error

Signed-off-by: Shi, Crane <[email protected]>

* revert the usage of k8scache

Signed-off-by: Shi, Crane <[email protected]>

* add custom storage group proposal draft

Signed-off-by: Shi, Crane <[email protected]>

* refine custom storage group proposal

Signed-off-by: Shi, Crane <[email protected]>

---------

Signed-off-by: CraneShiEMC <[email protected]>
@CraneShiEMC CraneShiEMC changed the title [ISSUE-1018] Refine Storage Group Feature Release OBS 1.3 [ISSUE-1018] Refine Storage Group Feature on OBS Release 1.3 branch Jul 31, 2023
@CraneShiEMC CraneShiEMC merged commit fc9d4f4 into rel/objectscale-1.3 Jul 31, 2023
3 of 5 checks passed
@CraneShiEMC CraneShiEMC deleted the feature-refine-storage-group-release-obs-1.3 branch July 31, 2023 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yimingwangdell yimingwangdell yimingwangdell approved these changes

@yanxiucai yanxiucai yanxiucai approved these changes

@libzhang libzhang Awaiting requested review from libzhang

@aiqiEMC aiqiEMC Awaiting requested review from aiqiEMC

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@CraneShiEMC @yanxiucai @yimingwangdell